Need a special offer?Find out if your project fits.
+

floating-point error vs numberformatting

Answered
Jaco asked on January 27, 2021

Hi Flexmonster,
 
I have got an issue with numbers returned from an ElasticSearch datasource.
 
Normally the bucket returned shows correct numbers:
 "total_sum_f_costtotal_f":{
"value":1313.75
},
But sometimes the number has a floating-point error. 
"total_sum_f_costtotal_f":{
"value":"1786.6999999999998"
},
 
Which should not be a problem, because Flexmonster has numberformatting....but in this case the entire column gets formatted, except the floating-point error numbers
 
Is this a known issue or is there workaround for problem?
 
Regards Jaco

3 answers

Public
Vera Didenko Vera Didenko Flexmonster January 28, 2021

Hello, Jaco,
 
Thank you for reaching out to us.
 
We have not managed to reproduce the issue on our end using the latest version of Flexmonster.
Please see a possible workaround described in the following Support forum thread.
Could you please let us know if this helps to resolve the issue?
 
If the issue persists, could you please provide your Flexmonster report configuration and, if possible, a dummy data set with which the issue is reproducible?
This will greatly help us to understand the situation better and make further progress in solving this case.
 
Thank you, and looking forward to your reply.
 
Kind regards,
Vera

Public
Jaco February 12, 2021

Hi Vera,
 
Thanks for your answer. I had prepared an example to share with you, but i noticed that despite receiving the following value from Elastic Search:
total_sum_f_costtotal_f{value: 1786.6999999999998}
Flexmonster parsed this value correctyle to 1787.70. 
But with the same data in our application, it stays "1786.6999999999998"
 
Since our application is inbetween Elastic Search and Flexmonster, the problem lies somewhere in this middlelayer. So for now, we have added a check that rounds all numbers from Elastic Search at 2 decimals and now we see the correct numbers in Flexmonster.
 
Regards Jaco

Public
Vera Didenko Vera Didenko Flexmonster February 15, 2021

Hello, Jaco, 
 
Thank you for the update.
 
We are glad to hear that you managed to resolve the issue.
Should any questions arise, please feel free to reach out.
 
Kind regards,
Vera

Please login or Register to Submit Answer