Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Most decimal numbers cannot be represented exactly by floating point, so there is error in decimal to floating point conversion. You can see how this conversion works using this calculator: https://www.h-schmidt.net/FloatConverter/IEEE754.html

Most real numbers cannot be presented exactly, including most decimals and some larger (positive) integers.

The smallest such integer in single-precision floating point is 16777217. This number cannot be represented exactly: the next floating point number after 16777216 is 16777218. Above that number you can no longer represent whole numbers.

Even at one decimal place of precision most numbers cannot be represented exactly. For example, 0.1, 0.2, 0.3 and 0.4 cannot and 0.5 is the first positive number that can be.

At two decimal places, there are virtually no numbers that can be represented exactly (0.25 is the first one that can be), so there's always some error and therefore the possibility of rounding errors. Even if you accept that, after 131072.01, some numbers cannot be represented exactly at all even with rounding: 131072 can be represented exactly, but the next number is 131072.015625 which rounds to 131072.02. No single-precision floating point numbers round to 131072.01.

Of course I'm using single-precision floating point here. These same problems exist for double-precision but at much larger numbers: the first integer that cannot be represented in double-precision is 9007199254740993. Either way, using floating point exposes you to the risk of errors in your calculations.

Hope that helps.



thank you,

my question however was about functionally, when is it advised not to you floating point arithmetic.


As mentioned, it's when you must make sure that you have perfect precision. Money and counting items is one such example. If I am a bank and get audited, if I have a transaction that has kept rounding up, and creating eg. 1 cent every time. After 10k transactions I will have "created" $100 from nowhere. The auditing firm will not be happy about this.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: