problems in floating point comparison [duplicate]

by

f is using float precision, but 0.98 is in double precision by default, so the statement f <= 0.98 is compared using double precision.

The f is therefore converted to a double in the comparison, but may make the result slightly larger than 0.98.

Use 

if(f <= 0.98f)

or use a double for f instead.

In detail… assuming float is IEEE single-precision and double is IEEE double-precision.

These kinds of floating point numbers are stored with base-2 representation. In base-2 this number needs an infinite precision to represent as it is a repeated decimal:

0.98 = 0.1111101011100001010001111010111000010100011110101110000101000...

A float can only store 24 bits of significant figures, i.e.

       0.111110101110000101000111_101...
                                 ^ round off here
   =   0.111110101110000101001000

   =   16441672 / 2^24

   =   0.98000001907...

A double can store 53 bits of signficant figures, so 

       0.11111010111000010100011110101110000101000111101011100_00101000...
                                                              ^ round off here
   =   0.11111010111000010100011110101110000101000111101011100

   =   8827055269646172 / 2^53

   =   0.97999999999999998224...

So the 0.98 will become slightly larger in float and smaller in double.

More Related Contents:

  1. Dividing 1/n always returns 0.0 [duplicate]
  2. C – Serialization of the floating point numbers (floats, doubles)
  3. Comparing float and double
  4. error: incompatible types for a function that returns float in C
  5. How do I restrict a float value to only two places after the decimal point in C?
  6. How to represent FLOAT number in memory in C
  7. What operations and functions on +0.0 and -0.0 give different arithmetic results?
  8. What is the instruction that gives branchless FP min and max on x86?
  9. Is the behaviour of casting a negative double to unsigned int defined in the C standard? Different behaviour on ARM vs. x86
  10. Checking if float is an integer
  11. Representing integers in doubles
  12. What is the fastest way to convert float to int on x86
  13. Why does adding a small float to a large float just drop the small one?
  14. How does printf and co differentiate between float and double
  15. Comparing floating point numbers in C
  16. Comparing float and double in C
  17. Can I make gcc tell me when a calculation results in NaN or inf at runtime?
  18. Is it safe to assume floating point is represented using IEEE754 floats in C?
  19. Count number of digits after `.` in floating point numbers?
  20. Status of __STDC_IEC_559__ with modern C compilers
  21. Is there a tool to know whether a value has an exact binary representation as a floating point variable?
  22. Is fmod() exact when y is an integer?
  23. Encoding int value as an IEEE-754 float (binary32)
  24. Trouble with float on C [duplicate]
  25. When does underflow occur?
  26. Serialize double and float with C
  27. C printf using %d and %f
  28. pow() cast to integer, unexpected result
  29. C dynamically printf double, no loss of precision and no trailing zeroes
  30. How can floating point calculations be made deterministic?

Leave a Comment