Why IEEE754 single-precision float has only 7 digit precision?

by

23 fraction bits (22-0) of the significand appear in the memory format but the total precision is actually 24 bits since we assume there is a leading 1. This is equivalent to log10(2^24) ≈ 7.225 decimal digits.

Double-precision float has 52 bits in fraction, plus the leading 1 is 53. Therefore a double can hold log10(2^53) ≈ 15.955 decimal digits, not quite 16.

Note: The leading 1 is not a sign bit. It is actually (-1)^sign * 1.ffffffff * 2^(eeee-constant) but we need not store the leading 1 in the fraction. The sign bit must still be stored

There are some numbers that cannot be represented as a sum of powers of 2, such as 1/9:

>>>> double d = 0.111111111111111;
>>>> System.out.println(d + "\n" + d*10);
0.111111111111111
1.1111111111111098

If a financial program were to do this calculation over and over without self-correcting, there would eventually be discrepancies.

>>>> double d = 0.111111111111111;
>>>> double sum = 0;
>>>> for(int i=0; i<1000000000; i++) {sum+=d;}
>>>> System.out.println(sum);
111111108.91914201

After 1 billion summations, we are missing over $2.

More Related Contents:

  1. biggest integer that can be stored in a double
  2. Which is the first integer that an IEEE 754 float is incapable of representing exactly?
  3. What is the difference between float and double?
  4. What range of numbers can be represented in a 16-, 32- and 64-bit IEEE-754 systems?
  5. Is floating-point math consistent in C#? Can it be?
  6. Why converting from float to double changes the value?
  7. Representing integers in doubles
  8. Extreme numerical values in floating-point precision in R
  9. Matlab vs C++ Double Precision
  10. Next higher/lower IEEE double precision number
  11. Why are floating point numbers inaccurate?
  12. Retain precision with double in Java
  13. Why can’t Rust infer the resulting type of Iterator::sum?
  14. What is a subnormal floating point number?
  15. How many double numbers are there between 0.0 and 1.0?
  16. How can I handle precision error with float in Java?
  17. Emulate “double” using 2 “float”s
  18. C- Floating point precision
  19. Python method-wrapper type?
  20. Set back default floating point print precision in C++
  21. Double vs float on the iPhone
  22. How do I print in Rust the type of a variable?
  23. Is there a functional difference between “2.00” and “2.00f”?
  24. Are the bit patterns of NaNs really hardware-dependent?
  25. Is there a way to create a data type that only accepts a range of values?
  26. Denormalized Numbers – IEEE 754 Floating Point
  27. Why does Excel not round according to 8-byte IEEE 754
  28. Exact decimal datatype for C++?
  29. What uses do floating point NaN payloads have?
  30. Does the C++ standard specify anything on the representation of floating point numbers?

Leave a Comment