floating-point-conversion

Half-precision floating-point arithmetic on Intel chips

related: https://scicomp.stackexchange.com/questions/35187/is-half-precision-supported-by-modern-architecture – has some info about BFloat16 in Cooper Lake and Sapphire Rapids, and some non-Intel info. Sapphire Rapids will have both BF16 and FP16, with FP16 using the same IEEE754 binary16 format as F16C conversion instructions, not brain-float. And AVX512-FP16 has support for most math operations, unlike BF16 which just has conversion to/from … Read more

Comparing float and double

This is because 1.1 is not exactly representable in binary floating-point. But 1.5 is. As a result, the float and double representations will hold slightly different values of 1.1. Here is exactly the difference when written out as binary floating-point: (float) 1.1 = (0.00011001100110011001101)₂ (double)1.1 = (0.0001100110011001100110011001100110011001100110011010)₂ Thus, when you compare them (and the float … Read more

C floating point precision [duplicate]

You can only represent numbers exactly in IEEE754 (at least for the single and double precision binary formats) if they can be constructed from adding together inverted powers of two (i.e., 2-n like 1, 1/2, 1/4, 1/65536 and so on) subject to the number of bits available for precision. There is no combination of inverted … Read more

C – Serialization of the floating point numbers (floats, doubles)

Assuming you’re using mainstream compilers, floating point values in C and C++ obey the IEEE standard and when written in binary form to a file can be recovered in any other platform, provided that you write and read using the same byte endianess. So my suggestion is: pick an endianess of choice, and before writing … Read more

problems in floating point comparison [duplicate]

f is using float precision, but 0.98 is in double precision by default, so the statement f <= 0.98 is compared using double precision. The f is therefore converted to a double in the comparison, but may make the result slightly larger than 0.98. Use if(f <= 0.98f) or use a double for f instead. … Read more

Why does str(float) return more digits in Python 3 than Python 2?

No, there’s no PEP. There’s an issue in the bug tracker, and an associated discussion on the Python developers mailing list. While I was responsible for proposing and implementing the change, I can’t claim it was my idea: it had arisen during conversations with Guido at EuroPython 2010. Some more details: as already mentioned in … Read more

Dividing 1/n always returns 0.0 [duplicate]

(1/i) i is an int, so that’s integer division, resulting in 0 if i > 1. Use 1.0/i to get floating point division.