For multiplication we simply add the number of decimal places in each argument together (using pen and paper) to work out output dec places. But division just blows your head apart. I’m off to lie down now. In SQL terms though, it’s exactly as expected. –Precision = p1 – s1 + s2 + max(6, s1 … Read more
For money, always decimal. It’s why it was created. If numbers must add up correctly or balance, use decimal. This includes any financial storage or calculations, scores, or other numbers that people might do by hand. If the exact value of numbers is not important, use double for speed. This includes graphics, physics or other … Read more
You are running into the old problem with floating point numbers that not all numbers can be represented exactly. The command line is just showing you the full floating point form from memory. With floating point representation, your rounded version is the same number. Since computers are binary, they store floating point numbers as an … Read more
Why would DateTime.Now be made less precise than what most CPU clocks could handle? A good clock should be both precise and accurate; those are different. As the old joke goes, a stopped clock is exactly accurate twice a day, a clock a minute slow is never accurate at any time. But the clock a … Read more
I know of no way to way to make normal floating points deterministic in .net. The JITter is allowed to create code that behaves differently on different platforms(or between different versions of .net). So using normal floats in deterministic .net code is not possible. The workarounds I considered: Implement FixedPoint32 in C#. While this is … Read more
For a given IEEE-754 floating point number X, if 2^E <= abs(X) < 2^(E+1) then the distance from X to the next largest representable floating point number (epsilon) is: epsilon = 2^(E-52) % For a 64-bit float (double precision) epsilon = 2^(E-23) % For a 32-bit float (single precision) epsilon = 2^(E-10) % For a … Read more
The reason it is only a suggestion is that you could quite easily write a print function that ignored the options value. The built-in printing and formatting functions do use the options value as a default. As to the second question, since R uses finite precision arithmetic, your answers aren’t accurate beyond 15 or 16 … Read more
Huge difference. As the name implies, a double has 2x the precision of float[1]. In general a double has 15 decimal digits of precision, while float has 7. Here’s how the number of digits are calculated: double has 52 mantissa bits + 1 hidden bit: log(253)÷log(10) = 15.95 digits float has 23 mantissa bits + … Read more
Because floating point arithmetic != real number arithmetic. An illustration of the difference due to imprecision is, for some floats a and b, (a+b)-b != a. This applies to any language using floats. Since floating point are binary numbers with finite precision, there’s a finite amount of representable numbers, which leads accuracy problems and surprises … Read more
These accuracy problems are due to the internal representation of floating point numbers and there’s not much you can do to avoid it. By the way, printing these values at run-time often still leads to the correct results, at least using modern C++ compilers. For most operations, this isn’t much of an issue.