For “more precise”: this recipe in the Python Cookbook has summation algorithms which keep the full precision (by keeping track of the subtotals). Code is in Python but even if you don’t know Python it’s clear enough to adapt to any other language. All the details are given in this paper.
The rounding error is not random and the way it is implemented it attempts to minimise the error. This means that sometimes the error is not visible, or there is not error. For example 0.1 is not exactly 0.1 i.e. new BigDecimal(“0.1”) < new BigDecimal(0.1) but 0.5 is exactly 1.0/2 This program shows you the … Read more
There are no integers in Javascript. Numbers are double precision floating point, which gives you a precision of 15-16 digits. This is consistent with your results.
I think it reflects more on your understanding of floating point types than on Python. See my article about floating point numbers (.NET-based, but still relevant) for the reasons behind this “inaccuracy”. If you need to keep the exact decimal representation, you should use the decimal module.
ApFloat is a library which contains arbitrary-precision approximations of trigometric functions and non-integer powers both; however, it uses its own internal representations, rather than BigDecimal and BigInteger. I haven’t used it before, so I can’t vouch for its correctness or performance characteristics, but the api seems fairly complete.
Binary floating point cannot represent the value 0.1 exactly, because its binary expansion does not have a finite number of digits (in exactly the same way that the decimal expansion of 1/7 does not). The binary expansion of 0.1 is 0.000110011001100110011001100… When truncated to IEEE-754 single precision, this is approximately 0.100000001490116119 in decimal. This means … Read more
If I was to implement some timer mechanism in C++, I would probably be using the std::chrono namespace together with std::priority_queue. #include <functional> #include <queue> #include <chrono> #include <sys/time.h> // for `time_t` and `struct timeval` namespace events { struct event { typedef std::function<void()> callback_type; typedef std::chrono::time_point<std::chrono::system_clock> time_type; event(const callback_type &cb, const time_type &when) : callback_(cb), … Read more
An IEEE754 64-bit double can represent any 32-bit integer, simply because it has 53-odd(a) bits available for precision and the 32-bit integer only needs, well, 32 🙂 It would be plausible for a (non IEEE754 double precision) 64-bit floating point number to have less than 32 bits of precision. That would allow truly huge numbers … Read more
You can find out with std::numeric_limits: #include <iostream> // std::cout #include <limits> // std::numeric_limits int main(){ std::cout << std::numeric_limits<long double>::digits10 << std::endl; }
Numeric precision refers to the maximum number of digits that are present in the number. ie 1234567.89 has a precision of 9 Numeric scale refers to the maximum number of decimal places ie 123456.789 has a scale of 3 Thus the maximum allowed value for decimal(5,2) is 999.99