R uses IEEE 754 double-precision floating-point numbers. Floating-point numbers are more dense near zero. This is a result of their being designed to compute accurately (the equivalent of about 16 significant decimal digits, as you have noticed) over a very wide range. Perhaps you expected a fixed-point system with uniform absolute precision. In practice fixed-point … Read more
Getting the number of digits to the left of the decimal point is easy: int(log10(x))+1 The number of digits to the right of the decimal point is trickier, because of the inherent inaccuracy of floating point values. I’ll need a few more minutes to figure that one out. Edit: Based on that principle, here’s the … Read more
They’re doubles. vpa() is simply choosing to display non-significant digits beyond the floating point relative accuracy, where printf() and disp() truncate or zero them out. You’re only getting your original four digits back out because the literal you chose to initialize num with just happens to be the exact decimal expansion of a binary double … Read more
I found the bug. .NET does the following in clr\src\vm\comnumber.cpp: DoubleToNumber(value, DOUBLE_PRECISION, &number); if (number.scale == (int) SCALE_NAN) { gc.refRetVal = gc.numfmt->sNaN; goto lExit; } if (number.scale == SCALE_INF) { gc.refRetVal = (number.sign? gc.numfmt->sNegativeInfinity: gc.numfmt->sPositiveInfinity); goto lExit; } NumberToDouble(&number, &dTest); if (dTest == value) { gc.refRetVal = NumberToString(&number, ‘G’, DOUBLE_PRECISION, gc.numfmt); goto lExit; } DoubleToNumber(value, … Read more
How can I exactly construct a time stamp of actual time with milliseconds precision? I suspect you mean millisecond accuracy. DateTime has a lot of precision, but is fairly coarse in terms of accuracy. Generally speaking, you can’t. Usually the system clock (which is where DateTime.Now gets its data from) has a resolution of around … Read more
The reason you’re not getting your exact value of a is that R is storing it as a double instead of as an integer. Because a is very large, there is some rounding that takes place when you assign a. Normally to store things as integers you would use L at the end of the … Read more
You can get the precision before you change it, with std::ios_base::precision and then use that to change it back later. You can see this in action with: #include <ios> #include <iostream> #include <iomanip> int main (void) { double d = 3.141592653589; std::streamsize ss = std::cout.precision(); std::cout << “Initial precision = ” << ss << ‘\n’; … Read more
Using lbc instead of lmapm would be easier because lbc is self-contained. local bc = require”bc” s=bc.pow(2,1000):tostring() z=0 for i=1,#s do z=z+s:byte(i)-(“0”):byte(1) end print(z)
The 8 refers to the number of bytes that the data type uses. So a 32-bit integer is integer*4 along the same lines. A quick search found this guide to Fortran data types, which includes: The “real*4” statement specifies the variable names to be single precision 4-byte real numbers which has 7 digits of accuracy … Read more
A BigDecimal is defined by two values: an arbitrary precision integer and a 32-bit integer scale. The value of the BigDecimal is defined to be . Precision: The precision is the number of digits in the unscaled value. For instance, for the number 123.45, the precision returned is 5. So, precision indicates the length of … Read more