Understanding floating point problems

by

Start by reading What Every Computer Scientist Should Know About Floating Point:
http://docs.sun.com/source/806-3568/ncg_goldberg.html

Short answer: double precision floats (which are the default in JavaScript) have about 16 decimal digits of precision. Rounding can vary from platform to platform. If it is absolutely essential that you get the consistently right answer, you should do rational arithmetic yourself (this doesn’t need to be hard – for currency, maybe you can just multiply by 100 to store the number of cents as an integer).

But if it suffices to get the answer with a high degree of precision, floats should be good enough, especially double precision.

More Related Contents:

  1. Why does “dtoa.c” contain so much code?
  2. How to capture drop-down, checkbox and radio selected text? [closed]
  3. What is the best workaround for the WCF client `using` block issue?
  4. When should I use double instead of decimal?
  5. Difference in months between two dates
  6. Way to have String.Replace only hit “whole words”
  7. How can I insert an image into a RichTextBox?
  8. Does C# have an equivalent to JavaScript’s encodeURIComponent()?
  9. How do I determine a mapped drive’s actual path?
  10. Get File Icon used by Shell
  11. How do I kill a process using Vb.NET or C#?
  12. Fastest method to remove Empty rows and Columns From Excel Files using Interop
  13. How to find overlapping controls that I added to form?
  14. Inconsistency in divide-by-zero behavior between different value types
  15. What is a predicate in c#? [duplicate]
  16. Start new process, without being a child of the spawning process
  17. .NET Configuration (app.config/web.config/settings.settings)
  18. How to Start a Process Unelevated
  19. Is String.Format as efficient as StringBuilder
  20. How to convert Decimal to Double in C#?
  21. Why is the “f” required when declaring floats?
  22. Save Settings in VB.Net or C#
  23. .NET graph library around? [closed]
  24. Server did not recognize the value of HTTP Header SOAPAction
  25. Asp .NET Button – OnClientClick=”return function()” vs OnClientClick=”function()”
  26. Validating Recaptcha 2 (No CAPTCHA reCAPTCHA) in ASP.NET’s server side
  27. How does the “Using” statement translate from C# to VB?
  28. Document.Ready() is not working after PostBack
  29. Objects implicitly instantiated in vb.net?
  30. JavaScript: Alert.Show(message) From ASP.NET Code-behind

Leave a Comment