Computing Standard Deviation in a stream

by

As outlined in the Wikipedia article on the standard deviation, it is enough to keep track of the following three sums:

s0 = sum(1 for x in samples)
s1 = sum(x for x in samples)
s2 = sum(x*x for x in samples)

These sums are easily updated as new values arrive. The standard deviation can be calculated as

std_dev = math.sqrt((s0 * s2 - s1 * s1)/(s0 * (s0 - 1)))

Note that this way of computing the standard deviation can be numerically ill-conditioned if your samples are floating point numbers and the standard deviation is small compared to the mean of the samples. If you expect samples of this type, you should resort to Welford’s method (see the accepted answer).

More Related Contents:

  1. I need help subtracting variables from one another, both being numbers
  2. Fastest way to list all primes below N
  3. Sieve of Eratosthenes – Finding Primes Python
  4. How to round a number to significant figures in Python
  5. Check if a number is a perfect square
  6. Is there a math nCr function in python? [duplicate]
  7. How can I check if two segments intersect?
  8. How to know the angle between two vectors?
  9. How can you determine a point is between two other points on a line segment?
  10. Python and Powers Math
  11. Math operations from string [duplicate]
  12. Difference between the built-in pow() and math.pow() for floats, in Python?
  13. Python: sort function breaks in the presence of nan
  14. How can I use numpy.correlate to do autocorrelation?
  15. How can I quickly estimate the distance between two (latitude, longitude) points?
  16. Exponentials in python: x**y vs math.pow(x, y)
  17. Efficiently detect sign-changes in python
  18. How do I find the distance between two points? [closed]
  19. scipy is not optimizing and returns “Desired error not necessarily achieved due to precision loss”
  20. Linear regression of arrays containing NANs in Python/Numpy
  21. Is cube root integer?
  22. Multiple variables in SciPy’s optimize.minimize
  23. Python floating-point math is wrong [duplicate]
  24. Equivalent of `polyfit` for a 2D polynomial in Python
  25. Get all numbers that add up to a number
  26. Calculating the angle between two vectors in Python
  27. How to perform bilinear interpolation in Python
  28. Getting the subsets of a set in Python
  29. Python cos(90) and cos(270) not 0
  30. How do I get a decimal value when using the division operator in Python?

Leave a Comment