Why does numpy std() give a different result to matlab std()?

by

The NumPy function np.std takes an optional parameter ddof: “Delta Degrees of Freedom”. By default, this is 0. Set it to 1 to get the MATLAB result:

>>> np.std([1,3,4,6], ddof=1)
2.0816659994661326

To add a little more context, in the calculation of the variance (of which the standard deviation is the square root) we typically divide by the number of values we have.

But if we select a random sample of N elements from a larger distribution and calculate the variance, division by N can lead to an underestimate of the actual variance. To fix this, we can lower the number we divide by (the degrees of freedom) to a number less than N (usually N-1). The ddof parameter allows us change the divisor by the amount we specify.

Unless told otherwise, NumPy will calculate the biased estimator for the variance (ddof=0, dividing by N). This is what you want if you are working with the entire distribution (and not a subset of values which have been randomly picked from a larger distribution). If the ddof parameter is given, NumPy divides by N - ddof instead.

The default behaviour of MATLAB’s std is to correct the bias for sample variance by dividing by N-1. This gets rid of some of (but probably not all of) of the bias in the standard deviation. This is likely to be what you want if you’re using the function on a random sample of a larger distribution.

The nice answer by @hbaderts gives further mathematical details.

More Related Contents:

  1. Need a python module for numerical and scientific computing other than NumPy and SciPy
  2. find length of sequences of identical values in a numpy array (run length encoding)
  3. A tool to convert MATLAB code to Python [closed]
  4. Weighted standard deviation in NumPy
  5. Python equivalent of MATLAB’s “ismember” function
  6. What is the equivalent of MATLAB’s repmat in NumPy
  7. reading v 7.3 mat file in python
  8. “Converting” Numpy arrays to Matlab and vice versa
  9. Numpy loading csv TOO slow compared to Matlab
  10. Performance: Matlab vs Python
  11. Selecting close matches from one array based on another reference array
  12. How can I efficiently process a numpy array in blocks similar to Matlab’s blkproc (blockproc) function
  13. read matlab v7.3 file into python list of numpy arrays via h5py
  14. Complex matlab-like data structure in python (numpy/scipy)
  15. Procrustes Analysis with NumPy?
  16. Numpy equivalent of MATLAB’s cell array
  17. Improve performance of converting numpy array to MATLAB double
  18. Why does numpy.histogram (Python) leave off one element as compared to hist in Matlab?
  19. How to print the full NumPy array, without truncation?
  20. Numpy array assignment with copy
  21. Repeating each element of a numpy array 5 times
  22. Calculate weighted average using a pandas/dataframe
  23. Comparing previous row values in Pandas DataFrame
  24. Ambiguity in Pandas Dataframe / Numpy Array “axis” definition
  25. How to create a numpy array of arbitrary length strings?
  26. Counting unique values in a column in pandas dataframe like in Qlik?
  27. How can I count the number of consecutive TRUEs in a DataFrame?
  28. How to read a v7.3 mat file via h5py?
  29. Weighted random sample without replacement in python
  30. Is there a MATLAB accumarray equivalent in numpy?

Leave a Comment