Pandas – Compute z-score for all columns

by

Using Scipy’s zscore function:

df = pd.DataFrame(np.random.randint(100, 200, size=(5, 3)), columns=['A', 'B', 'C'])
df

|    |   A |   B |   C |
|---:|----:|----:|----:|
|  0 | 163 | 163 | 159 |
|  1 | 120 | 153 | 181 |
|  2 | 130 | 199 | 108 |
|  3 | 108 | 188 | 157 |
|  4 | 109 | 171 | 119 |

from scipy.stats import zscore
df.apply(zscore)

|    |         A |         B |         C |
|---:|----------:|----------:|----------:|
|  0 |  1.83447  | -0.708023 |  0.523362 |
|  1 | -0.297482 | -1.30804  |  1.3342   |
|  2 |  0.198321 |  1.45205  | -1.35632  |
|  3 | -0.892446 |  0.792025 |  0.449649 |
|  4 | -0.842866 | -0.228007 | -0.950897 |

If not all the columns of your data frame are numeric, then you can apply the Z-score function only to the numeric columns using the select_dtypes function:

# Note that `select_dtypes` returns a data frame. We are selecting only the columns
numeric_cols = df.select_dtypes(include=[np.number]).columns
df[numeric_cols].apply(zscore)

|    |         A |         B |         C |
|---:|----------:|----------:|----------:|
|  0 |  1.83447  | -0.708023 |  0.523362 |
|  1 | -0.297482 | -1.30804  |  1.3342   |
|  2 |  0.198321 |  1.45205  | -1.35632  |
|  3 | -0.892446 |  0.792025 |  0.449649 |
|  4 | -0.842866 | -0.228007 | -0.950897 |

More Related Contents:

  1. How are iloc and loc different?
  2. How to convert index of a pandas dataframe into a column
  3. pandas loc vs. iloc vs. at vs. iat?
  4. What rules does Pandas use to generate a view vs a copy?
  5. Pandas split DataFrame by column value
  6. Selecting a row of pandas series/dataframe by integer index
  7. How to avoid Python/Pandas creating an index in a saved csv?
  8. What is the difference between using squared brackets or dot to access a column?
  9. how do I insert a column at a specific column index in pandas?
  10. Python Pandas: Get index of rows which column matches certain value
  11. Remove index name in pandas
  12. Accessing Pandas column using squared brackets vs using a dot (like an attribute)
  13. Python pandas slice dataframe by multiple index ranges
  14. Update row values where certain condition is met in pandas
  15. Constructing a co-occurrence matrix in python pandas
  16. Get column index from column name in python pandas
  17. How to avoid pandas creating an index in a saved csv
  18. Python Pandas: Get index of rows where column matches certain value
  19. Selecting pandas column by location
  20. Pandas max value index
  21. Get first row value of a given column
  22. Python: Removing Rows on Count condition
  23. Using replace efficiently in pandas
  24. T-test in Pandas
  25. In Python pandas, start row index from 1 instead of zero without creating additional column
  26. plotting a histogram on a Log scale with Matplotlib
  27. Selecting a range of columns in a dataframe
  28. How to efficiently assign unique ID to individuals with multiple entries based on name in very large df
  29. What’s the alternative to pandas chain indexing?
  30. start index at 1 for Pandas DataFrame

Leave a Comment