Find empty or NaN entry in Pandas Dataframe

by

np.where(pd.isnull(df)) returns the row and column indices where the value is NaN:

In [152]: import numpy as np
In [153]: import pandas as pd
In [154]: np.where(pd.isnull(df))
Out[154]: (array([2, 5, 6, 6, 7, 7]), array([7, 7, 6, 7, 6, 7]))

In [155]: df.iloc[2,7]
Out[155]: nan

In [160]: [df.iloc[i,j] for i,j in zip(*np.where(pd.isnull(df)))]
Out[160]: [nan, nan, nan, nan, nan, nan]

Finding values which are empty strings could be done with applymap:

In [182]: np.where(df.applymap(lambda x: x == ''))
Out[182]: (array([5]), array([7]))

Note that using applymap requires calling a Python function once for each cell of the DataFrame. That could be slow for a large DataFrame, so it would be better if you could arrange for all the blank cells to contain NaN instead so you could use pd.isnull.

More Related Contents:

  1. Python: create a pandas data frame from a list
  2. How are iloc and loc different?
  3. How to convert index of a pandas dataframe into a column
  4. How to display pandas DataFrame of floats using a format string for columns?
  5. Python pandas insert list into a cell
  6. What rules does Pandas use to generate a view vs a copy?
  7. Pandas split DataFrame by column value
  8. Selecting a row of pandas series/dataframe by integer index
  9. What is the difference between using squared brackets or dot to access a column?
  10. How do I create test and train samples from one dataframe with pandas?
  11. Pandas DataFrame to List of Dictionaries
  12. data.frame rows to a list
  13. How to map numeric data into categories / bins in Pandas dataframe
  14. Export from pandas to_excel without row names (index)?
  15. Pandas expand rows from list data available in column
  16. Get column index from column name in python pandas
  17. python pandas flatten a dataframe to a list
  18. Creating an R dataframe row-by-row
  19. Why does df.apply(tuple) work but not df.apply(list)?
  20. Python Pandas: Get index of rows where column matches certain value
  21. Convert list into a pandas dataframe
  22. Python: Removing Rows on Count condition
  23. Using replace efficiently in pandas
  24. Convert List to Pandas Dataframe Column
  25. rbind dataframes in a list of lists
  26. Python Pandas Data frame creation
  27. Group DataFrame in 5-minute intervals
  28. Selecting a range of columns in a dataframe
  29. How to efficiently assign unique ID to individuals with multiple entries based on name in very large df
  30. start index at 1 for Pandas DataFrame

Leave a Comment