Remove NaN values from pandas dataframe and reshape table [duplicate]

by

You need apply with dropna, only is necessary create numpy array and reassign Series for reset indices:

df.apply(lambda x: pd.Series(x.dropna().values))

Sample:

df = pd.DataFrame({'B':[4,np.nan,4,np.nan,np.nan,4],
                   'C':[7,np.nan,9,np.nan,2,np.nan],
                   'D':[1,3,np.nan,7,np.nan,np.nan],
                   'E':[np.nan,3,np.nan,9,2,np.nan]})

print (df)
     B    C    D    E
0  4.0  7.0  1.0  NaN
1  NaN  NaN  3.0  3.0
2  4.0  9.0  NaN  NaN
3  NaN  NaN  7.0  9.0
4  NaN  2.0  NaN  2.0
5  4.0  NaN  NaN  NaN

df1 = df.apply(lambda x: pd.Series(x.dropna().values))
print (df1)
     B    C    D    E
0  4.0  7.0  1.0  3.0
1  4.0  9.0  3.0  9.0
2  4.0  2.0  7.0  2.0

More Related Contents:

  1. How do I select rows from a DataFrame based on column values?
  2. Aggregation in Pandas
  3. Convert DataFrame column type from string to datetime
  4. How to apply a function to two columns of Pandas dataframe
  5. How to convert an XML file to nice pandas dataframe?
  6. Dynamically evaluate an expression from a formula in Pandas
  7. Constructing pandas DataFrame from values in variables gives “ValueError: If using all scalar values, you must pass an index”
  8. Can pandas automatically read dates from a CSV file?
  9. df.append() is not appending to the DataFrame
  10. How to get rid of “Unnamed: 0” column in a pandas DataFrame read in from CSV file?
  11. pandas dataframe str.contains() AND operation
  12. Pandas groupby mean – into a dataframe?
  13. pandas groupby without turning grouped by column into index
  14. Tilde sign in pandas DataFrame
  15. Drop columns whose name contains a specific string from pandas DataFrame
  16. Find the unique values in a column and then sort them
  17. how do I calculate a rolling idxmax
  18. Python Pandas How to assign groupby operation results back to columns in parent dataframe?
  19. Redefining the Index in a Pandas DataFrame object
  20. From password-protected Excel file to pandas DataFrame
  21. How to filter a pandas dataframe based on the length of a entry
  22. How can repetitive rows of data be collected in a single row in pandas?
  23. Group dataframe and get sum AND count?
  24. How is pandas groupby method actually working?
  25. Replace value for a selected cell in pandas DataFrame without using index
  26. Pandas groupby and aggregation output should include all the original columns (including the ones not aggregated on)
  27. Selecting a range of columns in a dataframe
  28. Replace NaN with empty list in a pandas dataframe
  29. Get weekday/day-of-week for Datetime column of DataFrame
  30. When is it appropriate to use df.value_counts() vs df.groupby(‘…’).count()?

Leave a Comment