Handling Variable Number of Columns with Pandas – Python

by

One way which seems to work (at least in 0.10.1 and 0.11.0.dev-fc8de6d):

>>> !cat ragged.csv
1,2,3
1,2,3,4
1,2,3,4,5
1,2
1,2,3,4
>>> my_cols = ["A", "B", "C", "D", "E"]
>>> pd.read_csv("ragged.csv", names=my_cols, engine="python")
   A  B   C   D   E
0  1  2   3 NaN NaN
1  1  2   3   4 NaN
2  1  2   3   4   5
3  1  2 NaN NaN NaN
4  1  2   3   4 NaN

Note that this approach requires that you give names to the columns you want, though. Not as general as some other ways, but works well enough when it applies.

More Related Contents:

  1. Filter pandas DataFrame by substring criteria
  2. Adding value labels on a matplotlib bar chart
  3. Pandas column of lists, create a row for each list element
  4. Compare two DataFrames and output their differences side-by-side
  5. Selecting a row of pandas series/dataframe by integer index
  6. pandas: merge (join) two data frames on multiple columns
  7. Remove duplicates from dataframe, based on two columns A,B, keeping row with max value in another column C
  8. How to count duplicate rows in pandas dataframe?
  9. Concatenate a list of pandas dataframes together
  10. How do I convert strings in a Pandas data frame to a ‘date’ data type?
  11. How to drop duplicates based on two or more subsets criteria in Pandas data-frame
  12. How to normalize json correctly by Python Pandas
  13. Replace invalid values with None in Pandas DataFrame
  14. How to show all columns’ names on a large pandas dataframe?
  15. Pandas monthly rolling operation
  16. Count number of words per row
  17. How to select all columns except one in pandas?
  18. Shift column in pandas dataframe up by one?
  19. In Pandas, does .iloc method give a copy or view?
  20. Move non-empty cells to the left in pandas DataFrame
  21. pandas logical and operator with and without brackets produces different results [duplicate]
  22. How to json_normalize a column with NaNs
  23. HDF5 – concurrency, compression & I/O performance [closed]
  24. Convert Pandas Series to DateTime in a DataFrame
  25. How to use a conditional statement based on DataFrame boolean value in pandas
  26. Stacked bar chart in Seaborn
  27. How to add/subtract time (hours, minutes, etc.) from a Pandas DataFrame.Index whos objects are of type datetime.time?
  28. How to easily share a sample dataframe using df.to_dict()
  29. Remove substring from column based on another column
  30. plotting value_counts() in seaborn barplot

Leave a Comment