double quoted elements in csv cant read with pandas

by

This will work. It falls back to the python parser (as you have non-regular separators, e.g. they are comma and sometimes space). If you only have commas it would use the c-parser and be much faster.

In [1]: import csv

In [2]: !cat test.csv
"column1","column2", "column3", "column4", "column5", "column6"
"AM", "07", "1", "SD", "SD", "CR"
"AM", "08", "1,2,3", "PR,SD,SD", "PR,SD,SD", "PR,SD,SD"
"AM", "01", "2", "SD", "SD", "SD"

In [3]: pd.read_csv('test.csv',sep=',\s+',quoting=csv.QUOTE_ALL)
pandas/io/parsers.py:637: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators; you can avoid this warning by specifying engine="python".
  ParserWarning)
Out[3]: 
     "column1","column2" "column3"   "column4"   "column5"   "column6"
"AM"                "07"       "1"        "SD"        "SD"        "CR"
"AM"                "08"   "1,2,3"  "PR,SD,SD"  "PR,SD,SD"  "PR,SD,SD"
"AM"                "01"       "2"        "SD"        "SD"        "SD"

More Related Contents:

  1. UnicodeDecodeError when reading CSV file in Pandas with Python
  2. Writing a pandas DataFrame to CSV file
  3. How to make separator in pandas read_csv more flexible wrt whitespace, for irregular separators?
  4. Skip rows during csv import pandas
  5. How to avoid Python/Pandas creating an index in a saved csv?
  6. datetime dtypes in pandas read_csv
  7. What is the fastest way to upload a big csv file in notebook to work with python pandas?
  8. python – Using pandas structures with large csv(iterate and chunksize)
  9. Pandas read csv file with float values results in weird rounding and decimal digits
  10. Using numpy.genfromtxt to read a csv file with strings containing commas
  11. How to keep leading zeros in a column when reading CSV with Pandas?
  12. Key error when selecting columns in pandas dataframe after read_csv
  13. Precision lost while using read_csv in pandas
  14. Python Pandas: How to read only first n rows of CSV files in?
  15. How to read a column of csv as dtype list using pandas?
  16. Prevent pandas read_csv treating first row as header of column names
  17. OSError: Initializing from file failed on csv in Pandas
  18. How to avoid pandas creating an index in a saved csv
  19. Pandas – write Multiindex rows with to_csv
  20. Read CSV into a dataFrame with varying row lengths using Pandas
  21. How to add header row to a pandas DataFrame
  22. Read specific columns with pandas or other python module
  23. Convert percent string to float in pandas read_csv
  24. UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0x8b in position 1: invalid start byte, while reading csv file in pandas
  25. Easiest way to read csv files with multiprocessing in Pandas
  26. Pandas read_csv dtype read all columns but few as string
  27. Calculate summary statistics of columns in dataframe
  28. pandas dataframe with 2-rows header and export to csv
  29. Pandas df.to_csv(“file.csv” encode=”utf-8″) still gives trash characters for minus sign
  30. start index at 1 for Pandas DataFrame

Leave a Comment