Escaping separator within double quotes, in awk

by

It’s easy, with GNU awk 4:

zsh-4.3.12[t]% awk '{ 
 for (i = 0; ++i <= NF;)
   printf "field %d => %s\n", i, $i
 }' FPAT='([^,]+)|("[^"]+")' infile
field 1 => filed1
field 2 => filed2
field 3 => field3
field 4 => "field4,FOO,BAR"
field 5 => field5

Adding some comments as per OP requirement.

From the GNU awk manual on “Defining fields by content:

The value of FPAT should be a string that provides a regular
expression. This regular expression describes the contents of each
field. In the case of CSV data as presented above, each field is
either “anything that is not a comma,” or “a double quote, anything
that is not a double quote, and a closing double quote.” If written as
a regular expression constant, we would have /([^,]+)|("[^"]+")/. Writing this as a string
requires us to escape the double quotes, leading to:

FPAT = "([^,]+)|(\"[^\"]+\")"

Using + twice, this does not work properly for empty fields, but it can be fixed as well:

As written, the regexp used for FPAT requires that each field contain at least one character. A straightforward modification (changing the first ‘+’ to ‘*’) allows fields to be empty:

FPAT = "([^,]*)|(\"[^\"]+\")"

More Related Contents:

  1. substitute words on a line only before first delimeter using sed or awk
  2. How do you delete all lines that contain no letters from the alphabet using either grep, sed, or awk? [closed]
  3. Printing with sed or awk a line following a matching pattern
  4. What are NR and FNR and what does “NR==FNR” imply?
  5. How do I find the text that matches a pattern?
  6. Is a /start/,/end/ range expression ever useful in awk?
  7. Can we use shell variables in awk?
  8. Convert commas decimal separators to dots within a Dataframe
  9. Summing values of a column using awk command
  10. Using multiple delimiters in awk
  11. Print all but the first three columns
  12. Using AWK to Process Input from Multiple Files
  13. Assigning system command’s output to variable
  14. Why does “1” in awk print the current line?
  15. How to split a delimited string into an array in awk?
  16. How to escape a single quote inside awk
  17. Random numbers generation with awk in BASH shell
  18. How to show only next line after the matched one?
  19. Print second-to-last column/field in `awk`
  20. How can I make awk not use scientific notation when printing small values?
  21. print unique lines based on field
  22. ignorecase in AWK
  23. Using awk to pull specific lines from a file
  24. get the user input in awk
  25. How to preserve the original whitespace between fields in awk?
  26. Adding thousands separator to variable with a period in its name in Javascript
  27. How to print lines between two patterns, inclusive or exclusive (in sed, AWK or Perl)?
  28. Delimiters in MySQL
  29. Why does my tool output overwrite itself and how do I fix it?
  30. Remove trailing delimiting character from a delimited string

Leave a Comment