awk

How to select lines between two marker patterns which may occur multiple times with awk/sed

Use awk with a flag to trigger the print when necessary: $ awk ‘/abc/{flag=1;next}/mno/{flag=0}flag’ file def1 ghi1 jkl1 def2 ghi2 jkl2 How does this work? /abc/ matches lines having this text, as well as /mno/ does. /abc/{flag=1;next} sets the flag when the text abc is found. Then, it skips the line. /mno/{flag=0} unsets the flag … Read more

Printing with sed or awk a line following a matching pattern

Never use the word “pattern” as is it highly ambiguous. Always use “string” or “regexp” (or in shell “globbing pattern”), whichever it is you really mean. The specific answer you want is: awk ‘f{print;f=0} /regexp/{f=1}’ file or specializing the more general solution of the Nth record after a regexp (idiom “c” below): awk ‘c&&!–c; /regexp/{c=1}’ … Read more

Save modifications in place with awk

In GNU Awk 4.1.0 (released 2013) and later, it has the option of “inplace” file editing: […] The “inplace” extension, built using the new facility, can be used to simulate the GNU “sed -i” feature. […] Example usage: $ gawk -i inplace ‘{ gsub(/foo/, “bar”) }; { print }’ file1 file2 file3 To keep the … Read more

What’s the most robust way to efficiently parse CSV using awk?

If your CSV cannot contain newlines then all you need is (with GNU awk for FPAT): $ echo ‘foo,”field,””with””,commas”,bar’ | awk -v FPAT='[^,]*|(“([^”]|””)*”)’ ‘{for (i=1; i<=NF;i++) print i ” <” $i “>”}’ 1 <foo> 2 <“field,””with””,commas”> 3 <bar> or the equivalent using any awk: $ echo ‘foo,”field,””with””,commas”,bar’ | awk -v fpat=”[^,]*|(“([^”]|””)*”)” -v OFS=’,’ ‘{ rec … Read more

How to print lines between two patterns, inclusive or exclusive (in sed, AWK or Perl)?

Print lines between PAT1 and PAT2 $ awk ‘/PAT1/,/PAT2/’ file PAT1 3 – first block 4 PAT2 PAT1 7 – second block PAT2 PAT1 10 – third block Or, using variables: awk ‘/PAT1/{flag=1} flag; /PAT2/{flag=0}’ file How does this work? /PAT1/ matches lines having this text, as well as /PAT2/ does. /PAT1/{flag=1} sets the flag … Read more

How do I use shell variables in an awk script?

#Getting shell variables into awk may be done in several ways. Some are better than others. This should cover most of them. If you have a comment, please leave below.                                                                                    v1.5 Using -v (The best way, most portable) Use the -v option: (P.S. use a space after -v or it will be less portable. E.g., awk … Read more

Awk – replace coumn 2 in table 1 from coumn 2 in table 2 based on matching data in column 1 (common between tables)

#!/usr/bin/env python with open(‘/etc/shadow’,’rb’) as file: for line in file: TargetLine = line.rstrip().split(“:”) with open(‘shadow.sync’,’rb’) as shadow: for row in shadow: SyncLine = row.rstrip().split(“:”) if TargetLine[0] == SyncLine[0]: TargetLine[1] = SyncLine[1] break print “NEW MODIFIED LINE: %s” % “:”.join(TargetLine) This will open the /etc/shadow file and loop through the lines. For each line on the … Read more