The UNIX Bash Scripting blog suggests: awk ‘!x[$0]++’ This command is telling awk which lines to print. The variable $0 holds the entire contents of a line and square brackets are array access. So, for each line of the file, the node of the array x is incremented and the line printed if the content … Read more
sort -u -t, -k1,1 file -u for unique -t, so comma is the delimiter -k1,1 for the key field 1 Test result: [email protected],2009-11-27 00:58:29.793000000,xx3.net,255.255.255.0 [email protected],2009-11-27 01:05:47.893000000,xx2.net,127.0.0.1
Helper method This method uses the helper: class Array def difference(other) h = other.each_with_object(Hash.new(0)) { |e,h| h[e] += 1 } reject { |e| h[e] > 0 && h[e] -= 1 } end end This method is similar to Array#-. The difference is illustrated in the following example: a = [3,1,2,3,4,3,2,2,4] b = [2,3,4,4,3,4] a – … Read more