Is there a way to 'uniq' by column?
sort -u -t, -k1,1 file
-
-u
for unique -
-t,
so comma is the delimiter -
-k1,1
for the key field 1
Test result:
[email protected],2009-11-27 00:58:29.793000000,xx3.net,255.255.255.0
[email protected],2009-11-27 01:05:47.893000000,xx2.net,127.0.0.1
awk -F"," '!_[$1]++' file
-
-F
sets the field separator. -
$1
is the first field. -
_[val]
looks upval
in the hash_
(a regular variable). -
++
increment, and return old value. -
!
returns logical not. - there is an implicit print at the end.
To consider multiple column.
Sort and give unique list based on column 1 and column 3:
sort -u -t : -k 1,1 -k 3,3 test.txt
-
-t :
colon is separator -
-k 1,1 -k 3,3
based on column 1 and column 3