Extract columns from a file based on header selected from another file
I have an idea, but since I'm not experienced in shell programming (and don't know awk) this looks like reinventing some wheels in a ridiculous way:
$ cat DATA.TXT
ID, head1, head2, head3, head4
1, 25.5, 1364.0, 22.5, 13.2
2, 10.1, 215.56, 1.15, 22.2
$ cat LIST.TXT
head1
head4
$ cols=($(sed '1!d;s/, /\n/g' DATA.TXT | grep -nf LIST.TXT | sed 's/:.*$//'))
$ cut -d ',' -f 1$(printf ",%s" "${cols[@]}") DATA.TXT
ID, head1, head4
1, 25.5, 13.2
2, 10.1, 22.2
P.S. I used some very basic ideas about bash arrays from this and this answers.
There is a useful awk
script here which you can use to extract specific column names from a csv file.
I have modified it slightly so that it can read column names from another file. Save the script below as dataExtractor.sh
.
#!/bin/bash
DATAFILE=${1:-data.txt}
COLUMNFILE=${2:-list.txt}
awk -F, -v colsFile="$COLUMNFILE" '
BEGIN {
j=1
while ((getline < colsFile) > 0) {
col[j++] = $1
}
n=j-1;
close(colsFile)
for (i=1; i<=n; i++) s[col[i]]=i
}
NR==1 {
for (f=1; f<=NF; f++)
if ($f in s) c[s[$f]]=f
next
}
{ sep=""
for (f=1; f<=n; f++) {
printf("%c%s",sep,$c[f])
sep=FS
}
print ""
}
' "$DATAFILE"
Running it:
$ cat data.txt
ID,head1,head2,head3,head4
1,25.5,1364.0,22.5,13.2
2,10.1,215.56,1.15,22.2
$ cat list.txt
ID
head1
head4
$ dataExtractor.sh data.txt list.txt
1,25.5,13.2
2,10.1,22.2