In bash, how to ingest lines from a file, and then that set of lines as a file itself
In a bash script, is there a way to ingest specific lines from a file, and then treat that set of lines as a file itself? For example, let's say file input_data
looks like
boring line 1 A
boring line 2 B
boring line 3 C
start of interesting block
line 1 A
line 2 B
line 3 C
end of interesting block
boring line 4 D
There's not much I can assume about the structure of the file except that 1) it will indeed have "start of interesting block" and "end of interesting block" and 2) it will generally be much larger and more complicated than this example. And let's say I want to do a great deal of processing in the "interesting block".
So I need to do something sort of like this:
interesting_lines=$(tail -n 6 input_file.txt | head -n 5)
process_1 interesting_lines #(Maybe process_1 is a "grep" or something more complicated.)
process_2 interesting_lines
etc.
Of course, that won't work, which is why I'm asking this question, but that's the idea that I need.
I guess one thing that would work would be
tail -n 6 input_file.txt | head -n 5 > tmpfile
process_1 tmpfile
process_2 tmpfile
etc.
rm tmpfile
but I am trying to avoid temporary files.
You can use process substitution.
interesting_lines=$(tail -n 6 input_file.txt | head -n 5)
process_1 <(printf '%s\n' "$interesting_lines")
process_2 <(printf '%s\n' "$interesting_lines")
Don't use head
or tail
. You can give a range with sed
or awk
:
process_1 <(sed -n '/start of interesting block/,/end of interesting block/p' input_file.txt)
process_2 <(awk '/start of interesting block/,/end of interesting block/' input_file.txt)
When you need better control over your boundaries, use awk
smarter. The next solution only seems to be more verbose, but now you can ad all kind of conditions
process_1 <(awk '/start of interesting block/ {interesting=1}
/end of interesting block/ {interesting=0}
interesting {print}
' input_file.txt)
When you have to look in a very long file for only a few interesting lines, you can do
sed -n '/start of interesting block/,/end of interesting block/p' input_file.txt |
tee >(process_1) | process_2
Demo:
printf "%s\n" {1..10} | tee >(sed 's/4/xxx/p') | sed 's/^/ /'