how to split input to two pipes
Solution 1:
The way question reads it sounds like you want one stdin redirected to two different commands. If that's the case, take advantage of tee
plus process substitution:
some-expensive-command | tee >(grep 'pattern' > output.txt) >(grep -v 'pattern' | another-command)
Process substitutions are in fact anonymous pipelines implemented within bash itself ( on subprocess level ). We can also make the use of a named pipeline + tee
. For instance, in terminal A do
$ mkfifo named.fifo
$ cat /etc/passwd | tee named.fifo | grep 'root'
And in another terminal B do
$ grep -v 'root' named.fifo
Another way to look at this is by recognizing that grep
is line pattern matching tool, so by reading line at a time and using that same line in multiple commands we can achieve exactly the same effect:
rm output.txt # get rid of file so that we don't add old and new output
some-expensive-command | while IFS= read -r line || [ -n "$line" ]; do
printf "%s\n" "$line" | grep 'pattern' >> output.txt
printf "%s\n" "$line" | grep -v 'pattern' | another-command
done
# or if another-command needs all of the output,
# place `| another-comand` after `done` clause
Yet another way is to abandon grep
and use something more powerful, like awk
:
some-expensive-command | awk '/pattern/{print >> "output.txt"}; !/pattern/{print}' | another-command.
Practically speaking, don't worry about using temporary files, so long as you clean them up after using. If it works, it works.
Solution 2:
Use bash
Process Substitution:
some-command | tee >(grep "pat" | another-command >>out1) | grep -v "pat" >>out2
The process substitution assigns some-command
’s output to grep "pat"
’s input, thus saving you the tempfile. Of course the data is still saved in a file (it’s always), just that you don’t have to take care of that. If you don’t want to save another-command
’s output in a file but rather print it I recommend to simply switch the two command lists.
Another nice source of information: man bash
/EXPANSION