Bash parallel pipelined a for loop
I would like to know if there is a way to execute the body of the loop as soon as my generator has emited IFS
. The default behaviour is for bash to slurp the loop and then execute it.
Take this code:
time for i in $(echo 3; sleep 2; echo 1); do sleep ${i}; done
It runs in 6s.
Because bash first execute echo 6; sleep 2; echo 1
fully and slurp the input.
And then once 2s are spent here and the echo 1
finished it follows it with the body of the loop.
Executing sleep 3
and then sleep 1
.
What I would expect is for it to run in 4s because:
- first
echo 3
is executed, triggeringsleep 3
. - while
sleep 3
execute thesleep 2; echo 1
code has time to run. - when
sleep 3
finishes, the body of the loop is free again andsleep 1
executes.
Note I don't want & done
, I want a single body of my loop executed at one time, just pipeline the generator + loop execution.
You might think that & do
does it (as it would be unambiguous vs & done
), but not it doesn't, it fails with a syntax error.
Note I know I can do this with xargs -P
or gnu-parallel but I would like to know if this is possible to do with loops.
Solution 1:
Well I've just found it, still posting.
To have this behaviour, you must use the while
loop with read
.
The equivalent to my earlier for loop is this:
time (echo 3; sleep 2; echo 1) | while read i; do sleep ${i}; done
Correctly execute in 4s.