Splitting command line args with GNU parallel

You are probably looking for --colsep.

generate_file_pairs | parallel --colsep ' ' ./prog {1} {2}  

Read man parallel for more. And watch the intro video if you have not already done so http://www.youtube.com/watch?v=OpaiGYxkSuQ

Quite late to the party here, but I bump into this problem fairly often and found a nice easy solution

Before passing the arg list to parallel, just replace all the spaces with newlines. I've found tr to be the fastest for this kind of stuff

Not working

echo "1 2 3 4 5"  | parallel echo --
-- 1 2 3 4 5

Working

echo "1 2 3 4 5" | tr ' ' '\n' | parallel echo --
-- 1
-- 2
-- 3
-- 4
-- 5

Protip: before actually running the parallel command, I do 2 things to check that the arguments have been split correctly.

1. Prepend echo in front of your bash command. This means that any commands that will eventually be executed will be printed for you to check first
2. Add a marker in the echo, this checks that the parallel split is actually working

> Note, this works best with small/medium argument lists. If the argument list is very large, probably best to just use a for loop to echo each argument to parallel

You are looking for -n option of parallel. This is what you are looking for:

./generate_file_pairs | parallel -n 2 ./prog {}

Excerpt from GNU Parallel Doc:

-n max-args
    Use at most max-args arguments per command line. Fewer than max-args 
    arguments will be used if the size (see the -s option) is exceeded, 
    unless the -x option is given, in which case GNU parallel will exit.

In Parallel's manual, it is said:

If no command is given, the line of input is executed ... GNU parallel can often be used as a substitute for xargs or cat | bash.

So take a try of:

generate command | parallel

Try to understand the output of this:

for i in {1..5};do echo "echo $i";done | parallel