Copy and Flatten directory contents with zsh

Solution 1:

The length of a command line in a shell is limited, probably to something like 65536 characters. So if your pattern expand to more characters than that, you'll get the "argument list too long" error you mentioned.

In such cases, find is the better approach (it will then also work for users of other shells). The copy can be accomplished with one of

find base_dir/ -iname '*.ftype' -print0 | xargs -0 -J% cp -np % dest_dir/
find base_dir/ -iname '*.ftype' -exec cp '{}' dest_dir/ \;

PS: The quoting in -iname '*.ftype' is important here to avoid expansion by the shell.

PPS: The error you get when using [A-Ma-m]**/*.ftype indicates that the pattern didn't get expanded. I don't know zsh very well, but most likely ** doesn't support additional patterns in front.

Solution 2:

“Argument list too long” is due to the total length of the command line (the command name and the arguments, plus one byte for each — plus the environment, actually). It is limited to sysctl kern.argmax bytes (KERN_ARGMAX in the C sysctl interface). The usual way to get around this error is to split the command into multiple calls.

Zsh commes with a function called zargs which helps with automatically running a command multiple times with successive sets of arguments. It's similar to the external command xargs but a different syntax.

autoload -U zargs
zargs -i -- **/*.ftype -- cp -- {} ../destination

Put the autoload line in your ~/.zshrc to avoid having to type it each time you want to use it. (Some zsh configuration frameworks may do it for you.)

The option -i causes zargs to replace {} in the command by each argument in turn. This runs cp once per argument. zargs can also run the command with multiple arguments at a time, which is faster, but a limitation is that the grouped arguments have to go at the end of the command line. This is not a good match for cp since it needs the destination to go at the end.

GNU cp has an option -T which allows the destination to be passed before the sources, precisely to facilitate the use of xargs (and zargs which is similar). Assuming you have GNU cp installed as gcp:

zargs -- **/*.ftype -- cp -T ../destination --

Alternatively, an advantage of zargs over xargs is that you can use it on a function.

f () { $@[2,-1] $1 }
zargs -- **/*.ftype -- f ../destination cp --

Note how f only reorders its arguments, it doesn't have the cp -- … ../destination built in. The point is to keep the f call at least as long as the cp call, otherwise zargs would build a command line that goes over the limit when the cp -- and ../destination parts are added. Alternatively, pass a smaller limit:

f () { cp -- "$@" ../destination/; }
zargs -s $(($(sysctl kern.argmax) - $#functions[f])) -- **/*.ftype -- f

(or just type a limit, e.g. zargs -s 999999 -- **/*.ftype -- f).

Alternatively, stop worrying about the command line length limit and use the zcp function to do fancy copies. As before, put the autoload and alias lines in your .zshrc to avoid having to type them each time.

autoload -U zmv
alias zcp='zmv -C'
zcp '**/(*.ftype)' '../destination/$1'