Can't pipe in bash's "mapfile" ... but why?

I just want to get all the files in a certain directory into a bash array (assuming that none of the files have a newline in the name):

So:

myarr=()
find . -maxdepth 1  -name "mysqldump*" | mapfile -t myarr; echo "${myarr[@]}"

Empty result!

If I do the roundabout way of using a file, temporary or otherwise:

myarr=()
find . -maxdepth 1  -name "mysqldump*" > X
mapfile -t myarray < X
echo "${myarray[@]}"

Result!

But why doesn't mapfile read properly from a pipe?

Solution 1:

From man 1 bash:

Each command in a pipeline is executed as a separate process (i.e., in a subshell).

Such subshells inherit variables from the main shell but they are independent. This means mapfile in your original command operates on its own myarr. Then echo (being outside the pipe) prints empty myarr (which is the main shell's myarr).

This command works differently:

find . -maxdepth 1 -name "mysqldump*" | { mapfile -t myarr; echo "${myarr[@]}"; }

In this case mapfile and echo operate on the same myarr (which is not the main shell's myarr).

To change the main shell's myarr you have to run mapfile in the main shell exactly. Example:

myarr=()
mapfile -t myarr < <(find . -maxdepth 1 -name "mysqldump*")
echo "${myarr[@]}"

Solution 2:

Bash runs the commands of a pipeline in a subshell environment, so any variable assignments etc. that take place within it aren't visible to the rest of the shell.

Dash (Debian's /bin/sh) as well as busybox's sh are similar, while zsh and ksh run the last part in the main shell. In Bash, you can use shopt -s lastpipe to do the same, but it only works when job control is disabled, so not in interactive shells by default.

So:

$ bash -c 'x=a; echo b | read x; echo $x'
a
$ bash -c 'shopt -s lastpipe; x=a; echo b | read x; echo $x'
b

_{(read and mapfile have the same issue.)}

Alternatively (and as mentioned by Attie), use process substitution, which works like a generalized pipe, and is supported in Bash, ksh and zsh.

$ bash -c 'x=a; read x < <(echo b); echo $x'
b

POSIX leaves it unspecified if the parts of a pipeline run in subshells or not, so it can't really be said that any of the shells would be "wrong" in this.

Solution 3:

As Kamil has pointed out, each element in the pipeline is a separate process.

You can use the following process substitution to get find to run in a different process, with the mapfile invocation remaining in your current interpreter, allowing access to myarr afterwards:

myarr=()
mapfile -t myarr < <( find . -maxdepth 1  -name "mysqldump*" )
echo "${myarr[@]}"

b < <( a ) will act similarly to a | b in terms of how the pipeline is wired - the difference is that b is executed "here".