Using read to capture multiple variables from a command's output

I have a youtube-dl command that when run outputs two new lines, and I'm trying to capture each of those to variables.

Here is what I have so far, but for some reason I can't understand, it only captures the first line of output while discarding the second ($audio is empty).

read -r video audio <<< "$(youtube-dl -g --youtube-skip-dash-manifest https://www.youtube.com/watch?v=-gcvAxJJiGo)"

How can I fix it to make it capture both inputs, or alternatively, is there a more reliable way to do so?

Solution 1:

read -r video audio will capture the first ($IFS separated) word into "video" and the rest of the line into "audio"

Another way to capture 2 lines is with command grouping and a process substitution:

{
  read -r video
  read -r audio
} < <(
  youtube-dl -g --youtube-skip-dash-manifest "$url"
)

The space between < <( is important: a <(process substitution) acts like a file, so you're using < to redirect input from a file.

It just occurred to me: another way to read lines of output is the mapfile command:

mapfile -t lines < <(youtube-dl ...)
declare -p lines  # just to inspect the array contents

video=${lines[0]}
audio=${lines[1]}

Solution 2:

read reads one line by design. The simplest way to read two lines is to use read twice.

exec 3< <(youtube-dl -g --youtube-skip-dash-manifest 'https://www.youtube.com/watch?v=-gcvAxJJiGo')
IFS= read -r -u3 video
IFS= read -r -u3 audio
exec 3>&-

read -u is not portable, neither is <(…). They work in Bash and your question is tagged bash, so they should work for you.

Note I single-quoted the string containing ? in case you had a directory https:/www.youtube.com/ and a matching file therein. It's only remotely possible, still quoting wildcard characters that should not be treated as wildcards is good practice in any shell. Compare what happens in Zsh.

Empty IFS is the right thing in general, when you want to read a line.

This other answer does not use exec and it's good. It's my personal preference to set input up visually before tools that will use it (compare this answer). Unfortunately <(…) { …; } does not work. With exec I can do it.

Or I can do it by piping youtube-dl to the rest of the script.

youtube-dl -g --youtube-skip-dash-manifest 'https://www.youtube.com/watch?v=-gcvAxJJiGo' \
| {
IFS= read -r video
IFS= read -r audio
}

Normally Bash runs the { … } fragment in a subshell, so the variables will not survive after }. shopt -s lastpipe can change this (if job control is not active). Without lastpipe (which is not portable) the entire approach is portable but you need to run everything that uses the variables inside the { } block because you don't know if it's a subshell or not (POSIX allows both behaviors). Note the stdin inside the block comes from youtube-dl; if anything inside the block needs to use the stdin of the entire script then you may need to use exec anyway to juggle the descriptors.

Another portable approach is like this:

input="$(youtube-dl -g --youtube-skip-dash-manifest 'https://www.youtube.com/watch?v=-gcvAxJJiGo')"
video="$(printf '%s\n' "$input" | head -n 1)"
audio="$(printf '%s\n' "$input" | head -n 2 | tail -n 1)"

Instead of head and tail you can use sed or awk to isolate the right line.

Note this solution does not use read. If you need features of read then this is not the way. It seems you don't need the features of read in this particular case though.

Even if youtube-dl was very fast, I would not advise video="$(youtube-dl … | head -n 1)" plus audio="$(youtube-dl … | …)". If video and audio come from the same input, from the same invocation of youtube-dl, then they are as coherent as the tool allows. Otherwise they may be incoherent. Frankly I don't know how incoherent they may be (if ever); but I know in general it's better to query once (e.g. date +%H:+%M should always be coherent; date +%H; date +%M will surprise you if you run it just before a full hour).