In what order does the shell execute commands and stream redirection?

I was trying to redirect both stdout and stderr to a file today, and I came across this:

<command> > file.txt 2>&1

This apparently redirects stderr to stdout first, and then the resulting stdout is redirected to file.txt.

However, why isn't the order <command> 2>&1 > file.txt? One would naturally read this as (assuming execution from left to right) the command being executed first, the stderr being redirected to stdout and then, the resulting stdout being written to file.txt. But the above only redirects stderr to the screen.

How does the shell interpret both the commands?

When you run <command> 2>&1 > file.txt stderr is redirected by 2>&1 to where stdout currently goes, your terminal. After that, stdout is redirected to the file by >, but stderr is not redirected with it, so stays as terminal output.

With <command> > file.txt 2>&1 stdout is first redirected to the file by >, then 2>&1 redirects stderr to where stdout is going, which is the file.

It may seem counter intuitive to start with, but when you think of the redirections in this way, and remember that they are processed from left to right it makes much more sense.

It might make sense if you trace it out.

In the beginning, stderr and stdout go to the same thing (usually the terminal, which I call here pts):

fd/0 -> pts
fd/1 -> pts
fd/2 -> pts

I'm referring to stdin, stdout and stderr by their file descriptor numbers here: they are file descriptors 0, 1 and 2 respectively.

Now, in the first set of redirections, we have > file.txt and 2>&1.

So:

> file.txt: fd/1 now goes to file.txt. With >, 1 is the implied file descriptor when nothing is specified, so this is 1>file.txt:
```
fd/0 -> pts
fd/1 -> file.txt
fd/2 -> pts
```

2>&1: fd/2 now goes to wherever fd/1 currently goes:

fd/0 -> pts
fd/1 -> file.txt
fd/2 -> file.txt

On the other hand, with 2>&1 > file.txt, the order being reversed:

2>&1: fd/2 now goes to wherever fd/1 currently goes, which means nothing changes:
```
fd/0 -> pts
fd/1 -> pts
fd/2 -> pts
```

> file.txt: fd/1 now goes to file.txt:

fd/0 -> pts
fd/1 -> file.txt
fd/2 -> pts

The important point is that redirection does not mean that the redirected file descriptor will follow all future changes to the target file descriptor; it will only take on the current state.

I think it helps to think that the shell will set up the redirection on the left first, and complete it before setting up the next redirection.

The Linux Command Line by William Shotts says

First we redirect standard output to the file, and then we redirect file descriptor 2 (standard error) to file descriptor one (standard output)

this makes sense, but then

Notice that the order of the redirections is significant. The redirection of standard error must always occur after redirecting standard output or it doesn't work

but actually, we can redirect stdout to stderr after redirecting stderr to a file with the same effect

$ uname -r 2>/dev/null 1>&2
$

So, in command > file 2>&1, the shell sends stdout to a file, then sends stderr to stdout (which is being sent to a file). Whereas, in command 2>&1 > file the shell first redirects stderr to stdout (ie displays it in the terminal where stdout normally goes) and then afterwards, redirects stdout to the file. TLCL is misleading in saying that we must redirect stdout first: since we can redirect stderr to a file first and then send stdout to it. What we cannot do, is redirect stdout to stderr or vice versa before redirection to a file. Another example

$ strace uname -r 1>&2 2> /dev/null 
4.8.0-30-generic

We might think this would dispose of stdout to the same place as stderr, but it doesn't, it redirects stdout to stderr (the screen) first, and then only redirects stderr, as when we tried it the other way round...

I hope this brings a little bit of light...

In what order does the shell execute commands and stream redirection?

Related

Recent Posts