Shell redirection i/o order

Solution 1:

The Bash manual has a clear example (similar to yours) to show that the order matters and also explains the difference. Here's the relevant part excerpted (emphasis mine):

Note that the order of redirections is significant. For example, the command

ls > dirlist 2>&1

directs both standard output (file descriptor 1) and standard error (file descriptor 2) to the file dirlist, while the command

ls 2>&1 > dirlist

directs only the standard output to file dirlist, because the standard error was made a copy of the standard output before the standard output was redirected to dirlist.

This post explains it from the POSIX viewpoint.

Confusions happen due to a key difference. > redirects not by making left operand (stderr) point to right operand (stdout) but by making a copy of the right operand and assigning it to the left. Conceptually, assignment by copy and not by reference.

So reading from left-to-right which is how this is interpreted by Bash: ls > dirlist 2>&1 means redirect stdout to the file dirlist, followed by redirection of stderr to whatever stdout is currently (which is already the file dirlist). However, ls 2>&1 > dirlist would redirect stderr to whatever stdout is currently (which is the screen/terminal) and then redirect stdout to dirlist.

Solution 2:

Redirections are:

• processed from left to right.
• interpreted iteratively:
  • an earlier redirection can affect a later one:
    • if an earlier redirection has redirected a given stream (identified by a file descriptor number, such as 1 for stdout (the default), and 2 for stderr), later redirections targeting that stream refer to the already-redirected version.
  • but not vice versa - a later redirection has no retroactive effect on the the target of an earlier redirection:
    • e.g., if you specify file descriptor 1 as the target in an earlier redirection, what 1 means at that time is locked in, even if 1 is redirected later.
• Note, however, that output isn't actually sent until all redirections are in place, and that any redirection-target output files are created or truncated before command execution begins (this is the reason why you can't read from and redirect output to the same file with a single command).

Applied to the example from the question:

• >file 2>&1:

  • >file first redirects stdout (file descriptor 1, implied by not prefixing > with a file descriptor number) to output file file
  • 2>&1 then redirects stderr (2) to the already redirected stdout (1).
  • The net effect is that both original streams end up in file.

• 2>&1 >file:

  • 2>&1 first redirects stderr to the then-original stdout; since file descriptor 2 participates in no further redirections, stderr output will therefore go to whatever stdout was defined as at that point - i.e., the original stdout, given that this is the first redirection.
    • Technically, the original stdout file descriptor is duplicated, and that duplicate is what stderr then refers to, which explains why it isn't affected by a later redirection of stdout.
  • >file then redirects the original stdout to file - but that has no effect anymore on the already locked-in redirection of stderr.
  • The net effect is that only sent-directly-to-stdout output is captured in file, while sent-to-stderr output is output to (the original, unredirected) stdout.

Solution 3:

This error:

ls: *.xyz: No such file or directory

is being written on stderr by ls binary.

However in this command:

ls -al *.xyz 2>&1 1> files.lst

You're first redirecting stderr to stdout which by default goes to tty (terminal)

And then you're redirecting stdout to a file files.lst, however remember that stderr doesn't redirected to file since you have stderr to stdout redirection before stdout to file redirection. Your stderr still gets written to tty in this case.

However in 2nd case you change the order of redirections (first stdout to file and then stderr to stdout) and that rightly redirects stderr to a file which is also being used by stdout.

Solution 4:

Because order does matter. In the first case, you first redirect stderr (2) to stdout (1). Then you redirect (1) to a file. But stderr (2) is still pointed to stdout of the shell running the command. Pointing (1) to a file in this case doesn't change the output device that (2) is directed at, so it still goes to terminal.

In the second case, you redirect stdout (1) to a file. Then you point stderr (2) to the same place 1 is pointed, which is the file, so the error message goes to the file.