What is a simple explanation for how pipes work in Bash?

I often use pipes in Bash, e.g.:

dmesg | less

Although I know what this outputs, it takes dmesg and lets me scroll through it with less, I do not understand what the | is doing. Is it simply the opposite of >?

• Is there a simple, or metaphorical explanation for what | does?
• What goes on when several pipes are used in a single line?
• Is the behavior of pipes consistent everywhere it appears in a Bash script?

Solution 1:

A Unix pipe connects the STDOUT (standard output) file descriptor of the first process to the STDIN (standard input) of the second. What happens then is that when the first process writes to its STDOUT, that output can be immediately read (from STDIN) by the second process.

Using multiple pipes is no different than using a single pipe. Each pipe is independent, and simply links the STDOUT and STDIN of the adjacent processes.

Your third question is a little bit ambiguous. Yes, pipes, as such, are consistent everywhere in a bash script. However, the pipe character | can represent different things. Double pipe (||), represents the "or" operator, for example.

Solution 2:

In Linux (and Unix in general) each process has three default file descriptors:

1. fd #0 Represents the standard input of the process
2. fd #1 Represents the standard output of the process
3. fd #2 Represents the standard error output of the process

Normally, when you run a simple program these file descriptors by default are configured as following:

1. default input is read from the keyboard
2. Standard output is configured to be the monitor
3. Standard error is configured to be the monitor also

Bash provides several operators to change this behavior (take a look to the >, >> and < operators for example). Thus, you can redirect the output to something other than the standard output or read your input from other stream different than the keyboard. Specially interesting the case when two programs are collaborating in such way that one uses the output of the other as its input. To make this collaboration easy Bash provides the pipe operator |. Please note the usage of collaboration instead of chaining. I avoided the usage of this term since in fact a pipe is not sequential. A normal command line with pipes has the following aspect:

    > program_1 | program_2 | ... | program_n

The above command line is a little bit misleading: user could think that program_2 gets its input once the program_1 has finished its execution, which is not correct. In fact, what bash does is to launch ALL the programs in parallel and it configures the inputs outputs accordingly so every program gets its input from the previous one and delivers its output to the next one (in the command line established order).

Following is a simple example from Creating pipe in C of creating a pipe between a parent and child process. The important part is the call to the pipe() and how the parent closes fd1 (writing side) and how the child closes fd1 (writing side). Please, note that the pipe is a unidirectional communication channel. Thus, data can only flow in one direction: fd1 towards fd[0]. For more information take a look to the manual page of pipe().

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>

int main(void)
{
    int     fd[2], nbytes;
    pid_t   childpid;
    char    string[] = "Hello, world!\n";
    char    readbuffer[80];

    pipe(fd);

    if((childpid = fork()) == -1)
    {
            perror("fork");
            exit(1);
    }

    if(childpid == 0)
    {
            /* Child process closes up input side of pipe */
            close(fd[0]);

            /* Send "string" through the output side of pipe */
            write(fd[1], string, (strlen(string)+1));
            exit(0);
    }
    else
    {
            /* Parent process closes up output side of pipe */
            close(fd[1]);

            /* Read in a string from the pipe */
            nbytes = read(fd[0], readbuffer, sizeof(readbuffer));
            printf("Received string: %s", readbuffer);
    }

    return(0);
}

Last but not least, when you have a command line in the form:

> program_1 | program_2 | program_3

The return code of the whole line is set to the last command. In this case program_3. If you would like to get an intermediate return code you have to set the pipefail or get it from the PIPESTATUS.