What is a "pipe" and how can it be "broken"?

I've gotten the "broken pipe" error from Xcode one time too many. I'm now curious to know exactly what a pipe is.

What is the concept of a "pipe" and how can it be "broken"?

Solution 1:

A pipe is simply an interprocess communication (IPC) mechanism used to connect the standard output of one process to the standard input of another.

An example is when you want to search a file for the word "pax":

cat filename | grep pax

and yes, I know you can grep the file directly but that doesn't explain how it works, does it?

This connects the standard output of the cat command to the standard input of the grep command. cat sends the contents of the file to its standard output, and grep reads its file (in this case) from its standard input. By connecting processes together like this, you can create your own tools consisting on any number of pipe segments. Things like:

show_users | grep pax | awk -F: '{print $4}' | tr '[a-z]' '[A-Z]' | cut -c1-20

A broken pipe is one where (usually) the receiver of the data has closed the connection while the sender is still trying to send stuff through.

For example, if you send a large file through a pager program (to view it one page at a time):

cat myfile | pager

and then do a CTRL-BREAK, this may cause the pager process to shut down its input pipe before cat has finished using it. That's one possibility for getting this broken pipe.

From a cursory Google search, this particular problem seems to be related to ad hoc deployments and the solutions given usually include exiting most of your software and rebooting most of your devices.

This is probably serious enough to report the issue to Apple. The more developers who complain about it, the more likely something will be done to fix it.

Solution 2:

The | character is often called a pipe. In the various UNIX shells (that I know of) it can be used to pipe the output of one command to the input of another.

cat myfile.txt | head

The head command only shows the first few lines of its input. At that point, it closes its input. This poses a problem for the command that was generating the input. Where does it write to? Whenever we have this situation, or the situation when the writing process ends before the reader is through, it's called a "broken pipe".

To prevent the cat command from lingering forever, the UNIX standard defines a special signal (SIGPIPE, signal 13) which it send to cat. The default action for this signal is to kill the process, which makes the cat end nicely.

It looks like the application you're using has installed a signal handler for all signals, including SIGPIPE, which creates the little popup message that you see.

Solution 3:

This error seems to come up fairly often. https://stackoverflow.com/questions/490366/ad-hoc-deployment-issue-putpkt-write-failed-broken-pipe it's "...an internal error in Xcode's ability to talk to your phone. It doesn't mean you've done anything wrong, it's a bug in the development system"

Solution 4:

A pipe is an IPC mechanism on Unix systems. A pipe has two ends, a read end, and a write end. Data that is written into the write end can be read from the read end and comes out in the order in which it was written.

In the Unix command line world, pipes are a very common way of hooking programs together to get a job done. For example sed 's/foo/bar/g' fred.txt | grep -e 'bar.*baz' will read in the file fred.txt replace all instances of the string foo with the string bar then search the result for lines that contain bar followed by some number of characters, then baz.

That, of course, doesn't seem terribly useful. But I'm sure if you think about it that you can see how you might be able to put that to all kinds of interesting uses, especially once you have programs like awk or perl at your disposal.

The pipe system has been a part of Unix from very early on. And if a process in your pipeline exits you usually want all the programs in the pipeline to exit. This means that, by default, a process that writes into a pipe were the process on the read end is gone will get a SIGPIPE signal. And if its blocked that signal, the write will still fail with a special kind of error indicating that the pipe has 'broken'.

The default handling of SIGPIPE kills the process that receives it. And if it isn't the 'head' of the pipeline, the whole SIGPIPE thing propagates back up the chain.

What Xcode is complaining about is that it started some sub-program to do something with a pipe leading to it, and that sub-program died unexpectedly leaving the pipe broken..