The Concept of 'Hold space' and 'Pattern space' in sed

Solution 1:

When sed reads a file line by line, the line that has been currently read is inserted into the pattern buffer (pattern space). Pattern buffer is like the temporary buffer, the scratchpad where the current information is stored. When you tell sed to print, it prints the pattern buffer.

Hold buffer / hold space is like a long-term storage, such that you can catch something, store it and reuse it later when sed is processing another line. You do not directly process the hold space, instead, you need to copy it or append to the pattern space if you want to do something with it. For example, the print command p prints the pattern space only. Likewise, s operates on the pattern space.

Here is an example:

sed -n '1!G;h;$p'

(the -n option suppresses automatic printing of lines)

There are three commands here: 1!G, h and $p. 1!G has an address, 1 (first line), but the ! means that the command will be executed everywhere but on the first line. $p on the other hand will only be executed on the last line. So what happens is this:

1. first line is read and inserted automatically into the pattern space
2. on the first line, first command is not executed; h copies the first line into the hold space.
3. now the second line replaces whatever was in the pattern space
4. on the second line, first we execute G, appending the contents of the hold buffer to the pattern buffer, separating it by a newline. The pattern space now contains the second line, a newline, and the first line.
5. Then, h command inserts the concatenated contents of the pattern buffer into the hold space, which now holds the reversed lines two and one.
6. We proceed to line number three -- go to the point (3) above.

Finally, after the last line has been read and the hold space (containing all the previous lines in a reverse order) have been appended to the pattern space, pattern space is printed with p. As you have guessed, the above does exactly what the tac command does -- prints the file in reverse.

Solution 2:

@Ed Morton: I disagree with you here. I found sed very useful and simple (once you grok the concept of the pattern and hold buffers) to come up with an elegant way to do multiline grepping.

For example, let's take a text file that has hostnames and some information about each host, with lots of junk in between that I dont care about.

Host: foo1
some junk, doesnt matter
some junk, doesnt matter
Info: about foo1 that I really care about!!
some junk, doesnt matter
some junk, doesnt matter
Info: a second line about foo1 that I really care about!!
some junk, doesnt matter
some junk, doesnt matter
Host: foo2
some junk, doesnt matter
Info: about foo2 that I really care about!!
some junk, doesnt matter
some junk, doesnt matter

To me, an awk script to just get the lines with the hostname and the corresponding info line would take a bit more than what I'm able to do with sed:

sed -n '/Host:/{h}; /Info/{x;p;x;p;}' myfile.txt

output looks like:

Host: foo1
Info: about foo1 that I really care about!!
Host: foo1
Info: a second line about foo1 that I really care about!!
Host: foo2
Info: about foo2 that I really care about!!

(Note that Host: foo1 appears twice in the output.)

Explanation:

1. -n disables output unless explicitly printed
2. first match, finds and puts the Host: line into hold buffer (h)
3. second match, finds the next Info: line, but first exchanges (x) current line in pattern buffer with hold buffer, and prints (p) the Host: line, then re-exchanges (x) and prints (p) the Info: line.

Yes, this is a simplistic example, but I suspect this is a common issue that was quickly dealt with by a simple sed one-liner. For much more complex tasks, such as ones in which you cannot rely on a given, predictable sequence, awk may be better suited.