Understanding Hold Buffer in Linux Sed Command

The hold buffer in sed allows you to temporarily store and retrieve lines of input.

Think of it as a secondary memory where you can place data while working with the main pattern space.

The hold buffer is especially useful when you need to perform operations involving more than one line at a time.

Instead of just immediately processing and printing each line, the hold buffer lets you save a line, process others, and then bring back the saved line when needed.

 

 

Difference between Pattern Space and Hold Space

In sed, when processing input, you’ll primarily deal with two main buffers: the pattern space and the hold space.

Pattern Space: This is the default working space for sed. Whenever sed reads a line from its input, it places the line into the pattern space.

Here, all editing commands act on the content of this space. After processing all the commands for a particular line, sed usually outputs the pattern space content, then clears it to read in the next line of input.

echo -e "apple\norange" | sed 's/apple/fruit/'

Output:

fruit
orange

In the command above, sed replaced “apple” with “fruit” in the pattern space before printing the result.

Hold Space: The hold space serves as a temporary storage that you can use in conjunction with the pattern space.

While the pattern space is constantly being overwritten with each new line of input, the hold space remains persistent until explicitly modified.

This allows you to save a line, or any data, to be used later in your sed processing.

echo -e "apple\norange" | sed -n '1h; 2{g; p;}'

Output:

apple

In this example, the first line “apple” is copied to the hold space with 1h.

On the second line, 2{g; p;} gets the content from the hold space and places it in the pattern space, then prints it.

Since the hold space had “apple”, that’s what gets printed.

Both pattern and hold spaces provide different functionalities. While the pattern space allows immediate processing, the hold space is ideal for holding data across multiple lines.

 

Copy the Pattern Space to the Hold Buffer

The h command in sed copies the content of the pattern space into the hold buffer.

When you use this command, whatever resides in the pattern space at that moment gets stored in the hold buffer, replacing anything previously held there.

Let’s walk through a practical example to demonstrate this:

echo -e "apple\norange\nbanana" | sed -n '1h; 2p; 3g; 3p'

Output:

orange
apple

Here’s what happens step-by-step:

  1. For the first line “apple”, the 1h command copies it into the hold buffer.
  2. For the second line “orange”, the 2p command prints it.
  3. For the third line “banana”, the 3g command gets the content from the hold buffer and places it in the pattern space. Since “apple” was stored in the hold buffer, the pattern space now contains “apple”.
  4. Still on the third line, the 3p command prints the current content of the pattern space, which is “apple”.

 

Append the Pattern Space to the Hold Buffer

The H command in sed allows you to append the content of the pattern space to the hold buffer.

It does not replace the existing content of the hold buffer.

Instead, it appends the pattern space content after what’s already in the hold buffer, and the two pieces of data are separated by a new line.

Here’s an example to clarify this operation:

echo -e "apple\norange" | sed -n '1h; 2H; 2g; 2p'

Output:

apple
orange

Breaking down the steps:

  1. On the first line “apple”, the 1h command copies it to the hold buffer.
  2. On the second line “orange”, the 2H command appends this line to the hold buffer. At this point, the hold buffer contains “apple\norange”.
  3. Still on the second line, the 2g command gets the entire content from the hold buffer and places it in the pattern space.
  4. Finally, the 2p command prints the content of the pattern space, which results in two lines: “apple” and “orange”.

 

Replace the Pattern Space with Hold Buffer Content

The g command in sed is used to replace the content of the pattern space with the contents from the hold buffer.

It’s a handy command when you wish to retrieve previously stored data from the hold buffer and operate on it within the pattern space.

Let’s delve into an example:

echo -e "apple\norange" | sed -n '1h; 2g; 2p'

Output:

apple

Understanding the process:

  1. On the first line, “apple”, the 1h command moves this content into the hold buffer.
  2. When processing the second line, “orange”, the 2g command replaces the content of the pattern space (which is “orange”) with the content from the hold buffer (which is “apple”).
  3. Immediately after, the 2p command prints the content of the pattern space, which is now “apple”.

 

Append the Hold Buffer to the Pattern Space

The G command in sed is used to append the content of the hold buffer to the pattern space, separated by a newline.

Let’s jump into an example to understand this concept:

echo -e "apple\norange" | sed -n '1h; 2G; 2p'

Output:

orange
apple

Step-by-step walkthrough:

  1. For the “apple” line, the 1h command deposits its content into the hold buffer.
  2. As we process the second line, “orange”, the 2G command appends the content of the hold buffer (which is “apple”) to the pattern space, ensuring they are separated by a newline. The pattern space then holds “orange\napple”.
  3. The subsequent 2p command prints the concatenated content of the pattern space.

 

Exchange Hold Buffer and Pattern Space

The x command in sed is used to swap the contents of the pattern space and the hold buffer.

Let’s dive into an example:

echo -e "apple\norange" | sed -n '1h; 2x; 2p'

Output:

apple

Here’s what happens:

  1. On the line “apple”, we use the 1h command to store its content into the hold buffer.
  2. As we reach the second line “orange”, the 2x command swaps the content of the pattern space (“orange”) with the content of the hold buffer (“apple”). Post-swap, the pattern space contains “apple”, and the hold buffer contains “orange”.
  3. Right after the swap, the 2p command prints the current content of the pattern space, which is “apple”.

 

Use Cases for the Hold Buffer

The hold buffer in sed is more than just a storage space; it’s a mechanism that opens the door to perform multi-line processing. Let’s explore some key use cases for the hold buffer:

Saving and Retrieving Lines for Multi-line Processing

You can store a line (or lines) and then apply operations on subsequent lines using the saved data.

echo -e "header\ncontent\nfooter" | sed -n '/header/h; /footer/{g; p;}'

Output:

header

In this instance, the “header” line is saved to the hold buffer. When we encounter the “footer” line, we retrieve the “header” and print it.

Storing Context or State between Lines

Sometimes, you need to recall context or a particular state when processing a later line. The hold buffer can serve as a memory, letting you reference prior lines for context.

echo -e "apple\norange\ninfo: banana" | sed -n '/apple/h; /info:/{x;G; s/info: //; p;}'

Output:

apple
banana
  • /apple/h: Copy the line containing “apple” to the hold buffer.
  • /info:/: Look for lines containing “info:”.
  • {x; ...}: For these lines, exchange the hold buffer and pattern space, which brings “apple” into the pattern space and puts “info: banana” into the hold buffer.
  • G: Appends the hold buffer (now containing “info: banana”) to the pattern space.
  • s/info: //: Remove the “info:” prefix.
  • p: Print the pattern space.

Rearranging the Order of Lines or Blocks

One of the more advanced uses of the hold buffer is to reorder lines or blocks of text.

If you need to shuffle the sequence of data based on a condition or pattern, the hold buffer becomes an indispensable tool.

echo -e "line2\nline1" | sed -n '1h; 2{p; x; p;}'

Output:

line1
line2
  • 1h: On the first line (line2), copy the content to the hold buffer.
  • 2{p; x; p;}: On the second line:
    • p: Print the current pattern space (line1).
    • x: Exchange the pattern space with the hold buffer, which brings line2 into the pattern space.
    • p: Print the pattern space now containing line2.

 

Using the Hold Buffer for Multi-pass Editing

Suppose you have a block of text that requires a sequence of different transformations.

Trying to achieve this in a single pass will be complex and error-prone. Multi-pass editing allows you to:

  1. Break down transformations into isolated steps.
  2. Ensure each transformation’s accuracy before moving to the next.
  3. Store the intermediate results securely for subsequent processing.

Let’s delve into an example where we want to:

  1. Convert a list of fruits to uppercase.
  2. Prefix each fruit with the word “FRUIT:”.

Command:

echo -e "apple\norange\nbanana" | sed -n -e '1,3{s/.*/\U&/;h}' -e '1,3{x;s/^/FRUIT: /;p}'

Output:

FRUIT: APPLE
FRUIT: ORANGE
FRUIT: BANANA

Breaking down the steps:

First -e (First pass for first three lines):

1,3{s/.*/\U&/;h}: For the lines 1 to 3, we convert the entire line to uppercase and then store it in the hold buffer.

Second -e (Second pass for first three lines):

1,3{x;s/^/FRUIT: /;p}: For the lines 1 to 3, we swap the content of the hold buffer with the pattern space, prefix the line with “FRUIT: “, and then print it.

 

Managing the Hold Buffer with Complex Scripts

When handling complex scripts, especially those that leverage the hold buffer intensively, it’s crucial to manage the hold buffer efficiently to prevent unintended outcomes and maintain readability.

Clear the Hold Buffer When Necessary

There are times when the hold buffer’s content is no longer needed, and to prevent its accidental use, you might want to clear it.

To clear the hold buffer:

s/.*//
h

This ensures that you’re starting with a clean slate, particularly if the buffer’s content might interfere with subsequent operations.

Debugging with the Hold Buffer

Debugging a sed script, especially with the hold buffer involved, can be challenging. One approach is to print both the pattern space and hold buffer at strategic points.

Use this:

# Print pattern space and hold buffer for debugging
l0
g
l0

This will provide a clear view of both buffers, assisting you in tracking the data flow through your script.

Leave a Reply

Your email address will not be published. Required fields are marked *