sed linux
Linux

31+ Examples for sed Linux command in text manipulation

In the previous post, we talked about bash functions and how to use them from the command line directly and we saw some other cool stuff. Today we will talk about a very useful tool for string manipulation called sed or sed Linux command.

Sed is one of the most common tools that people use to work with text files like log files, configuration files, and other text files.

If you need any type of text manipulation in your bash scripts, you need to become familiar with the sed and gawk tools. In this post, we are going to focus on sed Linux command and see its ability to manipulate text, which is a very important step in our bash scripting journey.

Sometimes you need to manipulate text in a text file like a huge log file to extract some text, delete, modify, append, or whatever. The Linux system provides some common tools for doing just that, one of those tools is sed.

We will discuss the 31+ examples with pictures to show the output of every example.

 

Understand sed Linux command

The sed command is called a stream editor, it is an interactive text editor, such as nano, you interactively use keyboard commands to insert, delete, or replace text in the data. Sed Linux command edits a stream of data based on a set of rules you provide and this is the format of the sed command:

$ sed options file

By default, the sed Linux command applies the specified commands to the STDIN. This allows you to pipe the data directly to the sed editor for processing like this:

$ echo "This is a test" | sed 's/test/another test/'

sed linux command

The sed command substitutes the second text string with the first text string pattern specified between the first forward slashes. In this example, the words “another test” was substituted for the word “test”, so the result will be as shown.

That’s the power of using the sed Linux command.

The above example was a very basic example to demonstrate the tool. We can use sed Linux command to manipulate files as well.

This is our file:

sed manipulate file

$ sed 's/test/another test' ./myfile

You’ll start seeing results before the sed editor completes processing the entire file because sed returns the data instantaneously.

You will notice that when your file is huge enough, you will see the result even before the processing is finished.

The sed editor doesn’t alter the data in the text file itself. It only sends the modified text to STDOUT. If you look at the text file, it still contains the original data. If you need to overwrite the file with the new content, you can check our previous post which was talking about redirections.

Using multiple sed Linux commands in the command line

To execute more than one command from the sed command line, just use the -e option like this:

$ sed -e 's/This/That/; s/test/another test/' ./myfile

sed multiple commands

Both commands applied to each line of data in the file. The commands must be separated by a semicolon, and there shouldn’t be any spaces between the end of the command and the semicolon.

Also, you can use a single quotation to separate commands like this:

sed separate commands

The same result, no big deal.

Reading commands from a file

If you have lots of sed commands you want to process, it is often easier to just store them in a separate file, and use the -f option to specify the file to the sed command like this:

$ sed -f mycommands myfile

sed read commands from file

Substituting flags

Look at the following example carefully:

$ cat myfile

$ sed 's/test/another test/' myfile

sed substitute flag

The substitute command works fine in replacing text in multiple lines, but it replaces only the first occurrence in each line. To substitute all occurrences of a pattern, you must use a substitution flag.

The flags are written like this:

s/pattern/replacement/flags

There are four types of substitutions:

  • A number, indicating the pattern occurrence for which new text should be substituted.
  • g, indicating that new text should be substituted for all occurrences of the existing text.
  • p, indicating that the contents of the original line should be printed.
  • w file, which means to write the results of the substitution to a file.

The first type of substitution, you can specify which occurrence of the matching pattern should be substituted.

$ sed 's/test/another test/2' myfile

sed number flag

As a result of specifying 2 as the substitution flag, the sed Linux command replaces the pattern only in the second occurrence in each line.

The g substitution means global, it enables you to replace every occurrence of the pattern in the text:

$ sed 's/test/another test/g' myfile

sed global flag

The p substitution flag prints the line that contains a matching pattern, and the -n option suppresses output from the sed command so it produces the output only for lines that have been modified by the substitute command.

$ cat myfile

$ sed -n 's/test/another test/p' myfile

sed supress lines

The w substitution flag generates the same output, but saves the output to the specified file:

$ sed 's/test/another test/w output' myfile

sed send output to file

The output of the sed command appears in STDOUT, but the lines that include the matching pattern are stored in the output file.

Replacing characters

Suppose that you want to substitute the C shell with the bash shell in the /etc/passwd file, you’d have to do this:

$ sed 's/\/bin\/bash/\/bin\/csh/' /etc/passwd

That looks confusing for some people because the forward slash is used as the string delimiter, you must use a backslash to escape it.

Luckily, there is another way to achieve that. The sed Linux command allows you to select a different character for the string delimiter in the substitute command:

$ sed 's!/bin/bash!/bin/csh!' /etc/passwd

The exclamation mark is used as a string delimiter. It is now easier to read.

Limiting sed

The commands you use in the sed command applied to all lines of the text data. If you want to apply a command only to a specific line or a group of lines, there are two forms:

  • A numeric range of lines.
  • A text pattern that filters out a line.

The address you specify in the command can be a single line number or a range of lines specified by a starting line and an ending line number:

$ sed '2s/test/another test/' myfile

sed restricted

Only line two is modified.

What about using a range of lines:

$ sed '2,3s/test/another test/' myfile

sed replace range of lines

Also, we can start from a line to the end of the file:

$ sed '2,$s/test/another test/' myfile

sed replace to the end

You can specify a text pattern to filter lines for the sed command. The pattern is written like this:

$ sed '/likegeeks/s/bash/csh/' /etc/passwd

sed pattern match

The command was applied only to the line that matches the text pattern.

You can use regular expressions to write this pattern to be more generic and useful.

Deleting lines

If you need to delete specific lines of text in a text stream, you can use the delete command.

When using the delete command, if you forget to include an addressing scheme, all the lines are deleted from the stream not the original file:

$ sed '3d' myfile

sed delete line

Here we delete the third line only from myfile:

$ sed '2,3d' myfile

sed delete multiple line

Here we delete a range of lines, the second and the third:

$ sed '3,$d' myfile

sed delete to the end

Here we delete from the third line to the end of the file.

Note that the sed command doesn’t touch the original file. It is only deleted from the output:

$ sed '/test 1/d' myfile

sed deletepattern match

Here we use a pattern to delete the line if matched on the first line.

You can also delete a range of lines using two text patterns like the following:

$ sed '/second/,/fourth/d' myfile

sed delete range of lines

The first to the third line deleted.

Inserting and appending text

The sed Linux command allows you to insert and append text lines to the data stream using the following commands:

  • The insert command (i) adds a new line before the specified line.
  • The append command (a) adds a new line after the specified line.

$ echo "Another test" | sed 'i\First test '

sed insert text

$ echo "Another test" | sed 'a\First test '

sed append

It works well for adding text before or after the text in the data stream, but what about adding text in the middle?

To insert or append data inside the data stream lines, you must specify where you want the data to be inserted.

You can match either a numeric line number or a text pattern, and of course, you cannot use a range of addresses.

$ sed '2i\This is the inserted line.' myfile

sed insert line

And the appending works the same way, but look at the position of the appended text:

$ sed '2a\This is the appended line.' myfile

sed append line

The difference is when appending, it places the new text line after the specified line number.

Modifying lines

The change command enables you to change the contents of an entire line of text in your data stream. All you have to do is to specify the line that you want to change.

$ sed '3c\This is a modified line.' myfile

sed modify line

You can also use a text pattern or a regular expression and all lines match that pattern will be modified.

$ sed '/This is/c This is a changed line of text.' myfile

sed pattern match

Transforming characters

The transform command (y) works on a single character like this:

$ sed 'y/123/567/' myfile

sed transform character

You can’t limit the transformation to a specific occurrence of characters.

Printing line numbers

The equal sign command prints the current line number of the line within the data stream:

$ sed '=' myfile

sed line numbers

The sed editor prints the line number before the actual line of text.

However, by using -n combined with the equal sign, the sed command displays the line number that contains the matching text pattern only.

$ sed -n '/test/=' myfile

sed hide lines

Reading data from a file

We’ve seen how to insert and append data to the data stream. Now we will read data from a file.

The read command (r) allows you to insert data from a separate file.

You can define a single line number or a text pattern address. The sed Linux command inserts the text from the file after the specified address.

$ cat newfile

$ sed '3r newfile' myfile

sed read data from file

The content is just inserted after the third line as expected.

The same technique works when using a text pattern:

$ sed '/test/r newfile' myfile

sed read match pattern

Cool right?

Useful examples

We can use the read command in conjunction with the delete command to replace a placeholder in a file with data from another file. Suppose we have a file called newfile.

The word DATA in that file is a placeholder for a real content which is stored in another file called data.

We will replace it with the actual content:

sed repalce placeholder

Now the placeholder text is replaced with the data file content. Super cool!!

This is just a very small intro about sed command. Actually, sed Linux command is another world by itself.

The only limitation is your imagination.

I hope you enjoy what’ve introduced today about the string manipulation using sed Linux command.

Thank you.