sed linux
Linux

31+ Examples for sed Linux command in text manipulation

On the previous post we’ve talked about bash functions and how to use it from the command line and we’ve seen some other cool stuff I recommend you to review it, Today we will talk about a very useful tool for string manipulation called sed, sed Linux command is one of the most common tools that people use to work with text files like log files, configuration files, and other text files. If you perform any type of data manipulation in your bash scripts, you want to become familiar with the sed and gawk tools in this post we are going to focus on sed Linux command and see its ability to manipulate text which is very important step in our bash scripting journey

Sometimes you’ll find yourself want to manipulate text in a text file like huge log file to extract some text, delete, modify or whatever. The Linux system provides some common tools for doing just that one of those tools is sed.

We will discuss the 31+ examples with pictures to show the output of everyone.

Our main points are:

Understand sed command

Using multiple sed commands in the command line

Reading commands from a file

Substituting flags

Replacing characters

Limiting sed

Deleting lines

Inserting and appending text

Modifying lines

Transforming characters

Printing line numbers

Reading data from a file

Useful examples

Understand sed Linux command

The sed command is called a stream editor, it is an interactive text editor, such as nano, you interactively use keyboard commands to insert, delete, or replace text in the data. Sed Linux command edits a stream of data based on a set of rules you provide and this is the format of the sed command

$ sed options file

By default, the sed Linux command applies the specified commands to the STDIN. This allows you to pipe data directly to the sed editor for processing like this

$ echo "This is a test" | sed 's/test/another test/'

sed linux command

The s command substitutes a second text string for the first text string pattern specified between the forward slashes. In this example, the words another test were substituted for the word test so the result will be like this

That’s the power of using the sed Linux command.

The above example was a very basic example to demonstrate the tool. We can use sed Linux command to manipulate files as well.

This is our file

sed manipulate file

$ sed 's/test/another test' ./myfile

You’ll start seeing results before the sed editor completes processing the entire file because sed returns the data instantaneously awesome!

You will notice that when your file is huge enough to see the result even before the processing is finished

The sed editor doesn’t modify the data in the text file itself. It only sends the modified text to STDOUT. If you look at the text file, it still contains the original data. You can overwrite the file with the new content very easy if you follow our previous posts we talk about redirections

Using multiple sed Linux commands in the command line

To execute more than one command from the sed command line, just use the -e option like this

$ sed -e 's/This/That/; s/test/another test/' ./myfile

sed multiple commands

Both commands are applied to each line of data in the file. The commands must be separated by a semicolon, and there shouldn’t be any spaces between the end of the command and the semicolon.

Also, you can use single quotation to separate commands like this

sed separate commands

The same result no big deal.

Reading commands from a file

If you have lots of sed commands you want to process, it is often easier to just store them in a separate file. Use the -f option to specify the file in the sed command like this

$ sed -f mycommands myfile

sed read commands from file

Substituting flags

Look at the following example carefully

$ cat myfile

$ sed 's/test/another test/' myfile

sed substitute flag

The substitute command works fine in replacing text in multiple lines, but it replaces only the first occurrence in each line. To get the substitute command to work on different occurrences of the text, you must use a substitution flag.

The flags are written like this

s/pattern/replacement/flags

There are four types of substitutions

  • A number, indicating the pattern occurrence for which new text should be substituted
  • g, indicating that new text should be substituted for all occurrences of the existing text
  • p, indicating that the contents of the original line should be printed
  • w file, which means to write the results of the substitution to a file

The first type of substitution, you can specify which occurrence of the matching pattern the sed Linux command should substitute new text for

$ sed 's/test/another test/2' myfile

sed number flag

As a result of specifying a 2 as the substitution flag, the sed Linux command replaces the pattern only in the second occurrence in each line.

The g substitution means global enables you to replace every occurrence of the pattern in the text

$ sed 's/test/another test/g' myfile

sed global flag

The p substitution flag prints a line that contains a matching pattern in the substitute command which used with -n option suppresses output from the sed command so it produces output only for lines that have been modified by the substitute command

$ cat myfile

$ sed -n 's/test/another test/p' myfile

sed supress lines

The w substitution flag produces the same output but stores the output in the specified file

$ sed 's/test/another test/w output' myfile

sed send output to file

The output of the sed command appears in STDOUT, but only the lines that include the matching pattern are stored in output file.

Replacing characters

Suppose that you want to substitute the C shell for the bash shell in the /etc/passwd file, you’d have to do this

$ sed 's/\/bin\/bash/\/bin\/csh/' /etc/passwd

That looks confusing for some people because the forward slash is used as the string delimiter, you must use a backslash to escape it.

Luckily there is another way to achieve that. The sed Linux command allows you to select a different character for the string delimiter in the substitute command

$ sed 's!/bin/bash!/bin/csh!' /etc/passwd

The exclamation point is used for the string delimiter. It is now easier to read.

Limiting sed

The commands you use in the sed command apply to all lines of the text data. If you want to apply a command only to a specific line or a group of lines, there are two forms

  • A numeric range of lines
  • A text pattern that filters out a line

The address you specify in the command can be a single line number or a range of lines specified by a starting line number, a comma, and an ending line number

$ sed '2s/test/another test/' myfile

sed restricted

Only line two is modified.

What about using a range of lines

$ sed '2,3s/test/another test/' myfile

sed replace range of lines

Also, we can start from a line to the end of the file

$ sed '2,$s/test/another test/' myfile

sed replace to the end

You can specify a text pattern to filter lines for the sed command. The pattern is written like this

$ sed '/likegeeks/s/bash/csh/' /etc/passwd

sed pattern match

The command was applied only to the line with the matching text pattern.

You can use regular expressions to write this pattern to be more generic and useful. We will talk about regular expressions in detail on future posts in a simple way.

Deleting lines

If you need to delete specific lines of text in a text stream, you can use the delete command

Be careful with the delete command, because if you forget to include an addressing scheme, all the lines are deleted from the stream

$ sed '3d' myfile

sed delete line

Here we delete the third line only from myfile

$ sed '2,3d' myfile

sed delete multiple line

Here we delete a range of lines the second and the third.

$ sed '3,$d' myfile

sed delete to the end

Here we delete from the third line to the end of the file

Note that the sed command doesn’t touch the original file. Any lines you delete are only gone from the output only.

$ sed '/test 1/d' myfile

sed deletepattern match

Here we use pattern to delete the line if matched on the first line

You can also delete a range of lines using two text patterns like the following

$ sed '/second/,/fourth/d' myfile

sed delete range of lines

The first to the third line deleted.

Inserting and appending text

The sed Linux command allows you to insert and append text lines to the data stream using the following commands

  • The insert command (i) adds a new line before the specified line
  • The append command (a) adds a new line after the specified line

You must specify the line to insert or append the line to insert on a separate line by itself So you can’t use these commands on a single command line.

$ echo "Another test" | sed 'i\First test '

sed insert text

As you can see the output comes before the data stream text

$ echo "Another test" | sed 'a\First test '

sed append

While in append mode the text appears after the data stream text.

This works well for adding text before or after the text in the data stream, but what about adding text in the middle?

To insert or append data inside the data stream lines, you must specify where the sed command where you want the data to appear.

You can match either a numeric line number or a text pattern, and of course, you cannot use a range of addresses.

$ sed '2i\This is the inserted line.' myfile

sed insert line

And the appending goes the same way but look at the position of the appended text

$ sed '2a\This is the appended line.' myfile

sed append line

The difference is it places the new text line after the specified line number.

Modifying lines

The change command allows you to change the contents of an entire line of text in the data stream. All you have to do is to specify the line that you want to change.

$ sed '3c\This is a modified line.' myfile

sed modify line

You can also use a text pattern or a regular expression and all lines match that pattern will be modified.

$ sed '/This is/c This is a changed line of text.' myfile

sed pattern match

Transforming characters

The transform command (y) works on a single character like this.

$ sed 'y/123/567/' myfile

sed transform character

You can’t limit the transformation to a specific occurrence of the character.

Printing line numbers

The equal sign command prints the current line number for the line within the data stream.

$ sed '=' myfile

sed line numbers

The sed editor prints the line number before the actual line of text nothing fancy here.

However by using –n combined with the equal sign the sed command display the line number that contains the matching text pattern only.

$ sed -n '/test/=' myfile

sed hide lines

Reading data from a file

We’ve seen how to insert and append data to the data stream. Now we will read data from a file.

The read command (r) allows you to insert data contained in a separate file.

You can only specify a single line number or text pattern address. The sed Linux command inserts the text from the file after the address specified.

$ cat newfile

$ sed '3r newfile' myfile

sed read data from file

The file called newfile content is just inserted after the third line as expected.

The same technique works when using a text pattern

$ sed '/test/r newfile' myfile

sed read match pattern

Cool right?

Useful examples

We can use the read command is to use it in conjunction with a delete command to replace a placeholder in a file with data from another file. Suppose we have the following file called newfile

The word DATA in the file is a placeholder for a real content which is stored on another file called data. We will replace it with the actual content

sed repalce placeholder

Now the placeholder text is replaced with the data file content. Super cool!!

This is just a very small intro about sed command. Actually, sed Linux command is another world by itself.

We can spend weeks to cover sed and its uses but you can use your mind with our previous posts to produce something great. As I said before the only limitation is your imagination.

I hope you enjoy what’ve introduced today about the string manipulation using sed Linux command.

This is just a beginning for sed command I will make another post about sed in future posts.

Thank you.