31+ Examples for sed Linux Command in Text Manipulation

In the previous post, we talked about bash functions and how to use them from the command line directly, and we saw some other cool stuff. Today we will talk about a handy tool for string manipulation called sed or sed Linux command.

We use sed to work with text files like log files, configuration files, and other text files.

In this post, we are going to focus on the sed Linux command, which is used for text manipulation, which is an essential step in our bash scripting journey.

Linux system provides some tools for text processing; one of those tools is sed.

We will discuss the 31+ examples with pictures to show the output of every example.

 

 

Understand sed Linux command

The sed command is a non-interactive text editor. Sed Linux command edits data based on the rules you provide; you can use it like this:

$ sed options file

You are not limited to use sed to manipulate files; you apply it to the STDIN directly like this:

$ echo "Welcome to LikeGeeks page" | sed 's/page/website/'

sed Linux command

The s command replaces the first text with the second text pattern. In this case, we replaced the string “website” with the word “page”.

The above example was a simple example to demonstrate the tool. We can use the sed Linux command to manipulate files as well.

Let’s create a sample file :

sed manipulate file

$ sed 's/test/another test/' ./myfile

The results are printed to the screen instantaneously; you don’t have to wait for processing the file to the end.

If your file is massive enough, you will see the result before the processing is finished.

Sed Linux command doesn’t update your data. It only sends the changed text to STDOUT. The file is still untouched. If you need to overwrite the existing content, you can check our previous post, which was talking about redirections.

 

Using multiple sed Linux commands in the command line

To run multiple sed commands, you can use the -e option like this:

$ sed -e 's/This/That/; s/test/another test/' ./myfile

sed multiple commands

You must separate sed commands by a semicolon without any spaces.

Also, you can use a single quotation to separate commands like this:

$ sed -e '
> s/This/That/
> s/test/another test/' myfile

sed separate commands

The same result, no big deal.

 

Reading commands from a file

You can save your sed commands in a file and use them by specifying the file using -f option.

$ cat mycommands

s/This/That/
s/test/another test/
$ sed -f mycommands myfile

read commands from file

Substituting flags

Look at the following example carefully:

$ cat myfile
$ sed 's/test/another test/' myfile

sed substitute flag

The above result shows that we replaced the first occurrence in each line. To substitute all occurrences of a pattern, use one of the following substitution flags.

We can write the flags like this:

s/pattern/replacement/flags

There are four types of substitutions:

  • g, replace all occurrences.
  • A number, the occurrence number for the new text that you want to substitute.
  • p, print the original content.
  • w file: means write the results to a file.

You can limit your replacement by specifying the occurrence number that should be replaced like this:

$ sed 's/test/another test/2' myfile

sed number flag

As you can see, we replaced only the second occurrence on each line.

The g flag means global, which means a global replacement for all occurrences:

$ sed 's/test/another test/g' myfile

sed global flag

The p flag prints each line contains a pattern match, you can use the -n option to print the modified lines only.

$ cat myfile
$ sed -n 's/test/another test/p' myfile

sed supress lines

The w flag saves the output to a specified file:

$ sed 's/test/another test/w output' myfile

send output to file

We printed the output on the screen, but we saved the matching lines to the output file.

 

Replace characters

Suppose that you want to search for bash shell and replace it with csh shell in the /etc/passwd file using sed, well, you can do it easily:

$ sed 's/\/bin\/bash/\/bin\/csh/' /etc/passwd

Oh!! that looks terrible.

Luckily, there is another way to achieve that. You can use the exclamation mark (!) as string delimiter like this:

$ sed 's!/bin/bash!/bin/csh!' /etc/passwd

Now it’s easier to read.

 

Limiting sed

Sed command processes your entire file. However, you can limit the sed command to process specific lines; there are two ways:

  • A range of lines.
  • A pattern that matches a specific line.

You can type one number to limit it to a specific line:

$ sed '2s/test/another test/' myfile

sed restricted

We modified only line two.

What about using a range of lines:

$ sed '2,3s/test/another test/' myfile

replace range of lines

Also, we can start from a line to the end of the file:

$ sed '2,$s/test/another test/' myfile

sed replace to the end

Or you can use a pattern like this:

$ sed '/likegeeks/s/bash/csh/' /etc/passwd

sed pattern match

Awesome!!

You can use regular expressions to write this pattern to be more generic and useful.

 

Delete lines

To delete lines using sed, you can use the delete (d) flag is your friend.

The delete flag deletes the text from the stream, not the original file.

$ sed '2d' myfile

sed delete line

Here we delete the second line only from myfile.

What about deleting a range of lines?

$ sed '2,3d' myfile

delete multiple line

Here we delete a range of lines, the second and the third.

Another type of ranges:

$ sed '3,$d' myfile

delete to the end

Here we delete from the third line to the end of the file.

All these examples never modify your original file.

$ sed '/test 1/d' myfile

delete pattern match

Here we use a pattern to delete the line if matched on the first line.

If you need to delete a range of lines, you can use two text patterns like this:

$ sed '/second/,/fourth/d' myfile

delete range of lines

We deleted from the second to the fourth line.

 

Insert and append text

You can insert or append text lines using the following flags:

  • The (i) flag.
  • The  (a) flag.
$ echo "Another test" | sed 'i\First test '

sed insert text

We added the text before the specified line.

$ echo "Another test" | sed 'a\First test '

sed append

We added the text after the specified line.

Well, what about adding text in the middle?

Easy, look at the following example:

$ sed '2i\This is the inserted line.' myfile

sed insert line

And the appending works the same way, but look at the position of the appended text:

$ sed '2a\This is the appended line.' myfile

sed append line

We used the same flags but with a location of insertion or appending.

 

Modifying lines

To modify a specific line, you can use the (c) flag like this:

$ sed '3c\This is a modified line.' myfile

sed modify line

You can use a regular expression pattern, and all lines match that pattern will be modified.

$ sed '/This is/c Line updated.' myfile

sed pattern match

 

Transform characters

The transform flag (y) works on characters like this:

$ sed 'y/123/567/' myfile

sed transform character

We applied the transformation to all data, and we cannot limit it to a specific occurrence.

 

Print line numbers

You can print line number using the (=) sign like this:

$ sed '=' myfile

sed line numbers

However, by using -n combined with the equal sign, the sed command displays the line number that contains matching.

$ sed -n '/test/=' myfile

hide lines

 

Read data from a file

You can use the (r) flag to read data from a file.

You can define a line number or a text pattern for the text that you want to read.

$ cat newfile
$ sed '3r newfile' myfile

read data from file

We inserted the content after the third line as expected.

And this is using a text pattern:

$ sed '/test/r newfile' myfile

read match pattern

Cool right?

 

Useful examples

Suppose that we have a file that contains text with a placeholder, and we have another file that contains the data that will fill the placeholder on the other file.

We will use the (r) and (d) flags to do the job.

The word DATA in that file is a placeholder for real content that we will store in another file called data.

We will replace it with the actual content:

$ Sed '/DATA>/ {
r newfile
d}' myfile

repalce placeholder

Awesome!! as you can see, we filled the placeholder location with the data from the other file.

This is just a small intro about the sed command. The sed Linux command is another world by itself.

The only limitation is your imagination.

36 thoughts on “31+ Examples for sed Linux Command in Text Manipulation
  1. I use sed on a daily basis for basic manipulation with some regular expressions, but it’s always great to get refreshers on it’s capabilities. Thanks for sharing !

    1. You are welcome.

      sed and awk are sys admin love daily use to process log files and much more.
      I think that I will write a post about regular expressions before continuing on text processing.

      Regards,

    2. hi Killaklik, can you please help me in a small transformation using sed,
      echo “Welcome To The ${geek.learn} Stuff” | sed ‘s/\./\_/g’
      please change and execute this to make OUTPUT as: Welcome To The ${GEEK_LEARN} Stuff

      The content in the flower bracket should change to upper case and . To transform to _

      1. You can replace the entire words between the brackets like this:
        $ sed -i 's/geek\.learn/GEEK_LEARN/g' myfile

        1. bro, will give you exact input,
          sp.config.fal.proxyhost=${sp.config.fal.proxyhost}
          saml.keystore.path=${liferay.home}/tomcat/conf/LiferayKeyStore.jks
          sp.config.uis.authcode=YUZoTjVMeVZLTlh2YTlKQ2RpQUZQdUJJaUxjRWk1RXM6dkh0WEw3UjhTSFg5Zk1kdVQwTXQ0aXJZM2NROVF4alU=

          # This is the definition that needs to be used for Cluster B (VW, VWN, Skoda, Seat)
          sp.config.mal.service.certificate.path.v=certificates/Systemuser_ServicePortletQA_VWPKI_34A533B96DC76C67.p12
          sp.config.fal.proxyhost=25003860

          1. and expected output would be:
            SP_CONFIG_FAL_PROXYHOST: ((SP_CONFIG_FAL_PROXYHOST))
            SAML_KEYSTORE_PATH: ((LIFERAY_HOME))/tomcat/conf/LiferayKeyStore.jks
            SP_CONFIG_UIS_AUTHCODE: YUZoTjVMeVZLTlh2YTlKQ2RpQUZQdUJJaUxjRWk1RXM6dkh0WEw3UjhTSFg5Zk1kdVQwTXQ0aXJZM2NROVF4alU=
            sp.config.fal.proxyhost: 25003860

            sharing you my sample script:(not fully completed though)
            the input is a properties file, so I started my code using :
            #/bin/bash
            if [ ! -f ./newscript/complete_manifest.yml ]; then
            echo -e “\nGenerating a complete_manifest.yml file”
            elif [ -f ./newscript/complete_manifest.yml ]; then
            rm -rf ./newscript/complete_manifest.yml
            echo -e “\nGenerating a complete_manifest.yml file”
            fi
            existing_file=”./manifest.yml”

            # Generate the file
            cat $existing_file > ./newscript/complete_manifest.yml
            export upper_case=`awk -F “[()]” ‘{ for (i=3; i<NF; i+=2) print $i }' ./newscript/complete_manifest.yml | tr '[:lower:]' '[:upper:]'`
            export lower_case=`awk -F "[()]" '{ for (i=3; i> ./newscript/complete_manifest.yml
            fi
            done < "$file"

            else
            echo "$file not found."

            fi

            sed -i 's/${/((/' ./newscript/complete_manifest.yml
            sed -i 's/}/))/' ./newscript/complete_manifest.yml
            sed -i -e 's/\r//g' ./newscript/complete_manifest.yml
            sed -i 's/=/: /' ./newscript/complete_manifest.yml
            sed -i '/^\s*[:@#]/ d' ./newscript/complete_manifest.yml
            sed -i -r '/^\s*$/d' ./newscript/complete_manifest.yml
            sed -i -e 's/$UPPER_CASE/$LOWER_CASE/g' ./newscript/complete_manifest.yml

          2. I’m a bit confused.
            What do you mean by the definition that needs to be used?

  2. >>> The s command replaces the second text string with the first text string pattern specified between the first forward slashes.

    This is wrong, you should fix this. You have some other similar places in your text.

    Thanks a lot for your bash tutorial series, I enjoy it very much.

    1. Thank for your interest.
      what other similar places? i didn’t got your point.

          1. You article says, that second string is replaced with first one. In fact, first one is replaced with second one (see you own screenshot!!). Please, fix you article so readers don’t get confused.

  3. I want to Add a line using sed in the 4th last line of a file. Can anyone please help me wth command.
    I am trying to use variables in sed but throwing an error.
    Considering that we do not know the total number of lines in a file.

    1. You can add a line in the same way as the examples above
      sed '4aThis is the appended line.' youfile.txt
      The above example will append a line after the fourth line.
      To insert a line, type it like this:
      sed '4iThis is the inserted line.' youfile.txt
      Check the difference in the examples above.
      Regards.

      1. I need to enter a line in the 4th LAST line of a file. Since the size of the file could vary, i mean the total number of lines, i am thinking i need to use variables.

        1. As you said, because the number of lines could vary, you have to count the total number of lines first then if there is a fourth line, you can insert your text after it.

          1. Yes right so what I was trying to do was..
            a=cat test.txt >> wc -l

            Post this i am not able to user this varaible a and then subtract 4 out of it and use

            sed ‘$(a-4) iThis is the inserted line.’ test.txt –> does not work.

          2. The code you’ve written is incomplete, also, your test file is unknown.
            Check your error and figure what line causes the problem.

        2. Better late then never….
          seq 1 9 > file.txt
          cat file.txt
          # magic:
          sed $(( $(cat file.txt|wc -l) – 4 ))’iinserted’ file.txt

          Yes, please sed a few digits to my bank account, too! 🙂

  4. I want to append some string after each line of the file except the header and footer of the files .
    So I have used below command to change the range of the lines except header and footer but its deleting header and footer and I do not want to delete the header and footer lines .

    sed -n ‘2,5p; ‘”$str_var”” abc.dat > abc.dat.temp

    even I have tried to skip the header and modified other lines by using below command but no options for footer/ trailer line to keep it as unchange .

    sed ‘1! s/$/'”$str_var”‘/g’ abc.dat > abc.dat.temp

    any suggestions how to change all the lines of the file except the header and footer line and the command should not be deleting the header/footer line .

    1. You can exclude lines like this:
      sed '10,20!s/xxx/yyy/' input > output
      This will exclude from line 10 to 20. and replace xxx with yyy

  5. The first statement under the title “Understand sed Linux Command” is: “The sed command is an interactive text editor like nano.”. That is hardly believable; sed is the abbreviation of “Stream EDitor”, or editor over a data stream: You instruct it on what you want to be done and get out of its way until it is finished, after the data stream is processed as a whole. This is the complete opposite of interactive! In fact, I remember from class -a long time ago, by the way- that sed was shown as an example of a NON-interactive program, and as a teacher I’ve continued showing sed as a NON-interactive program for a very long time. Am I wrong?

      1. Thank YOU for an excellent article. Wish I had it back when learning sed…

  6. Hi there i want to extract a part from a filename for example :
    NOIP2CONF=/etc/noip2.conf

    i need to save just “noip2″ in another variable
    i try with :
    PIDFILE=$(sed -n ‘s/^[[:space:]]*pid_file[[:space:]]*”\?\([^”]*\)\”\?/\1/p’ $NOIP2CONF)

    but is empty. well i keep searching how i can get it

    1. Can you elaborate more so I can help you?
      Also, can you write more lines from your files to see the pattern?

      Regards,

  7. You have a typo in an early example:
    ” $ sed ‘s/test/another test’ ./myfile”
    This returns an error, “unterminated substitute in regular expression”
    I was able to get the desired result like this:
    ” $ sed ‘s/test/another test/’ ./myfile”

  8. $ Sed ‘/DATA>/ { r newfile d}’ myfile
    to this command i am getting error as sed: -e expression #1, char 0: unmatched `{‘ please give me solution

    1. Check it creafully:
      $ Sed '/DATA>/ {
      r newfile
      d}' myfile

      You should use ‘ not ‘

    1. You can print specific lines like this:
      $ sed -n '14p;17p;33p' myfile.txt
      Hope that helps!

  9. Hi, guys!

    I have a problem because of different versions of a program in the generated outputs. The problem could

    be easily fixed if I can change two lines if they are inverse. However, both lines starting with space

    and the one contains normal characters, space and an = character too (pattern1), the other one is a

    fixed one starting with space containing an : character, too (pattern2). Furthermore, depending on the

    version of the output, pattern1 can be present more than once but I only want to have pattern2 before

    the first occurence of pattern1. I was thinking to simply delete pattern2 and insert before pattern1. It

    works fine if I have only one occurrence but if I have more and wanted to insert also 0, it changed

    further parts of the file it should not. Can you help me?

    ok for one occurrence:
    sed ‘/pattern2/d’ file1 | sed ‘/^pattern2.*/i \pattern1’ > file2

    problematic:
    sed ‘/pattern2/d’ file1 | sed ‘0,/^pattern2.*/i \pattern1’ > file2

    pattern1: ” word1 =”
    pattern2: ” word2 word3:”

    file:
    blabla (should not be changed, may contain word1 but not pattern1)
    pattern1 data or pattern2
    pattern2 or pattern1 data
    lots of other data
    pattern1 data (exactly the same line as the first occurrance)
    lots of further data can contain word1 many times but not pattern1

Leave a Reply

Your email address will not be published. Required fields are marked *