Practical Examples for Text Replacement Using Sed Command

In this tutorial, you will delve into practical examples of how sed command can be used in text replacement.

By the end of this tutorial, you’ll have a stronger knowledge about sed and its real-world applications in text substitution.

 

 

Change HTTP to HTTPS in URLs

Command:

echo "Visit my website at http://example.com" | sed 's/http:/https:/'

Output:

Visit my website at https://example.com

This command pipes an echo statement with the URL “http://example.com ” into sed, which then replaces “http:” with “https:”.

If you had a file named urls.txt containing multiple URLs, you can execute:

sed -i 's/http:/https:/g' urls.txt

With the -i option, sed edits the file in-place, and the g at the end of the expression ensures that all occurrences in each line are replaced, not just the first one.

 

Replace Spaces with Underscores

Command:

echo "This is a sample text" | sed 's/ /_/g'

Output:

This_is_a_sample_text

When dealing with files, the approach remains the same. Say you have a file named file.txt with multiple lines of text and you want to replace all spaces with underscores:

sed -i 's/ /_/g' file.txt

 

Change File Extensions

Command:

echo "image.jpg" | sed 's/\.jpg$/.png/'

Output:

image.png

In the given example, we target filenames with the .jpg extension and switch them to the .png extension. The key here is the \.jpg$ pattern.

The dot . is a special character in regular expressions, so we escape it using a backslash \.

The $ denotes the end of the line, ensuring we’re matching the file extension and not some other .jpg occurrence within a filename.

 

Replace Text Only in Odd or Even Lines

Replacing Text in Odd Lines

echo -e "Line1\nLine2\nLine3\nLine4" | sed '1~2s/Line/Odd/'

Output:

Odd1
Line2
Odd3
Line4

Using the pattern 1~2, we target every alternate line starting from the first, meaning all odd-numbered lines.

The provided sed command replaces the word “Line” with “Odd” but only on these odd lines.

Replacing Text in Even Lines

echo -e "Line1\nLine2\nLine3\nLine4" | sed '2~2s/Line/Even/'

Output:

Line1
Even2
Line3
Even4

With the pattern 2~2, this command affects every alternate line starting from the second, translating to all even-numbered lines.

The text “Line” is substituted with “Even” exclusively on these even lines.

 

Replace Line Starting with Specific Word

Command:

echo -e "Start: Begin the journey\nEnd: Conclude the trip\nStart: Initiate" | sed '/^Start:/c\Initiation Line'

Output:

Initiation Line
End: Conclude the trip
Initiation Line

The ^Start: pattern matches any line that begins with the word “Start:”.

The c\ command in sed tells it to replace (or change) the matched line with the text following it, in this case, “Initiation Line”.

Thus, all lines that begin with “Start:” are replaced in entirety with “Initiation Line”.

 

Remove Trailing Whitespaces

Command:

echo -e "This is a line with spaces    " | sed 's/[ \t]*$//'

Output:

This is a line with spaces

The pattern [ \t]*$ matches zero or more spaces or tabs ([ \t]*) right before the end of a line ($).

When we pair this with the substitution command in sed, it effectively deletes those trailing whitespaces.

 

Replace Newlines with Spaces

Command:

echo -e "This is the first line\nThis is the second line" | sed ':a;N;$!ba;s/\n/ /g'

Output:

This is the first line This is the second line

In the above command, we transform newlines (\n) into spaces, joining lines together.

Let’s understand its parts:

1. :a: Define a label named a.
2. N: Read/append the next line of input into the pattern space.
3. $!ba: If we’re not at the end of the file ($!), branch (or go back) to the label a. This ensures we’ve read the whole file into the pattern space.
4. s/\n/ /g: Once the whole file is loaded, replace every new line with a space.

 

Replace Text in Specific Lines

Command:

echo -e "Line1\nLine2\nLine3" | sed '2s/Line/Text/'

Output:

Line1
Text2
Line3

For this command, we’re explicitly targeting the second line.

The 2 before the s tells sed to only apply the substitution command to the second line.

Replace Text Between Two Lines

Command:

echo -e "Line1\nLine2\nLine3\nLine4" | sed '2,3s/Line/Text/'

Output:

Line1
Text2
Text3
Line4

This example demonstrates a range operation. By specifying 2,3 before the s, we instruct sed to make the replacement from lines 2 through 3.

Both “Line2” and “Line3” are transformed to “Text2” and “Text3” respectively.

 

Replace Uppercase with Lowercase

Command:

echo "HELLO World" | sed 's/\(.*\)/\L\1/'

Output:

hello world

The goal here is to transform all uppercase letters into their lowercase.

The pattern \(.*\) captures the entire line. The \L then converts the matched portion (the whole line, in this case) to lowercase.

 

Replace Tabs with Spaces

A common practice is to replace tabs with four spaces for consistency.

echo -e "This is\ttabbed content" | sed 's/\t/    /g'

Output:

This is    tabbed content

The provided sed command takes a tab character (\t) and replaces it with four spaces.

 

Remove/Strip HTML Tags

HTML tags identified by the < and > characters.

echo "<h1>This is a title</h1>" | sed 's/<[^>]*>//g'

Output:

This is a title

The given sed command matches any sequence starting with < and ending with >, and replaces it with nothing, effectively removing it.

The pattern <[^>]*> breaks down as:

• <: Match the opening <.
• [^>]*: Match zero or more characters that are not >.
• >: Match the closing >.

 

Convert HTML Entities to Characters

HTML entities are a means of representing special characters in HTML. Some common entities include & for “&”, < for “<“, and > for “>”.

echo "This is &amp; example" | sed 's/&amp;/\&/g; s/&lt;/</g; s/&gt;/>/g'

Output:

This is & example

The sed command above replaces these entities with their corresponding characters.

The semicolons in the command (;) allow for multiple substitutions in a single sed command.

 

Replace Multiple Words

You can use semicolons to write multiple replacements in a single command:

echo "The cat sat on the mat" | sed 's/cat/lion/g; s/mat/rock/g'

Output:

The lion sat on the rock

With sed, multiple substitutions can be combined using semicolons (;). In this command, we replace both “cat” with “lion” and “mat” with “rock”.

 

Replace Text After a Match

Command:

echo "Title: Linux Basics" | sed 's/\(Title: \).*/\1Intro to Linux/'

Output:

Title: Intro to Linux

In this command, we’re targeting text that comes after a specific pattern, in this case “Title: “.

The idea is to retain the “Title: ” prefix while replacing everything after it.

The pattern \(Title: \).* captures “Title: ” into a group (thanks to the parentheses) and then matches everything that follows with .*.

In the replacement part \1Intro to Linux, \1 refers back to the matched group, and we append our new text “Intro to Linux” after it.

 

Replace Text Before a Specific Word

Command:

echo "Training for marathon" | sed 's/.* \(marathon\)/Preparation for \1/'

Output:

Preparation for marathon

In this example, the aim is to replace text that precedes a specific word, in this case, “marathon”.

The pattern .* \(marathon\) matches everything up to and including the word “marathon”, capturing “marathon” into a group with the parentheses.

The replacement part Preparation for \1 uses \1 to refer back to the captured group, allowing us to keep the word “marathon” while replacing the text preceding it.

 

Replace Text Inside a Parenthesis

Command:

echo "This is a (sample) text." | sed 's/(\(.*\))/[modified]/'

Output:

This is a [modified] text.

The pattern (\(.*\)) captures the content inside the parentheses.

The parentheses ( and ) are used literally, while the inner \( and \) form a capturing group around the content.

The replacement part ([modified]) replaces the entire captured content with the word “modified” enclosed within square brackets.

 

Change Date Format

Command:

echo "Today's date is 08/31/2023." | sed 's/\([0-9]\{2\}\)\/\([0-9]\{2\}\)\/\([0-9]\{4\}\)/\3-\1-\2/'

Output:

Today's date is 2023-08-31.

In this case, we’re converting the MM/DD/YYYY format to YYYY-MM-DD. The sed pattern consists of three capturing groups, each capturing a segment of the date:

1. \([0-9]\{2\}\): Matches two digits, representing the month.
2. \([0-9]\{2\}\): Matches another two digits, representing the day.
3. \([0-9]\{4\}\): Matches four digits, indicating the year.

The replacement part, \3-\1-\2, reorders these captured groups to form the new date format.

 

Convert JSON to YAML

Using sed to convert JSON to YAML is a complex and potentially error-prone task due to the nesting structures of both formats.

For advanced substitution, a dedicated tool like jq or yq would be more suitable for conversions.

However, here’s a simple example:

echo '{ "name": "John", "age": 30, "city": "New York" }' | sed 's/{//;s/}//;s/,/\n/g;s/":"/: /g;s/"//g'

Output:

 name: John
 age: 30
 city: New York

In this command, 's/,/\n/g' replaces all commas with newline characters, and 's/":"/: /g' replaces all occurrences of ":" with ": ". This will work for the simple JSON object provided

For robust JSON to YAML conversions in files or complex JSON structures, using a dedicated utility like jq in tandem with yq is highly recommended.

 

Replace JSON value

Let’s replace the value for a specific JSON key.

echo '{ "name": "John", "age": 30, "city": "New York" }' | sed 's/\("name": "\)[^"]*/\1Doe/'

Output:

{ "name": "Doe", "age": 30, "city": "New York" }

In this command, we’re targeting the value associated with the “name” key and replacing “John” with “Doe”.

The pattern \("name": "\)[^"]* captures the key “name” and its associated value until the next double-quote.

In the replacement, \1Doe, the \1 recalls the captured group (which is “name”: “), and “Doe” serves as the new value.

Manipulating JSON content is best done with specialized tools like jq. But for simpler tasks, sed can still be used.

 

Wrap Text in Quotes

If you want to wrap text strings within quotes using sed, you can do it like this:

echo This is a sample text | sed 's/\(.*\)/"\1"/'

Output:

"This is a sample text"

In this command, the pattern \(.*\) captures the entire line of text.

In the replacement section, "\1", the \1 refers back to the captured content, which we wrap in double quotes.

 

Replace First Character in Every Line

Here’s how you can replace the first character of every line with sed:

echo -e "Aapple\nBbanana\nCcherry" | sed 's/^.//'

Output:

apple
banana
cherry

The sed command used here targets the first character of every line and removes it. The pattern ^. matches the initial character of each line.

The replacement pattern is empty, so the matched character is essentially deleted.

 

Replace Last Character in Every Line

Here’s a straightforward method to alter or remove the last character of every line using sed:

echo -e 'apple.\nbanana!\ncherry?' | sed 's/.$//'

Output:

apple
banana
cherry

The pattern .$ matches the last character of each line.

The following replacement pattern is empty, meaning the matched character gets deleted.

 

Add Line Numbers

You can use sed to add line numbers like this:

echo -e "apple\nbanana\ncherry" | sed = | sed 'N; s/\n/ /'

Output:

1 apple
2 banana
3 cherry

sed 'N; s/\n/ /': This merges the line number and the corresponding line, replacing the new line with a space.

 

Replace Single Quote

By enclosing the entire sed command in double quotes, we can refer to the single quote within the command.

echo "It's a wonderful day!" | sed "s/'//g"

Output:

Its a wonderful day!

The pattern s/'//g searches for all single quotes and replaces them with nothing (removes them).

To replace single quotes with another character or string, for example, with an underscore:

echo "It's a wonderful day!" | sed "s/'/_/g"

Output:

It_s a wonderful day!

 

Replace Double Quote

By wrapping the entire sed command in single quotes, we can refer to the double quotes inside the command.

echo 'He said, "Hello!"' | sed 's/"/-/g'

Output:

He said, -Hello!-

The pattern s/"/-/g seeks all double quotes and replaces them with dashes.

If you wish to completely remove double quotes:

echo 'He said, "Hello!"' | sed 's/"//g'

Output:

He said, Hello!

 

Replace Backslash

The backslash is an escape character, so to represent a single backslash in a sed expression, you need to use four backslashes (\\\\).

echo 'This is a backslash: \\' | sed 's/\\\\/-/g'

Output:

This is a backslash: -

In the example above, \\\\ matches a single backslash, and the pattern s/\\\\/-/g replaces it with a dash.

 

Replace XML tags

Let’s replace the opening and closing <tag> elements with <newTag>:

echo '<tag>Content goes here</tag>' | sed 's/<\/\?tag>/<newTag>/g'

Output:

<newTag>Content goes here<newTag>

The pattern <\/\?tag> matches either <tag> or </tag>, thanks to the optional slash (\? denotes that the preceding character, /, is optional).

To remove the content between XML tags:

echo '<tag>Content goes here</tag>' | sed 's/<tag>.*<\/tag>/<tag><\/tag>/'

Output:

<tag></tag>

 

Convert Markdown headers to HTML Tags

Markdown is a popular lightweight markup language, with its headers indicated by # symbols.

Converting these headers into their corresponding HTML tags can make your content web-ready.

Convert # (h1) headers:

echo '# Header 1' | sed 's/^# \(.*\)/<h1>\1<\/h1>/'

Output:

<h1>Header 1</h1>

Convert ## (h2) headers:

echo '## Header 2' | sed 's/^## \(.*\)/<h2>\1<\/h2>/'

Output:

<h2>Header 2</h2>

You can follow the same pattern for h3, h4, h5, and h6 headers by increasing the number of # symbols and adjusting the h value in the <hx> tag accordingly.

 

Replace Home Directory Path with Tilde

In Unix-like systems, the tilde (~) is often used as a shorthand for the user’s home directory.

If you want to replace the explicit home directory reference with a tilde, here’s how to do that with sed:

Assuming a home directory like /home/username, you would do:

echo '/home/username/Documents/myfile.txt' | sed "s~/home/username~\~~g"

Output:

~/Documents/myfile.txt

In this example, we used the ~ as the delimiter for the sed command instead of the typical / to avoid clashing with the forward slashes in the path.

The pattern s~/home/username~\~~g replaces all occurrences of /home/username with ~.

 

Remove Duplicate Slashes in a Path

If you’re manipulating paths or URLs, you might encounter extraneous repeated slashes (//).

To remove these duplicates, you can do it like this:

echo '/home//username///Documents/myfile.txt' | sed 's~//\+~/~g'

Output:

/home/username/Documents/myfile.txt

In this example, we’re utilizing the ~ as the delimiter for the sed command to avoid conflicts with the forward slashes in the path.

The pattern s~//\+~/~g finds sequences of one or more slashes (//\+) and replaces them with a single slash.