Find & Replace Strings In Multiple Files Using Linux awk

In this tutorial, you’ll learn how to use awk command to find and replace text across multiple files.

From simple text replacements to more complex examples using regular expressions, conditional replacements, and using shell variables.

 

 

Simple Find and Replace

First, let’s consider you have multiple files containing the string “PlanA” and you need to replace it with “PlanB”. Here’s how to do it using awk:

awk '{gsub(/PlanA/, "PlanB"); print}' file1.txt > temp.txt && mv temp.txt file1.txt

This command uses awk gsub function to globally substitute PlanA with PlanB.

After the substitution, the output is redirected to a temporary file temp.txt.

Then, mv is used to replace the original file with the updated one.

 

Find and Replace in Files with a Specific Extension

Let’s say you want to replace “OldPackage” with “NewPackage” in all .cfg files in a directory.

You can use awkand find:

find . -type f -name "*.cfg" -exec sh -c 'awk "{gsub(/OldPackage/, \"NewPackage\"); print}" {} > {}.tmp && mv {}.tmp {}' \;

This command used find to locate all files with the .cfg extension. For each file found, awk is executed to replace “OldPackage” with “NewPackage”.

The output is temporarily saved in a file with the same name plus .tmp. Finally, the original file is replaced by this temporary file.

 

Case-Insensitive Search

Suppose you want to replace the string “ServiceOption” with “PremiumFeature” regardless of its case (e.g., “serviceoption”, “Serviceoption”, “SERVICEOPTION”, etc.) in all .txt files.

Command:

find . -type f -name "*.txt" -exec awk 'BEGIN{IGNORECASE=1}{gsub(/ServiceOption/, "PremiumFeature"); print}' {} > {}.tmp && mv {}.tmp {} \;

By setting IGNORECASE=1, awk performs a case-insensitive match.

The find command locates all .txt files, and for each file, awk replaces any case variation of “ServiceOption” with “PremiumFeature”.

The output is first written to a temporary file and then the original file is replaced with this updated version.

 

Find and Replace with Regex

Imagine you need to update phone numbers in multiple files from an old format to a new one.

For example, transforming phone numbers from the format (123) 456-7890 to 123-456-7890. Here’s how to apply regex:

find . -type f -name "*.data" -exec awk '{gsub(/\([0-9]{3}\) [0-9]{3}-[0-9]{4}/, "123-456-7890"); print}' {} > {}.tmp && mv {}.tmp {} \;

This command uses a regex pattern in awk gsub function to match phone numbers in the specified format.

The pattern \([0-9]{3}\) [0-9]{3}-[0-9]{4} locates phone numbers with the old format and replaces them with the new format.

As in previous examples, the find command locates the relevant files, and the changes are first made in a temporary file before overwriting the original.

 

Find and Replace Based on a Specific Column

Suppose you want to replace the string “BasicTier” with “StandardTier” in the third column of multiple .csv files.

Command:

find . -type f -name "*.csv" -exec awk -F, '{if($3 == "BasicTier") $3="StandardTier"; print}' OFS=, {} > {}.tmp && mv {}.tmp {} \;

This command uses awk with the -F, option which sets the field separator to a comma.

The awk script checks if the third field ($3) is “BasicTier” and replaces it with “StandardTier”.

The output field separator (OFS) is also set to a comma, ensuring the CSV format remains intact.

The find command identifies all .csv files and the changes are applied via a temporary file for each.

 

Using Variables in Replacement

Suppose you have multiple .log files, and you want to replace the term “Error” with a variable that contains the current date, like “Error_2024_03_24”.

Command:

current_date=$(date +%Y_%m_%d)
find . -type f -name "*.log" -exec awk -v dateVar="$current_date" '{gsub(/Error/, "Error_"dateVar); print}' {} > {}.tmp && mv {}.tmp {} \;

This command first stores the current date in the current_date variable in a specific format (+%Y_%m_%d).

In the awk command, the -v flag is used to pass this variable as dateVar.

Inside the awk script, gsub replaces “Error” with the concatenated string “Error_” followed by the content of dateVar.

The find command handles the file search, and the substitution is executed on a temporary file before replacing the original.