Replace Columns in Files Using Linux awk

In this tutorial, you will learn various methods to replace columns in a file using awk.

From basic replacements to more advanced operations like conditional replacements, integrating external variables, and much more.

 

 

Basic Column Replacement

First, let’s take a sample data file, customer_data.txt, which looks like this:

101, Taylor, Premium
102, Alex, Basic
103, Morgan, Standard

To replace the Plan Type column with new data, use this command:

awk -F, '{$3=" NewPlan"; print}' OFS=, customer_data.txt

Output:

101, Taylor, NewPlan
102, Alex, NewPlan
103, Morgan, NewPlan

This command sets the field separator as a comma -F,. The awk script {$3="NewPlan"; print} replaces the third column with NewPlan and prints the entire line.

The OFS=, sets the output field separator to a comma.

 

Replace Multiple Columns at Once

To replace the Name and Plan Type columns, use the following command:

awk -F, '{$2=" NewName"; $3=" NewPlan"; print}' OFS=, customer_data.txt

Output:

101, NewName, NewPlan
102, NewName, NewPlan
103, NewName, NewPlan

In this command, $2="NewName" and $3="NewPlan" instruct awk to set the second and third columns to NewName and NewPlan respectively.

 

Conditional Column Replacement

To replace the third column with “Premium” for “Basic” plans and print all rows. Here’s how:

awk -F, '{$3=($3==" Basic"?"Premium":$3); print}' OFS=, customer_data.txt

Output:

101, Taylor, Premium
102, Alex, Premium
103, Morgan, Standard

In this command, awk uses a ternary operator ($3==" Basic"?"Premium":$3). It checks if the third column is “Basic”.

If true, it changes it to “Premium”; otherwise, it retains the original value.

 

Column Replacement Based on External Variable

Imagine you have a variable in your shell and you want to update customer_data.txt based on this variable.

Let’s set a shell variable first:

new_plan=" Elite"

Then, using this variable in awk to replace the Plan Type in our data file:

awk -v plan="$new_plan" -F, '{$3=plan; print}' OFS=, customer_data.txt

Output:

101, Taylor, Elite
102, Alex, Elite
103, Morgan, Elite

Here, -v plan="$new_plan" passes the shell variable new_plan to awk as a variable named plan.

 

Replacing with Incremented Values

To replace the first column with a sequential number starting from 1, use this awk command:

awk -F, '{$1=NR; print}' OFS=, customer_data.txt

Output:

1, Taylor, Premium
2, Alex, Basic
3, Morgan, Standard

In this command, NR represents awk‘s built-in variable that holds the current record number (which increments with each line processed).

 

Replace with a Computed Value

Let’s say you want to create a loyalty score in the last column, based on a simple calculation: (length of customer's name) * 10.

To compute and replace the last column with the loyalty score:

awk -F, '{$3=length($2) * 10; print}' OFS=, customer_data.txt

Output:

101, Taylor, 70
102, Alex, 50
103, Morgan, 70

In this script, length($2) calculates the length of the second column and multiply it by 10.

 

Replace Using a Substring

Suppose you want to replace the third column with the first three letters of the second column.

The awk command for this will be:

awk -F, '{$3=substr($2,1,4); print}' OFS=, customer_data.txt

Output:

101, Taylor, Tay
102, Alex, Ale
103, Morgan, Mor

This command uses the substr($2,1,3) function to extract the substring from the second column, starting at the first character and taking three characters in length.

 

Replace with External File Content

Assume you have a file new_plan.txt with new plan types you want to use:

Gold
Silver
Bronze

And the customer_data.txt:

101, Taylor, Premium
102, Alex, Basic
103, Morgan, Standard

To replace the third column in customer_data.txt with the lines from new_plan.txt, use the following awk command:

awk -F, 'NR==FNR{plans[NR]=$0; next} {print $1, $2, plans[FNR]}' OFS=, new_plan.txt customer_data.txt

Output:

101, Taylor, Gold
102, Alex, Silver
103, Morgan, Bronze

In this command, awk first reads new_plan.txt.

NR==FNR is a condition true only while reading the first file, where it stores each line in the plans array.

When processing customer_data.txt, it replaces the third column with the corresponding line from the plans array.

 

Combine Columns

Suppose you want to combine the first and second columns into the third column separated by a dash.

The awk command for this will be:

awk -F, '{$3=$1 "-" $2; print}' OFS=, customer_data.txt

Output:

101, Taylor,101- Taylor
102, Alex,102- Alex
103, Morgan,103- Morgan

 

Replace Based on a Match

Consider the customer_data.txt file:

101, Taylor, Premium
102, Alex, Basic
103, Morgan, Standard
104, Casey, Basic

Imagine you want to offer a special “Elite” plan to customers whose current plan is “Basic”.

The awk command will be:

awk -F, '$3~/Basic/{ $3="Elite" }; { print }' OFS=, customer_data.txt

Output:

101, Taylor, Premium
102, Alex, Elite
103, Morgan, Standard
104, Casey, Elite

Here, the pattern /$3~/Basic/ checks if the third column contains the word “Basic”.

If it does, it gets replaced with “Elite”.