Linux

Shell Scripting Part4 – Input, Output, and Redirection

In the previous post, we talked about the parameters and options in detail. Today, we will talk about something essential in shell scripting, which are Input, Output, and Redirection.

You can display the output from your shell scripts in two ways:

  • Display output on the screen.
  • Send output to a file.

 

 

Standard file descriptors

Everything is a file in Linux, and that includes input and output.

Each process can have nine file descriptors opened at the same time. The file descriptors 0, 1, 2 are kept for the bash shell usage.

0              STDIN.

1              STDOUT.

2              STDERR.

You can use the above file descriptors to control input and output.

You need to fully understand these three because they are like the backbones of your shell scripting. So we are going to describe every one of them in detail.

 

STDIN

STDIN stands for standard input, which is the keyboard by default.

You can replace the STDIN, which is the keyboard and replace it with a file by using the input redirect symbol (<), it sends the data as keyboard typing. No magic!!

When you type the cat command without anything, it accepts input from STDIN. Any line you type, the cat command prints that line to the screen.

 

STDOUT

The STDOUT stands for the standard output, which is the screen by default.

You can redirect output to a file using the >> symbol.

If we have a file contains data, you can append data to it using this symbol like this:

pwd >> myfile

The output generated by pwd is appended to myfile without deleting the existed content.

Linux shell scripting Input, Output, and Redirection

The following command tries to redirect the output to a file using > symbol.

ls –l xfile > myfile

redirect error

I have no file called xfile on my PC, and that generates an error that is sent to STDERR.

 

STDERR

The shell sends errors to the screen by default.

If you need to redirect the errors to a log file instead of sending it to the screen, you can redirect errors using the redirection symbol.

 

Redirecting errors

We can redirect the errors by placing the file descriptor which is 2 before the redirection symbol like this:

ls -l xfile 2>myfile
cat ./myfile

redirect error to file

As you can see, the error now is in the file and nothing on the screen.

 

Redirecting errors and normal output

To redirect errors and the normal output, you have to precede each with the proper file descriptor like this:

ls –l myfile xfile anotherfile 2> errorcontent 1> correctcontent

redirect error and data

The ls command result is sent to the correctcontent file using the 1> symbol. And error messages were sent to the errorcontent file using the 2> symbol.

You can redirect normal output and errors to the same file using &> symbol like this:

ls –l myfile xfile anotherfile &> content

redirect-all to file

here, we redirect the errors and normal output to a file named content.

 

Output redirection

There are two ways for output redirection:

  • Temporarily redirection.
  • Permanently redirection.

Temporary redirections

For temporary redirections, you can use the >&2 symbol like this:

#!/bin/bash

echo "Error message" >&2

echo "Normal message"

temp redirection

So if we run it, we will see both lines printed normally because, as we know, errors go to the screen by default.

You can redirect errors to a file like this:

./myscript 2> myfile

redirect error to file

Shell scripting is Awesome! Normal output is sent to the screen, while the echo message which has >&2 symbol sends errors to the file.

Permanent redirections

If you have much data that need to be redirected, you can have a permanent redirection using the exec command like this:

#!/bin/bash

exec 1>outfile

echo "Permanent redirection"

echo "from a shell to a file."

echo "without redirecting every line"

redirect all to file

If we look at the file called outfile, we will see the output of the echo lines.

We redirect the STDOUT at the beginning, what about in the middle of a script like this:

#!/bin/bash

exec 2>myerror

echo "Script Begining ..."

echo "Redirecting Output"

exec 1>myfile

echo "Output goes to the myfile"

echo "Output goes to myerror file" >&2

permanent redirection

The exec command redirects all errors to the file myerror, and we send the standard output to the screen.

We use the statement exec 1>myfile to redirect output to the myfile file, and finally, errors go to the myerror file using >&2 symbol.

 

Redirecting input

You can redirect input to a file instead of STDIN using exec command like this:

exec 0< myfile

This command tells the shell to take the input from a file called myfile instead of STDIN and here is an example:

#!/bin/bash

exec 0<testfile

total=1

while read line; do

	echo "#$total: $line"

	total=$(($total + 1))

done

redirect input

Shell scripting is easy.

You know how to use the read command to get user input. If you redirect the STDIN to a file, the read command will try to read from STDIN, which points to the file.

Some Linux system administrators use this technique to read the log files for processing, and we will discuss more ways to read the log on the upcoming posts in a professional way.

 

Creating custom redirection

You know that there are nine file descriptors, you use only 3 of them for input, output, and error.

The remaining six file descriptors are available for use for input and output redirection.

We can use the exec command to assign a file descriptor for output like this:

#!/bin/bash

exec 3>myfile

echo "This line appears on the screen"

echo "This line stored on myfile" >&3

echo "This line appears on the screen"

create redirection

 

Creating input file descriptors

To redirect input file descriptors do the following:

1-  Save the STDIN to another file descriptor.

2- Redirecting it to a file.

3- Revert STDIN to its original location.

Look at the following code to understand these steps:

#!/bin/bash

exec 7<&0

exec 0<myfile

total=1

while read line; do

	echo "#$total: $line"

	total=$(($total + 1))

done

exec 0<&7

read -p "Finished? " res

case $res in

y) echo "Goodbye" ;;

n) echo "Sorry, this is the end." ;;

esac

create input file descriptor

We saved the STDIN to file descriptor 7, and redirect the STDIN to a file.

The STDIN reverted to its original location after iterating over file lines.

The last read command just to make sure that STDIN is reverted, and you can use the keyboard normally.

 

Close file descriptors

When the script exits, the file descriptors will close automatically. If you want to close the file descriptor yourself, redirect the file descriptor to this symbol &- it will be closed.

#!/bin/bash

exec 3> myfile

echo "Testing ..." >&3

exec 3>&-

echo "Nothing works" >&3

closing file descriptor

As you can see, it gives an error of bad file descriptor because it is no longer exist.

 

lsof Command

The lsof command is used to list all the opened files on the system and background processes.

On many Linux systems like Fedora, the lsof command is located under /usr/sbin.

This is some of the important options for lsof command:

-p: for process ID.

-d: for the file descriptor.

You can get the process PID using the $$ variable.

list opened descriptors

We can use the -a to combine the results of -p option and -d option.

Now, testing the command from a script:

#!/bin/bash

exec 4> myfile1

exec 5> myfile2

exec 6< myfile3

lsof -a -p $$ -d 0,1,2,4,5,6

list custom descriptors

The shell script creates the file descriptors 4 and 5 for writing and 6 for reading.

 

Suppressing command output

Sometimes you don’t want to see any output. We redirect the output to the black hole, which is /dev/null.

For example, we can suppress errors like this:

ls -al badfile anotherfile 2> /dev/null

You can truncate a file without deleting it completely using the same command.

cat /dev/null > myfile

Now you understand the input, output, how to redirect them, how to create your file descriptor, and redirect to it.

I hope you enjoy it. Keep coming back.

Thank you.

Mokhtar Ebrahim
Founder of LikeGeeks. I'm working as a Linux system administrator since 2010. I'm responsible for maintaining, securing, and troubleshooting Linux servers for multiple clients around the world. I love writing shell and Python scripts to automate my work.

2 thoughts on “Shell Scripting Part4 – Input, Output, and Redirection

  1. I didn’t understand the example of creating file descriptors.
    Why did we “exec 0<myfile"? Not "exec 7 myfile”? Shouldn’t it be “cat myfile > /dev/null”?

    1. The demonstrated example shows how to save the STDIN to a file descriptor and redirect STDIN to a file temporarily for testing purpose.
      Then we revert STDIN back.

Leave a Reply

Your email address will not be published. Required fields are marked *