Shell Scripting

Objective

• Write simple shell scripts and invoke them.
• Work with variables and parameters.
• Understand quoting and escaping.
• Use exit and exit status.
• Use redirection operators.
• Perform conditional tests.
• Execute loops.
• Call subroutines.

Summary

Shell Scripts

• A shell script is a sequence of commands stored in a file for which you have a repeated use.
• The script can easily be modified or customized for a particular application.
• The script is typically executed by entering the name of the script on the command line.
• To create and execute a shell script:
  1. Open a new empty file in your favorite text editor, for example, gnome-text-editor, nano, vim.
  2. Put commands in the new empty file, like you would enter them on the command line.
  3. Give your script a sensible name that gives a hint about what the script does. Script names often end in .sh.
  4. Comment your script code using #. Lines beginning with a # (except for #!) are comments and will not be executed. Comments may also occur following the end of any command providing a whitespace before the #.
  5. Obtain the execute permissions for the correct owners to have the script runnable using chmod u+x script_name.sh.
  6. Execute the script using ./script_name.sh.

Running Scripts

Run the following command from the terminal to execute a simple shell statement:

> echo "Hello world!"
Hello world!

Or you can write it in a shell script called script.sh and run the script from the terminal:

#!/bin/sh
echo "Hello world!"

Now run the script from the terminal:

> chmod u+x script.sh
> ./script.sh
Hello world!
> sh script.sh
Hello world!

Example 1

Here is an example of simple shell script:

#!/bin/sh
clear
 
echo "The script starts now."
 
echo "Hi, $USER!"
echo
 
echo "I will now fetch you a list of connected users:"
echo
w
echo
 
echo "I'm setting two variables now."
COLOR="black"
VALUE="9"
echo "This is a string: $COLOR"
echo "This is a number: $VALUE"
echo
 
echo "I'm giving you back your prompt now. See you!"
echo

Variables

• Variables are how programming and scripting languages represent data.

• Variables appear in arithmetic operations and manipulation of quantities, and in string parsing.

• The name of a variable is a placeholder for its value, the data it holds. Referencing (retrieving) its value is called variable substitution $.

• If variable1 is the name of a variable, then $variable1 is a reference to its value, the data item it contains:

  > variable1=23
  > echo variable1
  variable1
  > echo $variable1
  23

• The only time a variable appears naked—without the $ prefix—is when declared or assigned. Assignment may be with an =, as in var1=27; in a read statement: read var2; and at the head of a loop: for var3 in 1 2 3.

• Enclosing a referenced value in double quotes ("...") does not interfere with variable substitution. This is called partial quoting, sometimes referred to as weak quoting. Using single quotes ('...') causes the variable name to be used literally, and no substitution will take place. This is full quoting, sometimes referred to as strong quoting.

• Note that $variable is actually a simplified form of ${variable}.

Example 2

#!/bin/sh
a=375
hello=$a
 
echo hello # hello
# not a variable reference, just the string "hello" .
 
echo $hello # 375
# this is a variable reference
 
echo ${hello} # 375
# also a variable reference, as above
 
# quoting
echo "$hello" # 375
echo "${hello}" # 375
echo
 
hello="A B    C D"
echo $hello # A B C D
echo "$hello" # A B    C D
# echo $hello and echo "$hello" give different results
# why? because quoting a variable preserves whitespace
 
echo
echo '$hello' # $hello
# variable referencing is disabled (escaped) by single quotes, which
# causes the "$" to be interpreted literally
# notice the effect of different types of quoting
 
hello= # setting it to an empty value

Command Substitution

• The output of commands can be stored and processed using command substitutions.
• var=$(ls) for example executes ls and stores the output in var.
• var=`ls` does the same, but this is the older syntax. The $(...) syntax is always preferred.
• Can be directly used in other commands, for example, echo "$(ls)".

Quoting

• Enclosing a string in quotes. This has the effect of protecting special characters in the string from reinterpretation or expansion by the shell.

• A character is special if it has an interpretation other than its literal meaning.

• Quoting can also suppress echo’s appetite for newlines:

  $ echo $(ls -l)
  total 8 -rw-rw-r-- 1 tux tux 13 Aug 21 12:57 t.sh -rw-rw-r-- 1 tux tux 78 Aug 21 12:57 u.sh
  $ echo "$(ls -l)"
  total 8
  -rw-rw-r-- 1 tux tux 13 Aug 21 12:57 t.sh
  -rw-rw-r-- 1 tux tux 78 Aug 21 12:57 u.sh

• Use double quotes to prevent word splitting. An argument enclosed in double quotes presents itself as a single word, even if it contains whitespace separators.

Example 3

#!/bin/sh
list="one two three"
for a in $list # splits the variable in parts at whitespace
do
  echo $a
done
# one
# two
# three
echo "---"
for a in "$list" # preserves whitespace in a single variable
do
  echo $a
done
# one two three

Exiting

• The exit command terminates a script, just as in a C program. It can also returns a value, which is available to the script’s parent process.
• Every command returns an exit status, sometimes referred to as a return status or exit code.
• A successful command returns a 0, while an unsuccessful one returns a non-zero value that usually can be interpreted as an error code.
• When a script ends with an exit that has no parameter, the exit status of the script is the exit status of the last command executed in the script.
• $? reads the exit status of the last command executed.

Example 4

#!/bin/sh
echo hello
echo $? # exit status 0 returned because the command executed successfully
lskdf # unrecognized command.
echo $? # non-zero exit status returned because the command failed to execute
echo
exit 113 # will this return 113 to the shell?
# to verify this, type "echo $?" after the script terminates
echo $? # this will print 113
echo $? # this will print 0

Special Characters

Redirection

Most commands consume input and produce output/error through the following standard streams:

Most commands allow you to specify input, output, and error channels through redirection of one or more of the standard files.

Standard Output

Use > to redirect stdout, > will overwrite and >> will append:

ls
ls > output_file
ls 1> output_file
cat output_file
ls .. >> output_file
cat output_file

Standard Input

Use < to redirect stdin:

wc < input_files
wc -l < input_files

Standard Error

Use 2> to redirect stderr:

sort file1
sort file1 2> error_log
cat error_log
sort file2 2>> error_log
cat error_log

Use &> to redirect both stdout and stderr, and >&2 to redirect stdout to stderr.

Exercise 1

1. Enter the command to list all current users: who. The output of the command is written to standard output, displayed at your terminal.
2. Send the output to a file called whoson: who > whoson. What are the contents of the whoson file now?
3. Execute: date > whoson. What are the contents of the whoson file now?
4. Append the results of who to the end of whoson: who >> whoson. What are the contents of the whoson file now?

Piping

Connects the standard output of one process to the standard input of another, and prevents using unnecessary temporary files to communicate between processes”

ls /bin > output
more < output
ls /bin | more

Use the pipe operator | to chain two commands together:

ls /bin | more
ls /bin | sort
ls /bin | sort | more
find / | grep iostream | more

Tests

We can test for a condition then act according to the result of the test. The syntaxes for the if / then / else and if / then / elif conditional blocks are:

if [ condition ]
then
  statements
fi

if [ condition ]
then
  statements
else
  statements
fi

if [ first-condition ]
then
  statements
elif [ second-condition ]
then
  statements
else
  statements
fi

true is a built-in command that returns a successful (zero) exit status, while false is a built-in command that returns an successful (one) exit status.

Example 5

#!/bin/sh
true
echo $? # prints 0
false
echo $? # prints 1
 
if true; then
  echo true
elif false; then
  echo false
fi
 
if [ 0 ]; then
  echo 0
fi
if [ 1 ]; then
  echo 1
fi

Comparison Operators

String

• = for equal to: if [ $a = $b ]
• != for not equal to: if [ $a != $b ]
• < for less than, in lexicographic order: if [ $a \< $b ] or if [[ $a < $b ]]
• > for greater than, in lexicographic order: if [ $a \> $b ] or if [[ $a > $b ]]
• -z for string is null, that is, has zero length
• -n for string is not null

Example 6

#!/bin/sh
string='' # zero-length ("null") string variable
if [ -z $string ]
then
    echo "\$string is null."
else
    echo "\$string is not null."
fi # $string is null.

Integer

• -eq for equal to: if [ $a -eq $b ]
• -ne for not equal to: if [ $a -ne $b ]
• -gt for greater than: if [ $a -gt $b ]
• -ge for greater than or equal to: if [ $a -ge $b ]
• -lt for less than: if [ $a -lt $b ]
• -le for less than or equal to: if [ $a -le $b ]
• < for less than: if (($a < $b))
• <= for less than or equal to: if (($a <= $b))
• > for greater than: if (($a > $b))
• >= for greater than or equal to: if (($a >= $b))

Example 7

#!/bin/sh
# printf "%s\n" {0..100} > work.txt
num=$(wc -l < work.txt)
echo $num
if [ $num -gt 150 ]
then
  echo "You have worked hard for today."
  echo
fi

Compound

• ! for logical not: if [ ! exp ]
• -a for logical and: if [ exp1 -a exp2 ]
• -o for logical or: if [ exp1 -o exp2 ]

Example 8

#!/bin/sh
if ! grep $USER /etc/passwd # no square brackets
# if ! grep $1 /etc/passwd; # > /dev/null
then
  echo "Your user account is not managed locally."
fi

• [ is just a command that returns zero as an exit code if the condition is true and non-zero if the condition is false. It is equivalent to the command test.
• [[ is a bash built-in and is not compatible with the POSIX shell. POSIX (Portable Operating System Interface) is a set of standard operating system interfaces based on the Unix operating system.

Arithmetic Operators

• + addition, - subtraction, * multiplication, / division, ** exponentiation, % modulus
• += plus-equals, -= minus-equals, *= times-equals, /= divide-equals, %= mod-equals
• ! not, && and, || or

Example 9

#!/bin/sh
a=4
b=5
 
echo $a $b
 
if [ $a != $b ]
then
  echo "\"$a\" is not equal to \"$b\"."
  echo "(string comparison)"
fi
echo
 
if [ $a -ne $b ]
then
  echo "$a is not equal to $b."
  echo "(integer comparison)"
fi
echo
 
if [ $a -gt 0 ] && [ $a -lt 5 ]
then
  echo "The value of \"a\" lies between 0 and 5."
fi
echo

Exercise 2

Write a shell script to prompt the user for entering a filename in your system, then view the contents of the specified file using the cat command if its number of lines are below 30, otherwise view the file contents using the more command.

Loops

A loop is a block of code that iterates a list of commands as long as the loop control condition is true.

The for loop syntax is:

for arg in [ list ]
do
    statements
done

During each pass through the for loop, arg takes on the value of each successive variable in the list. Having the entire 'list' enclosed in quotes creates a single variable.

Example 10

#!/bin/sh
# listing the planets
for planet in Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune
do
  echo $planet # each planet on a separate line
done
echo; echo
 
# all planets on same line
# entire 'list' enclosed in quotes creates a single variable
for planet in "Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune"
do
  echo $planet
done
echo; echo
 
exit 0

The while loop syntax is:

while [ condition ]
do
    statements
done

This construct tests for a condition at the top of a loop and keeps looping as long as that condition is true (returns a 0 exit status).

Example 11

#!/bin/sh
# count to 10 using a while loop
LIMIT=10
a=1
while [ $a -le $LIMIT ]
do
  echo -n "$a "
  a=$((a + 1))
done
echo; echo

Example 12

#!/bin/sh
echo
while [ "$var1" != "end" ] # the quotes are required to avoid a syntax error
do
  echo -n "Input variable #1 (end to exit): "
  # printf "Input variable #1 (end to exit): "
  read var1
  echo "variable #1 = $var1"
  echo
done
 
exit 0

Exercise 3

Using a for loop, write a script that goes through all files stored in the /etc directory. The loop terminates when the /etc/resolv.conf file is found. Count how many nameservers are defined in that file.

Functions

A function is a set of commands used to perform a specific task within a script. This function can be reused anywhere in your script to perform that particular task several times.

Shell functions are like subroutines, procedures, and functions in other programming languages. The syntax for defining functions is as follows:

function_name() {
  statements
}
 
function_name

Exercise 4

Create a hello-world.sh shell script with the following function definition then run it:

#!/bin/sh
# defining a basic function
func_helloworld() {
  # sequence of statements
  echo "Hello World"
}
 
# invoking the function
func_helloworld

Arguments

You can define a function that will accept parameters while calling the function. These parameters would be represented by $1, $2, and so on. $0 contains the filename of the script that is currently being executed. We supply the arguments after we call a function in a script.

$* and $@ can be used to capture all supplied arguments, and $# to get the number of arguments. $* is equivalent to "$1 $2 $3 ..." and $@ is equivalent to "$1" "$2" "$3" .... $* can be double-quoted to expand to a list of individual tokens rather than one string containing all positional parameters separated by spaces.

Example 13

Following is an example where we supply two arguments Dane and Doe, and then we capture and print them in the function:

#!/bin/sh
# define your function
hello() {
  echo "Hello $1 $2!"
}
 
# invoke your function
hello Dane Doe # Hello Dane Doe!

Escaping

• Escaping is a method of quoting single characters. The escape (\) preceding a character tells the shell to interpret that character literally.

• With certain commands and utilities, such as echo, escaping a character may have the opposite effect as it can toggle on a special meaning for that character.

• Special meanings of certain escaped characters used with echo:

  Character	Meaning	Code
  \a	Alert (BELL)	0x07
  \b	Backspace (BACKSPACE)	0x08
  \t	Tab (CHARACTER TABULATION)	0x09
  \n	Newline (LINE FEED)	0x0A
  \v	Vertical tab (LINE TABULATION)	0x0B
  \r	Return (CARRIAGE RETURN)	0x0C
  \0nn	translates to the octal ASCII equivalent of 0nn, where nn is a string of digits

• \" gives the quote its literal meaning:

  $ echo "Hello"
  Hello
  $ echo "\"Hello\" ... he said."
  "Hello" ... he said.

• \$ gives the dollar sign its literal meaning; a variable name following \$ will not be referenced:

  $ echo "\$variable01"
  $variable01
  $ echo "The book cost \$7.98."
  The book cost $7.98.

Example 14

#!/bin/sh
echo; echo
# escaping a newline
 
echo "This will print
as two lines."
# this will print
# as two lines.
 
echo "This will print \
as one line."
# this will print as one line
 
echo; echo
echo "\v\v\v\v" # prints \v\v\v\v literally
 
# use the -e option with 'echo' to print escaped characters
echo -e "\v\v\v\v" # prints 4 vertical tabs
 
echo -e "\042" # prints " (quote, octal ASCII character 42)
 
# the $'\?' construct makes the -e option unnecessary
echo $'\n' # newline.
echo $'\a' # alert (beep)
 
# in this case, '\nnn' is an octal value
echo $'\t \042 \t' # quote (") framed by tabs
 
quote=$'\042' # " assigned to a variable.
echo "$quote This is a quoted string, $quote and this lies outside the quotes."
echo
echo; echo
 
escape=$'\033' # 033 is octal for escape
echo "\"escape\" echoes as $escape"
# no visible output
echo; echo

Example 15

#!/bin/sh
cmd0() { echo -n "[$@-0]"; return 0; }
cmd1() { echo -n "[$@-1]"; return 1; }
second() { echo "[second]"; }
 
doit() { echo "case: $@"; eval "$@"; echo; }
 
doit 'cmd0 start && cmd0 first && second'
doit 'cmd0 start && cmd0 first || second'
doit 'cmd0 start || cmd0 first && second'
doit 'cmd0 start || cmd0 first || second'
 
doit 'cmd0 start && cmd1 first && second'
doit 'cmd0 start && cmd1 first || second'
doit 'cmd0 start || cmd1 first && second'
doit 'cmd0 start || cmd1 first || second'
 
doit 'cmd1 start && cmd0 first && second'
doit 'cmd1 start && cmd0 first || second'
doit 'cmd1 start || cmd0 first && second'
doit 'cmd1 start || cmd0 first || second'
 
doit 'cmd1 start && cmd1 first && second'
doit 'cmd1 start && cmd1 first || second'
doit 'cmd1 start || cmd1 first && second'
doit 'cmd1 start || cmd1 first || second'

Resources

• Bash Reference Manual (opens in a new tab)
• Bash Guide (opens in a new tab)
• Shell Field Guide (opens in a new tab)
• Shell Style Guide (opens in a new tab)
• Awesome Shell (opens in a new tab)
• BashGuide (opens in a new tab)
• Advanced Bash Scripting Guide (opens in a new tab)