Bailee Egan - Bash Scripts for Beginners

If you're someone like me who learned coding in higher-level languages, like Python, R, and Java, bash can be quite strange. Luckily for you, bash has the same structures as other languages, albeit with different syntax. Note that this tutorial assumes you're familiar with basic bash commands for file manipulation.

This tutorial will cover the following:

Bash script basics: Creating bash scripts, Variables, Backticks, User input, Arguments
Control structures: For loops, If/else statements, While loops
Intermediate: Reading a file line by line, Flags and options

Creating Bash Scripts

Bash scripts are simply a list of bash commands that the computer executes line by line, much like you use commands line by line. Create a bash script like any file.

nano myscript.sh

Let's add these commands

# Inside myscript.sh
echo Hello world!
touch newfile.txt
echo Hello new file >> newfile.txt

We can run bash scripts like so:

bash myscript.sh
# Hello world!
# Creates newfile.txt with 'Hello new file' inside

It's as simple as that! But we can make our scripts more complicated!

Variables

Variables hold information, like numbers and strings (which include file names and paths). Think of them as placeholders.

You can set values to variables as in the following. Note that variable names are case-sensitive

a=10
A=5
# Note: a does not equal A

file=/path/to/file.txt
path=/another/path
newFile=test.txt

You must ensure that there are no spaces between the variable name, the equals sign, and the value

# Good: a=10
# Bad: a = 10

You use variables in scripts by adding a dollar sign ($) at the front of the variable name

echo $a
# 10
echo $file
# /path/to/file.txt

Combine variables with strings

touch "$path"/created_file.txt
# Creates a new file called created_file.txt at /another/path

Or combine variables

mv $file "$path"/"$newFile"
# Saves /path/to/file.txt at /another/path/test.txt

In general, it is good practice to place quotes around your variable names to prevent them from blending into your strings, e.g. $varName vs "$var"Name may have different outputs.

Backticks (`)

Commands within backticks will be run before the main command For example, you know that wc -l file will output the number of lines within a given file, but if you are writing a script, you might not want the output to simply be a number. You can do the following:

echo The number of lines is `wc -l file.txt` in file.txt
# The number of lines is 1303 in file.txt

Backticks also work for declaring variables

count=`grep -c "Apples" fruitList.txt`
echo The number of apples is $count
# The number of apples is 107

User input

Getting user input is actually quite easy. Use the read command in front of the desired variable name, and you can use the input like a variable.

echo What is your name?
read name
echo Hello "$name"!

# What is your name
# $ Bailee
# Hello Bailee!

Note that this is single line only. Pressing enter will submit the input.

Arguments

You know how commands like ls can take additional arguments such as ls newDir to list the files within newDir? You can make your script to accept arguments as well. Arguments are denoted as $1 for the first parameter, $2 for the second, and so on. You use them like variables. Let's say you want to make a script that counts the number of items in a given directory

# counter.sh
numItems=`ls $1 | wc -l`
echo The number of items in $1 is $numItems

You can run it using:

bash counter.sh newDir
# The number of items in newDir is 125

Note that flags (like ls -l -h) will be covered below.

For Loops

We use for loops to perform the same set of commands for a particular list of things (such as a sequence of numbers or a list of files).

Let's loop over this list of numbers. We generate this list using seq

for x in `seq 1 5`
do
    echo My number is $x
done
# My number is 1
# My number is 2
# My number is 3
# My number is 4
# My number is 5

You can use a list of strings separated by spaces such as egg milk flour. If you have a string with spaces already, you must surround the string with quotes, such as "brown egg" milk "wheat flour"

for str in `"brown egg" milk "wheat flour"`
do
    echo Don't forget to add the $str
done
# Don't forget to add the brown egg
# Don't forget to add the milk
# Don't forget to add the wheat flour

Output from commands, such as ls, can be used in loops. Here you can output the first line within each file of a given directory

dir=$1
for file in `ls $dir`
do
    echo File: $file
    head -n 1
done
# File: dog_names.txt
# Spot
# File: cat_names.txt
# Whiskers
# File: fish_names.txt
# Goldie

If/Else Statements

At this moment, you're scripts are linear, meaning that A -> B -> C -> D. With if/else statements, you can create more complex scripts where A -> B and depending on B, you can go to C or D. Here is a simple if-else statement.

echo What is your name?
read name

if [[ $name == "Tom" ]]; then
    echo Hello, you must be Tom
else
    echo I don't know you
fi

# What is your name?
# > Tom
# Hello, you must be Tom

Like declaring variables, spacing matters. You must have a space between the condition and the surrounding brackets, or you will get an error.

You can also omit the else statement if you don't need a command for that case.

For multiple if conditions within the same stack, use elif (i.e. else-if) following the initial if. The else statement can be omitted here as well, but if it is included, it must be at the end.

if [[ $name == "Tom" ]]; then
    echo Hello, you must be Tom
elif [[ $name == "Brad" ]]; then
    echo Brad, please, come in
else
    echo I don't know you
fi

Here are some examples of condition statements that would be useful. There are plenty more if you search for them on the internet.

if [[ $name == "Tom" ]]; then   # If $name equals Tom
if [[ $name != "Tom" ]]; then   # If $name does not equal Tom
if [[ -d raw_files ]]; then # If directory raw_files exists and is a directory
if [[ -f testFile.txt ]]; then  # If testFile.txt exists and is a file
if [[ ! -f testFile.txt ]]; then    # If testFile.txt does NOT exist OR testFile.txt is NOT a file
if [[ $x -ge 10 ]]; then    # If $x is Greater or Equal than 10
if [[ $n -lt 15 ]]; then    # If $n is Less Than than 15

Combine multiple conditions into one if-statement using && (AND) and || (OR).

if [[ $name == "Tom" || $name == "Brad" ]]; then    # If $name equals Tom OR $name equals Brad
if [[ $x -gt 10 && $x -lt 20 ]]; then   # If $x is greater than 10 AND less than 20

Finally, you can put if-else statements with for loops. Let's say you ran an analysis on a batch different data files. Each analysis generates a separate results folder and a log inside that folder. You might want to read the logs of each analysis without having to go into each folder manually. You also want to note if the log file doesn't exist, suggesting that the analysis might have an error.

for dir in `ls Results`
do      
    echo $dir            
    if [[ -f Results/$dir/log.txt ]]; then
        cat Results/$dir/log.txt
    else
        cat ERROR: NO LOG FOUND
    fi
done | less

You can note the pipe after the for-loop statement. Like head and sort, output from loops can be shunted to other commands. Output from other commands can also be piped into loops.

While-loops

While-loops are similar to for-loops in that they repeat a set of commands. However, for-loops are finite. On the other hand, while-loops will repeat commands until their condition is met (while <condition is true>; do something; done). Think of them as if-statements that repeat forever until the condition is not true.

secret=''
while [[ $secret != "there is no cost without sacrifice" ]]
do
    echo "What is the password?"
    read secret
done
echo "Welcome!"

Here, you are asked for a password. If you put in the wrong password, you will be asked again until it is correct. Then you will receive a warm welcome. Note that this is not a good way to code passwords because a user can simply view the file for the right phrase.

Reading a file line by line

While loops can be used to read files line by line. To read a file, you must pipe the output of a command such as cat into the while loop. The format of the while loop should now be while read varName, shown below.

# file.txt
TO DO LIST:
1) Eat
2) Clean
3) Walk the dog

cat file.txt | while read line; do
    echo $line
done
# TO DO LIST:
# 1) Eat
# 2) Clean
# 3) Walk the dog

Flags and options

Adding flags and options can make your script not only more complex but also more user friendly, given that users don't need to input arguments in an exact order.

report=0
file=''

while getopts 'rf:' flag
do
    case "$flag" in
        r)  report=1 ;;
        f)  file="$OPTARG" ;;
        *)  echo "Unknown option. Exiting"
            exit ;;
    esac

    if [[ $report == "1" ]]; then
        echo "Deleting" $file >> report.log
    fi
    rm $file
done

Let's go through this in chunks. The following below are your variables. They don't necessarily need to match your flags, but they are useful for holding information if a flag is used or if a flag has an argument

report=0
file=''

This is another type of while loop. Here, flag will be the variable, and you're going to use getopts to loop through abf:, setting $flag as each flag. Note that options followed by a colon (:) take their own arguments. Here the -f flag will take a file argument while -a and -b don't need arguments

while getopts 'rf:' flag

This is a case statement. Think of it as a large if-else stack, where we check if $flag matches any of the strings denoted as <string>). Each case must end with double semicolons (;;). Use $OPTARG to get the argument of a flag. Finally *) denotes the commands for erroneous options. If there is an error, we exit. Otherwise, the rest of the script is run normally.

case "$flag" in
    r)  report=1 ;;
    f)  file="$OPTARG" ;;
    *)  echo "Unknown option. Exiting"
        exit ;;
esac

if [[ $report == "1" ]]; then
    echo "Deleting" $file >> report.log
fi
rm $file

Conclusions

This tutorial should hopefully set you up to writing more complex bash scripts. I know this isn't everything, but thanks for reading, and let me know if there are any typos, errors, or things can be added.