Lab46 Wiki

To become familiar with the basics of shell scripting.

The following websites, in addition to the bash(1) manual page, will likely have some useful information:

• http://freeos.com/guides/lsst/
• http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html
• http://www.linuxconfig.org/Bash_scripting_Tutorial
• http://www.gnu.org/software/bash/manual/bashref.html

As we've explored and familiarized ourselves with the various tools available to us on the command-line, our capabilities have increased to enable solving of increasingly complex and detailed problems. What were once impossible or data-entry intensive tasks have become more doable, feasible, and even time-saving.

With shell scripting, we continue that trend, by increasing our abilities to solve problems by delegating the actual grunt work (processing, calculation, data entry, routine task(s)) to the computer, freeing us up to solve more meaningful tasks.

Much like a play or movie script, a shell script is basically a collection of commands, algorithms, and logic that is, by default, read from start to finish, that unfolds the tale of a particular story.

In our case, we're telling the story of a particular task or problem to solve, and the computer plays it out for us. We are the playwright.

Shell scripts can range from a simple progression of a few commands, to a sophisticated algorithm, to something which automates regular/routine work, to even a fully interactive experience to be used by some end user.

Shell scripts and their abilities provided by their corresponding scripting languages differ from programming languages in that we are operating on an entirely different (and higher) level.

Where languages like machine code and assembly language are among the lowest level languages, and offer the best performance and intrinsic capabilities exposing the greatest details of the hardware, higher level languages abstract away pieces of that detail, taking capabilities away from us, but enabling other solutions that would have been tedious to implement having to take into account so many details.

Programming languages like C and C++ offer some middle ground… C especially is considered a “hybrid” language, having both low and high level features, and although there are still things that machine or assembly language could outperform, C offers more sanity.

Higher level languages like Java and Visual Basic abstract away more details, enabling some solutions to be easily coded, but making others more difficult as it removes the ability for the user to expose themselves to the details of the machine.

Scripting languages are typically among the very top of the high level language scale, removing pretty much all underlying detail considerations, and possessing typically the most “human readable” syntax, enabling some quick and easy solutions to problems.

Again, the sacrifice we make going higher level is an inevitable loss in performance and efficiency for the benefit of having to deal with less detail getting in the way of directly coding a solution.

The lower we go, we gain that performance and efficiency at the cost of more detail, which slows down overall development, but if the solution calls for performance and/or efficiency, a lower level tool is the tool for the job.

A shell script is really as simple as a literal script for the computer to follow. We begin by illustrating a simple script listing files in our home directory, checking available disk space, and getting a list of who is currently on the system:

ls ~
df
who

These commands should go in a regular text file, perhaps named something like script1.sh or script1.

With the above commands in script1.sh, and us saved and exited out of our text editor, it now comes time to run our script:

lab46:~$ ./script1.sh
-bash: ./script1.sh: Permission denied
lab46:~$ 

Well, not quite yet. There's one additional detail we need to take into account, and that is we desire to run our script, but will the system allow us to run it?

As you will see from a successful setting of permissions, the script should now run when invoked, and basically provide the output of the commands therein, in the order listed. The computer just processed those commands for us, without us having to type them in.

The benefits of this become apparent when we can just run the script again, and again, and again. And the computer follows those same set of instructions diligently each time.

Although pretty much any command or capability is fair game for inclusion into a script, we will focus on some common basics that will get you started.

The beauty of shell scripting is that it opens the door to near limitless possibilities. In some cases, there simply cannot be just one solution to a problem. Just as with programming, we now enter into a realm that requires unstructured thought and creativity in order to accomplish structured tasks, and such begins a more effective utilization of computing resources.

The intent for shell scripting is to free us from manually performing tasks. Whenever you find yourself doing a similar task over and over again on the computer, stop and think of ways the task could be automated, potentially freeing you of some or all of it. Our job turns more to managing the computing resources, and the more time you spend scripting, the more you'll be amazed at the problems you're now able to accomplish.

And also remember– ANYTHING you are able to type or concoct on the command-line is able to be a part of a shell script. The command-line therefore makes for an excellent prototyping environment, trying things out to verify their operation before embedding within a script.

Two commands will be among our simplest for input and output:

• echo
• read

The echo command, as we've used in the past, is a simple and effective means of displaying variable contents and strings.

For example:

lab46:~$ echo "Hello, World!"
Hello, World!
lab46:~$ 

We can use echo in many cases when we need to display something.

There are two common uses for echo, the above example and the following:

lab46:~$ echo -n "Hello, World!"
Hello, World!lab46:~$ 

With the addition of the “-n” argument to echo, we disable the automatic newline character at the end of the string. We see its immediate impact in the second example as the string is displayed, then immediately followed by the prompt.

In our scripts, disabling the newline may help enable more stylized output, or helping to display variable data inline amongst some other command incantation.

We will use read to get input from the user. It basically accepts a variable (or variables) to place input into.

For example:

lab46:~$ read name
bob
lab46:~$ echo $name
bob
lab46:~$ 

or…

lab46:~$ read num1 num2
4 7
lab46:~$ echo $num2
7
lab46:~$ echo $num1
4
lab46:~$ 

The read command is a quick and effective way of obtaining input from STDIN.

One of the classic ways to do arithmetic operations in the Bourne shell is with the let keyword.

As there are no data types in shell variables, we must indicate when we wish to use data in a particular way. The let keyword enables this, by treating variable contents as if they are numeric.

To illustrate the use of the let keyword, try the following:

lab46:~$ read num1 num2
6 4
lab46:~$ let num1=$num1+$num2
lab46:~$ echo $num1
10
lab46:~$ 

The let keyword enables the simple arithmetic operations (certainly addition, subtraction, and multiplication), but is rather limited in its abilities. If you plan to do more in-depth calculations, you should use a tool that is designed to effectively calculate numbers (write a program in a lower-level language, use a tool such as bc(1), or use some other facility that has expanded arithmetic capabilities).

But again, we're aiming for an introduction, and an exposure to traditionally classic functionality. bash(1) in particular has made a lot of enhancements that may improve greatly upon the capabilities we are covering, so it would behoove you to familiarize yourself with the functionality of the particular shell you're working with.

Before we go any further, it is important to point out a potential problem we could face, and a solution that was implemented to avoid that problem.

In our first script we wrote above, when we ran it, it ran under our current shell (which by default is bash(1)). There exist many shells, and it is rather easy to switch to a different one. Different shells have different shell syntax, and if you end up using any bash(1) specific code in your script, someone who does not run bash(1) could end up having problems running your script.

To avoid this problem, the shabang facility was created, which launches a particular shell environment to run your script within.

If you write in bash(1), you want to ensure that an instance of bash is run.

The format of the shabang is as follows:

#!/path/to/shell -options

In the case of bash, we'd want to do the following, to have as the very first line in our scripts:

#!/bin/bash

This (the SHH character followed by the BANG character), will launch the specified command, then all script contents will be run within the environment of that command.

Often, a common evaluation to make in processing a task is to compare two (or more) values, and to execute a portion of code conditionally dependent upon the result of the comparison.

Whenever we use the words “if”, “then”, and/or “else”, we are making a choice (or selection)… something happens as the result of that condition.

Pretty much every programming and scripting language implements the venerable if statement, and most notably so, as it forms the basis for much of your script's ability to solve tasks.

To use an if statement, we first need a condition to evaluate.

We'll start off by having the following script logic, in a file called script2.sh:

#!/bin/bash
 
echo -n "Pick a number: "
read num1
 
echo -n "Pick another number: "
read num2

In the code so far, we've prompted the user for two separate numbers. We've stored those responses in the variables $num1 and $num2.

Now, what we'd like to do is perform a comparison to see which of the two numbers if higher. For starters, we'll compare to see if $num1 is less than $num2, so append the following to script2.sh:

if [ "$num1" -lt "$num2" ]; then
    echo "$num2 is greater than $num1"
fi

Here we have among the simplest of if statements. It is actually a symbiotic relationship with the [(1) utility (yes, the square brackets are commands on the system– look up their manual page).

In the if, we evaluate “$num1” and “$num2”, seeing if the left parameter is less than the parameter on the right. If the result of this comparison is true, we run the code within the if body, that is, the echo line.

When our if statement, and all its supporting code has concluded, we must terminate the if code block with the keyword fi (just the word “if” spelled backwards). This forms the syntax for our simple singular if statement.

We can of course be more capable. We will now expand our if statement to include an else if, or secondary if, within the same if block. Replace the above if block with the following in your script:

if [ "$num1" -lt "$num2" ]; then
    echo "$num2 is greater than $num1"
elif [ "$num2" -lt "$num1" ]; then
    echo "$num1 is greater than $num2"
fi

As you can see, else if statements in bash are elif, and we follow it up with another comparison, this time comparing to see if $num2 is less than $num1. We then print out a corresponding statement if that condition is true.

Finally, we should address the third and final condition that can take place in our current example.. and that is if both numbers are equal. If you look at the logic of our two conditions, you see we are comparing less than conditions ONLY… if both are equal, neither of the first two if conditions will be true.

So, we will follow this up with a catch-all statement, the else. So replace the if block with the following in your script:

if [ "$num1" -lt "$num2" ]; then
    echo "$num2 is greater than $num1"
elif [ "$num2" -lt "$num1" ]; then
    echo "$num1 is greater than $num2"
else
    echo "$num1 and $num2 are equal in value"
fi

A couple things to keep in mind regarding if statements:

• within an if block, you can have multiple elif statements. You're not just limited to one.
• within an if block, you can ONLY have one if, and one else
• within an if block, there is only one terminating fi.
• there NEED to be spaces surrounding the [ and ] commands from conditional logic.
• there NEED to be spaces between each element of the condition.
• the double quotes aren't required, but are a good means of preventing more serious errors if either variable is NULL.
• the semi-colon is a command separator. It actually has nothing to do with the if block (merely allows us to have the then on the same line as the if; some prefer it on the next line.

So feel free to utilize if blocks in your scripts, to enable whatever comparisons necessary.

Obviously, we aren't just limited to the less than comparison. There are a number of comparisons at our disposal. A list of some common ones (check manual pages for a more complete list):

To finish our introductory journey, we'll take a look at iteration structures, which allow us to repeat a task some number of times.

In programming languages, these typically take the form of for and while loops, and sure enough, both are present to us in shell scripting. In addition, bash has a number of additional iteration structures that could be useful in certain situations. The bash manual page would be a good reference for this information.

For this lab, we will only be taking a look at the for iteration structure.

In bash, there are two forms of for loops, the numeric for loop, and the list-based for loop. We will take a look at both.

First up, the numeric for loop.

This loop, for those who have experience programming in other languages, will recognize similarity to existing knowledge. The numeric for loop has 3 values:

1. the starting condition
2. the looping condition (ie it will keep looping as long as that condition is true)
3. the stepping value

We will start with a simple script to display the numbers 1-10 to the screen, one per line:

#!/bin/bash
 
for((i=1; i<=10; i++)); do
    echo "$i"
done

The i++ is shorthand for the equation: i=i+1. Either could be used in that field.

If you'd like to alter the stepping value, put in the appropriate equation.

In addition to the numeric for loop, we have the list-based for loop. This is still a loop, it still iterates a number of times, but it does so not on a condition to evaluate, but instead a list of provided values.

Check out the following script:

#!/bin/bash
 
echo -n "First color is: "
for color in red orange yellow green blue indigo violet; do
    echo "$color"
    echo -n "Next color is: "
done

Go ahead and run this script, analyze the output, and make sure everything in it makes sense. Be sure to ask questions.

With your scripting knowledge, please write for me the following scripts, and answer any pertinent questions:

Sample output for the above script could resemble:

lab46:~$ ./myscript
Please enter directory to process: /some/dir
Processing ...

Directory: /some/dir
Total Files: 42

(a-g):***************
(h-m):********
(n-s):*****************
(t-z):**

lab46:~$ 

Aside from the requirement to use asterisks, you are free to format the output of your script as you see fit. The requested information needs to be clearly visible.

This assignment has activities which you should tend to- document/summarize knowledge learned on your Opus.

As always, the class mailing list and class IRC channel are available for assistance, but not answers.