Lab46 Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
<WRAP centeralign round box>
<WRAP><color red><fs 200%>Corning Community College</fs></color></WRAP>
<WRAP><fs 150%>CSCS1730 UNIX/Linux Fundamentals</fs></WRAP>
<fs 125%>Lab 0x6: Shell Scripting Concepts</fs>
</WRAP>

~~TOC~~
=====Objective=====
To become familiar with the basics of shell scripting.
=====Reference=====
The following websites, in addition to the **bash**(**1**) manual page, will likely have some useful information:

  * http://freeos.com/guides/lsst/
  * http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html
  * http://www.linuxconfig.org/Bash_scripting_Tutorial
  * http://www.gnu.org/software/bash/manual/bashref.html

=====Background=====
As we've explored and familiarized ourselves with the various tools available to us on the command-line, our capabilities have increased to enable solving of increasingly complex and detailed problems. What were once impossible or data-entry intensive tasks have become more doable, feasible, and even time-saving.

With shell scripting, we continue that trend, by increasing our abilities to solve problems by delegating the actual grunt work (processing, calculation, data entry, routine task(s)) to the computer, freeing us up to solve more meaningful tasks.

Much like a play or movie script, a shell script is basically a collection of commands, algorithms, and logic that is, by default, read from start to finish, that unfolds the tale of a particular story.

In our case, we're telling the story of a particular task or problem to solve, and the computer plays it out for us. We are the playwright.

Shell scripts can range from a simple progression of a few commands, to a sophisticated algorithm, to something which automates regular/routine work, to even a fully interactive experience to be used by some end user.

Shell scripts and their abilities provided by their corresponding scripting languages differ from programming languages in that we are operating on an entirely different (and higher) level.

Where languages like machine code and assembly language are among the lowest level languages, and offer the best performance and intrinsic capabilities exposing the greatest details of the hardware, higher level languages abstract away pieces of that detail, taking capabilities away from us, but enabling other solutions that would have been tedious to implement having to take into account so many details.

Programming languages like C and C++ offer some middle ground... C especially is considered a "hybrid" language, having both low and high level features, and although there are still things that machine or assembly language could outperform, C offers more sanity.

Higher level languages like Java and Visual Basic abstract away more details, enabling some solutions to be easily coded, but making others more difficult as it removes the ability for the user to expose themselves to the details of the machine.

Scripting languages are typically among the very top of the high level language scale, removing pretty much all underlying detail considerations, and possessing typically the most "human readable" syntax, enabling some quick and easy solutions to problems.

Again, the sacrifice we make going higher level is an inevitable loss in performance and efficiency for the benefit of having to deal with less detail getting in the way of directly coding a solution.

The lower we go, we gain that performance and efficiency at the cost of more detail, which slows down overall development, but if the solution calls for performance and/or efficiency, a lower level tool is the tool for the job.

=====A simple script=====
A shell script is really as simple as a literal script for the computer to follow. We begin by illustrating a simple script listing files in our home directory, checking available disk space, and getting a list of who is currently on the system:

<code bash>
ls ~
df
who
</code>

These commands should go in a regular text file, perhaps named something like **script1.sh** or **script1**.

<WRAP round info box>Although UNIX/Linux doesn't require file extensions, the traditional convention is to suffix a shell script with the extension ".sh"

This is done more for end user recognition of the presence of a shell script. If desired, an extension need not be given, and it would not affect the operation of the script at all.
</WRAP>

====Running our script====
With the above commands in **script1.sh**, and us saved and exited out of our text editor, it now comes time to run our script:

<cli>
lab46:~$ ./script1.sh
-bash: ./script1.sh: Permission denied
lab46:~$ 
</cli>

Well, not quite yet. There's one additional detail we need to take into account, and that is we desire to run our script, but will the system allow us to run it?

^  1.  ^|Thinking about how we'd make our script runnable:|
| ^  a.|View the permissions of **script1.sh**. How did you do it?|
|:::^  b.|What are the permissions?|
|:::^  c.|How could you change the permissions to allow the script to be runnable?|
|:::^  d.|What did you type?|
|:::^  e.|Test it by attempting to run the script again. Did it work?|

As you will see from a successful setting of permissions, the script should now run when invoked, and basically provide the output of the commands therein, in the order listed. The computer just processed those commands for us, without us having to type them in.

The benefits of this become apparent when we can just run the script again, and again, and again. And the computer follows those same set of instructions diligently each time.

=====Scripting Philosophy=====
Although pretty much any command or capability is fair game for inclusion into a script, we will focus on some common basics that will get you started.

The beauty of shell scripting is that it opens the door to near limitless possibilities. In some cases, there simply cannot be just one solution to a problem. Just as with programming, we now enter into a realm that requires unstructured thought and creativity in order to accomplish structured tasks, and such begins a more effective utilization of computing resources.

The intent for shell scripting is to free **us** from manually performing tasks. Whenever you find yourself doing a similar task over and over again on the computer, stop and think of ways the task could be automated, potentially freeing you of some or all of it. Our job turns more to managing the computing resources, and the more time you spend scripting, the more you'll be amazed at the problems you're now able to accomplish.

And also remember-- **ANYTHING** you are able to type or concoct on the command-line is able to be a part of a shell script. The command-line therefore makes for an excellent prototyping environment, trying things out to verify their operation before embedding within a script.

=====Simple I/O=====
Two commands will be among our simplest for input and output:

  * echo
  * read

====echo====
The **echo** command, as we've used in the past, is a simple and effective means of displaying variable contents and strings.

For example:

<cli>
lab46:~$ echo "Hello, World!"
Hello, World!
lab46:~$ 
</cli>

We can use **echo** in many cases when we need to display something.

There are two common uses for echo, the above example and the following:

<cli>
lab46:~$ echo -n "Hello, World!"
Hello, World!lab46:~$ 
</cli>

With the addition of the "**-n**" argument to **echo**, we disable the automatic newline character at the end of the string. We see its immediate impact in the second example as the string is displayed, then immediately followed by the prompt.

In our scripts, disabling the newline may help enable more stylized output, or helping to display variable data inline amongst some other command incantation.
====read====
We will use **read** to get input from the user. It basically accepts a variable (or variables) to place input into.

For example:

<cli>
lab46:~$ read name
bob
lab46:~$ echo $name
bob
lab46:~$ 
</cli>

or...

<cli>
lab46:~$ read num1 num2
4 7
lab46:~$ echo $num2
7
lab46:~$ echo $num1
4
lab46:~$ 
</cli>

The **read** command is a quick and effective way of obtaining input from STDIN.
=====arithmetic=====
One of the classic ways to do arithmetic operations in the **Bourne** shell is with the **let** keyword.

As there are no data types in shell variables, we must indicate when we wish to use data in a particular way. The **let** keyword enables this, by treating variable contents as if they are numeric.

<WRAP round info box>Care should be taken to ensure that data utilized in a **let** statement IS actually numeric, otherwise errors could take place.</WRAP>

To illustrate the use of the **let** keyword, try the following:

<cli>
lab46:~$ read num1 num2
6 4
lab46:~$ let num1=$num1+$num2
lab46:~$ echo $num1
10
lab46:~$ 
</cli>

The **let** keyword enables the simple arithmetic operations (certainly **addition**, **subtraction**, and **multiplication**), but is rather limited in its abilities. If you plan to do more in-depth calculations, you should use a tool that is designed to effectively calculate numbers (write a program in a lower-level language, use a tool such as **bc**(**1**), or use some other facility that has expanded arithmetic capabilities).

But again, we're aiming for an introduction, and an exposure to traditionally classic functionality. **bash**(**1**) in particular has made a lot of enhancements that may improve greatly upon the capabilities we are covering, so it would behoove you to familiarize yourself with the functionality of the particular shell you're working with.

^  2.  ^|To play with the **let** statement, create **age.sh** and have it do the following:|
| ^  a.|Prompt the user for their birth year.|
|:::^  b.|Store the current year, as determined by **date**(**1**), in a variable.|
|:::^  c.|How did you store the current year, as output by **date**(**1**), in a variable?|
|:::^  d.|In your script, subtract the birth year variable from the current year variable. Display the result.|
|:::^  e.|How did you perform the subtraction?|
|:::^  f.|Provide your script.|

=====shabang=====
Before we go any further, it is important to point out a potential problem we could face, and a solution that was implemented to avoid that problem.

In our first script we wrote above, when we ran it, it ran under our current shell (which by default is **bash**(**1**)). There exist many shells, and it is rather easy to switch to a different one. Different shells have different shell syntax, and if you end up using any **bash**(**1**) specific code in your script, someone who does not run **bash**(**1**) could end up having problems running your script.

To avoid this problem, the **shabang** facility was created, which launches a particular shell environment to run your script within.

If you write in **bash**(**1**), you want to ensure that an instance of **bash** is run.

The format of the **shabang** is as follows:

<code bash>
#!/path/to/shell -options
</code>

In the case of **bash**, we'd want to do the following, to have as the **very first line** in our scripts:

<code bash>
#!/bin/bash
</code>

This (the SHH character followed by the BANG character), will launch the specified command, then all script contents will be run within the environment of that command.

=====selection=====
Often, a common evaluation to make in processing a task is to compare two (or more) values, and to execute a portion of code conditionally dependent upon the result of the comparison.

Whenever we use the words "if", "then", and/or "else", we are making a choice (or selection)... something happens as the result of that condition.

Pretty much every programming and scripting language implements the venerable **if** statement, and most notably so, as it forms the basis for much of your script's ability to solve tasks.
====if====
To use an **if** statement, we first need a condition to evaluate.

We'll start off by having the following script logic, in a file called **script2.sh**:

<code bash>
#!/bin/bash

echo -n "Pick a number: "
read num1

echo -n "Pick another number: "
read num2
</code>

In the code so far, we've prompted the user for two separate numbers. We've stored those responses in the variables **$num1** and **$num2**.

Now, what we'd like to do is perform a comparison to see which of the two numbers if higher. For starters, we'll compare to see if **$num1** is less than **$num2**, so append the following to **script2.sh**:

<code bash>
if [ "$num1" -lt "$num2" ]; then
    echo "$num2 is greater than $num1"
fi
</code>

Here we have among the simplest of **if** statements. It is actually a symbiotic relationship with the **<nowiki>[</nowiki>**(**1**) utility (yes, the square brackets are commands on the system-- look up their manual page).

In the **if**, we evaluate "$num1" and "$num2", seeing if the left parameter is //less than// the parameter on the right. If the result of this comparison is true, we run the code within the **if** body, that is, the **echo** line.

When our **if** statement, and all its supporting code has concluded, we must terminate the **if** code block with the keyword **fi** (just the word "if" spelled backwards). This forms the syntax for our simple singular **if** statement.

We can of course be more capable. We will now expand our **if** statement to include an //else if//, or secondary **if**, within the same **if** block. Replace the above **if** block with the following in your script:

<code bash>
if [ "$num1" -lt "$num2" ]; then
    echo "$num2 is greater than $num1"
elif [ "$num2" -lt "$num1" ]; then
    echo "$num1 is greater than $num2"
fi
</code>

As you can see, //else if// statements in **bash** are **elif**, and we follow it up with another comparison, this time comparing to see if **$num2** is less than **$num1**. We then print out a corresponding statement if //that// condition is true.

Finally, we should address the third and final condition that can take place in our current example.. and that is if both numbers are equal. If you look at the logic of our two conditions, you see we are comparing less than conditions ONLY... if both are equal, neither of the first two if conditions will be true.

So, we will follow this up with a catch-all statement, the //else//. So replace the **if** block with the following in your script:

<code bash>
if [ "$num1" -lt "$num2" ]; then
    echo "$num2 is greater than $num1"
elif [ "$num2" -lt "$num1" ]; then
    echo "$num1 is greater than $num2"
else
    echo "$num1 and $num2 are equal in value"
fi
</code>

A couple things to keep in mind regarding **if** statements:

  * within an **if** block, you can have multiple **elif** statements. You're not just limited to one.
  * within an **if** block, you can ONLY have one **if**, and one **else**
  * within an **if** block, there is only one terminating **fi**.
  * there NEED to be spaces surrounding the <nowiki>[</nowiki> and <nowiki>]</nowiki> commands from conditional logic.
  * there NEED to be spaces between each element of the condition.
  * the double quotes aren't required, but are a good means of preventing more serious errors if either variable is NULL.
  * the semi-colon is a command separator. It actually has nothing to do with the **if** block (merely allows us to have the **then** on the same line as the **if**; some prefer it on the next line.

So feel free to utilize **if** blocks in your scripts, to enable whatever comparisons necessary.

Obviously, we aren't just limited to the //less than// comparison. There are a number of comparisons at our disposal. A list of some common ones (check manual pages for a more complete list):

^  Symbol  ^  Description  |
|  -lt  |  is less than  |
|  -gt  |  is greater than  |
|  -eq  |  is equal to  |
|  -le  |  is less than or equal to  |
|  -ge  |  is greater than or equal to  |
|  -ne  |  is not equal to  |

^  3.  ^|To play with the **if** statement, create **guess1.sh** and have it do the following:|
| ^  a.|After the shabang line, put the following line: **<nowiki>pick=$(($RANDOM % 20))</nowiki>**|
|:::^  b.|Prompt the user to guess a number, between 1 and 20. Store the response in a variable.|
|:::^  c.|With only 4 guesses total, compare the user's guess against the value in **$pick**.|
|:::^  d.|If the user guesses correctly, display a message indicating so. If not, tell them.|
|:::^  e.|What do you suppose the **<nowiki>pick=$(($RANDOM % 20))</nowiki>** line does?|
|:::^  f.|Provide your script.|

=====iteration=====
To finish our introductory journey, we'll take a look at iteration structures, which allow us to repeat a task some number of times.

In programming languages, these typically take the form of **for** and **while** loops, and sure enough, both are present to us in shell scripting. In addition, **bash** has a number of additional iteration structures that could be useful in certain situations. The **bash** manual page would be a good reference for this information.

For this lab, we will only be taking a look at the **for** iteration structure.

====numeric for loop====
In **bash**, there are two forms of **for** loops, the numeric **for** loop, and the list-based **for** loop. We will take a look at both.

First up, the numeric **for** loop.

This loop, for those who have experience programming in other languages, will recognize similarity to existing knowledge. The numeric **for** loop has 3 values:

  - the starting condition
  - the looping condition (ie it will keep looping as long as that condition is true)
  - the stepping value

We will start with a simple script to display the numbers 1-10 to the screen, one per line:

<code bash>
#!/bin/bash

for((i=1; i<=10; i++)); do
    echo "$i"
done
</code>

The **i++** is shorthand for the equation: **i=i+1**. Either could be used in that field.

If you'd like to alter the stepping value, put in the appropriate equation.

^  4.  ^|Looking at our **numeric** for loop script above, modify it to do the following:|
| ^  a.|Instead of one per line, put the count all on one line, each separated by a space. How did you do this?|
|:::^  b.|Instead of going from 1 to 10, go from 20 to 2, stepping 2 values per iteration. How did you do this?|
====list-based for loop====
In addition to the numeric **for** loop, we have the list-based for loop. This is still a loop, it still iterates a number of times, but it does so not on a condition to evaluate, but instead a list of provided values.

Check out the following script:

<code bash>
#!/bin/bash

echo -n "First color is: "
for color in red orange yellow green blue indigo violet; do
    echo "$color"
    echo -n "Next color is: "
done
</code>

Go ahead and run this script, analyze the output, and make sure everything in it makes sense. Be sure to ask questions.

^  5.  ^|Looking at our **list-based** for loop script:|
| ^  a.|How many times does the loop iterate?|
|:::^  b.|What is the value of **$color** the fifth time through?|
|:::^  c.|Modify the script so that for the case of the color **violet**, it prepends the text "Final color is: " instead of "Next color is: "... output for all the other colors remains unchanged.|
|:::^  d.|How did you do this?|

=====Scripts=====
With your scripting knowledge, please write for me the following scripts, and answer any pertinent questions:

^  6.  ^|Write a script to do the following:|
| ^  a.|Creates 8 files: file1 file2 file3 file4 file5 file6 file7 file8|
|:::^  b.|Place the number of the filename within the file (file5 has a 5 in it, etc.)|
|:::^  c.|Prompt the user to pick a number between 1 and 8|
|:::^  d.|Have your script insert that number in a new file of that numbered name, shifting all the other file names out of the way.|
|:::^  e.|When you are done, there should be 9 files, two of which contain the same number.|
|:::^  f.|Please put your script on your Opus or Portfolio page.|

^  7.  ^|Write a script to do the following:|
| ^  a.|Display a 4 line histogram of all the files in a given directory, sized to fit a standard screen.|
|:::^  b.|Prompt the user for a directory, capture it into a variable and change to that directory in your script.|
|:::^  c.|List and count all the files that begin with lowercase **a-g**, store that number in a variable.|
|:::^  d.|List and count all the files that begin with lowercase **h-m**, store that number in another variable.|
|:::^  e.|List and count all the files that begin with lowercase **n-s**, store that number in another variable.|
|:::^  f.|List and count all the files that begin with lowercase **t-z**, store that number in another variable.|
|:::^  g.|Cap each number at a max of 60, if it goes above just truncate it to 60, and keep that result for each.|
|:::^  h.|Display a row heading (which section of the alphabet) followed by the histogram for that line. Use asterisks.|
|:::^  i.|Your script should output the directory it is processing, total files, and then 4 lines of histogram.|
|:::^  j.|Please put your script on your Opus or Portfolio page.|

Sample output for the above script could resemble:

<cli>
lab46:~$ ./myscript
Please enter directory to process: /some/dir
Processing ...

Directory: /some/dir
Total Files: 42

(a-g):***************
(h-m):********
(n-s):*****************
(t-z):**

lab46:~$ 
</cli>

Aside from the requirement to use asterisks, you are free to format the output of your script as you see fit. The requested information needs to be clearly visible.

=====Conclusions=====
This assignment has activities which you should tend to- document/summarize knowledge learned on your Opus.

As always, the class mailing list and class IRC channel are available for assistance, but not answers.
Lab46 Wiki

User Tools

Site Tools

Page Tools
