UNIX Tutorials, Tips, Tricks and Shell Scripts

Shell Programming Guide for Absolute Beginners...with Examples!!! - Part I


Shell Programming Guide for Absolute Beginners...with Examples!!! - Part I Welcome to our Shell Programming Guide for Absolute Beginners! I sincerely hope you find it to be informative and helpful as you seek to better understand what shell programming is and how shell programs can be used to make your system and network operations more efficient and productive.

The following is a list of shell programming topics that will be covered in this guide. My intent is not to go too deep on any specific topic, but to only introduce you to key shell programming concepts and provide basic shell program examples where applicable to demonstrate usage. There should be sufficient information provided with each topic so that you can perform additional research on it if needed.

          · What is shell programming and why it is useful
          · Which shell should you use when writing shell programs
          · Naming your shell program files
          · Writing and running your first shell program
          · Comments: Your future self, and others, will thank you
          · Simple shell program variables
          · Common shell programming constructs

My hope is that you will benefit from my 15+ years of shell programming experience and that this guide will help you to focus on only those concepts which are most useful to shell programming beginners.

Let's get started ...

What is shell programming and why it is useful

Simply stated, shell programming is the creation of an executable file (the shell program or script) that contains commands, assignments and data that work together to accomplish a higher-level task.

TIP: The terms "shell programming" and "shell scripting" are used interchangeably, so try substituting the word "scripting" for "programming" in your search phrases if you are looking for something and cannot find anything about it or would like to see additional information about a particular topic.

Shell programs are often developed to automate a repetitive task or for monitoring purposes.

Examples of common repetitive tasks include adding new users (that require customization not provided by native Linux and UNIX user account management utilities) to a system, data backup, and system and network configuration work.

Although the list of things which can be monitored is endless, here are a few of the most common targets for ongoing monitoring:

          · system processes
          · disk space usage
          · server/host availability
          · system performance and non-disk resource utilization

In addition to traditional system management type tasks, shell programs are frequently used in the product development space (think of an enterprise-class disk storage array vendor) to automate the testing or qualification of new product features and functionality. They are also used to automate standard back-end business processes such as the processing of new sales for an ecommerce site.

Shell programs are not limited to the world of non-interactive scheduled (batch) jobs and complex automation programs. The shell programming language is robust enough to create user-space applications that allow end users to enter data, retrieve data and perform simple to complex front-end business operations.

Which shell should you use when writing shell programs

There are multiple shells to choose from that can be used as your default user account shell for interactive logins on a Linux or UNIX system, and different shells can also be used when writing shell programs. This guide will discuss the most frequently-used shells in both the Linux and UNIX space.

For Linux, the Bash (Bourne Again Shell) is typically used for both interactive logins and shell programming. The Korn shell is most often used on enterprise-class UNIX servers, especially on systems running AIX (IBM's flavor of UNIX).

I personally like the Korn shell because it has had more time to mature and it is the shell that I started with as a shell programming beginner, but both Bash and Korn are powerful scripting languages and either should be sufficient for the shell programs you will need to write.

Many times the shell used for shell programming is the one the original shell program author knew best or the one that has "always" been used in a given work environment, so you may not have a choice if you are inheriting legacy shell programs to maintain.

The good thing is that although the Bash shell is typically used on Linux systems and the Korn on UNIX systems, both shells can easily be installed on the other platform which make shell programs extremely portable. Also, a program written for Bash can often be run with the Korn shell and vice versa, but there are exceptions that may be need to be resolved - e.g., the "print" command is supported by Korn but not Bash.

Naming your shell program files

Historically, shell programmers have used the name of the shell binary/executable file (i.e., bash for Bash or ksh for Korn) for the file name extension for their shell programs. Some examples are addnewuser.bash or calculate_disk_usage.ksh.

Over time, I have typically dropped the file extension completely when creating a new shell program opting for names such as addnewuser or calculate_disk_usage. This is especially true when the shell program includes a shebang (discussed below) but there are many times when I will not use a file extension even if the script does not start with a shebang. (When a shell program does not have a shebang, the program has to be run by specifying the name of the shell when the program is invoked either at the command prompt or from within another program or cron job entry.)

Since integrated development environments (IDEs) are becoming more popular for source code editing, to include shell programming, the practice of using file name extensions is becoming more common because IDEs provide editing features and select the correct compiler or interpreter to use by keying off of the extension used for the file name. For example, an IDE will invoke the Python compiler when running a file that has .py as the extension.

Writing and running your first shell program

For this section, I will make the assumption that you know how to use either a native editor such as vim or vi, or that you have a favorite IDE for editing program files.

The first line of your shell program will contain a shebang. A shebang consists of the number sign and exclamation point characters ("#!"). Immediately following these two characters will be the full path name of the interpreter (executable program) that will be used to run the commands in your shell program.

If you want to use Bash to run your program, your shebang line will look like this ...

          #!/bin/bash

and for Korn this ...

          #!/bin/ksh

TIP: To find out the full path to your preferred shell, just run "which bash" or "which ksh" at your system's command prompt.

*Again, the shebang line MUST be put on the FIRST line in your shell program.

Consider this very simple first shell program which simply displays "Hi Mom!" ...

          #!/bin/bash

          echo "Hi Mom!"

If you try to run it at this time you will receive the following error:

          # /home/jp1/tmp/myshellprogram
          -bash: /home/jp1/tmp/myshellprogram: Permission denied

To eliminate this error you will either need to run this program from the command line specifying the shell ...

          # bash /home/jp1/tmp/myshellprogram
          Hi Mom!

OR set the execute permission bit on the file ...

          # chmod u+x /home/jp1/tmp/myshellprogram
          # /home/jp1/tmp/myshellprogram
          Hi Mom!

TIP: If you are in the same directory as your shell program you can simply type ../myshellprogram. to run it. That is a period and forward slash ("./") followed by the name of your program - e.g., "./myshellprogram"

Congratulations! You have just learned how to write and run your first shell program. Nice work.

Comments: Your future self, and others, will thank you

Shell program comment lines are for human consumption. A comment line is preceded with the number sign ("#") and is ignored by the shell ...

          #!/bin/bash

          #display a line of text
          echo "Hi Mom!"

Comments are not only good for other people who may need to understand or make modifications to your shell program, they can save you much time down the road when you come back to update a program that you have not looked at for 6 months. This is especially true for more complex shell programs.

Simple shell program variables

As you will quickly find out, a shell program is not very useful without one or more variables. In basic terms, a variable is a character string (the variable name) to which a value is assigned. The value may be a number, a string of text (characters) or other types of data.

To access the value in a shell program variable, all you need to do is put a dollar sign ("$") immediately before the variable name. *There are no spaces between the dollar sign and the variable name.

TIP: It may be useful to use all capital letters for your variable names. This way you can easily recognize the variables used in a line of code since they stand out from the commands used in your program.

Our sample shell program has been updated to assign and access the variable MESSAGE, which contains the string we want to display ...

          #!/bin/bash

          MESSAGE="Hi Mom!"
          #display a line of text
          echo $MESSAGE

Note that quotes are used around string variable assignments. You would not use quotes for a number variable assignment such as this:

          NUMBER=12345

There are other variable types used in shell programs, such as arrays, but those are beyond the scope of this guide.

Common shell programming constructs

Since this is a basic guide to shell programming, I will only cover two of the most frequently used programming language constructs...the if-then and the while loop constructs.

The If-then Construct

If-then constructs are used to check a condition...is shell program variableX equal to a given number for example, is variableY equal to string "abc", is variableZ greater than variableX, and so on.

This article will provide examples of three different if-then constructs. (1) if-then with no else; (2) if-then-else; and (3) if-then-elif-else.

First, we will modify our shell program example to include a simple if-then construct ...

          #!/bin/bash

          MESSAGE="Hi Mom!"
          NUMBER=12345

          #display a line of text
          echo $MESSAGE

          if [ $NUMBER -eq 12345 ]
          then
                    echo "Yes, the value in the variable named NUMBER equals 12345"
          fi


this produces the following output when run ...

          # ./myshellprogram
          Hi Mom!
          Yes, the value in the variable named NUMBER equals 12345

Things you should note is there are brackets around the condition you are checking AND a single space (" ") after the opening bracket and one before the closing bracket, and the if statement is terminated with if spelled backwards ("fi").

Now let us include an else path to our shell program and update the value in the variable NUMBER to cause the else path to be executed ...

          #!/bin/bash

          MESSAGE="Hi Mom!"
          NUMBER=55555

          #display a line of text
          echo $MESSAGE

          if [ $NUMBER -eq 12345 ]
          then
                    echo "Yes, the value in the variable named NUMBER equals 12345"
          else
                    echo "No, the value in the variable named NUMBER does not equal 12345"

          fi

which produces this output ...

          # ./myshellprogram
          Hi Mom!
          No, the value in the variable named NUMBER does not equal 12345

We can also introduce an else-if ("elif") which provides another code path for our shell program to check ...

          #!/bin/bash

          MESSAGE="Hi Mom!"
          NUMBER=55555

          #display a line of text
          echo $MESSAGE

          if [ $NUMBER -eq 12345 ]
          then
                    echo "Yes, the value in the variable named NUMBER equals 12345"
          elif [ $NUMBER -eq 55555 ]
          then
                    echo "Yes, the value in the variable named NUMBER equals 55555"

          else
                    echo "No, the value in the variable named NUMBER does not equal 12345 or 55555"
          fi

which outputs ...

          # ./myshellprogram
          Hi Mom!
          Yes, the value in the variable named NUMBER equals 55555

*Note that the "elif" also needs its own "then" on the next line.

Before you continue reading... Has this article been helpful to you? Would it benefit others? If you answered "yes" to either question, kindly share the page.


MORE READERS = MORE FUTURE ARTICLES
Thank you for sharing.


The while Loop Construct

The other must-know shell programming construct is the while loop. Each shell program line inside the body of a while loop will be run until the condition statement (similar to the condition statement used with the if-then construct) evaluates to false.

Our example shell program has been updated with a while loop to display our message ten times before the loop terminated ...

          #!/bin/bash

          MESSAGE="Hi Mom!"
          NUMBER=55555

          #define a variable to count the loop interation
          LOOPCOUNTER=0

          #display a line of text until the while statement evaluates to false
          while [ $LOOPCOUNTER -lt 10 ]
          do
                    echo $MESSAGE
                    (( LOOPCOUNTER=LOOPCOUNTER+1 ))
          done


          if [ $NUMBER -eq 12345 ]
          then
          echo "Yes, the value in the variable named NUMBER equals 12345"
          elif [ $NUMBER -eq 55555 ]
          then
                    echo "Yes, the value in the variable named NUMBER equals 55555"
          else
          echo "No, the value in the variable named NUMBER does not equal 12345 or 55555"
          fi

In addition to the while loop, two additional shell programming concepts were utilized in our updated program. "-lt" was used to check if the value stored in the variable LOOPCOUNTER was less than the desired number of loop iterations (10 in our example).

The second new concept was the use of double parenthesis to perform an arithmetic operation. The (( LOOPCOUNTER=LOOPCOUNTER+1 )) statement will add 1 to the value stored in the variable LOOPCOUNTER.

The output for our while loop shell program looks like this ...

          # ./myshellprogram
          Hi Mom!
          Hi Mom!
          Hi Mom!
          Hi Mom!
          Hi Mom!
          Hi Mom!
          Hi Mom!
          Hi Mom!
          Hi Mom!
          Hi Mom!
          Yes, the value in the variable named NUMBER equals 55555

TIP: If needed, a "continue" statement can be used to terminate the current loop iteration and start a new one (*as long as the while statement does not evaluate to false). A "break" statement is used to terminate the while loop completely. No additional loop iterations are performed. Typically, a continue or break is used if you need to check for a particular condition within the loop and take action when that condition is met.

Thank you for reading our beginner's guide to shell programming for UNIX and Linux. I hope that the guide's basic shell program examples contained just enough code to help you understand each new shell programming concept as it was being introduced, and that the guide in general will provide you with a solid foundation to build on.

Do you need to learn Linux or UNIX shell programming and get practice writing & running shell programs...on a REAL SERVER? If you are ready to move past the basics, either of these online courses is a good place to start ...

UNIX and Linux Operating System Fundamentals contains a very good "Introduction to UNIX Shell Programming" module, and should be taken if you are new to the UNIX and Linux operating system environments or need a refresher on key concepts.

UNIX Shell Programming is a good option if you are already comfortable with UNIX or Linux and just need to sharpen your knowledge about shell programming and the shell in general.

Both courses include remote access to an Internet Lab system for completing the course's hands-on exercises, which are used to re-enforce the key concepts presented in the course. Any questions you may have while taking the course are answered by an experienced shell programmmer.

Thanks for reading, and happy programming!!!