UNIX Tutorials, Tips, Tricks and Shell Scripts

4 Methods for Running a Shell Script on Linux or UNIX with Examples...and BONUS debugging techniques!!! - Part I


Now that you understand how to write a basic shell script, let's look at the four different methods typically used to run a shell script.

An explanation of some of the behind the scenes of a shell script will be followed by a couple of options for actually making a shell script executable, and a few other things that we need to consider when executing a shell script.

Let's get started ...
Shebang - what's that about and how it relates to running a shell script?

Why in the world do we have to start our shell scripts with the #! characters - the "shebang"? That is a good question, one that many people do not even ask, they just assume "that's the way it is". Consider this VERY simple two line shell script:

#! /usr/bin/env bash
echo "Hello from myscript"

Shebang is the sequence "#!" that we find at the very beginning of most shell scripts. (We will talk about those situations where we can get by with not using the shebang later on.)

### PLEASE READ AN IMPORTANT MESSAGE FROM OUR FOUNDER ###

The time has come to END the social experiment known as Facebook.

It is breaking our world and is hurting people.

(1) People are dying as the result of genocide (ie. the Rohingya in Myanmar), bullying and mental health problems enabled by Facebook

(2) People's behavior, including yours if you use Facebook, is being manipulated by Facebook's subjective Newsfeed algorithm

(3) Families, communities and countries are becoming more divided, contrary to Facebook's stated mission of building community

READ MORE >>

The shebang sequence is in reality a human readable instance of a magic number in the executable file. The magic number - 0x23 0x21 is in fact the ASCII of #!. This magic number is detected by the execve call that actually executes the shell script. (We can find out more about execve, which is part of the "exec" family of functions, on the exec man page.) So the exec call receives the command line and determines from the magic number whether the file is a script or an executable binary.



If the magic number is '#!' then "exec" knows the file is a shell script. If the magic number is '.ELF' then "exec" knows the file is an executable binary file. When a shell script is found, then exec will launch the proper shell interpreter.

Once "exec" knows to handle our file as a shell script, it then looks at the command specified on the shebang line. This command is most commonly the interpreter that we want to execute our shell script.

One caveat to this in that by using the command ...

#! /usr/bin/env bash

as the first line of our shell script, we make our script more portable. Not all Linux systems have the shell interpreters - sh, bash, csh, ksh - located in /bin. By using the /usr/bin/env command we ensure that the proper shell interpreter executes our shell script.


More about 'exec' ...

The "exec" family of functions are actually front ends for the actual function that executes either binary executable files or shell scripts in Linux. The Linux function execve executes the filename that is specified in the function call. There is actually a lot that goes on behind the scenes when a file is executed, and most of it is handled by the execve function call.

Below we will investigate ways to totally circumvent the need for a shebang line in our shell scripts, but it is suggested that shell script authors include this line to ensure that others will know what shell was intended to run this script.

Methods for Executing a Shell Script

We have 4 different methods for running a shell script, which we will review below. A simple script named "myscript" will be used to demonstrate usage. It contains the following lines ...

#! /usr/bin/env bash
echo "Hello from myscript"

Method 1 - Use the filename to run the shell script

This is probably the most common, as well as the simplest and most straight forward, method to execute shell scripts. This method requires us to have the shebang line in our script. We simply use the absolute pathname or the relative pathname to specify the shell script to execute.

$ cd /home/pbmac/scripts
$ ./myscript

or

$ /home/pbmac/scripts/myscript
The output appears on the terminal:

Hello from myscript

Changing Permissions to Make a Shell Script Executable

Let's jump in here with a side note about running the shell script. To run a shell script using just the filename the execute bit must be set on the file permissions. If you are going to run the shell script by one of the other methods, then the execute bit does NOT have to be set.

Consider this example ...

$ ls -l
-rwxrwxrwx 1 pbmac pbmac 48 Sep 25 16:02 myscript
-rw-rw-r-- 1 pbmac pbmac 48 Sep 25 18:57 testscript
The first script has the execute bit set for everyone - notice the x, while the testscript has no execute bits set. So, it depends on how you want to run the shell script as to whether the execute bit is set.

To change the permissions you use the chmod command. Notice that in the above example the myscript file has 3 sets of rwx in its permissions, on the left side of the output. The leftmost rwx are the read/write/execute permissions for the owner. The middle rwx is the read/write/execute permissions for the group and the third rwx are the read/write/execute permissions for others. The testscript file has rw- which is the read and write permissions for the owner, rw- for the group and only r-- for others.

The format of the chmod command is "chmod <permissions> <file>". For example ...

$ chmod u+x testscript
$ ls -l
-rwxrwxrwx 1 pbmac pbmac 48 Sep 25 16:02 myscript
-rwxrw-r-- 1 pbmac pbmac 48 Sep 25 18:57 testscript
The chmod above "says" change the permissions for 'u' - the owner and to '+' - to add the 'x' - execute bit. So, we add the execute permission for the user, that is the owner of the file. In the ls -l output you can see that the first set of permissions now has an 'x', which signifies the owner of the file has execute permissions.

We can also combine permissions

$ chmod ug+g testscript
sets execute permission for user and group.

$ chmod +x testscript
set execute permissions for user, group and other.

Using chmod you can set r, and w permissions as well. You just need to determine who needs access your script and who needs to run the shell script, then set the permissions accordingly.



Method 2 - Specify the interpreter as a way to execute the shell script

We can also execute a shell script by specifying the interpreter on the command line.

$ bash myscript
Hello from myscript
This tells "exec" to use the bash shell to run the myscript file. Specifying the interpreter on the command line forces Linux to use that shell to run the script, no matter what is specified on the shebang line inside of the script. Using the wrong interpreter from the command line may cause the script to fail, or to run in an unintended manner.

Since we are specifying which shell interpreter will be used on the command line THIS is an instance where we do not have to include a shebang line in the file. However, it is a wise habit to include the shebang line, including the appropriate shell interpreter in your file to avoid any confusion by someone else who may be running or making updates to your script.


Method 3 - Using . ./ (dot space dot slash) to execute the shell script

This syntax is the least used syntax for running a shell script, mainly due to its somewhat odd syntax. Using this method will execute the shell script in the current shell without forking a new shell instance to run the script.


What does Forking / Fork do?

In Linux the term "fork" means that a running process creates a new child process which is identical to its parent except that it has a different system process id. When the child process completes then control is returned to the parent process - which may be the command line interpreter.


This method (and the source command below) actually runs the shell script in the same process id as the current command line shell. What is the advantage of this?

There are two files that control your login environment .bashrc or .bash_profile (sometimes you will not find both of these files in your home directory - that's okay). If you edit one or both of these files the changes do NOT take effect until the files are executed. Now, if you simply run them as a shell script the new shell that is created will have the new settings, but NOT your login shell. After the changes are made we can either logout and login back in to allow the changes to take effect, or we can use the "dot space dot slash" method to execute the altered file for the changes to take place.

For instance ...

$ . ./.bashrc

or

$ . ./.bash_profile
However we can execute any shell script in this manner ...

$ . ./myscript
Hello from myscript

Method 4 - Using the source command to execute the shell script

The last method is probably the second most used method to run a shell script. It uses the built-in source command. This is the same as the .(dot) method explained above. For some people using the dot space dot slash seems odd, so the alternative is this source command ...

$ source ./myscript
Hello from myscript





Other Considerations when executing a shell script

There are a couple of things we need to keep in mind as we create shell scripts for our own use or for the use of others.

Where should I keep my scripts?

If you are writing lots of shell scripts, and especially if you want others to have access to them, you should decide on a central location for your scripts. Some system administrators create a scripts library in a /usr/local/scripts directory, or someplace similar. If you are making such a library simply for personal use you can create a directory somewhere in your home directory and place all of your scripts there.

Also, make sure that your script names are not the same as Linux system commands. If that happens whichever appears first in your PATH variable will be the one that gets executed. An easy way to solve this is to add a unique suffix at the end of your script name.

Do I have to specify the entire path every time to run my shell script?

It can be a real hassle to continually type the entire path to your script, or having to cd to the directory where it is located to run it from there. The solution is to set your PATH environment variable to include the directory where your scripts are stored.

I keep my scripts in a scripts directory within my home directory. Then I edit my PATH variable. Each user should have a PATH variable set in their .bash_profile, or some systems place it in .bashrc. If it is not in either of these locations, then you are running with the default system PATH. No problem!! It is quite easy change your PATH.

$ cd
$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:
This is a typical PATH for a Linux user. To change this we open .bash_profile (I do not have this file so I edit .bashrc) with your favorite text editor and add the following line:

export PATH="$HOME/scripts:$PATH"
Save the changes, close the editor, and source whichever file you just edited

$ source .bashrc
$ echo $PATH
/home/pbmac/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:
Notice that my script library is now at the beginning of my PATH.

If you decide to create a system wide script library so that all users can run the shell scripts you write then you have to change the system wide PATH. This is usually found in either the /etc/environment file or the /etc/profile file. To make this change simply edit the file where PATH variable is define, on my system it is the /etc/environment file, and change the PATH line to include your system's script library location, for instance /usr/local/scripts:

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/scripts"
Then source whichever file you just edited

$ source /etc/environment
$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/scripts
Again, notice that the /usr/local/scripts has now been added to our PATH. Every user who logins in will now have this directory in their PATH.

Can I execute a shell script from another script?

One last question that is often asked, "Can I run one of my shell scripts from another script?". The answer is "Yes". It is actually quite easy, especially when you have the script location set in the PATH as we discussed above.

We simply run the shell script that we are writing, and include a line that runs the library script that we already had developed and placed in our handy dandy library. Let's take a look at an example ...

$ cat myscript
#! /usr/bin/env bash
echo "Hello from myscript"
# The following line calls another script found in my library
two_script
echo "We are done"
We also have a new script, called two_script, that resides in our library directory which is now in our PATH ...

$ cat two_script
#! /usr/bin/env bash
echo "This is two_script"
exit 1
So, if we execute myscript as it is above we will see the output:

$ ./myscript
Hello from myscript
This is two_script
We are done
We can follow the echo statements that produce the output. Notice the first echo output is from myscript, the second output is from two_script, and the last output is again, from myscript.

Now if we change myscript to execute two_script using the source command we see something different about the output ...

$ cat myscript
#! /usr/bin/env bash
echo "Hello from myscript"
# The following line calls another script found in my library
source two_script
echo "We are done"

$ ./myscript
Hello from myscript
This is two_script

Notice we are missing the "We are done" output. Why is that? Well, if we go back to what we learned about the source command - that it actually runs the shell script in the same shell as the calling script - it makes sense. If we do not spawn a separate shell, which is the case with the source command, then when two_script executes the exit statement it exits the original shell. In the first instance two_script actually spawns a separate shell to run the two_script in and it is that shell that ends when the exit command is run. So if we call either:

$ ./myscript
or use
$ bash myscript

a new shell is spawned to run the script, and when that shell exits, control is returned to the myscript. Using "dot space dot-slash" or the source command we will execute two_script in the same shell and the exit of two_script will exit out of that shell back to the command prompt.


In conclusion ...

Some of this just takes practice to remember all of these steps and keep in mind all of the various aspects of writing and executing shell scripts.

Hopefully this mini-tutorial has provided you with a solid understanding of the typical methods for executing shell scripts and the various considerations that come into play.


BONUS: How to execute a shell script in debug mode

Most scripts are obviously much more complex that our simple myscript. It can be difficult at times to figure out what is going on. We do have the ability to debug scripts, as shown below.

$ bash -x ./myscript
Notice we added an '-x' argument to the command line. This '-x' tells bash to run in debug mode. Let's add a few lines to myscript to try and get a better understanding of what this can do for us.

$ cat myscript
#! /usr/bin/env bash
number=1 + 2
echo "Number is: $number"
echo "Hello from myscript"
two_script
echo "We are done"

$ cat two_script
#! /usr/bin/env bash
echo "Uname is: " `uname`
echo "This is two_script"
exit 1
We have added two lines to myscript and one line to two_script. Now when we run the shell script we get output

$ ./myscript
./myscript: line 2: +: command not found
Number is:
Hello from myscript
Uname is: Linux
This is two_script
We are done
But what does that error message mean? If we simply run the shell script in debug mode it is easy to see

$ cat myscript
#! /usr/bin/env bash
number=1 + 2
echo "Number is: $number"
echo "Hello from myscript"
bash -x two_script
echo "We are done"
Notice we add a line to run two_script in debug mode as well, even though there is not an error in that script, this will enable debug mode. If we did not run the two_script in debug mode we would simply NOT see any debug output for that script. When we run myscript in debug mode we can easily find the error: All of the debug output is preceded by a '+' character.

$ bash -x myscript
+ number=1
+ + 2
myscript: line 2: +: command not found
+ echo 'Number is: '
Number is:
+ echo 'Hello from myscript'
Hello from myscript
+ bash -x two_script
++ uname
+ echo 'Uname is: ' Linux
Uname is: Linux
+ echo 'This is two_script'
This is two_script
+ exit 1
+ echo 'We are done'
We are done
Right away we can see that the line where we were trying to add 1+2 is failing. Notice that the line which is suppose to read "number=1+2" is broken into to separate line by the shell interpreter, so our debug output looks like:

+ number=1
+ + 2
myscript: line 2: +: command not found
The interpreter is seeing the '+' as a new command, not as an arithmetic operator. To resolve this syntax error we must edit the addition statement

number=`expr 1 + 2`
and this resolves the issue and the script runs properly.

There are other script debug utilities, but the capabilities built into the shell can go a long way towards resolving many simple bugs.





Do you need to learn Linux/UNIX shell scripting and get practice writing & running scripts...on a REAL SERVER? If you are ready to move past the basics, either of these online courses is a good place to start...

UNIX and Linux Operating System Fundamentals contains a very good "Introduction to UNIX Shell Scripting" module, and should be taken if you are new to the UNIX and Linux operating system environments or need a refresher on key concepts.

Linux/UNIX Shell Scripting is a good option if you are already comfortable with UNIX or Linux and just need to sharpen your knowledge about shell scripting and the UNIX shell in general.

Both courses include access to an Internet Lab system for completing the course's hands-on exercises, which are used to re-enforce the key concepts presented in the course. Any questions you may have while taking the course are answered by an experienced UNIX technologist.

Thanks for reading, and happy scripting!!!

Popular Online Courses

Select a course for enrollment details.