Bash scripting basics
COMMANDS
Basic commands
To display command one after another “;” :
Command1 ; command2 ; command3
To change to current shell prompt, modify the PS1 and $PS2 variables.
Debugging a script:
Display argument and commands:
bash –x ./script
Display input as they are read:
bash –v ./script
Working with files
diff
-y display lines side to side
sdiff –compare files side by side
-s
-s
-l display only differences in the left colun
-b ignore space change
sort
-n - to sort numbers
-M – to sort month names
-r sort values in descending order
-t – specify field separating character, -k – which field to sort on (e.g. for csv files). Example: sort –t ‘:’ –k 3 /etc/passwd – to sort by user id
cut - remove sections from the beginning and the end of the file
Using Environmental variables
Global variables are visible from the shell session and the any child processes, while local variables are only visible in the shell that creates them. Linux defines standard local variables by default.
printenv – to see global variables
set – displays all the local variables for a particular process, but also the global variables.
Setting variables
Use single quotation to mark a string (‘ ‘).
To define a local variable
test=’test variable’
To view:
echo $test
To export to global:
export test
To remove corresponding variable or function:
unset test
Using command aliases
alias –p – to print current aliases
alias il=’ls –il’ – to create an alias
Basic Scripts
1.Create a script file, always begins with: #!/bin/bash
2.Set executable permissions.
3.Execute script:
The PATH environment variable is set to look for commands only in a handful of directories. To get the shell to find my test1 script, I need to do one of two things:
▪ Add the directory where my shell script file is located to the PATH environment variable.
▪ Use an absolute or relative filepath to reference my shell script file in the prompt.
Command input
Using multiplecommands “;” :
command1 ; command2 ; command3
To ignore the new line (equal to the one line output above):
To ignore the new line (equal to the one line output above):
command1 ; \
command2 ; \
command3
command2 ; \
command3
Use back tick (`) to specify a command:
today=`date +%y%m%d`
Setting the date
$ date "+%d %B %Y"
We can set the date and time as follows:
# date -s "Formatted date string"
For example:
# date -s "21 June 2009 11:01:22"
Input redirection- Sending output from a command to file: command > file
Append inputto existing file command >> file
Output redirection– takes contents of a file and redirects to a command: command < file
The inline input redirection symbol is the double less-than symbol (<<).Besides this symbol, you must specify a text marker that delineates the beginning and end of the data used for input. You can use any string value for the text marker, but it must be the same at the beginning of the data and the end of the data:
command << marker
data
marker
wc << EOF
text 1
text 2
text3
EOF
Piping ( | ) provides a way to link commands to provide more detailed output. The Linux system actually runs both commands at the same time, linking them together internally in the system.
command1 | command2 | command3 etc.
Replace string 1 with 2 in text file with one line command:
/usr/bin/perl -p -i -e "s/string1/string2/g" text.txt
It can also be used with input variables:
/usr/bin/perl -p -i -e "s/$1/$2/g" $3
$1/$2/g" $3
Replace string 1 with 2 in text file with one line command:
/usr/bin/perl -p -i -e "s/string1/string2/g" text.txt
It can also be used with input variables:
/usr/bin/perl -p -i -e "s/$1/$2/g" $3
$1/$2/g" $3
Calculations
exp 1 + 2
Some symbols however may need to excluded in a script: exp 1 \* 2
var3 = ‘ exp $var1 / $var2’
Square bracketed expressions can be used within scripts:
var3 = $[ var1 * var2]
echo $[1+2+3]
The above support only integer. The bccan be used for floating point calculations. The argument “scale” need to be set to the number of decimal spaces:
bc –q
scale=2
1/3
0.33
quit
Using bc in scripts:
variable=`echo "options; expression" | bc`
var1=`echo " scale=4; 3.44 / 5" | bc`
bc also recongnises input redirection:
variable=`bc << EOF
options
statements
expressions
EOF `
var5=`bc << EOF
scale = 4
a1 = ( $var1 * $var2)
b1 = ($var3 * $var4)
a1 + b1
EOF
`
Arrays
Bash supports regular arrays as well as associative arrays. Regular arrays are arrays which can use only integers as its array index. But associative arrays are arrays which can take a string as its array index.
Associative arrays are very useful in many types of manipulations. Associative array support came with version 4.0 of Bash. Therefore, older versions of Bash will not support associative arrays.
How to do it...
An array can be defined in many ways. Define an array using a list of values in a line, as follows:
array_var=(1 2 3 4 5 6)
#Values will be stored in consecutive locations starting from index 0.
Alternately, define an array as a set of index-value pairs as follows:
array_var[0]="test1"
array_var[1]="test2"
array_var[2]="test3"
array_var[3]="test4"
array_var[4]="test5"
array_var[5]="test6"
Defining associative arrays
In an associative array, we can use any text data as an array index. However, ordinary arrays can only use integers for array indexing.
Initially, a declaration statement is required to declare a variable name as an associative array. A declaration can be made as follows:
$ declare -A ass_array
After the declaration, elements can be added to the associative array using two methods, as follows:
1. By using inline index-value list method, we can provide a list of index-value pairs:
$ ass_array=([index1]=val1 [index2]=val2)
2. Alternately, you could use separate index-value assignments:
$ ass_array[index1]=val1
$ ass_array[index2]=val2
Listing of array indexes
Arrays have indexes for indexing each of the elements. Ordinary and associative arrays differ in terms of index type. We can obtain the list of indexes in an array as follows:
$ echo ${!array_var[*]}
Or, we can also use:
$ echo ${!array_var[@]}
Exiting the script
Every command uses exit status to report to the shell that it is done processing. The exit status is a value between 0 and 255. The $? special variable holds the exit status of the last command executed:
date
echo $?
0
The exit command can be used in a script to specify the exit status.
exit 5
Structured commands
if
command
then
commands
fi
The then command is only executed if the statements under the if return and exit value of 0.
elif – nested if
if
command1
then
commands
elif
command2
then
more commands
fi
The bash shell provides an alternative way of declaring the test command in an if-then
statement:
if [ condition ] then
commands
fi
The square brackets define the condition that’s used in the test command. Be careful; you must have a space after the first bracket, and a space before the last bracket or you’ll get an error message.
There are three classes of conditions the test command can evaluate:
· Numeric comparisons (works only with integers)
· String comparisons
· File comparisons
Comparing strings
■ The greater-than and less-than symbols must be escaped, or the shell will use them as redirection symbols, with the string values as filenames.
■ The greater-than and less-than order is not the same as that used with the sort command.
Notice that the test command uses the standard mathematical comparison symbols
for string comparisons, and text codes for numerical comparisons. This is a subtle feature that many programmers manage to get reversed. If you use the mathematical comparison symbols for numeric values, the shell interprets them as string values and may not produce the correct results.
Empty and uninitialized variables can have catastrophic effects on your shell script
tests. If you’re not sure of the contents of a variable, it’s always best to test if the variable contains a value using -n or -z before using it in a numeric or string comparison.
File comparisons
-n – check if the string is non-zero
Using Boolean operators
▪ [ condition1 ] && [ condition2 ]
▪ [ condition1 ] || [ condition2 ]
The first Boolean operation uses the AND Boolean operator to combine two conditions. Both conditions must be met for the then section to execute.
The second Boolean operation uses the OR Boolean operator to combine two conditions. If either condition evaluates to a true condition, the then section is executed.
Advanced features
((expression))
The double parentheses command allows you to incorporate advanced mathematical formulas in your comparisons. The test command only allows for simple arithmetic operations in the comparison. The double parentheses command provides more mathematical symbols that programmers from other languages are used to using.
[[expression]]
The double bracket command provides advanced features for string comparisons.
The CASE command
case variable in
pattern1 | pattern2) commands1;;
pattern3) commands2;;
*) default commands;;
esac
If the variable matches the pattern, the shell executes the commands specified for the pattern.
The FOR command
for var in list do
commands
done
Specifying values in the list:
for test in Alabama Alaska Arizona Arkansas California Colorado
do
echo "The next state is $test"
done
Reading a list from a variable:
list="Alabama Alaska Arizona Arkansas Colorado"
Reading values from a command – use `
Changing the field separator
IFS envirmental variable tells the shell which value to use as separators. The defaults are: space, tab and new line. If the shell sees any of these, it treats them as new data field value separators. IFS can be locally specified in a script and the shell will ignore the other values. This will allow space between the characters:
IFS=$’\n’
When working on long scripts, it’s possible to change the IFS value in one place, then
forget about it and assume the default value elsewhere in the script. A safe practice to get into is to save the original IFS value before changing it, then restore it when you’re done.
This technique can be coded like this:
IFS.OLD=$IFS
IFS=$’\n’
‹use the new IFS value in code›
IFS=$IFS.OLD
Example for reading the fields in the /etc/passwd:
IFS.OLD=$IFS
IFS=$’\n’
for entry in `cat /etc/passwd`
do
echo "Values in $entry -"
IFS=:
for value in $entry
do
echo " $value"
done
done
Using wildcards to read files and directories
for file in /home/rich/test/*
do
if [ -d "$file" ]
then
echo "$file is a directory"
elif [ -f "$file" ]
then
echo "$file is a file"
fi
done
Using C-style loops
for (( variable assignment ; condition ; iteration process ))
for (( a = 1; a ‹ 10; a++ ))
Using multiple vairables:
for (( a=1, b=10; a ‹= 10; a++, b-- )) do
echo "$a - $b"
done
The while command
while test command do
other commands
done
The key to the while command is that the exit status of the test command specified must change, based on the commands run during the loop. If the exit status never changes, the while loop will get stuck in an infinite loop.
var1=10
while [ $var1 -gt 0 ]
do
echo $var1
var1=$[ $var1 - 1 ]
done
The until command
It is the opposite of the while command:
until [ $var1 -eq 0 ]
do
echo $var1
var1=$[ $var1 - 25 ]
done
Controlling the loop
break - can be used to exit any loop while in progress
for var1 in 1 2 3 4 5 6 7 8 9 10
do
if [ $var1 -eq 5 ]
then
break fi
echo "Iteration number: $var1" done
When you want to stop an outer loop from an inner loop: break n
continue is a way to prematurely stop processing commands inside of a loop but not terminate the loop completely.
Processing output of a loop
Can be redirected to a file using the pipe commands:
for file in /home/rich/*
do
if [ -d "$file" ]
then
echo "$file is a directory"
elif
echo "$file is a file"
fi
done > output.txt
handling user input
Command line paramaters
./script value1 valu2 ….
The values are read by the positional parameters – these are special values: $0 – name of the program (the entire path, to avoid that basename `$0`) , $1 – the first variable, $2- the second …. $9, ${10}, ${11}….
$# - provides how many command line parameters are entered in the command line
$ can not be used inside braces and has to be replaced by !
This one will show the last command line input: ${!#}
$* or $@ grab all command line parameters and put them in a single variable. $* takes all parameters as a single word, while $@ takes all parameters as multiple words in the same string
shift – assigns the value of $3 to $2, $2 to $3, etc.
count=1
while [ -n "$1" ]
do
echo "Parameter #$count = $1" count=$[ $count + 1 ]
shift
done
Processing options
Testing parameters
use –n to check if the string is non-zero
if [ -n "$1" ]
then
echo Hello $1, glad to meet you.
else
echo "Sorry, you didn’t identify yourself."
fi
Processing options
Can be done using the case and shift commands:
while [ -n "$1" ]
do
case "$1" in
-a) echo "Found the -a option" ;;
-b) echo "Found the -b option";;
-c) echo "Found the -c option" ;;
*) echo "$1 is not an option";;
esac
shift
done
Separating options from parameters – double dash (- -) – is used to specify the end of options list.
Standard linux command line options
These are well known command line options and using them in scripts is recommended.
Common Linux Command Line Options Option Description
-a Show all objects
-c Produce a count
-d Specify a directory
-e Expand an object
-f Specify a file to read data from
-h Display a help message for the command
-i Ignore text case
-l Produce a long format version of the output
-n Use a non-interactive (batch) mode
-o Specify an output file to redirect all output to
-q Run in quiet mode
-r Process directories and files recursively
-s Run in silent mode
-v Produce verbose output
-x Exclude and object
-y Answer yes to all questions
Getting User Input
Basic Reading
read command takes input from the standard input (keyboard) and places it into a standard variable.
read variable
read –p allows to you to specify the prompt directly on the command line
read -p "Please enter your age:" age
read –t secodns – timing out is required, so the prompt don’t stay forever
read –t 5 –p “Please print your age: “ age
Since the read command exits with a non-zero exit status if the timer expires, it’s easy to use the standard structured statements, such as an if-then statement or a while loop to track what happened.
if read -t 5 -p "Please enter your name: " name
then
echo "Hello $name, welcome to my script"
else
echo
echo "Sorry, too slow!"
fi
-s – silent reading – prevents data from being displayed on the monitor (e.g. for passwords)
Reading from a file
Each call of read reads a single line of input. Most common method is to pipe the result using the cat command, using while. At the end of the file, read will exit with non-zero.
count=1
cat test | while read line
do
echo "Line $count: $line"
count=$[ $count + 1]
done
echo "Finished processing the file"
Presenting Data
The Linux system handles every object as a file. This includes the input and output process. Linux identifies each file object using a file descriptor. The file descriptor is a non-negative integer, which uniquely identifies open files in a session. Each process is allowed to have up to nine open file descriptors at a time. The bash shell reserves the first three file descriptors (0 - STDIN, 1 - STDOUT, and 2 – STDERR)
STDIN (<) - The shell receives input from the keyboard on the STDIN file descriptor, and processes each character as you type it.
When you use the input redirect symbol (<),Linux replaces the standard input file descriptor with the file referenced by the redirection. It reads the file and retrieves data just as if it were typed on the keyboard.
STDOUT (>) file descriptor references the standard output for the shell. On a terminal interface, the standard output is the terminal monitor.
STDERR The shell handles error messages using the special STDERR file descriptor. The STDERR file descriptor references the standard error output for the shell. This is the location where the shell sends error messages generated by the shell or programs and scripts running in the shell.
STDERR file descriptor is set to the value 2. You can select to redirect only error messages by placing this file descriptor value immediately before the redirection symbol. The value must appear immediately before the redirection symbol or it won’t work:
$ ls -al badfile 2> test4
If you want to redirect both errors and the normal output, you’ll need to use two redirection symbols or alternatively &>can be used to redirect the both to the same file.
$ ls -al test test2 test3 badtest 2> test6 1> test7
Redirecting output in scripts
Temporary redirection
To purposely generate error messages in your script, you can redirect an indi- vidual output line to STDERR. All you need to do is use the output redirection symbol to redirect the output to the STDERR file descriptor. When you redirect to a file descriptor, you must precede the file descriptor number with an ampersand sign (&):
echo "This is an error message" >&2
Permanent redirections
It can get tedious having to redirect every echo statement. Instead, you can tell the shell to redirect a specific file descriptor for the duration of the script by using the exec command:
#!/bin/bash
exec 1>testout
Redirecting input
The same technique as above can be used for input:
exec 0< testfile count=1
while read line
do
echo "Line #$count: $line"
count=$[ $count + 1 ]
done
Creating Your Own Redirection
When you redirect input and output in your script, you’re not limited to the three default file descriptors. I mentioned that you could have up to nine open file descriptors in the shell. The other six file descriptors are numbered from three through eight and are available for you to use as either input or output redirection. You can assign any of these file descriptors to a file, then use them in your scripts as well.
Suppressing command output
There are times when you don’t want to display any output from your script. This often occurs if you’re running a script as a background process.
You can redirect STDERR to a special file called the null file. The standard location for the null file on Linux systems is /dev/null.
$ ls -al > /dev/null
Using Temporary Files
The Linux system contains a special directory location reserved for temporary files. Linux uses the /tmp directory for files that don’t need to be kept indefinitely. Most Linux distributions configure the system to automatically remove any files in the /tmp directory at bootup.
There are 7 commands to create temporary files:
mktemp (-d to create a directory)
Logging messages
Sometimes it’s beneficial to send output both to the monitor and to a file for logging. Instead of having to redirect output twice, you can use the special tee command.
The tee command is like a T-connector for pipes.
tee filename
*********************************************************************************
*********************************************************************************
*********************************************************************************
To create a file and enter text in it:
cat>>text.txt
line1
line2
Ctr+D
or equally
cat <>text.txt
line1
line2
EOF
To append a line at the end of the file:
echo line1 ; echo line2,etc >> file.txt
***********************
Multiple pings Script
for i in $(cat IPADDR.txt); doping ${i}
done
or equally
fping -f IPADDR.txt
***************
SSH to multiple and execute commands at multiple hostst:
ssh server2 'command and arguments' >> /path/to/local_output_file
ssh server3 'command and arguments' >> /path/to/local_output_file
or
for server in $(cat my_servers.txt)
do
ssh $server 'command and arguments' >> /path/to/local_output_file
done
**************
Print Lines
# cat test.csv
1.1.1.1,2.2.2.2 2.2.2.2 443
1.1.1.1,2.2.2.3 2.2.2.3 444
1.1.1.1,2.2.2.4 2.2.2.4 445
1.1.1.1,2.2.2.5 2.2.2.5 446
1.1.1.1,2.2.2.6 2.2.2.6 447
1.1.1.1,2.2.2.7 2.2.2.7 448
# cat test.csv | awk '{print "source "$1" destination "$2" port "$3" end "}'
source 1.1.1.1,2.2.2.2 destination 2.2.2.2 port 443 end
source 1.1.1.1,2.2.2.3 destination 2.2.2.3 port 444 end
source 1.1.1.1,2.2.2.4 destination 2.2.2.4 port 445 end
source 1.1.1.1,2.2.2.5 destination 2.2.2.5 port 446 end
source 1.1.1.1,2.2.2.6 destination 2.2.2.6 port 447 end
source 1.1.1.1,2.2.2.7 destination 2.2.2.7 port 448 end
Join Lines
# cat test.csv
1.1.1.1,2.2.2.2 2.2.2.2 443
1.1.1.1,2.2.2.3 2.2.2.3 444
# for i in "`cat test.csv`" ; do echo ${i} ; done
1.1.1.1,2.2.2.2 2.2.2.2 443 1.1.1.1,2.2.2.3 2.2.2.3 444
# for i in "`cat test.csv`" ; do echo ${i} ; done | awk '{print "source "$1" destination "$2" port "$3" end "$4,$5,$6}'
source 1.1.1.1,2.2.2.2 destination 2.2.2.2 port 443 end 1.1.1.1,2.2.2.3 2.2.2.3 444
New Line
# for i in "`cat test.csv`" ; do echo ${i} ; done | awk '{print "source "$1" destination "$2" port "$3" end "$4,$5"\n"$6}'
source 1.1.1.1,2.2.2.2 destination 2.2.2.2 port 443 end 1.1.1.1,2.2.2.3 2.2.2.3
444
0 Response to "Bash scripting basics"
Post a Comment