Scripting on Linux
Origins of scripting languages Learning scripting languages A simple example of a shell script A script with a loop and debugging Processing a table of data by selecting rows and columns Doing arithmetic internally within the shell Perl, Python, Ruby etc.
Origins of scripting languages
Hardwired programs and embedded OS allowed for very little system control. Job control language JCL allowed IBM mainframe operators to "batch" a set of programs into a routine operation or job, e.g. deducting payments made from accounts outstanding. Job parameters allowed rerun of failed (partially completed) jobs. Benefits: * greater accuracy of job control, * less typing, * ability to restart job from initial data when system goes down part way through. Mainframe features then repeatedly reinvented on smaller and cheaper machines.
Command languages developed
These included TSO/Clist (IBM on-line), PCL (Prime Primos), DCL (DEC VAX/VMS), Aegis (Apollo DomainOS). MS-DOS Batch File (.BAT) etc. Microsoft froze Batch file language in early 1990s. Very early versions of Microsoft Basic were packaged with MS-DOS, while later versions were sold seperately. Unix saw competitive shell development and transfer of ideas between shell languages. * Bourne Shell: /bin/sh * 'C' Shell: /bin/csh * Korn Shell: /bin/ksh * Ash Shell: /bin/ash * GNU Bash Shell (Bourne Again SHell): /bin/sh and /bin/bash Unix shell scripts have all the usual 3rd generation features, including loop constructs, variables, arrays, branches and functions.
Learning scripting languages
Many features shared between shell languages and 'C', Perl, Python etc.
The 80/20 rule: 80% of the benefit derives from knowledge of the 20% most useful subset. Programmers learn by:
* Reading tutorials, books, reference material * Studying code examples * Running modified code examples * Many small experiments * Knowing how to program using a similar type of language
Learning scripting languages 2
Those requiring an in-depth understanding of these languages will need to read the appropriate books and online tutorials and carry out a comprehensive series of programming exercises. In other cases a useable subset of knowledge can be obtained by reading the source code of existing programs and executing these, and by conducting a number of small experiments supplemented with tactical use of the reference information provided with these languages. By this means, your programming knowledge can grow on an as-needed basis.
A simple shell script example
Return values and tests
The Unix shell convention is for a program which exits successfully to end with a return of 0 (which the shell considers as true), and for an error exit to result in a return of 1 or more (which the shell considers false). This is the opposite way round to how this is done within 'C'. The number N returned by a program (e.g. using the bash exit N statement) is different from the standard output of the same program. The shell command: echo $? gives you the return code of the last foreground command. You can also obtain this value within shell scripts using the special variable $? .
A script with a loop and debugging
menu script part 2
A table with rows and columns
Here are some records from the may.logins input file: reboot shutdown usr11361 usr11187 usr11187 usr11187 usr11513 usr11451 usr11456 usr11138 usr12069 usr12069 ~ ~ console console console console console console console console console console Tue May 11 13:14 Tue May 11 13:15 Tue May 11 09:04 - 10:22 (01:18) Mon May 10 18:53 - 20:30 (01:36) Mon May 10 18:50 - 18:53 (00:02) Mon May 10 18:38 - 18:50 (00:12) Mon May 10 15:15 - 16:27 (01:11) Mon May 10 12:11 still logged in Mon May 10 10:53 - 15:14 (04:21) Mon May 10 09:03 - 10:40 (01:36) Sat May 8 11:05 - 09:01 (1+21:55) Sat May 8 11:00 - 11:04 (00:04)
This gives us time logged, one record per login session.
Login analysis script 2
This example combines a number of the features of previous examples, using awk, grep and sed filters to access specific rows from columns, and to exclude unwanted data from the analysis. The input data is a set of login records. This application was used to analyse average usage of 20 workstations during particular months.
Login analysis script 3
Doing arithmetic internally within the shell
Some of the earlier Unix shells didn't have builtin arithmetic operators, so they farmed this job out to external programs such as expr as we saw above. This can be done more quickly using the Bash shell let builtin, as in the following example script:
Advanced scripting: Perl, Python, Ruby etc.
Shell scripts provide glue logic together with other utilities, pipelines and redirection.
Advantage: Very rapid development of systems/network administration and automated operations. Disadvantage: Slow and difficult to develop very large programs, possibly system dependant. Have to load and execute other small programs very many times ? Adding 100 users to system:
* Shell: 1 hour to run (1992), helped us learn how to do it * Perl: 2 minutes to run, we already knew how to do it * 'C' few seconds but 10 times as long to write the program ?
Advanced scripting: Perl, Python, Ruby 2
Fully portable scripting languages: Perl, Tcl, Python and Ruby can be used to handle simple scripting-type applications together with more complex applications, e.g. requiring object-oriented active website development. These languages build upon the features available in Unix shell languages, using almost identical syntax for many purposes, e.g. handling regular expressions.
These languages trade off machine for programmer efficiency, compare against 'C', 'C++' . Can combine benefits by using 'C' modules in scripting interpreter or split application design after profiling to find where cycles are used.
Advanced scripting: Perl, Python, Ruby 3
The login analysis program described above was rewritten in Perl.
Advanced scripting: Perl, Python, Ruby 4
This program runs much faster than the shell script, because everything is done inside the same process. There are many syntactic similarities, but Perl borrows array and loop notation from 'C' and some other notation from grep, awk and sed. As in Bash, $ is used to introduce single (scalar) variables and @ is used for arrays. $_ is used for a default scalar variable, so that the substitution operation e.g: s/\)//; which strips a closing round bracket from a string, doesn't need to specify which string it operates upon.