Perl and the OS by odn41067

VIEWS: 51 PAGES: 32

									                                                                  Perl and the OS
                                                                        Remember all those Perl built in functions
 More Perl                                                              shown earlier –
                                                                            Access to file and directory information
                                                                            Process management
                                                                            Ports and Sockets


                                                                        Perl scripts exploiting these functions are
                                                                        often used to automate repetitive tasks for
                                                                        the “system’s administrator”.




                                                                  Information on files and directories …

                                                                        File (and subdirectory) names in a
 Perl : files and directories                                           directory
                                                                        Properties of an individual file




                                                               Inevitably some system dependent features – Unix files and Windows files do
                                                               have different properties.




Reading a directory                                               Basic properties of a file
 Directory handle                                                       Perl has “file tests” similar to those that
   Obtained using opendir                                               exist in shell.
   Read via readdir
                                                                        Used as
 Returns list of names of all entries in directory
                                                                            <test operator> filename
   Subdirectories
   Links                                                                    Mostly return true/false result
   Files, including hidden files whose names start with “.”
   References to current and parent directory (“.” and “..”)




                                                                                                                                             1
File tests                                                  Example: accessing files …
     File test operators                                        Input: fully qualified path name of
        -x      is file (or directory) “executable”?
                                                                directory of interest
        -r      is it readable?
        -w      is it writeable?                                Output: a listing of directory contents,
        -d      is it a directory?                                Directories and links: just identify by type
        -l      is it a link?
                                                                  Files: highlight those that are executable;
        -f      just a plain old file maybe?
                                                                  those that are “text” should have length
        -T      is it “text”?
        -e      does it even exist?
                                                                  given in report.
        -s      size in bytes
        -M      days (as real number) since last modified
        …




Osutil1.pl                                                  Osutil1.pl
 #!/share/bin/perl                                          while(1) {
                                                             print "Enter pathname of directory : ";
 while(1) {
                                                             $directory = <STDIN>;
   print "Enter pathname of directory : ";
   $directory = <STDIN>;                                     chomp($directory);
   …                                                         if($directory =~ /^Quit$/i) { last; }
     #get list of names of contents of directory             if(!opendir(DIRHANDLE, $directory) ) {
     …                                                             print "Couldn't open that directory\n";
     foreach $name (@names) {                                      next;
         …                                                          }
         }
                                                                $directory =~ s#/$##;
     }
 }                                                              @names = readdir DIRHANDLE;




Osutil1.pl                                                  Osutil1.pl
 opendir(DIRHANDLE, $directory);                                @names = readdir DIRHANDLE;
   First argument is a Directory Handle (like file
     handles, directory handles are usually given names         foreach $name (@names) {
     consisting entirely of capital letters)                        if($name =~ /^\.+$/) { next; }
     Second argument is pathname of directory                       $fullname = $directory . "/" . $name;
 $directory =~ s#/$##;                                              …
   Simply tidying up for later. Some people enter               }
   directory names with a trailing /, some don’t. This
   removes a trailing slash if one is there so that later
   formatting can be done more consistently.                }




                                                                                                                 2
                                                               Of course, there is another
Osutil1.pl                                                     way …
    if( -d $fullname) { print "Subdirectory: $fullname\n"; }
                                                                  Actually, there are two other ways of
    elsif( -l $fullname) { print "$fullname is a link\n"; }
    else {                                                        getting lists of files in directories
          print "$fullname\n";                                    And you can use “stat” function on files
          if( -x $fullname) { print "\texecutable\n"; }
          if( -T $fullname) {                                     to get lots and lots of extraneous
                   $size = -s $fullname;                          information about a file.
                   print "\tText with $size bytes\n";
                   }
    }




globbing                                                       globbing
    Use simple shell style pattern to specify                     Differs from readdir in that get back
    entry names of interest                                       fully qualified file names
        *        everything
        *.pl     all perl scripts                                 Shell is used to do pattern matching to
        [AB]*.cc         cc programs whose names start            select the files that you want (note *
        with A or with B (note * means something quite            doesn’t return files beginning with “.”)
        different here than in regular expression)
    Invoke either by
        <>
        glob




Osutil2.pl                                                     Osutil2.pl
                                                               while(1) {
#!/share/bin/perl                                                …
while(1) {
                                                                 foreach (<*>) {
  print "Enter pathname of directory : ";
                                                                     if( -d ) { print "Subdirectory: $_\n"; }
  $directory = <STDIN>;
                                                                     elsif( -l ) { print "$_ is a link\n"; }
  chomp($directory);
                                                                     else {
  if($directory =~ /^Quit$/i) { last; }
                                                                              print "$_\n";
  unless(chdir( $directory) ) {
                                                                              if( -x) { print "\texecutable\n"; }
       print "Couldn't change to that directory\n";
                                                                              if( -T) {
       next;
                                                                                       $size = -s;
       }
                                                                                       print "\tText with $size bytes\n";
    …
                                                                                       }
    }
}
                                                                     }
                                                                 }




                                                                                                                            3
Caution …                                                   Other file operations
  Code using implicit unnamed variable ($_)                   Perl core has functions:
  can be cutely concise                                         rename
(<*>)
                                                                unlink
if( -d )
$size = -s;                                                     chmod
  But remember                                                  mkdir
      Code is generally “WORM”y                                 rmdir
         Write once                                           Mostly take filename (file in current working
         Read many
                                                              directory or file with fully qualified pathname)
  Too many linguistic tricks will result in
  problems for readers




                                                            system(), fork(), exec()
                                                              Since Perl interpreter is a big C program,
 Perl : processes                                             naturally have access to the C functions in
                                                              stdlib.h and unistd.h
                                                                system
                                                                   Argument is string with command(s) interpreted by sh
                                                                   (or passed directly to execvp)
                                                                   Subprocess shares stdin, stdout, stderr with perl program
                                                                   that launched it
                                                                   Perl process waits for subprocess to terminate
                                                                   On return $? holds information about termination
                                                                       $? >> 8           exit code




… fork,                                                     After a fork
  fork()                                                      Child process goes and does its stuff
      The usual (almost)                                      Parent can continue, or can wait for
         Parent process (that performed fork()) gets          child to terminate
         process id of child
                                                                Usual problem with “zombie” processes if
         Child gets a zero value as its return value from
         fork                                                   don’t wait for child terminations
         If attempt to fork fails (process table full or        Can play with signal mechanisms if this is a
         similar problem) then instead of negative value,       problem
         get “undefined” returned




                                                                                                                               4
… exec                                                              Using other processes
   The usual                                                          Fork – exec
      Replace code segment of current process                           Well, if you want
      with executable specified in exec call                          System
      Other arguments build up argv argument                            Commonly used to run small subtasks
      vector, and if desired the envp                                 Backticks
      environment vector                                                Like “system”, but a better way of collecting the
                                                                        output from a subprocess so that it is available in
                                                                        to the Perl program that launched the subtask




System and backticks                                                Example: getting system data
   system(“date”)                                                     Your organization
      Invokes date operation in shell, which prints to
      stdout shared with Perl process                                   Uses Sun’s NIS+ system to store data like
   $date=`date`;                                                        users and hosts
   $str = “Today is $date”;                                             Has its machines named:
      Invokes date operation in shell, grabbing the                        Machine.dept.company_URL
      output and in this case interpolating it into a string
      (Just an example; this isn’t the way to do it for               You want a printout
      real as perl has its own functions to get at dates.)              Grouped by department
      Note, you can’t embed a backtick operation
      directly into a string.                                           Lists machines and their IP addresses




Data is available from niscat                                       Hosts.pl
 niscat hosts.org_dir                                                 Program needs to grab output from
 Returns list in format:                                              niscat command
Canonical_host_name alias IP comments                                   Split the line to get name, alias, ip
                                                                        Examine the name for machine and
 Example                                                                department fields (things like “localhost”
red.accounting.ourorg.com red.accounting.ourorg.com 209.208.207.1
red.accounting.ourorg.com red 209.208.207.1                             will also appear, and should be ignored)
blue.accounting.ourog.com blue 209.208.207.2                            Stores unique [dept, name, ip]
   Machine may appear several times with                                combinations
   different aliases                                                  Prints report of sorted data




                                                                                                                              5
Need some data structures!                                   Hosts.pl
  Data structures and references not part of                 #!/share/bin/perl
  this introductory Perl component.
                                                             system("echo $LOGNAME");
  perldoc perldsc                                            $date = `date`;
    Examples of lists of lists, lists of hashes, hashes of   $str = "Today is $date";
    list, and what is needed here a hash of hashes           print $str , "\n";
  Hash 1
                                                             foreach (`niscat hosts.org_dir`) {
    Indexed by department name                                  …
    Value is Hash 2                                          }
  Hash 2
                                                             foreach $dept (keys %machines) {
    Indexed by machine name
                                                                    …
    Value is IP address                                      }




Hosts.pl                                                     Hosts.pl
  System call; executed while Perl waits                     foreach (`niscat hosts.org_dir`) {
system("echo $LOGNAME");                                       ($name, $alias, $ip ) = split / /;
                                                               if($name =~ /^(\w+)\.(\w+)/) {
  Using backticks to grab output …                                 $machine = $1;
$date = `date`;
                                                                   $dept = $2;
                                                                   $machines{$dept}{$machine} = $ip;
foreach (`niscat hosts.org_dir`) {
                                                                   }
  …
                                                             }
}




Hosts.pl
foreach $dept (keys %machines) {
  print "$dept\n";                                            Example: processes etc
  foreach $machine
          (keys %{ $machines{$dept} } ) {
      $ip = $machines{$dept}{$machine};
      printf "\t%-20s\t%16s\n" ,$machine, $ip;
  }
}




                                                                                                       6
                                                                    Perl script for processing
Examples                                                            assignments …
 Change all occurrences of “cat” to “dog” (dogastrophe)               This example meant to illustrate a more
                                                                      realistic use of Perl for a tedious
  Use system call to run “date” piping output to lp                   system’s administration task.
                                                                      “Unix shell” (or Python) scripts would
 Find all words in the dictionary that contain all the vowels         be as good – the Perl is (in my opinion)
                                                                      a little easier to understand than a “sh”
                                                                      script.
Uhm, why can’t we have REAL examples




The task …                                                          Task …
  Process assignments submitted for a lab class.
                                                                      Typical problems
     Assignment submission system creates a directory
     for each student who submitted work for                            Students submitting incorrect files
     assessment;                                                        Students who submit code that results in 1000
     this directory holds the file(s) submitted by the                  lines of compiler error messages
     student.                                                           Students submitting programs that handle I/O
     Student directories grouped in “lab class”                         tasks incorrectly; example:
     directories.                                                          program is supposed to read from a file
     Tutors need printouts for students in their labs:                     Instead program “prompts for input” and waits (for
        Listing of code                                                    ever?)
        Report on test compilation                                      Programs that loop and generates 10000 lines of
        Report on test run                                              output
        Output to be clearly labeled for each student in lab.




Perl script                                                         Specific example task …
    Part is general purpose, used for any                             Students writing Java
    assignment                                                          File A.java provided by lecturer, possibly modified
       Get tutor name, lab name (to identify printout)                  by student
       Switch to correct lab directory                                  File B.java, written by student, contains main() for
       For each student subdirectory in lab directory,                  Java program
       process submission                                               Other Java files (depending on student’s design of
                                                                        Java program)
    Part is specific to requirements of an
                                                                        README.txt documentation file
    individual assignment
       This assignment requires a Java program to be compiled and       All files to be submitted as a single “tarred and
       then run reading input from file …                               gzipped” file A1.tar.gz
       This assignment has a C++ main.cc and a Graph(.h,.cc)
       files, to be compiled with options … and run …




                                                                                                                                7
Specific Example task …                                              Specific Example task …
  Processing of one submission:                                        Processing of one submission:
    Unpack files                                                         If compiled successfully
    If student included an A.java that is unchanged                         Run the program, with a time limit of 5 seconds to catch
    from that supplied by lecturer, remove it.                              those infinite loops
                                                                            Program requires a single command line parameter, and
    List README.txt and .java files                                         no “stdin” input.
    Copy default A.class file                                               Send program output to file.
    Compile the java code                                                Print last 100 lines of output file resulting from
       Limit output to the last 100 lines of any compilation error       test run.
       report
                                                                         Remove all files created; restore original tar gzip
    If compiled successfully                                             file
       Run the program, with a time limit of 5 seconds to catch          Make sure that all error messages (to stderr) are
       those infinite loops
                                                                         merged with stdout at all stages of processing.




Specific Example task …                                              Lablist.pl
  General                                                              Main program
    Start printout with a banner page with tutor’s                       Input of tutor name and directory name
    name                                                                 Change to that directory
    List identifiers of all students who submitted in                       Holds submissions for a lab class (one subdirectory per
    that lab                                                                student)
    List student output (output and error output from                       “Control” subdirectory used by electronic submission
                                                                            system
    steps just described) with identification
                                                                            Files: A.java, A.class
    information
                                                                         For each entry in directory
    Send to console a brief indication of progress as
                                                                          if is a subdirectory with student submission then
    script runs together with identification of any
                                                                                   process that submission
    students whose submitted files appear corrupt




Lablist.pl                                                           Lablist.pl
  Main line                                                            system() call used to launch subtasks
  Two subroutines                                                        Unpack student files
    “valid”                                                              Compare student’s A.java with default; maybe
                                                                         delete file
       identifies student subdirectories (must be a
       directory not a file, must not be called                          List to java and other files
       “control”)                                                        Compile student’s B.java (and rest of java code)
    processsubmission                                                    Run student’s code
       Performs processing steps for individual                        Output from script must merge correctly with
       submission
                                                                       output from system calls.




                                                                                                                                       8
                                                               Lablist.pl: main line, command
Lablist.pl                                                     line arguments
 #!/share/bin/perl -w
                                                                $tutor = $ARGV[0];
 sub processsubmission { … }                                    $dirname = $ARGV[1];
                                                                if(! ((defined $dirname) && (defined $tutor))) {
 sub valid { … }                                                    print STDERR <<MSG;
                                                                Tutor:
 $tutor = $ARGV[0];                                             This script requires two command line arguments:
 $dirname = $ARGV[1];                                           1) your name as a banner for printout
 …                                                              2) the directory with the submissions for your lab.
 …                                                              MSG
 foreach (@valid) {                                                 exit 1;
    …                                                           }
 }




Lablist.pl : change to lab directory,
prepare for output, …                                          More Perl magic
 …                                                              $| = 1;
                                                                  Switches current default output
 if(!chdir($dirname)) {
                                                                  (STDOUT) to autoflush mode
     print STDERR "Couldn't cd to the directory $dirname\n";
     exit 1;                                                         Output buffers flushed after each write
 }                                                                Script will print header line, then invoke system();
                                                                  want header line to appear before output from
 $| = 1;
                                                                  system()!
 system("banner $tutor");
                                                                  Recent Perl implementations would do a flush before
 print "\f";                                                      an exec call (like system) but older interpreters don’t.




Lablist.pl : check
subdirectories                                                 Lablist.pl
 opendir(CWD, ".");                                             sub valid {
 @entries = readdir(CWD);
                                                                  $name = $_[0];
 closedir(CWD);
                                                                  if($name =~ /^\.+$/) { return 0; }
 @entries = sort @entries;                                        if($name eq "control") { return 0; }
                                                                  if(-d $name) { return 1; }
 print "Submissions for your lab received from:\n";
 @valid = ();                                                     return 0;
 foreach (@entries) {                                           }
    if(valid($_)) {
         print $_, "\n";
         push(@valid, $_);
         }
 }




                                                                                                                             9
Lablist.pl                                                  Lablist.pl: process submission
 foreach (@valid) {                                          sub processsubmission {
   printf "\fStart of record for $_\n";                        #Prepare files
   print STDERR "$_\n";                                        #Deal with A.java
   chdir($_) ||                                                #Print required files
        die "Failed to cd into student's subdirectory\n";      #Try compile
     processsubmission;                                        #Try run
     chdir("..");                                              #Tidy remaining files
 }                                                           }




Lablist.pl : prepare files                                  Lablist.pl: Deal with A.java
sub processsubmission {                                     sub processsubmission {
  system("gunzip A1.tar.gz");                                 …
  $errcode = $? >> 8;                                         #Remove any .class files
  if($errcode) {                                              system("rm -f *.class");
       print "Corrupted gzip file?\n";                        system("ls -l");
       print STDERR "$_ has corrupt gzip file\n";             #has student submitted an A.java
       return;                                                if(-r "A.java") {
  }                                                                 # if same as that supplied, remove it
  system("tar -xf A1.tar");                                         system("diff A.java ../A.java 1>/dev/null 2>/dev/null");
  if($errcode) {                                                    $code = $? >> 8;
       print "Corrupted tar file?\n";                               if($code == 0) {
       print STDERR "$_ has corrupt tar file\n";                           system("rm A.java");
       return;                                                             }
  }                                                           }
 …                                                            system("cp ../A.class .");




Lablist.pl : print files                                    Lablist.pl : compile Java
 sub processsubmission {                                     sub processsubmission {
   …                                                           …

   system("cp ../A.class .");
                                                                system("javac B.java 2>&1 > compile_errs");
   @files = <*.{java,txt}>;
   foreach $file (@files) {                                     $compile_error = $? >> 8;
       print "\f$file:\n";                                      if($compile_error) {
       system("cat $file");                                          system("tail -100 compile_errs");
                                                                     print "\fAssignment not run because of compilation errors\n";
   }
                                                                     }
                                                                else {
                                                                     …
                                                                }




                                                                                                                                     10
Lablist.pl : Run Java                      Lablist.pl : Tidy up
sub processsubmission {                     sub processsubmission {
   …                                          …
   if($compile_error) { … }
   else {                                       system("rm -f *.java *.class *.txt compile_errs output");
        $command = '
                                                system("gzip A1.tar");
ulimit -t 5
                                            }
java B 17 2>&1 < /dev/null > output
tail -100 output
';
        system($command);
   }




                                           use Socket;
                                                Perl core does contain a number of the Unix
 Perl : network stuff                           standard network functions
                                                   socket(), gethostbyname(), …
                                                Problem
                                                   Many of these functions require obscure constants
                                                   (e.g. integers representing different protocols)
                                                   and/or data structures
                                                Solution
                                                   The constants, structures, functions etc are
                                                   defined in the Socket module.




Use a module …                             Network example 1: http client

  More on modules shortly …                     A minimal http client!
  Using a module –
     Like #include for C/C++                       Sends a “GET” request for a specified file
     Like import for Java                          Prints all the response that it receives
  Program starts with “use” directives.            Command line arguments identify host and
                                                   port number.
  Then can utilize functions declared in
  that module.




                                                                                                            11
Client.pl                                        Client.pl
 #!/share/bin/perl -w                             if(@ARGV < 1) {
                                                     print "Invoke with one or two arguments,
 use Socket;                                         hostname and optional port\n";
                                                     exit(1);
 # get arguments with host/port
 # build “address” data structures                }
 # open socket connection
 # set I/O options on socket connection           $server = shift @ARGV;
 # get user to enter file name                    $port = shift @ARGV || 80 ;
 # send “Get” request
 # loop printing all lines of response




Client.pl                                        Client.pl
 $iaddr = inet_aton($server)                      connect(SOCK, $paddr)       || die "connect: $!\n";
    || die "no host: $server\n";
 $paddr = sockaddr_in($port, $iaddr);             select(SOCK);
                                                  $|=1;
 $proto = getprotobyname('tcp');                  select(STDOUT);
 socket(SOCK, PF_INET, SOCK_STREAM, $proto)
    || die "socket: $!\n";                        print "File :";
 connect(SOCK, $paddr) || die "connect: $!\n";    $input = <STDIN>;
                                                  chomp($input);




Client.pl                                        Network example 2: “Analyze” a web log

 print SOCK "GET $input HTTP/1.0\n\n";              A minimal analyzer
 while($line = <SOCK>) {                              Read access log
      print $line;                                    Extract client IP addresses, forming list of unique
 }                                                    IP addresses
                                                      Lookup hostnames corresponding to IP addresses
 close (SOCK)           || die "close: $!\n";         Strip machine name to get domain
 exit;                                                Form list of unique domains identifying our clients
                                                      Print sorted list




                                                                                                            12
Input data                                                       Web.pl
    Access log with entries like:                                 #!/share/bin/perl
210.84.124.193 - - [20/Jun/2001:00:30:50 +1000] "GET /subjects    use Socket;
  HTTP/1.1" 301 253
203.132.226.245 - - [20/Jun/2001:00:36:22 +1000] "GET
                                                                  #read file, saving unique client IPs
  /images/staff.gif HTTP/1.0" 304 –
                                                                  #loop using “get host by address” to try to get host name
    Take client address:                                          # strip machine, and keep list of unique domains
       203.132.226.245
                                                                  #sort and print
       P53-max4.wgg.ihug.com.au
    Machine not significant, so simplify to
    wgg.ihug.com.au




Web.pl                                                           Web.pl
while(<STDIN>) {                                                  print "Sorting and reporting\n";
  @data = split;
                                                                  foreach (sort (keys %names)) {
  $callers{$data[0]} = 1;
}                                                                   print "$_\n";
                                                                  }
foreach (keys %callers) {
                                                                  $val = scalar keys %names;
   $addr = inet_aton($_);
    $name = gethostbyaddr($addr, AF_INET);
                                                                  print "Our clients were from $val different domains\n";
    if($name eq "") { next; }

    $pos = index($name, ".");
    $name = substr($name,$pos+1);
    $names{$name} = 1;
}




                                                                 Whose is bigger? Java’s or Perl’s?
                                                                    “Class library” that is.
 Whose is bigger?
                                                                    You “sort of” know about Java’s packages.
                                                                    Perl also makes a substantial effort.
                                                                       Not a “class library” – a collection of modules.
                                                                       www.cpan.org/modules
                                                                       There are hundreds




                                                                                                                              13
  Modules …                                                And more modules
     Language extensions and documentation tools             File utilities
     Experimental thread support                             Text manipulation
     Utilities (perl programs for file and directory         Natural language helpers (spell check, word
     manipulations)                                          stemming, hyphenation, …)
     Networking: Socket wrappers, ICQ, Gnutella, LDAP,       XML and XSLT
     Japper, SMTP, SSL, .…                                   Encryption, digests, etc
     Data types and algorithms                               WWW, CGI, and HTTP
     Database interfaces                                     MS-Windows stuff
     User interfaces (including TCL/TK, Xwindows, …          Junk, …




  Modules                                                  Our interest in modules …
      Modules have to be “installed” into your               Databases
      Perl environment                                       WWW, CGI stuff
         e.g. PPM tool
            Download zip file with module you want (from
            www.cpan.org or www.activerperl.com)
            Use ppm to install




  Databases                                                DBI and DBD
      Perl’s approach to using database is                   DBI “Data base interface”
      very similar to that you learnt with                   DBD “Data base driver”
      Java.                                                    Many DBD variants, one (or more) for each
         High level API providing a uniform                    make of database
         mechanism for submitting SQL queries                     DBD-Adabas, DBD-Excel (yes can treat Excel
         Low level, database specific drivers that                spreadsheet as a database), DBD-Ingres, DBD-
                                                                  Informix, DBD-Interbase, DBD-ODBC, DBD-
         accept requests via high level interface and
                                                                  Oracle, …
         invoke operations on the actual database

www.oreilly.com/catalog/perldbi/ch04.html




                                                                                                                 14
 DBD                                                       What drivers do you have?
                                                           use DBI;
     Fortunately, average Perl programmer                  @drivers = DBI->available_drivers;
     does not have to bother much about                    print “Available drivers : @drivers\n”;
     the DBD level.                                          On my Windows PC, I got things like “Proxy”,
        Find what drivers you have installed                 “ADO”, and “ODBC”
                                                             On Unix, I got things like “Proxy”, and
        Install additional drivers as needed.                “Oracle”
                                                             Useful ones were ODBC and Oracle
                                                                 I had to install the DBD-ODBC component on my
                                                                 PC, it doesn’t come by default with PC-Perl.




 What data sources do you have?                            Datasources
     OK, you have a ODBC on your PC, or an
     Oracle driver on Unix.
                                                             On PC, I got response
                                                                 DBI:ODBC:Visual FoxPro Database
     What databases can you use these to
                                                                 DBI:ODBC:dBase Files
     access?
                                                                 ….
   use DBI;
   print “Enter driver name : “;                                 DBI:ODBC:epalfinder
   $drivername = <STDIN>;                                           This is the meaningful one, it is the entry that I
                                                                    had created (Control Panel/ODBC Data
   chomp($drivername);                                              sources) for a Microsoft Access database with
   @sources = DBI->data_sources($drivername);                       my data table
   print “sources: @sources\n”;




 Datasources                                               Datasources
     On Unix I got                                           Typically, you know the name of the
        List of various Oracle databases on                  datasource that your program uses.
        systems in uni                                       It gets coded as a constant string in
           dbi:Oracle:CSCI.CS.UOW.EDU.AU
                                                             your program.
           This was the important one – it is the Oracle
           system used for database assignments
                                                             You rarely need to use those DBI calls
Note: current Oracle for student assignments is
                                                             in actual programs.
dbi:Oracle:CSCI




                                                                                                                         15
Use of datasource                                         Connecting to a datasource
                                                            Get “database handle”
  Similar to your previous use of JDBC                         Provide “data source” name string
                                                               If necessary, provide user-name and password
  You connect to a datasource and get a                        Preferably, also supply attributes (e.g. specify that
  “database handle” (Java’s Connection object)                 want “autocommit” of operations)
  You use your database handle to prepare an                DBI->connect(…)
  SQL statement for execution.
                                                               Returns a “scalar”, the database handle (or fails)
  You run your query, getting in return an                $dbh = DBI->connect(“DBI:ODBC:epalfinder”);
  “object” that you can use to get each row of            $dbh = DBI->connect(
  a response (for select queries etc)                       “dbi:Oracle:CSCI.CS.UOW.EDU.AU”,”HSimpson”, “Doh”);
                                                          $dbh = DBI->connect(“dbi:Oracle:CSCI.CS.UOW.EDU.AU”,
                                                            “Hsimpson”, “Doh”, { AutoCommit => 1} );




Preparing statements                                      More statements:
  Simple examples:                                           Get subset of stuff from Table1,
    Get everything in Table1                                 depending on user input
  $sth = $dbh->prepare(
                                                              print “Enter ‘Grade’ cutoff : “;
      “SELECT * FROM Table1”);
  …                                                           $grade = <STDIN>;
  $sth->execute;                                              chomp($grade);
    Get subset of stuff from Table1                           $sth = $dbh->prepare(
  $sth = $dbh->prepare(                                           “SELECT * FROM Table1 WHERE GRADE > $grade”);
      “SELECT * FROM Table1 WHERE GRADE > 3”);                …
  …
                                                              $sth->execute;
  $sth->execute;




Still more statements                                     Still more statements …
  Advice is to “prepare statements” with                      $sth = $dbh->prepare(
                                                                  “SELECT GRADE FROM Table1 WHERE NAME=‘Smith’”);
  placeholders for arguments, rather than build
  string for statement each time.                            As usual, need single quotes around string
  $sth = $dbh->prepare(                                      data in an SQL statement.
      “SELECT * FROM Table1 WHERE GRADE >   ?”);              print “Enter name : “;
  …                                                           $name= <STDIN>;
  …                                                           chomp($name);
  print “Enter ‘Grade’ cutoff : “;                            $sth = $dbh->prepare(
  $grade = <STDIN>;                                               “SELECT * FROM Table1 WHERE name= ‘$name’”);
  chomp($grade);                                                Oops, suppose the name is O’Brien!
  $sth->execute($grade);                                        Don’t have problems if use alternative with “?”
    Partly, this is for efficiency; but there are other         placeholders (which is another reason for using them)
    reasons …




                                                                                                                        16
Statements to insert, update, and
delete data …                                         Statement preparation
$sth = $dbh->prepare(
      “INSERT INTO TABLE1 VALUES (?, ?, ?)”);
                                                        If SQL is faulty, then prepare will fail,
$sth = $dbh->prepare(                                   get “undef” as result.
      “UPDATE TABLE SET GRADE= ? WHERE NAME = ?”);    $sth = $dbh->prepare(…);
$sth = $dbh->prepare(                                 unless defined $sth die “SQL problem?”;
      “DELETE FROM TABLE1 WHERE NAME=‘Smith’”);




Executing statements                                  Executing statements
   Inserts, Updates, Deletes                            Getting selected data
                                                           Fetch successive rows of a result table
      Succeed or fail                                      @row = $sth->fetchrow_array;
                                                           Returns undef when have processed all data.
      Can check
                                                        Typical code:
      $sth->execute($data) || die “delete failed”;
                                                      $sth = $dbh->prepare(“select … = ?“);
      Can get more information from DBI::errstr.      …
        … die “Delete failed because $DBI::errstr”;   $sth->execute($data1, $data2);
                                                      while(@row = $sth->fetchrow_array) {
                                                        …
                                                      }




Close your database handle                            What are the -> things?
   It will be closed when your program                  Objects in Perl of course.
   terminates.                                             Invoke method of object
   But best if explicitly close connection as           Mostly beyond the scope of our
   soon as possible                                     introductory treatment
                                                        Easy to use for simple standard
   $db->close;
                                                        database operations.

                                                        So, don’t worry about them!
 er … shouldn’t that be disconnect




                                                                                                         17
My usual DB test                                  Connect to db and run
                                                  use DBI;
  Table “Teams”                                   $dbh = DBI->connect("dbi:Oracle:CSCI",
create table Teams (                                   “homer","doh") || die "It didn't connect";

     Team1 varchar(32),                           print "Connected\n";
                                                  $searchhandle = $dbh->prepare("select * from teams");
     Team2 varchar(32),
     Score1 number,                               $searchhandle->execute || die "Select request failed because
                                                        $DBI::errstr";
     Score2 number
                                                  while(@row=$searchhandle->fetchrow_array) {
);                                                      foreach $item (@row) { print "$item "; }
insert into teams values (‘norths’,                     print “\n“;
                                                  }
  ‘souths’, 1, 0);




It should run

                                                   Database example
$ perl testdb.pl
Connected
norths souths 2 0
Easts Wests 3 2
City Country 2 2
River Forest 0 0
$




e-pal example                                     e-pal
  Simple service for finding email- pen-             Database records
                                                         Your email identifer (unchecked character string of
  friends with similar interests                         32 characters)
  Database                                               Your “gender” (male, female, “eperson” – if you
                                                         don’t want to specify gender)
    Insert a record that describes you and your
                                                         “Gender” of desired correspondents (male, female,
    interests                                            eperson, any)
    Search against existing records to find              Interest1, interest2, interest3, interest4, interest5
    someone with common interests                            Integer values
                                                             Index numbers of interests in a predefined array of
                                                             approximately 100 topics – Aerobics, Archery, Ballet,
                                                             Cats, Cars, Drinks, …, Yoga, Zen




                                                                                                                     18
e-pal                                                       e-pal
    Unique key                email                           Match criteria are difficult to specify in SQL
    To add data                                                 “any” matches “male”, “female”, or “eperson”
      Submit email, gender, wanted gender, five                 Interests represented as 5 integers in random
      interests picked from list                                order
    To search                                                 So don’t really exploit database capabilities!
      Submit your gender, wanted gender, five interests
                                                                Retrieve complete records
      picked from list
         Match must satisfy you wanted gender                   Examine records procedurally in program code
         You must satisfy match’s desired gender
         You must have some common interests.




e-pal : program structure                                   e-pal : program structure
  Initialization:
     Create lists and hashes with predefined interests        Handle search results
     Open database connection                                   For each retrieved record, match against
     Prepare a statement that will be used for insertion        specified requirements; print emails of
  DoAdd                                                         those who match
     Prompt for email, gender, desired gender, list of        Main
     interests (each part handled by subroutine)                Initialize
     Run parameterized insertion request
                                                                While not quitted
  DoSearch                                                          Ask user for add or search request
     Prompt for gender, desired gender, list of interests           DoAdd or DoSearch
     Run search request




The epal table                                              Epal code
CREATE TABLE EPAL
   (email      VARCHAR(32)      NOT NULL,                     General:
   type        varchar(8)       NOT NULL,
   want        varchar(8)       NOT NULL,                       use DBI
   interest1 number(4),                                             “include” function definitions for database
   interest2 number(4),                                         use strict
   interest3 number(4),
   interest4 number(4),
                                                                    Really an interpreter switch.
   interest5 number(4),                                             It checks that all variables are properly
   CONSTRAINT id_pkey PRIMARY KEY(email),                           declared and scoped (this is a slightly larger
   CONSTRAINT type_check CHECK (type in ('MALE',
                                                                    program, several subroutines, tendency to use similar
   'FEMALE', 'EPERSON')),
                                                                    names “interest”, “want” etc in different subroutines –
                                                                    don’t want any confusions!)
   CONSTRAINT want_check CHECK (want in ('MALE',
   'FEMALE', 'EPERSON', 'ANY'))
);




                                                                                                                              19
Epal code                                                         epal
                                                                  #!/share/bin/perl -w
    Structure                                                     use strict;
                                                                  use DBI;
       “uses”
                                                                  my $data_source = "dbi:Oracle:CSCI";
       Declaration of filescope “my” variables,                   …
       some initialized at declaration                            sub initialize { … }
       Functions                                                  sub owntype { … }
                                                                  sub getinterests { … }
       main                                                       sub wanttype { … }
                                                                  sub doSearch { … }
                                                                  sub doAdd { … }
                                                                  initialize;
                                                                  while(1) { … }




Epal filescope “my” variables                                     Epal : “main()”
                                                                   initialize;
 my $data_source = "dbi:Oracle:CSCI";
 my $dbh;                                                          while(1) {
 my $searchHandle;                                                   my $cmd;
                                                                     print "Enter command (add,search, list (interests), quit): ";
 my $insertHandle;
                                                                     $cmd = <STDIN>;
 my %interesttable;
                                                                     if($cmd =~ /quit/i) { last; }
 my @interestlist = ( "Abseiling", "Aerobics", …
                                                                     elsif($cmd =~ /search/i) { doSearch; }
   …                                                                 elsif($cmd =~ /add/i) { doAdd; }
  "WebDesign", "WineTasting", "Yoga", "Zen"                          elsif($cmd =~ /list/i) {
   );                                                                     print "Interest list @interestlist\n"; }
                                                                     else { print "Command not recognized\n"; }
                                                                   }




Epal : initialize                                                 Epal : doAdd
sub initialize {
  my ($id, $interest);                                             sub doAdd {
  $id = 0;                                                           my ($you, $desire, $email, @interests);
  foreach $interest (@interestlist) {                                $you = owntype;
        $interesttable{$interest} = $id; $id++;                      $desire = wanttype;
  }                                                                  @interests = getinterests;
  $dbh = DBI->connect($data_source,
        “HSimpson", “Doh", { AutoCommit => 1})                         print "Your email address : ";
         || die "Couldn't connect to db\n";                            $email = <STDIN>;
  $searchHandle = $dbh->prepare(                                       chomp($email);
                "SELECT * FROM epal");                                 $insertHandle->execute(
  $insertHandle = $dbh->prepare(                                            $email, $you, $desire, @interests)
        "INSERT INTO epal VALUES ( ?, ?, ?, ?, ?, ?, ?, ? ) ");             || die "Failed to insert record because $DBI::errstr";
}                                                                  }




                                                                                                                                     20
Epal : doSearch : 1 :                        SQL request         Epal : doSearch : 2 :                      check results
                                                                 sub doSearch {
 sub doSearch {                                                    …
   my ($you, $desire, @interests, @row);                           $score = 0;
   my ($theiremail, $theirtype, $theirdesire, @theirinterest);     while(@row = $searchHandle->fetchrow_array) {
   my $score;                                                           ( $theiremail, $theirtype, $theirdesire, @theirinterest)
   $you = owntype;                                                                      = @row;
   $desire = wanttype;                                                  …
   @interests = getinterests;                                           print "Contact mail $theiremail:\tCommon interests: ";
                                                                        foreach (@common) {
     $searchHandle->execute                                                     print "\t$interestlist[$_]";
         || die "Select request failed because $DBI::errstr";                   }
     …                                                                  print "\n";
 }                                                                 }
                                                                   unless ($score > 0) { print "Sorry, we currently don't have any
                                                                   contacts for you\n"; }
                                                                 }




Epal : doSearch : 3 :                        matching            Epal: minor subroutines
while(@row = $searchHandle->fetchrow_array) {
                                                                  sub owntype {
  ( $theiremail, $theirtype, $theirdesire, @theirinterest)
                                                                   my $type;
               = @row;
                                                                   my @allowed;
  next unless (
                                                                   while(1) {
       ($desire eq "ANY") || ($desire eq $theirtype));
                                                                     print "Your type : ";
  next unless (
                                                                     $type = <STDIN>;
       ($theirdesire eq "ANY") || ($theirdesire eq $you));
                                                                     chomp $type;
  my @common = (); my $interest;
                                                                     @allowed = qw( MALE FEMALE EPERSON);
  foreach $interest (@interests) {
                                                                     if(grep { /$type/i } @allowed) { last; }
       push(@common, $interest)
                                                                     print "Unrecognized type; allowed values are @allowed\n";
               if (grep {$interest == $_ } @theirinterest);
                                                                   }
       }
                                                                   return uc $type;
  next if ((scalar @common) == 0);
                                                                  }
  $score++;
  print "Contact mail $theiremail:\tCommon interests: ";




Epal: minor subroutines                                          Epal: minor subroutines
 sub getinterests {                                               sub getinterests {
  my @list; my @nlist; my $temp;                                   …
  while(1) {                                                       while(1) {
    print "Enter five personal interests : ";                       …
    $temp = <STDIN>;                                                $temp = 1;
    @list = split /\s/, $temp;                                      my $item;
    my $length;                                                     foreach $item (@list) {
    $length = @list;                                                     unless (exists $interesttable{$item}) {
    if($length != 5) { print "5 interests required\n"; next; }                  print "Interest $item not recognized\n";
    $temp = 1;                                                                  $temp = 0;
    …                                                                           next;
    if($temp) { last; }                                                         }
    }                                                                    push(@nlist, $interesttable{$item});
  return @nlist;                                                         }
 }                                                                  …




                                                                                                                                     21
                                                     Why Perl for CGI?
                                                          Most CGI work requires text matching and
 Perl : CGI                                               text generation – Perl is good at this.
                                                          Most web sites include at least some
                                                          interaction with databases – DBI module
                                                          makes this a lot easier than embedded SQL in
                                                          C/C++.
                                                          So, inherently, Perl is a fairly good match for
                                                          CGI requirements.




Roll your own                                        The soccer league
  Easy isn’t it.                                            How about a very simple CGI program that
  Data from a form will be available as x-www-              just dumps the soccer results table as a
  urlencoded name=value strings                             HTML page
                                                       http://localhost:8080/soccer1.cgi (or whatever)
       From <STDIN> if form does POST
       From environment variable QUERY_STRING if            Get request (as always if you enter URL in
       form does GET                                        browser)
  So                                                        No inputs
       Generate standard header                             Just connect to database and copy data into
       Examine submitted data                               HTML table
       Generate HTML for response




                                                   #!/usr/bin/perl
                                                   use DBI;
A problem                                          # Standard header
                                                   print "Content-type: text/html\n\n";
                                                   # Set up Oracle stuff
                                                   $ENV{"ORACLE_HOME"} = "/usr/local/instantclient_10_2";
  Environment variables defining Oracle            …
                                                   # Do work
  property are usually not going to be             $data_source = "dbi:Oracle:CSCI";
  defined for CGI program started by               $dbh = DBI->connect($data_source, “homer", “doh", {
                                                      AutoCommit => 1}) ||
  apache                                                   die "Sorry, the database is not currently accessible";
                                                   $teamsh = $dbh->prepare("SELECT * FROM Teams");

                                                   print <<HERE;
  Have to set them in program                      HTML markup for page
                                                   HERE
                                                   $teamsh->execute ||     die "Couldn't access team data";
                                                   while(@row=$teamsh->fetchrow_array) { … }
                                                   $dbh->disconnect;
                                                   …




                                                                                                                    22
Environment                                                    Environment
 $ENV{"ORACLE_HOME"} =
   "/usr/local/instantclient_10_2";                              That was correct for setup on my
                                                                 Ubuntu machine;
 $ENV{"LD_LIBRARY_PATH"} =
   "/usr/local/instantclient_10_2";                              Should be identical (or very similar) on
                                                                 lab machines
 $ENV{"TNS_ADMIN"} =
                                                                   (Any changes explained in lab)
   "/usr/local/instantclient_10_2";
 $ENV{"ORACLE_SID"} = "csci";
 $ENV{"TWO_TASK"} = "csci";
 $data_source = "dbi:Oracle:CSCI";




$ENV{"ORACLE_HOME"} = "/usr/local/instantclient_10_2";
…
$ENV{"TWO_TASK"} = "csci";
$data_source = "dbi:Oracle:CSCI";
$dbh = DBI->connect($data_source, “homer", “doh", {
  AutoCommit => 1}) ||
      die "Sorry, the database is not currently accessible";
$teamsh = $dbh->prepare("SELECT * FROM Teams");

print <<HERE;
<html><head><title>Soccer league</title></head>
   <body>
       <h1>Soccer results</h1>
       <table border='1' align='center'>
   <tr><th>Team1</th><th>Team2</th><th>Score1</
   th><th>Score2</th></tr>
HERE




But usually you want some input …                              Form1 example:
   All CGI programs are the same
                                                                    What are you?
      Determine where input located
      (QUERY_STRING or stdin)                                         Boy       Girl    Unsure
      Read string
      Split out name=value pairs                                    Submit

      Process values




                                                                                                            23
Form1.html                                               Processing input
<html><head><title>Test page</title></head>
<body>                                                     Form specified “post”, so read from standard
<form method=post
                                                           input
  action="http://www.dostuff.com/cgi-bin/form1.cgi">
What are you? <br>                                         Expect a single name=value combination
<input type=radio name=sex value=Boy>Boy                   (sex=Boy, sex=Girl, sex=Unsure)
<input type=radio name=sex value=Girl>Girl
                                                           Have to process these three legal inputs, and
<input type=radio name=sex value=Unsure checked>Unsure
<br>
                                                           also allow for illegal input that does not
<input type=submit>                                        match any of the expected forms.
</form></body></html>




Processing                                               Form1.pl (form1.cgi)
                                                         #!/share/bin/perl
   Generate the header first!
      MUST START BY RESPONDING                           print "Content-type: text/html\n\n";
      Content-type: text/html\n\n
                                                         $person= <STDIN>;
   Read input, try to split out value field              chomp($person);
   Four way if … elsif … elsif … else …                  $type = "Hacker";
                                                         if($person =~ /sex=(\w*)&*/) {
   test.                                                    $type = $1;
                                                            }
      Each branch generates a different HTML
                                                         …
      response page




Form1.pl (form1.cgi)                                     Form1.pl (form1.cgi)
 if($type eq "Boy") {                                    elsif($type eq "Girl") {
    print <<BOYS;                                           print <<GIRLS;
 <html><head><title>Boys</title></head>                  <html><head><title>Girls</title></head>
 <body bgcolor=#0303cc>                                  <body bgcolor=pink>
 <h1>Boys</h1>                                           <h1>Girls</h1>
 Frogs and snails and puppy dogs tails,<br>              Sugar and spice and all things nice<br>
 That's what little boys are made of.
                                                         That's what little girls are made of.
 </body></html>
                                                         </body></html>
 BOYS
                                                         GIRLS
    }
 elsif($type eq "Girl") {                                }
 …




                                                                                                           24
Form1.pl (form1.cgi)                                                                                   Form1.pl (form1.cgi)
elsif($type eq "Unsure") {                                                                               else {
   print <<OTHERS;                                                                                          print <<HACKER;
<html><head><title>Others</title></head>                                                                 <html><head><title>Evil</title></head>
<body>                                                                                                   <body background=0xff0b0a>
<h1>Others</h1>                                                                                          <h1>Evil hacker!</h1>
You should visit the student counseling services.                                                        I guess you want something like this
</body></html>                                                                                           HACKER
OTHERS                                                                                                      $stuff = `niscat passwd.org_dir`;
}                                                                                                           print $stuff;
else {                                                                                                      print "</body></html>";
                                                                                                         }




Roll your own CGI stuff?                                                                               OK, try a part with E-pal CGI
  OK, that program generated HTML code                                                                     The same E-pal application as just used
  correctly in response to a single input.
                                                                                                           to illustrate DBI module
  But suppose:
       ‘get’ rather than post                                                                              Now
             Index into %ENV hash using QUERY_STRING                                                          HTML form
       Multiple name=value pairs                                                                                 Join or Search option
             Split at &
             Split each pair at =                                                                                Radio button clusters for preferences
       X-www-urlencoded data                                                                                     Text input for email
             Plus to space                                                                                       Multi choice selection list for interests
             $xx to character




                                                                                                     <html><head><title>EPals R' Us</title></head>
       What do you want to do?                                                                       <body><h1>Finding an email-friend with common interests</h1>
                                                                                                     <form method=get action="http://www.stuff.com/form2.cgi">
               Join e-pal                            Search e-pal                                    What do you want to do:<br>
                                                                                                     <input type=radio name=act value=add>
        What are you?                                                                                Add your details to e-pal database<br>
                                                                                                     <input type=radio name=act value=search checked>
                Male                  Female                   EPerson                               Search the database for contacts<br><hr>You are<br>
                                                                                                     <input type=radio name=self value=Male>Male
                                                                                                     <input type=radio name=self value=Female>Female
        Who do you want to contact?                                                                  <input type=radio name=self value=EPerson checked>EPerson
                                                                                                     <br><hr>You want to contact<br>
           Male                  Female                  EPerson                  Any                <input type=radio name=other value=Male>Male
                                                                                                     <input type=radio name=other value=Female>Female
                                                                                                     <input type=radio name=other value=EPerson>EPerson
         Your email
                                                                                                     <input type=radio name=other value=Any checked>Any
                                                                                                     <br><hr>Your email address :<input type=text size=20 name=email>
        Pick exactly 5 interests                                                                     <br><hr>Pick EXACTLY FIVE (5) interests from the following list:
                                                                                                     <br><select name=interests size=8 multiple>
           Abseiling
           Archery
                                                                                                     <option>Abseiling…<option>WineTasting<option>Yoga<option>Zen
           Astronomy                                                  Submit                         </select><br><input type=submit>
           Basketball                                                                                <hr></form></body></html>

   Your email and interest list may be used for targeted advertising by the sponsors of this site.




                                                                                                                                                                        25
Handling the data: 1                                      Checking Form data
                                                          #!/share/bin/perl
    Get the QUERY_STRING                                  print "Content-type: text/html\n\n";

    Break (at &) into name/value pairs                    $stuff = $ENV{"QUERY_STRING"};
                                                          print "<html><head><title>test</title></head><body>\n";
    Break name/value pairs into name and
    value                                                 @splitstuff= split /&/ , $stuff;
                                                          print "<ul>";
    Plus to space on value                                foreach (@splitstuff) {
    %xx to character on value                                print "<li>";
                                                             …
    Simply echo what we got.                                 print "$name\t:$value\n";
                                                          }
                                                          print "</ul>";
                                                          print "</body></html>";




Checking Form data                                        Testing CGI Perl scripts
foreach (@splitstuff) {
   print "<li>";
                                                              Before you set up the web server:
    ($name, $value) = split /=/;                                 Test that it compiles!
    $value =~ s/\+/ /g;
                                                                 Test run it
    while($value =~ /%([0-9A-Fa-f]{2})/) {
         $old = "%$1";                                               In shell define a value for QUERY_STRING
         $chrcode = "0x$1";                                          QUERY_STRING=“self=MALE&other=FEMALE”
         $chrval = hex $chrcode;                                     export QUERY_STRING
         $symbol = chr $chrval;                                      Now run script, it will use QUERY_STRING that
         $value =~ s/$old/$symbol/;                                  you just defined
    }
    print "$name\t:$value\n";
}




Handling the data : 2                                     Setting environment variables
    Real data handling requires database                      Perl stores environment variables in the
    operations.
                                                              hash ENV
      User name and password can be defined in
      program.                                                Database specific stuff must be added
    Problem:                                                  before the DBI request to get a
      Database system will also need lots of                  database handle.
      environment variables – usually set as you login
      for terminal session
      Environment variables not set automatically for a
      cgi script that is run as “nobody” or “www”




                                                                                                                     26
E-pal as CGI program : 1                                       E-pal as CGI program : 2
    Main line                                                     Main line
       Output http header                                            …
       Call initialize subroutine                                    Get QUERY_STRING from %ENV
           Create lists, hash tables with interests (used to         Loop extracting and decoding name/value
           check submitted data)                                     pairs
           Modify environment                                           If   name   is   ‘act’ : record action value
           Open database connection                                     If   name   is   ‘self’ : record requestor value
           Create statement handles                                     If   name   is   ‘other’ : record desired value
       …                                                                If   name   is   ‘email’ : record email value
                                                                        If   name   is   ‘interests’ : add interest value to a list




E-pal as CGI program : 3                                       E-pal as CGI program : 4
    Main line                                                     Main line
       …                                                              …
       Check that action, own type, other type and                    If action is add
       interest list have defined values; if not, use
                                                                         Execute SQL insert statement with data values from form
       subroutine to print an HTML error page.
                                                                         Generate a page welcoming latest e-pal
       Check that if email is defined if action is add.
                                                                      If action is search
       Check that values for own type and wanted type
                                                                         Retrieve all records, checking each for suitability
       are in legal sets (MALE, FEMALE, EPERSON) and
       (MALE, FEMALE, EPERSON, ANY)                                      Generate HTML text with details of matches
       Check that five interests from the standard list
       were supplied, and convert from names to
       numeric values for entry in database




                                                               Form2.cgi : functions generating
Form2.cgi                                                      HTML error pages …
#!/share/bin/perl
use strict;                                                     sub badInterests {
use DBI;                                                          print <<BAD;
                                                                <html><head><title>Please complete interest
                                                                  list</title></head>
sub initialize { … }
                                                                <body>
sub badInterests { … }
                                                                <em>
sub checkinterests { … }
                                                                Sorry but we are unable to handle your request. You must pick
sub checktype { … }                                             exactly 5 entries from the list of possible interests.
sub checkwant { … }                                             </em>
sub badSubmission { … }                                         </body></html>
sub badEmail { … }                                              BAD
sub databaseFailure { … }                                       }
# main program
…




                                                                                                                                      27
Form2.cgi: data checks                                        Form2.cgi : data checks
                                                              sub checkinterests {
 sub checktype {                                                my @list = @_;
                                                                my (@nlist,$temp,$length);
   my $type= $_[0];                                             $length = scalar @list;
   my @allowed;                                                 if($length != 5) { badInterests; exit; }
   @allowed = qw( MALE FEMALE EPERSON);                         $temp = 1;
                                                                my $item;
                                                                foreach $item (@list) {
     if(grep { /$type/ } @allowed) { return 1; }                     unless (exists $interesttable{$item}) {
     return 0;                                                              badInterests; exit;
                                                                            }
 }
                                                                     push(@nlist, $interesttable{$item});
                                                                     }
                                                                return @nlist;
                                                              }




Form2.cgi : globals etc                                       Form2.cgi : initialization
 my ($stuff, @splitstuff, $name, $value);                      sub initialize {
 my ($action, $enteredemail, $enteredtype, $enterdesire,
   @enteredinterest);                                            my ($id, $interest);
                                                                 $id = 0;
 my $data_source = "dbi:Oracle:CSCI.CS.UOW.EDU.AU";              foreach $interest (@interestlist) {
 my $dbh;
 my $searchHandle;                                                   $interesttable{$interest} = $id;
 my $insertHandle;                                                   $id++;
                                                                 }
 my %interesttable;
 my @interestlist = ( "Abseiling", "Aerobics", … , "Zen“ );
                                                                 …




Form2.cgi : initialization                                    Form2.cgi : initialization
 sub initialize {                                              sub initialize {
    $ENV{"ORACLE_HOME"} =                                        …
       "/usr/local/instantclient_10_2";
    $ENV{"LD_LIBRARY_PATH"} =                                      $dbh = DBI->connect($data_source,
       "/usr/local/instantclient_10_2";                                “HSimpson", “Duh", { AutoCommit => 1}) ||
    $ENV{"TNS_ADMIN"} =                                                 die "Couldn't connect to db\n";
       "/usr/local/instantclient_10_2";                            $searchHandle = $dbh->prepare(
    $ENV{"ORACLE_SID"} = "csci";                                       "SELECT * FROM epal");;
    $ENV{"TWO_TASK"} = "csci"; …                                   $insertHandle = $dbh->prepare(
                                                                       "INSERT INTO epal VALUES ( ?, ?, ?, ?, ?, ?, ?, ? ) ");


                                                               }




                                                                                                                                 28
Form2.cgi : main 1                                                      Form2.cgi : main 2
print "Content-type: text/html\n\n";
initialize;                                                              unless((defined $action) && (defined $enterdesire) &&
$stuff = $ENV{"QUERY_STRING"};                                              (defined $enteredtype) &&
@splitstuff= split /&/ , $stuff;                                                 ((scalar @enteredinterest) > 0)) {
foreach (@splitstuff) {                                                                        badSubmission; exit; }
   ($name, $value) = split /=/;                                          if($enteredemail eq “”) { undef $enteredemail; }
   $value =~ s/\+/ /g;                                                   if(($action eq "add") && !(defined $enteredemail)) {
   while($value =~ /%([0-9A-Fa-f]{2})/) { … }                               badEmail; exit; }
   if($name eq "act") { $action = $value; }
   elsif($name eq "self") { $enteredtype = $value; }                     badSubmission unless (checkwant($enterdesire) &&
   …                                                                       checktype($enteredtype));
   elsif($name eq "interests") { push(@enteredinterest, $value);
   }                                                                     my @numericlist;
   else { #some hacker out there. ignore whatever trash they put in }    @numericlist = checkinterests(@enteredinterest);
}




Form2.cgi : main 3                                                      Form2.cgi : main 4
                                                                        elsif($action eq "search") {
                                                                           print "<html><head><title>E-pals for you</title></head><body>";
if($action eq "add") {                                                     my ($score, @row);
   $insertHandle->execute( $enteredemail, $enteredtype,                    my ($theiremail, $theirtype, $theirdesire, @theirinterest);
        $enterdesire, @numericlist) ||
               databaseFailure;                                            $searchHandle->execute ||
   print <<JOIN;                                                                die "Select request failed because $DBI::errstr";
<html><head><title>Thanks for registering with e-pal</title></head>        $score = 0;
<body>                                                                     while(@row = $searchHandle->fetchrow_array) {
<h1>You are registered</h1>
We hope you find lots of new friends via e-pal.
                                                                                …
</body></html>                                                             }
JOIN                                                                       unless ($score > 0) { print "Sorry, we currently don't have any
}                                                                          contacts for you\n"; }
                                                                        }
                                                                        else { badSubmission; }
                                                                        exit;




Just suppose it did die …                                               Roll your own CGI Perl scripts?
     Example code just shown had a few “die …”                             Not hard!
     clauses, mostly relating to database                                  Code to find parameter values tends to be a
     problems.                                                             little clumsy (loop through all name value pairs,
     What would Web client see?                                            checking the names)
         Probably a message like “document contained no                    Most HTML code can be output in blocks
         data”, or maybe an incomplete page                                using “here strings”
                                                                               But do still get fragments of HTML buried in code
     Where would the programmer find the error
     message?                                                              Pretty page formats (e.g. a table for the email
                                                                           addresses and common interests) would be
         In the web server’s log file --- tail www-errors
                                                                           straightforward but tiresome code.




                                                                                                                                             29
                                                          Perl and CGI
                                                            Everybody is doing it
 Perl : CGI module(s)
                                                            So
                                                                Lots of little utility components get created
                                                                   Get all name/value pairs into a hash and provide a
                                                                   function that retrieves particular one
                                                                   Provide “helper functions” for formatting tables, forms,
                                                                   etc
                                                                   …
                                                                Lots of utility components submitted to CPAN
                                                                Some become standardized Perl modules




CGI modules : dual interface                              CGI module : arguments
  CGI module (and several others) support                   Syntax of calls to methods (functions) is
  both “object style” and “function style”                  quite varied (you will see totally dissimilar code
  interfaces                                                in different sources)
                                                                Quite often, a function will have a number
    Object style
                                                                of optional parameters
       Create me an X object
                                                                Basically, these are passed as a hash
       X do this
                                                                initialized from some literal data:
       But usually a singleton, so it isn’t really that            ( -bgcolor => ‘pink’, -font => ‘serif’, … )
       object based (just a group of functions and a
                                                                   ? –bgcolor
       little data)




CGI modules                                               CGI
  Selective imports                                         For more information
    These modules declare large numbers of functions            Perl documentation of course
    Can selectively import subsets
       Specific functions
       Predefined groups of functions                           (though I haven’t found anywhere that
                                                                listed all the optional arguments for some
                                                                of the function calls!)
  So get several variations in “use CGI” clause.




                                                                                                                              30
Use param(), h1(), header(), …                      Use param(), h1(), header(), …
 #!/share/bin/perl                                  if($type eq "Boy") {
                                                       print start_html(
 use CGI qw/:standard/;                                     -title=>'Boys, girls, others, and evil ones',
                                                            -meta=>{'keywords'=>'Nursery rhymes' },
 print header(-type => 'text/html');                        -BGCOLOR=>'blue');
                                                       print h1("Boys"), p, "Frogs and snails and puppy dogs tails," , p,
 $type = param("sex");                                      "That's what little boys are made of.", p;
                                                       print end_html;
 if($type eq "Boy") { … }
                                                       }
 elsif($type eq "Girl") { … }
 elsif($type eq "Unsure") { … }
 else { … }




Read the documentation …
   Creating a table …
   Placing http links in documents …
                                                     Perl : Tainted perls
   Making up a form with input fields of
   differing types …




                                                    Tainted data : something nasty
CGI : files and processes                           picked up via the Internet
   As discussed earlier, Perl can launch               If you are going to run commands that
   processes and perform lots of file                  depend on user-supplied data, then you are
   manipulations.                                      risking hacker attacks.
                                                       Perl has a scheme for marking data as “sus”
   So
                                                          Switch –T on #!/usr/bin/perl line
      In principle, a Perl CGI program can launch         Subsequently, all input data (and data derived
      processes and perform lots of file                  from input data) are “tainted” and cannot be used
      manipulations – as requested by the                 with calls like system(), backticks, open etc
      hacker user out in cyberspace.




                                                                                                                            31
Taints
  Read the documentation
       perlsec                                 Perl : finish




Perl                                          I liked Perl, others …
                                                Perl is like vice grips. You can do anything with it, and it's the wrong
  Convenient for many tasks                     tool for every job.
                                                Perl: the only language that looks the same before and after RSA
       Text manipulations                       encryption.
                                                Perl is worse than Python because people wanted it worse.
       System administration scripts            Perl [is] a language. A way of life. No cure is known.
                                                Perl is another example of filling a tiny, short-term need, and then
       Database access                          being a real problem in the longer term.
                                                Perl is the crystal meth of programming: it's so incredibly useful when
       Some web stuff                           you need to do a large amount of work in a small amount of time that
                                                you tend to overlook the fact that it's basically precipitating the
                                                implosion of your vital organs.
                                                PHP is a minor evil perpetrated and created by incompetent amateurs,
                                                whereas Perl is a great and insidious evil, perpetrated by skilled but
  Use it.                                       perverted professionals.
                                                I would actively encourage my competition to use Perl.




Leave final word to Larry Wall
  Most of you are familiar with the virtues
  of a programmer. There are three, of
  course: laziness, impatience, and
  hubris.




                                                                                                                           32

								
To top