A Perl Wrapper for (Some) Vista 4.1 Web APIs

Document Sample
A Perl Wrapper for (Some) Vista 4.1 Web APIs Powered By Docstoc
					A Perl Wrapper for
(Some) Vista 4.1 Web APIs


                Brent Shaw
    Course Development & Web Services
        University of Central Florida
             Orlando, Florida
About the
University of Central Florida
 • 46,719 students (Fall 2006)
 • 1,686 Faculty & Adjuncts
   (1,392 FTEs)

 • 12 regional campuses
 • 2006 - 2007 Budget:
   $926,866,266
 • College of Medicine opening
   Fall 2008

  Page 2 / 274        Brent Shaw   2/1/2010
About Me
CMS Administrator for UCF
   – Keep WebCT CE 4.1.5.8 running
        • 5000 courses - 85%
   – Help with Vista 4.1
        • Conversion from CE to Vista
        • Oracle


Using WebCT since it was in Beta the first time
  (1997 or so).
Worked for WebCT for a year.

 Page 3 / 274                Brent Shaw   2/1/2010
About WebCT at UCF: CE 4.1.5.8
• 5,000 courses /        77,000 users
• Server:
   – Solaris 8  Solaris 10
   – SunFire 1280 - 12 x 900Mhz Sparc CPUs
   – 24 Gigabytes of RAM
   – 1 TB of Sun 6920 SAN storage currently at 59%
Flat files / .csv from PeopleSoft.
Old, old, old file system (~1997).

 Page 4 / 274          Brent Shaw             2/1/2010
About WebCT at UCF: Vista 4.1
• 300 sections            /       61,000 users
• Servers:
   – Solaris 10 and Oracle 9i
   – App. Servers (4-1)                     Database
        •   Sunfire v240                    E20K Domain Board
        •   2 x 1.5 Ghz CPUs                4 x 1.5 Ghz CPUs
        •   4 Gigabytes RAM                 16 Gigabytes RAM
        •   75 Gigabytes local storage      1 Terabyte SAN storage
PeopleSoft XML exports

 Page 5 / 274                  Brent Shaw                    2/1/2010
Original Motivation
 We needed a way for our university help
  desk to reset Vista passwords.
   – Help desk used a web page to reset
     WebCT CE passwords.
   – Not set passwords, reset to a default.
   – Not reset everyone’s password.
     (Like MINE! Or my bosses, etc…)


 Page 6 / 274        Brent Shaw               2/1/2010
Original Motivation
In other words:
I had this…       but           I wanted this…




 Page 7 / 274      Brent Shaw               2/1/2010
Why use the Web based API?
For the password reset tool, the default password was
  available from WebCT CE 4.1.

Either way, some part of my password tool was going
   to have to talk to a different server.




       CE                                    Vista



 Page 8 / 274           Brent Shaw               2/1/2010
My Mindset: Web vs Command Line
• If it’s command line driven, you’re tied to a
  server with Vista installed.
        • weblogic81/config/WebCTDomain/siapi.sh …
        • (Relatively high level of security)


• With the Web based APIs, you can use it
  from any computer with a network
  connection.
        • Server, desktop, anywhere
        • (Potentially less secure)

 Page 9 / 274               Brent Shaw               2/1/2010
Using the Web API
• Pros                            • Cons
   – Can be used on any              – Almost have to be
     computer with a                   running Vista on SSL.
     network connection.
        • Server, desktop,
          laptop.
   – Great for one off                    – Harder to get
     transactions.                          running at first.
                                             • But that is what this
                                               talk is for.


 Page 10 / 274               Brent Shaw                       2/1/2010
Using the Command Line API
• Pros                        • Cons
   – Because you’re on a         – Limited to a server
     Vista server, it’s            with Vista installed.
     more secure.

   – Easier to get running            – For CGI, either
     at first.                          modify WebLogic, or
                                        run separate Apache
   – Better for batch jobs.             on a Vista managed
                                        node.

 Page 11 / 274           Brent Shaw                  2/1/2010
Why Perl?
• Perl is flexible.
• Perl can be written platform
  independently.
        • Develop on Windows (Activestate)
        • Deploy on Solaris
• Perl plays well with Apache.
        • In my case, the password CGI is on
          WebCT CE 4.1 and written in Perl.
• I already knew Perl.
 Page 12 / 274           Brent Shaw            2/1/2010
What’s Required
• Perl 5.8
        • Might work on lower versions, but I haven’t tested on
          anything but 5.8.
• LWP::UserAgent
• HTTP::Request::Common
• XML::Simple
   – XML::Parser (or XML::SAX?)
   – Expat
        • It just worked on Windows.
        • I had to compile Expat statically on Solaris.


 Page 13 / 274                Brent Shaw                   2/1/2010
What’s Required
• Date::Manip
  – Only a single usage:
        • Timestamp format specified by IMS
        • 1996-10-24T17:40:58 +0100
        • Date::Manip::UnixDate("today", '%O%z')


• Digest::MD5 qw(md5_hex)                 (Core)
• Data::Dumper                            (Core)
• Crypt::SSLeay            (if you're using SSL)

 Page 14 / 274              Brent Shaw             2/1/2010
What’s Required
Oh yeah, a copy of Blackboard Learning
 System Vista 4 License.

I think it will work on the Blackboard
   Learning System CE 6 License, but I
   don’t write for it nor do I test on it.

SSO links to Learning Contexts require an
  Authentication Module (Deployable
  Component) to be written.

 Page 15 / 274         Brent Shaw            2/1/2010
This Is Alpha Code at Best
I guarantee that things will change.
In some cases, major things will change.
I can almost promise these changes will
  break things.
Error handling is not the greatest.
Feedback is greatly appreciated.


 Page 16 / 274    Brent Shaw        2/1/2010
The Set Up
Servers are defined in an Array of Hashes:
{name        => 'PRODUCTION',
default      => 'yes',
server       => 'https://web.vista.server.edu/',
glcid        => 'URN:X-WEBCT-VISTA-V1:9195260f-
                     0aaa-ef12-002a-c92f51d29850', #
user         => 'departmentAdminUser',         #
password     => 'departmentUserPW',            #
macSecret => 'superSecretSharedSecret',
macSecretSSO=> 'otherSuperSecretSharedSecret',
oraclehost => 'db.vista.server.edu',           #
oraclesid    => 'webct',                       #
oracleuser => 'webctReadOnlyAccount',          #
oraclepass => 'superSecretPassword',           #},
 Page 17 / 274         Brent Shaw              2/1/2010
The Set Up
Information common to 'all' Vista servers:
%data = (
siapi =>'webct/systemIntegrationApi.dowebct',
sso    =>'webct/urw/ssUCFSSO/cobaltMainFrame.dowebct',
autosignon => 'webct/public/autosignon',
        );

+ Server Hash


   Page 18 / 274        Brent Shaw            2/1/2010
sub setServer(['servername'])
setServer()
   1. If a server is already declared, continue to
      use it.
   2. If there is no server declared, setup the
      default server (default => 'yes').
setServer('default')
setServer('DEVELOPMENT')

 Page 19 / 274        Brent Shaw            2/1/2010
sub _getServer('servername')
• Leading underscore means it's a
  "private" function.
• Returns the hash of server information
  for the server requested.
   – If the server can't be found, returns the
     default server.
   – If it can't find a default server, it returns
     the last server in the list of servers.

 Page 20 / 274         Brent Shaw              2/1/2010
sub _getDefaultServer()
• Returns the hash of server information
  for the first default server in the list of
  servers. (If there are multiple
  default =>'yes' declarations.)

   – If it can't find a default server, it returns
     the last server in the list of servers.


 Page 21 / 274         Brent Shaw              2/1/2010
WebCT CGI: MACs, Keys & Values
What is a MAC?
• Message Authentication Code.

• It's an MD5 hash based on a shared
  secret and some of the values of a CGI
  request.

• 32 character alphanumeric.

 Page 22 / 274    Brent Shaw        2/1/2010
WebCT CGI: MACs, Keys & Values

http://server.com/script.cgi?key1=value1
  &key2=value2&key3=value3

• The manual says use all values in
  calculating the MAC.

• That’s WRONG!

 Page 23 / 274    Brent Shaw          2/1/2010
WebCT CGI: MACs, Keys & Values
Retrieving a Person Record:
   ?adaptor=ims&operation=configure&
     option=get_person_ims_info&webctid=userID&
     timestamp=epochtime&auth=MAC
Used in MAC:
   – configure, get_person_ims_info, userID,
     epochtime
Not Used in MAC:
   – ims, MAC

 Page 24 / 274        Brent Shaw            2/1/2010
WebCT CGI: MACs, Keys & Values
# Stuff needed to calculate the MAC.
my %MACpairs = ( operation       => 'configure',
                    option       => 'get_person_ims_info',
                    webctid      => $webctid,
                    timestamp => time,);

# Stuff needed to make the URL,
# but not used in the MAC calculation.
my %URLpairs = ( adapter         => 'ims',
                    auth         => '',);

# We're missing the MAC…

   Page 25 / 274          Brent Shaw             2/1/2010
sub calculateMac(values %hash, 'secret')

 Submit an array of values needed to
   calculate the MAC (+ the secret):
   values         (operation     =>    'configure',
                  option         =>    'get_person_ims_info',
                  webctid        =>    $webctid,
                  timestamp      =>    time,), $data{macSecret}
 Returns a MAC of the form:
  ef6a1c68f88f48dcacb6bb507d67f8e7

  Page 26 / 274                Brent Shaw               2/1/2010
sub _buildURL(\%Mac, \%others)

Receives two hash refernces: MAC data,
 and "other" data.

Returns the CGI parameter string for the key
 & value pairs in the hashes:
operation=configure&option=get_person_ims_info&
  webctid=bshaw&timestamp=1173703811&adapter=ims&
  auth=ef6a1c68f88f48dcacb6bb507d67f8e7


 Page 27 / 274      Brent Shaw          2/1/2010
Building a URL to Get a Person Record
 my $URLstring =
 $data{'server'} .
 $data{'siapi'} .
 "?" .
 _buildURL(\%MACpairs, \%URLpairs);
 http://vista.server.edu/webct/systemIntegrationApi.dowebct?operation=co
    nfigure&option=get_person_ims_info&
    webctid=bshaw&timestamp=1173703811&adapter=ims&
    auth=ef6a1c68f88f48dcacb6bb507d67f8e7

 So let's fetch at the Person Record!

   Page 28 / 274                Brent Shaw                      2/1/2010
sub _httpGET('url')
Accepts a URL.

Returns the HTTP GET of the URL.

(Fetches a web page using GET.)



 Page 29 / 274   Brent Shaw        2/1/2010
sub _removeDoctype
To make it HTML, HTTP responses from
 Vista are prefaced with:
        <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01
          Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

        <!-- systemintegrationapi/result.jsp -->
        <!-- applicationframework/CommonHeader.jspi -->
        <<< Snipped 12 New Lines >>>
        <!-- applicationframework/contextPath.jspi -->
        <<< Snipped 9 New Lines >>>


 Page 30 / 274              Brent Shaw                    2/1/2010
sub getConsortiaID('userID')
• Accepts a WebCT ID.
• Returns the same WebCT ID (if successful).
• "Consortia ID" is now depreciated.
• Currently, fetching a Consortia ID returns the
  WebCT ID.
• If the Consortia ID != WebCT ID, the user
  doesn't exist.
   – Quick and dirty check for existence.


 Page 31 / 274          Brent Shaw          2/1/2010
sub getSourced('userID')
• Accepts a WebCT ID.

• Returns the IMS sourced.source and
  sourced.id.

• Most requests require both the WebCT
  ID and the source (i.e. sourced.id and
  sourced.source).

 Page 32 / 274    Brent Shaw         2/1/2010
sub getHomeareaURL('userID')
• Accepts a WebCT ID.

• Returns a SSO URL to the user's "My Vista"
  homearea.
        • https://webcourses.ucf.edu/webct/public/autosignon?times
          tamp=1173710186&wuui=bshaw&url=/webct/viewMyWeb
          CT.dowebct&mac=dec026ec8125ed46899582bf83fe25f2


• Works out of the box.
  Does not require extra Authentication Module.

 Page 33 / 274              Brent Shaw                   2/1/2010
sub makeSSOurl('userid', 'path')
 •       Accepts a WebCT ID and a path to a tool.
 •       Returns a SSO link to the tool for the user.
 •       Requires a SSO Deployable Content module
         to be written.

 Q. Where do you get a path to a tool?
 A. getHomeareaXML() or better yet
    getHomeareaXMLasHash()

     Page 34 / 274        Brent Shaw            2/1/2010
sub getHomeareaXML('userID')

• Accepts a WebCT ID.
• Returns the XML representing the user's
  "My Vista" homearea.

• Useful for creating a portal object that
  displays new course mail, new quizzes,
  new grades, etc.

 Page 35 / 274     Brent Shaw          2/1/2010
sub getHomeareaXMLasHash
• Same thing as before, but returns the
  XML as a hash instead of raw XML.
• Basically:
  XML::Simple::XMLin(getHomeareaXML())
• Use Data::Dumper to look at the hash.
• Perhaps "ForceArray=>1" in XML::Simple
  – That will really change things…


 Page 36 / 274       Brent Shaw       2/1/2010
IMS Person Record
• IMS defines a "Person Record."
• Vista Person Record includes:
   – sourced.source              name.fn
     sourced.id                  name.n.family
                                 name.n.given
                                 name.n.other
                                 name.n.suffix
   – email
   – Roles (Institution Admin, Domain Admin, etc.)
   – Custom columns in the "global database"


 Page 37 / 274          Brent Shaw              2/1/2010
sub getPersonRecordXML('userID')

• Accepts a WebCT ID.


• Returns the XML representing a user's
  Person Record.

                                  XML



                                  hash


 Page 38 / 274     Brent Shaw        2/1/2010
sub setPassword('userid', 'pw')
Accepts WebCT ID and a password.
The only way I've found to set a user's
  password is to submit XML via an HTTP
  POST.
• Password is plain text. (SSL)
• This should also be modified to accept
  encrypted passwords.

 Page 39 / 274    Brent Shaw        2/1/2010
sub setPassword('userid', 'pw')
Minimal, required XML includes:
• sourced.source sourced.id
• WebCT ID
• name.fn (full name, but not the other names)
• password


setPassword() retrieves the extra information
  from a getPersonRecord() call.

 Page 40 / 274       Brent Shaw           2/1/2010
sub _httpPOST
• Currently accepts just the XML.
• Only submits an HTTP POST to the
  Vista server in the form of "ims import
  restrict".
• This will change.
   – Needs to accept a URL.
   – Needs to accept the POST content (XML).


 Page 41 / 274      Brent Shaw          2/1/2010
What’s Next?
• Pull server config data into it's own file.
• Pull out makeSSOurl() into it's own
  module.
   – Because it depends on a Vista
     Authentication Module.
• POD.
• Pull XML out into HTML::Template.

 Page 42 / 274       Brent Shaw         2/1/2010
What's Next?
• Add GLCID / User / Password support.
  – Multiple Institutions.
• Add username and password support
  for various LCIDs. (College, Dept. etc.)
• Add a function to return the currently
  designated server.
• Modify setPassword() to accept
  encypted passwords.

 Page 43 / 274     Brent Shaw         2/1/2010
What’s Next?
Module Layout                                     Blackboard




                                 Vista                    CE             Blackboard
                                                                           Blackboard::
                           Blackboard::Vista      Blackboard::CE
                                                                           Blackboard




   Web               SSO                 Powersight             Config
Blackboard::      Blackboard::          Blackboard::           Blackboard::
 Vista::Web       Vista::SSO          Vista::Powersight        Vista::Config



  Page 44 / 274                     Brent Shaw                                 2/1/2010
What’s Next?
• Release on CPAN.


• **Add a function to add / drop enrollments.**
• Add a function to retrieve single user’s
  enrollment data (saw request on the webct-
  users@webct.com listserv)
• DBI access to Powersight tables and views.
   – Separate module due to DBI dependency.


 Page 45 / 274        Brent Shaw              2/1/2010
Gotcha’s
• Set the GLCID, username and password in the
  GUI.
   – Administration > Utilities > Settings > System
     Integration API IMS Adapter
   – Administration > Utilities > Settings >
     System Integration API Standard Adapter System
     Integration
• EXPAT might have to be statically linked.
• Sometimes it just stops working … (reboot)

 Page 46 / 274         Brent Shaw             2/1/2010
Gotcha’s
 my $req = HTTP::Request::Common::POST
"$data{'server'}$data{'siapi'}",
Content_Type   => 'form-data',
Content        => [%MACpairs, %URLpairs,
                   "FILE" => [undef,

  "filename.xml",
                                 Content =>
  $XML,]
                    ],;
# You _cannot_ use "FILENAME" instead of
  "FILE" –
 Page 47 / 274     Brent Shaw        2/1/2010
# Vista will barf if the parameter name is
 Questions?
 Comments?
Suggestions?

         Brent Shaw
       CD&WS at UCF
    bshaw@mail.ucf.edu
http://reach.ucf.edu/~shaw/
How I Program
• Whitesmiths style
           while (x == y)
               {
               something();
               somethingelse();
               }
           finalthing();

•    Komodo IDE by Activestate for my editor
•    Komodo Edit is free
•    WinMerge for diffs
•    Stylus Studio for XML


    Page 49 / 274                 Brent Shaw   2/1/2010
INSTALLING EXPAT
# cd to the expat source
# http://sourceforge.net/projects/expat/
# http://superb-east.dl.sourceforge.net/sourceforge/expat/expat-2.0.0.tar.gz
cd /home/reliant/shaw/downloads/expat-2.0.0

# to add the correct "ar"
PATH=/usr/ccs/bin:$PATH
# add gcc to the path so configure can find it.
PATH=/local/gnu/bin:$PATH

# if needed
/local/gnu/bin/make clean

# staticly link so CGI scripts can find the all the .so files without having to mess with
#      LD_LIBRARY_PATH env variable
./configure --enable-static --disable-shared --prefix=/home/reliant/shaw/.local
./configure --enable-static --disable-shared --
    prefix=/coursedev/webctpro/webct/webct/webct/generic/ucf/expat
/local/gnu/bin/make
/local/gnu/bin/make install




  Page 50 / 274                        Brent Shaw                              2/1/2010
INSTALLING XML::PARSER
First, delete .cpan/build and .cpan/sources
SHELL: # to add the correct "ar"
SHELL: PATH=/usr/ccs/bin:$PATH
SHELL: # add gcc to the path so configure can find it.
SHELL: PATH=/local/gnu/bin:$PATH
SHELL: perl -MCPAN -e shell
CPAN: get XML::Simple
SHELL: pico .cpan/build/XML-Parser-2.34/Makefile.PL
PICO: $expat_libpath =
    '/coursedev/webctpro/webct/webct/webct/generic/ucf/expat/lib';
PICO: $expat_incpath =
    '/coursedev/webctpro/webct/webct/webct/generic/ucf/expat/include';
PICO: save and exit
SHELL: cd .cpan/build/XML-Parser-2.34/
SHELL: perl ./Makefile.PL
    PREFIX=/coursedev/webctpro/webct/webct/webct/generic/ucf/utils
    LIB=/coursedev/webctpro/webct/webct/webct/generic/ucf/utils
SHELL: /local/gnu/bin/make
SHELL: /local/gnu/bin/make install



  Page 51 / 274                 Brent Shaw                      2/1/2010