Soaplab SOAP-based Analysis Web Services by vev19514

VIEWS: 9 PAGES: 22

									            Soaplab
SOAP-based Analysis Web Services

      Martin Senger <senger@ebi.ac.uk>


    http://industry.ebi.ac.uk/soaplab
         What is the task?
1. You have (or someone else has) one or
   more command-line analysis tools
2. You want to access them from other
   computers, eventually over an Internet
3. You want to access them using your (or
   someone else’s) programs, not just by
   filling the forms and clicking on web
   pages
How it fits with the rest of the world
• Phase 1: Describe your command-line tool
  (preferable using an ACD file)
  – http://www.hgmp.mrc.ac.uk/Software/EMBOSS/Acd/index.html

• Phase 2: Soaplab
  – this whole presentation is about this phase
• Phase 3: Write your own programs (GUIs and
  others) to access tools via Soaplab
  – part of this presentation is about this
  – Taverna: an example of a workflow system using
    Soaplab (and other) web services as components
What Web Services are in Soaplab?
• By resources that are being accessed:
   – command-line tools (sub-project: AppLab)
   – web pages (sub-project: Gowlab)
• By results that are being delivered:
   – Analysis/Gowlab Factory Service (misnomen)
       • produces a list of all available services
   – Analysis Service
       • represents one analysis tool and allows to start it, to control it, and
         to exploit it, or it represents a remote web page accessible using the
         same API as other Soaplab analysis services
       • does it in loosely type manner using the same API for all analysis
   – Derived Analysis Service
       • does the same as the Analysis Service above but in a strongly
         typed manner having individual API for any individual analysis tool
                       Graphically…                                                Your own
                                                                                   client
                                                                                   programs
                                                                             

                           Tomcat                                                 Talisman
     
                          (or any servlet engine)
                                                                                   BioPerl
     XML
       XML                                                                         modules
        ACD
     (analysis
       (analysis
                           Apache Axis                                             (Bio::Tools::Run::Analysis)
    description)
        (analysis             (SOAP Toolkit)
     description)
      description)                                                           
                           Soaplab                  MySQL with
                                                                                   Provided ready
                                                                                   clients
   acd2xml                 services                   results

       (convertor)

                             Files: IOR/analyses/…                          Create analysis description
                                                                            Convert to XML
        XML                                                                 Start AppLab server
          XML
            XML
        (analysis
                           AppLab                    Filesystem            Start Tomcat server                   This is
          (analysis
      description)
          (analysis
       description)          server
                                                     with analysis
                                                        results
                                                                            Deploy Web Services                   Soaplab
        description)
                                                                            Use existing client, or/and
                                                                            write your own client
Important:                    Command-line
• how to write ACD               driven
                              analysis tools
• what API to use
  by the clients                                                                  Downoad: soaplab-clients-version




                                                                                                                            distribution
                                                                                                                              Binary
                                                             Downoad: applab-cembalo-soaplab-version (for EMBOSS)

                                                                               Downoad: analysis-interfaces-version
                                                                                  http://industry.ebi.ac.uk/soaplab/dist/
                        Or…                                               Your own
                                                                          client
                                                                          programs
                                                                     

                        Tomcat                                           Talisman
     
                       (or any servlet engine)
                                                                          BioPerl
     XML
       XML                                                                modules
        ACD
     (analysis
       (analysis
                        Apache Axis                                       (Bio::Tools::Run::Analysis)
    description)
        (analysis          (SOAP Toolkit)
     description)
      description)                                                   
                        Soaplab                  MySQL with
                                                                          Provided ready
                                                                          clients
   acd2xml              services                   results

       (convertor)

                                                                    Create web resource description
                         Gowlab                   Filesystem        Convert to XML
        XML
                                                 with “analysis”    Start AppLab server
          XML
            XML
        (analysis
                        classes                     results
                                                                    Start Tomcat server           This is
          (analysis
      description)
          (analysis
       description)                                                 Deploy Web Services           Soaplab
        description)
                                                                    Use existing client, or/and
                                                                    write your own client
Important:                    Remote
• how to write ACD         web resources
                            (web pages)
• what API to use
  by the clients                                                         Downoad: soaplab-clients-version




                                                                                                                   distribution
                                                                                                                     Binary
                                                                      Downoad: analysis-interfaces-version
                                                                         http://industry.ebi.ac.uk/soaplab/dist/
         A. Client developer
• Needs to know the Soaplab API
• Can develop in any programming
  language
• Needs to know a place (an endpoint) with
  a running Soaplab services
  – http://industry.ebi.ac.uk/soap/soaplab   (EMBOSS)

• Can use the ready, pre-prepared Java and
  Perl clients

                                             Downoad: soaplab-clients-version
                                             http://industry.ebi.ac.uk/soaplab/dist/
                                                 Soaplab API
                                       http://industry.ebi.ac.uk/soaplab/API.html

      The main methods for running an analysis                                            The List (factory) service
     createJob (inputs)                                                                      getAvailableAnalyses()
     run()                      createAndRun (inputs)                                        getAvailableCategories()
     waitFor()                                               runAndWaitFor (inputs)          getAvailableAnalysesInCategory()
                                                                                             getServiceLocation()
     getResults()
     destroy()

     getSomeResults()


The methods for asking                           The methods for asking                    An example of (additional)
about the whole analysis                           what is happening                      methods for a derived service
getAnalysisType()                             getStatus()                                    createEmptyJob()
getInputSpec()            describe()          getLastEvent()
getResultSpec()                                                                              set_sequence_usa (datum)
                                              getCreated()                                   set_format (datum)
                                              getStarted()                                   set_firstonly (datum)
                                                                   getCharacteristics()
                                              getEnded()                                     …
                                              getElapsed()                                   get_report()
                                                                                             get_outseq()
Once you know API…
import org.apache.axis.client.*;
import javax.xml.namespace.QName;
import org.apache.axis.AxisFault;
import java.util.*;

Call call = null;
try {

  // specify what service to use
  call = (Call) new Service().createCall();
  call.setTargetEndpointAddress ("http://industry.ebi.ac.uk/soap/soaplab/display::showdb");

  // start the underlying analysis (without any input data)
  call.setOperationName (new QName ("createAndRun"));
  String jobId = (String)call.invoke (new Object[] { new Hashtable() });

  // wait until it is completed
  call.setOperationName (new QName ("waitFor"));
  call.invoke (new Object[] { jobId });

  // ask for result 'outfile'
  call.setOperationName (new QName ("getSomeResults"));
  Map result = (Map)call.invoke (new Object[] { jobId, new String[] { "outfile" } });

  // ..and print it out
  System.out.println (result.get ("outfile"));

} catch (AxisFault e) {
   AxisUtils.formatFault (e, System.err, target.toString(),
                          (call == null ? null : call.getOperationName()));
   System.exit (1);
} catch (Exception e) {
   System.err.println (e.toString());
   System.exit (1);
}
       Once you know API…
• Direct usage of the Soaplab API…
          use SOAP::Lite;

          my ($soap) = new SOAP::Lite
           -> proxy ('http://industry.ebi.ac.uk/soap/soaplab/display::showdb');

          my $job_id = $soap->createAndRun()->result;
          $soap->waitFor ($job_id);
          my $results = $soap->getSomeResults ($job_id, ['outfile'])->result;

          print $results->{'outfile'} . "\n";




• …or using BioPerl modules
              use Bio::Tools::Run::Analysis;
              print new Bio::Tools::Run::Analysis (-name => 'display::showdb')
                 ->wait_for ( {} )
                 ->result ('outfile');
  What names to use for inputs?
• Ask getInputSpec()
    – some inputs are mutually exclusive (but
      Soaplab does not tell you )
    – supported types: basic types
• And the results? Ask getResultSpec()
    – expected types: basic types, byte[], byte[][]
    – better check org.embl.ebi.SoaplabClient.AxisUtils.aa2bb()
                       What else…
• Code examples at:
  – http://industry.ebi.ac.uk/soaplab/UserGuide.html

• Play also with existing clients:
      run-analysis-client –h
      run-factory-client –h
      run-registry-client –h
      run-tutorial




                                                       Downoad: soaplab-clients-version
                                                       http://industry.ebi.ac.uk/soaplab/dist/
         B. Service provider
• EMBOSS provider
                                    Downoad: applab-cembalo-soaplab-version (for EMBOSS)
                                                         http://industry.ebi.ac.uk/soaplab/dist/




  – contains pre-generated XML files for all
    EMBOSS analysis (but not EMBOSS itself)
• Provider of your own analysis tools
                                                      Downoad: analysis-interfaces-version
                                                         http://industry.ebi.ac.uk/soaplab/dist/



  – contains   acd2xml
                (convertor)
                              for creating XML files
                  Installation
• Run perl INSTALL.pl
  – and you will be asked many questions
  – or prepare your answers in environment
    variables and run: perl INSTALL.pl –q
• Do you want to add your analyses?
  – write an ACD file (e.g. metadata/bigdeal.acd)
  – run: generator/acd2xml –d –l MyApps.xml bigdeal
  – see generator/README for details
    What questions to expect?
         basics
JAVA_HOME                       where is Java installed
PROJECT_HOME                    where is everything unpacked
RESULTS_URL                     URL pointing to the server-side results
         tomcat and axis locations
TOMCAT_HOME                     directory where is a Tomcat servlet engine
TOMCAT_URL                      URL accessing Tomcat (e.g. http://localhost:8080)
AXIS_IN_TOMCAT                  directory where Axis is installed under TOMCAT
         mySQL locations (the whole mySQL stuff is, however, optional)
MYSQL_URL                       location of mySQL Soaplab database
MYSQL_USER                      username of mySQL Soaplab database
MYSQL_PASSWD                    password for mySQL Soaplab database
         rarely needed, only for your own analyses
ADD_TO_PATH                     will be added to PATH before the AppLab server starts
         nothing to do with web-services (so ignore it…)
ALWEB_SCRIPTS_URL               where are alweb cgi-bin scripts
ALWEB_DOCS_URL                  where are alweb related documents
                   And run it!
• ./run-AppLab-server
   – check AppLabServer.log
• (start your Tomcat)
• ./ws/deploy-web-services
• try any client
   – check soaplab.log
• …complain to senger@ebi.ac.uk
         Deploying services
• That can be tricky…because it includes
  many things:
  – move your classes to Tomcat
  – create service descriptors (files *.wsdd)
  – generate and compile code for the derived
    services
  – etc.
• Therefore, use a Soaplab admin client:
  – run-admin-client –help
  – which is used by ws/deploy-web-services, anyway
Practical session
                A. Main steps
i.    Download various Soaplab distributions
ii.   Try several ready clients

1.    Code your own client (calling EMBOSS at EBI)
2.    Run Soaplab on top of your own EMBOSS
      installation on your machine
3.    Code your own ACD file and run Soaplab
      service on top of it
                   Code your own client
• Calling EMBOSS at EBI:
  – class Seqret.java
  – usage:java Seqret [format] < input_sequence > reformatted_sequence
  – where format represents output sequence format
    and is used as parameter osformat, and input_sequence
    is used as parameter sequence_direct_data or
    sequence_usa (check for double colon)
                                                                       >gnl|embl|MMIGK7 (V00808.1) Part of the murine gene for kappa-immunoglobulin

  – check it by calling:                                                 leader peptide and variable part (cell line MOPC41).
                                                                       cgtgaccaatcctaactgcttcttaataatttgcataccctcactgcatcgccttgggga
                                                                       cttctttatataacagtcaaacatatcctgtgccattgtcattgcagtcaggactcagca
      • java Seqret ncbi < embl:v08008                                 tggacatgagggctcctgcacagatttttggcttcttgttgctcttgtttcaaggtaaaa
                                                                       tgaaacttaaaattgggaattttccactgtttccaactgtggttagtgttgactggcatt

  – it does the same as:                                               tgggggatgtcctcttttatcatgcttatctatgtggatattcattatgtctccactcct
                                                                       aggtaccagatgtgacatccagatgacccagtctccatcctccttatctgcctctctggg
                                                                       agaaagagtcagtctcacttgtcggccaagtcaggacattggtagtagcttaaactggct
     run-analysis-client -e http://industry.ebi.ac.uk/soap/soaplab \   tcagcaggaaccagatggaactattaaacgcctgatctacgccacatccagtttagattc
                                                                       tggtgtccccaaaaggttcagtggcagtaggtctgggtcagattattctctcaccatcag
                          -name edit::seqret \
                                                                       cagccttgagtctgaagattttgtagactattactgtctacaatatgctagttctccgtg
                          -w -r outseq –q \                            gacgttcggtggaggcaccaagctggaaatcaaacgtaagtagaatccaaagtctctttc
                          sequence_usa embl:v00808 osformat ncbi       ttccgttgtctagtctgtgtctatg
     Run Soaplab on top of your
            EMBOSS
• Install EMBOSS
• Install the rest 
• run AppLab and Tomcat
  servers
• deploy Soaplab services
• check with Soaplab and your
  clients
          Add your own analysis

• add a “real” analysis
  – cal [–m] [–j] [–y] [[month] year]
  –   write an ACD file
  –   convert it to XML, create MyApp.xml
  –   edit run-AppLab-server script
  –   restart AppLab and Tomcat server
       • deploy:
       • ws/deploy-web-services –n cal –l
  – try by: ws/run-analysis-client
• add a Gowlab-based resource

								
To top