Main Title 32pt by yurtgc548


									SciDAC SSS Quarterly Report

                             Sandia Labs

                              January 25, 2005

                            William McLendon

 Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,
       for the United States Department of Energy’s National Nuclear Security Administration
                               under contract DE-AC04-94AL85000.
                APItest - Overview

• Current Release : v0.2.5

• Test Driver tool for testing software
  – Ordered tests
  – API Testing

• Available for download:
       Work Since Previous Meeting

• October : Development meeting at ORNL
 – Imported APItest into OSCAR
 – Started developing some APItests

• Fixing bugs

• Finishing up User Guide
                     APItest - Features

• Scripted Test Driver            • Can set environment
   – XML based tests                variables for tests.
• Object Oriented                 • SUID Test Capability
   – Easy to define new test         – CAUTION!
     types.                       • Timeouts
• Conditional test ordering       • User Interface Types
   – Batch Scripts                   – Web Browser
       • Can call other batches      – Command Line
• Built In Test Types                – Archived Results
   – Shell                             Browsing
   – Script                       • Test Results Export
   – SSS                             – Save Test results to an
• Fully scripted in Python             XML file.
• Portable                        • Open Source
                                     – LGPL
               APItest - Requirements

• Packages required to run APItest:

  – Python 2.3 or greater
      • lower versions can work but install is not as easy.

  – ElementTree (

  – Twisted (

  – SSSlib (
      • optional, for SSSlib tests.
                   APItest - Execution

• There are two ways to execute APItest

• Command Line Only
   – $> apitest -f test_file.[apt|apb]

• Web browser GUI (Interactive)
  – $> apitest httpd
      • -i [--iroot] : root directory to look for tests
      • -p [--port] : port number (default = 2112)
      • -h [--host] : host (default = localhost)

• Running with --help will display command line usage.
APItest - GUI (Top Page)

             • Top level index page for

             • APItest recurses from the
               [testroot] to find tests.

             • File Extensions:
                – TEST : .apt
                – BATCH : .apb

             • Batch tests shown in bold

             • Organized by directory
APItest - GUI (Test Run Execution)

                  • Quick status view of test

                  • This screen loads when the
                    EXECUTE button is selected.

                  • Color coded for easy viewing.

                  • Currently running test is

                  • Refreshes every 15 seconds
                    until tests complete.
APItest - GUI (Test Run Summary)

                 • Summary of test results

                 • Color coding

                 • Batch Features
                    – Cumulative summaries
                        • Total # of tests
                        • # that passed (green)
                        • # that failed (red)

                 • Hyperlinks to see more
                   details about a test or batch.
APItest - GUI (Test Details - PASSED)

                    • Example of a test that was
                      executed and passed.

                    • Summary Data

                    • Dependency Data
                       – Only shown if test was run
                         as part of a batch.

                    • Output
                       – Side-by-side comparison
                       – Dependent on type of test.
APItest - GUI (Batch Summary)

                • Clicking a hyperlink into a
                  batch test gives us a
                  summary view.

                • In this example, one test
                  failed a dependency
                   – FAILDEP in the status
                      column indicates this.

                • Clicking on the
                  scriptTest2.apt link will show
                  us more data about this test.
APItest - GUI (Test Details - FAILDEP)

                     • Detailed view of
                       scriptTest2.apt from
                       previous slide.

                     • Failed a dependency on
                        – Expected : FAIL
                        – Actual : PASS
APItest - GUI (List of Test Runs)

                  • Clicking the View All Runs
                    button gives this screen.

                  • Test Runs are sorted by
                    date and time.

                  • Click links to view details
                    of a particular run.
SSS - Service Directory Startup Test

     PREP           TEST SDOFF


   RM EMNG            SDSTAT

          APItest - Ongoing & TODO

• User Guide
• Validation via file matching
• Bug fixing
• Configuration file
• More SSS tests
• Test Developer GUI
• Remote test execution
• Additional native test types
• Session management and security (Cookies?)
• DB Connectivity
             APItest - Test Types

 – Execute a command via the command line.
 – Execute a script.
 – Execute a ssslib communication.
             APItest - Example CMD Test


  Example test. Runs ls -lr from /tmp, expects 0 exit status and any input.

  <test type="cmd” wdir="/tmp">

    <command interpreter="/bin/ksh">ls</command>

    <input name="argument" format="literal">-l</input>
    <input name="argument" format="literal">-r</input>
    <input name="stdin" format="literal" />

    <output name="stdout" format="regexp">.*</output>
    <output name="stderr" format="literal" />
    <output name="status" format="literal">0</output>

    APItest - Example Script Test

  <info>Timeout test</info>

  <test name="timestamp1" type="script" timeout="4">

      <command interpreter="/bin/bash" wdir="/tmp">
for x in 1 2 3 4 5 6 7 8 9 10
   echo "timeout_test :${x}: `date`”
   sleep 1

    <output name="stdout" format="regexp">.*timeout.*</output>
    <output name="stderr" format="literal" />
    <output name="status" format="literal">0</output>

             APItest - Example ENVVAR Test
  This script executes but adds some stuff to the environment.
  NOTE: setting envvar OVERRIDES any inherited environment variables.

  <test type="script">

    <command interpreter="/bin/csh">#!/bin/csh env</command>

    <input name="envvar" format="literal" key="test_envvar_1">foobar</input>
    <input name="envvar" format="literal" key="test_envvar_2">foo</input>

    <output name="stdout" format="regexp">
    <output name="stderr" format="literal" />
    <output name="status" format="literal">0</output>

              APItest - Example SUID Test


  <test type="script">

     <command interpreter="python" uname="wcmclen" gid="500">
#!/usr/bin/env python
import os
print os.getuid()
print os.getgid()

    <input name="argument">/tmp/testuser/</input>
    <input name="argument">-l</input>

     <output name="stdout" format="regexp">^27030$\n^500$</output>
     <output name="stderr" format="literal" />
     <output name="status" format="literal">0</output>
              APItest - Example SSS Test

  <info>Sample sss query of the service directory</info>

  <test type="sss" destination="service-directory">

     <input name="sendbuf">
&lt;get-location>&lt;location component='service-directory' host='*' port='*'
protocol='*' schema_version='*' tier='*'/>&lt;/get-location></input>

    <output name="recvbuf" format="regexp">.*</output>



To top