Condor - AP roject and a System by 1PHef8h0


									Packaging & Testing:
    NMI & VDT

             Alain Roy
 Computer Sciences Department
 University of Wisconsin-Madison
  In Brief: What is NMI?
› NSF Middleware Initiative
› Funding for middleware infrastructure
› Build, package, and test grid software

  In Brief: What is VDT?
› Virtual Data Toolkit—GriPhyN project
› Grid middleware + tools for virtual data
› Supports physicists in GriPhyN, usable
 by anyone
  Currently in active use in CMS, Atlas, EDG,
    and more
› Very easy to install

       Why do you care?
› Condor participates in grid activities
› Condor team members are part of
› More testing means better Condor
  and Condor-G
› NMI & VDT make it easy for you to
  start with grid software

         NMI at Condor
› Three full-time Condor team
  members work on NMI
› Build & package grid software
  Only quality software is included.
  Ensure that versions that work together
   (not always easy to do!)
› Test infrastructure

 NMI Testing: Verification
› NMI does basic verification testing
  Can we submit a Globus single job?
  Can we submit a Condor-G job?
  Can we transfer a single file?

 NMI Testing: Local Grids
› NMI has seven computers that can be
 organized in a local-area grid

› Will expand with more computers,
 more architectures

 NMI Testing: Local Grids
› Stress testing with DAGMan
  Coordinated tests of thousands of jobs
  Jobs distributed across grid sites
  Tests Condor-G, Condor, Globus
› Real-life testing with CMS application
  Tests everything: Globus, Condor-G,
   Condor, file transfers…
  Real-life workload, high stress

NMI Testing: Larger Grids
› We can attach more resources to
 local grids for stress testing
  UW Condor pool with hundreds of nodes
  Remote resources
› Distributed testing across NMI
 partner sites

         VDT at Condor
› Two full-time Condor team members
› Packaging and installation
  NMI packages software
  VDT installs it beautifully
› Packaging and installation are not
  glamorous, but are essential
› NMI & VDT work together and make
  each other better
         VDT installation
› VDT installation goal:
  You hit a button, the software is
   correctly installed and configured.
  (You can customize it afterwards.)
› What does this depend on?
  Excellent packaging
  Excellent installation
  Excellent testing

     VDT example installation
› Download Pacman
› pacman –get VDT-Server
   Answer a few questions
› pacman –get VDT-Client
› ls
condor/   ftsh/     Pacman.db        replica/        vdt/
doc/      globus/   perl/            setup.csh       vdt-install.log
edg/      gpt/      post-install/

            VDT Testing
› Wouldn’t it be nice if VDT could
 share with NMI?
  VDT and NMI people are both on the
   Condor staff
  Collaboration to share:
    • Test harness
    • Tests
    • Test infrastructure

Where can you learn more?
› NMI:
  Bill Taylor:

› VDT:
  Alain Roy:

     Take-home message
› NMI & VDT are pushing us towards:
  Heavily tested grid middleware
  Supported software infrastructure

  Better support, better software


To top