Condor - AP roject and a System by 1PHef8h0

VIEWS: 5 PAGES: 15

									Packaging & Testing:
    NMI & VDT

             Alain Roy
 Computer Sciences Department
 University of Wisconsin-Madison
         roy@cs.wisc.edu
  http://www.cs.wisc.edu/condor
  In Brief: What is NMI?
› NSF Middleware Initiative
› Funding for middleware infrastructure
› Build, package, and test grid software




                     www.cs.wisc.edu/condor
  In Brief: What is VDT?
› Virtual Data Toolkit—GriPhyN project
› Grid middleware + tools for virtual data
› Supports physicists in GriPhyN, usable
 by anyone
  Currently in active use in CMS, Atlas, EDG,
    and more
› Very easy to install

                         www.cs.wisc.edu/condor
       Why do you care?
› Condor participates in grid activities
› Condor team members are part of
  NMI & VDT
› More testing means better Condor
  and Condor-G
› NMI & VDT make it easy for you to
  start with grid software


                      www.cs.wisc.edu/condor
         NMI at Condor
› Three full-time Condor team
  members work on NMI
› Build & package grid software
  Only quality software is included.
  Ensure that versions that work together
   (not always easy to do!)
› Test infrastructure

                       www.cs.wisc.edu/condor
 NMI Testing: Verification
› NMI does basic verification testing
  Can we submit a Globus single job?
  Can we submit a Condor-G job?
  Can we transfer a single file?




                       www.cs.wisc.edu/condor
 NMI Testing: Local Grids
› NMI has seven computers that can be
 organized in a local-area grid




› Will expand with more computers,
 more architectures

                      www.cs.wisc.edu/condor
 NMI Testing: Local Grids
› Stress testing with DAGMan
  Coordinated tests of thousands of jobs
  Jobs distributed across grid sites
  Tests Condor-G, Condor, Globus
› Real-life testing with CMS application
  Tests everything: Globus, Condor-G,
   Condor, file transfers…
  Real-life workload, high stress


                       www.cs.wisc.edu/condor
NMI Testing: Larger Grids
› We can attach more resources to
 local grids for stress testing
  UW Condor pool with hundreds of nodes
  Remote resources
› Distributed testing across NMI
 partner sites


                      www.cs.wisc.edu/condor
         VDT at Condor
› Two full-time Condor team members
› Packaging and installation
  NMI packages software
  VDT installs it beautifully
› Packaging and installation are not
  glamorous, but are essential
› NMI & VDT work together and make
  each other better
                       www.cs.wisc.edu/condor
         VDT installation
› VDT installation goal:
  You hit a button, the software is
   correctly installed and configured.
  (You can customize it afterwards.)
› What does this depend on?
  Excellent packaging
  Excellent installation
  Excellent testing


                         www.cs.wisc.edu/condor
     VDT example installation
› Download Pacman
   http://physics.bu.edu/~youssef/pacman
› pacman –get VDT-Server
   Answer a few questions
› pacman –get VDT-Client
› ls
condor/   ftsh/     Pacman.db        replica/        vdt/
doc/      globus/   perl/            setup.csh       vdt-install.log
edg/      gpt/      post-install/    setup.sh



                                    www.cs.wisc.edu/condor
            VDT Testing
› Wouldn’t it be nice if VDT could
 share with NMI?
  VDT and NMI people are both on the
   Condor staff
  Collaboration to share:
    • Test harness
    • Tests
    • Test infrastructure



                            www.cs.wisc.edu/condor
Where can you learn more?
› NMI: www.nsf-middleware.org
  Bill Taylor: bt@cs.wisc.edu


› VDT: www.griphyn.org/vdt
  Alain Roy: roy@cs.wisc.edu




                       www.cs.wisc.edu/condor
     Take-home message
› NMI & VDT are pushing us towards:
  Heavily tested grid middleware
  Supported software infrastructure


  Better support, better software




                      www.cs.wisc.edu/condor

								
To top