Lessons from MDC1 -- Preparing for MDC2

Document Sample
Lessons from MDC1 -- Preparing for MDC2 Powered By Docstoc
					                  Lessons from MDC1:
                   Preparing for MDC2

Stephen Johnson
           MDC1 Accomplishments (1)

• Overall: qualified success
• Institutional Contributions
   – CU, SB, VU, LNL, BNL, etc.
• On nearly all measures, PHENIX contributed heartily to RCF
  plans, exercises, etc.
   – instrumental in making MDC1 a success for RCF!

• “Sinking” -- Goal: place ‘data’ on HPSS
   –   500GB of simulated data into HPSS
   –   9 MB/sec -- saturation of connection/tape writing
   –   PRDF in muon arms
   –   Not PRDF format for central arm (more on this later)
          MDC1 Accomplishments (2)

• Reconstruction -- Goal: run reconstruction jobs on the farm,
  accessing data and returning results to HPSS
   – Successfully used RCF-developed reconstruction control
       • Improved software (Dave!)
           – automatic submission, monitoring
       • implemented by RCF
   – Of the 100K pisa events -- only a few thousand made it to
       • some pisa files didn’t make it to HPSS
       • many files were not reconstructed entirely
           – file truncation
           – reconstruction code problems
           – afs hiccups
          MDC1 Accomplishments (3)

• “Mining”
   – ORNL tape optimizing software for retrieving DST’s
       • 1MB/s -> 5MB/s (tape reads at ~10MB/s)
   – Currently all DST’s fit on disk
   – Preliminary analyses of PWG and individuals
• Databases:
   – production database (OBJY)
   – geometry database for drift chamber -- proof in principle
                  Lessons: Manpower

• At any one time, had a handful of people working on MDC
   –   Reconstruction: 1-2
   –   Sinking: <1
   –   PRDF: 1-2
   –   Keeping STAF running, database: 1
   –   Analyzing DST’s: ~1
   –   General coordination, communication: 1-2
• Need more before/during/after MDC2!!
   – if we’re not working on it now, we’re behind
        • luckily we are
                 Lessons: Preparation
• Need MDC coordinator
    – Pick one prior to the week before!
• Stephen’s perfect world: Need at least a few
  people thinking/working predominantly on MDC
    – This does not include Dave, Mark, Jeff, Charlie!! who
      already have predefined and time-consuming jobs
• Opinion: Lack of sufficient preparation led to
    – No PRDF for central arms during MDC1 (contrary to
      our best efforts during)
    – Minimal database use
        • Great to have one expert on DB (Mark) -- this is a BIG
          job though and he’s also the STAF expert and the … and
          … and …
•   Note: lack of manpower         lack of preparation
•   Not: lack of working           lack of preparation
                        Lessons: RCF
• Understaffed!! Reactionary.
• Personal experience:
   – most RCF’ers sincerely success oriented (weekend workers)
   – respond well to personal questions/requests
       • especially to PHENIX/Dave (walk down the hall!)
• Communication, Communication, Communication
   – e.g. /phenix/workarea fiasco
   – RCF problem? Solution?
• Still some environmental changes on the horizon
   – some expected instabilities
   – Fixed: HPSS, farm of linux boxes
• Learning from their mistakes?

                                   FOR MORE INFO...
                         For MDC2

• The plan:
   – only 2 weeks in ~February -- 3 short months away!

• Blitzkrieg
   – Exclusive use of HPSS and linux farm
   – Sink, reconstruct, analyze simultaneously
   – Need to prove the principle behind day-1 operation

• Different format from MDC1
   – lack of sufficient preparation spells disaster (as opposed to
                 MDC2 plans:Sinking

• Large set of PRDF files
   – How many? Central? Muon? Together?
• Determine pisa/response parameters
   – e.g. Expected detector resolutions, t0, etc.
• Event mixing
   – at PRDF level
   – needs extra subsystem work (?)
       • then should be done soon
• Integration with offline -- how much?
   – Calibrations? Signals from actual detectors? Other?
         MDC2 plans: Reconstruction
• Currently has majority of attention
• Although many (expected) subtleties/bugs, I’m not worried
  about algorithm development, etc.
• Databases:
   – access to detector parameters/calibrations, etc.
   – preliminary discussions have begun
   – will we be ready for MDC2??
• Monitoring software
   – more sophisticated (e.g. ability to restart jobs that have failed,
     start them at the point they failed, etc.)
• Process of feeding events to farm
   – currently a simple model -- is that what we need/want?
            MDC2 plans: DST analysis

• Continuing small efforts
    – mostly focused on reconstruction problems
• Some efforts by PWG, individuals
    – Hadron, LVM, muon (?) others??
• Efforts here will refine reconstruction, develop mature
  physics analysis
• Data carousels/Grand Challenge ?
    – Lots of murmuring -- need to study options and make a decision
•   mDST production -- major mode of operation
    – very little (if at all) during MDC1

• MDC went better than I expected
   – simple format of RCF = lack of preparation not a killer
   – development of structure
   – majority of basic tasks performed
• But
   – PRDF woes held us up
   – lacking in sophistication -- too much babysitting.
   – good feedback -- back to the drawing board
• MDC2 preparation needs to start now
   – with real manpower and effort

Shared By: