Lessons from MDC1:
Preparing for MDC2
MDC1 Accomplishments (1)
• Overall: qualified success
• Institutional Contributions
– CU, SB, VU, LNL, BNL, etc.
• On nearly all measures, PHENIX contributed heartily to RCF
plans, exercises, etc.
– instrumental in making MDC1 a success for RCF!
• “Sinking” -- Goal: place ‘data’ on HPSS
– 500GB of simulated data into HPSS
– 9 MB/sec -- saturation of connection/tape writing
– PRDF in muon arms
– Not PRDF format for central arm (more on this later)
MDC1 Accomplishments (2)
• Reconstruction -- Goal: run reconstruction jobs on the farm,
accessing data and returning results to HPSS
– Successfully used RCF-developed reconstruction control
• Improved software (Dave!)
– automatic submission, monitoring
• implemented by RCF
– Of the 100K pisa events -- only a few thousand made it to
• some pisa files didn’t make it to HPSS
• many files were not reconstructed entirely
– file truncation
– reconstruction code problems
– afs hiccups
MDC1 Accomplishments (3)
– ORNL tape optimizing software for retrieving DST’s
• 1MB/s -> 5MB/s (tape reads at ~10MB/s)
– Currently all DST’s fit on disk
– Preliminary analyses of PWG and individuals
– production database (OBJY)
– geometry database for drift chamber -- proof in principle
• At any one time, had a handful of people working on MDC
– Reconstruction: 1-2
– Sinking: <1
– PRDF: 1-2
– Keeping STAF running, database: 1
– Analyzing DST’s: ~1
– General coordination, communication: 1-2
• Need more before/during/after MDC2!!
– if we’re not working on it now, we’re behind
• luckily we are
• Need MDC coordinator
– Pick one prior to the week before!
• Stephen’s perfect world: Need at least a few
people thinking/working predominantly on MDC
– This does not include Dave, Mark, Jeff, Charlie!! who
already have predefined and time-consuming jobs
• Opinion: Lack of sufficient preparation led to
– No PRDF for central arms during MDC1 (contrary to
our best efforts during)
– Minimal database use
• Great to have one expert on DB (Mark) -- this is a BIG
job though and he’s also the STAF expert and the … and
… and …
• Note: lack of manpower lack of preparation
• Not: lack of working lack of preparation
• Understaffed!! Reactionary.
• Personal experience:
– most RCF’ers sincerely success oriented (weekend workers)
– respond well to personal questions/requests
• especially to PHENIX/Dave (walk down the hall!)
• Communication, Communication, Communication
– e.g. /phenix/workarea fiasco
– RCF problem? Solution?
• Still some environmental changes on the horizon
– some expected instabilities
– Fixed: HPSS, farm of linux boxes
• Learning from their mistakes?
FOR MORE INFO... http://www.rhic.bnl.gov/RCF
• The plan:
– only 2 weeks in ~February -- 3 short months away!
– Exclusive use of HPSS and linux farm
– Sink, reconstruct, analyze simultaneously
– Need to prove the principle behind day-1 operation
• Different format from MDC1
– lack of sufficient preparation spells disaster (as opposed to
• Large set of PRDF files
– How many? Central? Muon? Together?
• Determine pisa/response parameters
– e.g. Expected detector resolutions, t0, etc.
• Event mixing
– at PRDF level
– needs extra subsystem work (?)
• then should be done soon
• Integration with offline -- how much?
– Calibrations? Signals from actual detectors? Other?
MDC2 plans: Reconstruction
• Currently has majority of attention
• Although many (expected) subtleties/bugs, I’m not worried
about algorithm development, etc.
– access to detector parameters/calibrations, etc.
– preliminary discussions have begun
– will we be ready for MDC2??
• Monitoring software
– more sophisticated (e.g. ability to restart jobs that have failed,
start them at the point they failed, etc.)
• Process of feeding events to farm
– currently a simple model -- is that what we need/want?
MDC2 plans: DST analysis
• Continuing small efforts
– mostly focused on reconstruction problems
• Some efforts by PWG, individuals
– Hadron, LVM, muon (?) others??
• Efforts here will refine reconstruction, develop mature
• Data carousels/Grand Challenge ?
– Lots of murmuring -- need to study options and make a decision
• mDST production -- major mode of operation
– very little (if at all) during MDC1
• MDC went better than I expected
– simple format of RCF = lack of preparation not a killer
– development of structure
– majority of basic tasks performed
– PRDF woes held us up
– lacking in sophistication -- too much babysitting.
– good feedback -- back to the drawing board
• MDC2 preparation needs to start now
– with real manpower and effort