Document Sample
phenix_plenary_2004_12_10 Powered By Docstoc
					                     The DAQ is Back!

                            John Haggerty
                    Brookhaven National Laboratory

December 10, 2004           John Haggerty            1
                         Executive Summary

It’s back!

It’s Linux!

It’s faster (we think)

December 10, 2004            John Haggerty   2
                               Event Builder

•   David Winter deserves credit and our gratitude for pulling the Linux Event
    Builder project together… he’s done a great job
•   Network communication within the EvB is now entirely by TCP, allowing the
    network gurus to guarantee delivery of our packets; that’s as important a
    change as Linux
•   There was a frightening episode around Thanksgiving of apparently random
    kernel panics in the seb’s after switching to TCP which looked like it might
    have been a modulo a modulo ruckus in the kernel (extremely difficult for
    civilians to diagnose and repair), but Brian and David found workarounds
•   The reasons for multiple downloads (operationally clunky) have been
    understood and fixed
•   Speeds up to 6.2 kHz have been observed, albeit with very small and well
    zero suppressed events, but that means the per-event overheads have been

December 10, 2004                 John Haggerty                                3
                            Detector Readiness

•   Everything except EMCAL has joined the Big partition, and that is now
    possible to add
•   I think we can say (quietly) that the data corruption in the jseb is basically
    eliminated, and the padding banks have been removed from all the
•   There have still been problems reading some of the larger groups, notably
    MUTR, and sometimes TEC

December 10, 2004                  John Haggerty                                     4
                           Multievent Buffering

•   Looks good to go as far as I can see
•   Data taken with noise (random) trigger has not shown any signs of problems
    in MUTR and EMCAL
•   One oddity of Forced Accepts in the EMCAL I believe shows a failure of
    some kind of the old FPGA code (cell numbers should be periodic for FA’s)
•   We just this week realized that a feature of the new firmware prevented
    zero suppression from taking place in the DCM’s, but Chi seems to have
    found a fix for that
•   Next software release should make 4 event buffering the running default
•   It would be good to take data for an extended period of time with the noise
    trigger or a high rate cosmic trigger to see if there is any evidence of cell

December 10, 2004                 John Haggerty                                 5
                         Run Control and Servers

•   Ed has done a lot of housecleaning and improvements; everything is better
    integrated now, and with David has eliminated some of the operational
    annoyances (multiple downloads)
•   The initialization speed (particularly the EvB server) is much faster now
•   We have only rarely done an oncsCleanup recently
•   The next release (before Christmas) will include writing to a PostgreSQL
    runcontrol database (I’ve got php that reads it almost ready)
•   Control of the buffer boxes has been moved to its own gui; it’s better, but
    still needs some operational kinks ironed out (i.e., it works if you do exactly
    the right thing, but is prone to errors)
•   Ed has new cdev data format just about ready

December 10, 2004                  John Haggerty                                      6
                               Buffer Boxes

•   Five 6.4 Tbyte disk arrays arrived in the counting house yesterday, after
    some purchasing kerfuffles
•   We bought some additional switch modules for the Foundry switch so the
    buffer boxes and the CORBA server can be right on that switch (old Dell’s
    are getting Gigabit, too)
•   We have been logging to the 4 buffer boxes we had last year, and they are
    all Scientific Linux
•   We should blank the disks soon of accumulated detritus
•   Some furious system management when the new machines arrive, but
    everything should be in place for them

December 10, 2004                John Haggerty                                  7
                               Other Projects

•   Splitting the GL1 and LL1 readout is done; seems to work, and David and
    Brian integrated it into the EvB this week—this gets the LL1 data into
    another seb, after some DCM modifications by Jamie
•   Martin has done a lot of system management in 1008 to bring all our
    systems to SL 3.0 (including those nifty new Dell desktops)
•   New granule (TOF.W) has so far not caused us any pain
•   CORBA services got moved to a machine safely tucked away on UPS and
    Gigabit without too much fuss
•   Achim has done a lot of work on TEC noise, and there is still one idea to
    work on with ARCNET to reduce the noise further
•   STAR scalers…

December 10, 2004                 John Haggerty                                 8
                           So How fast Is It?

•   There’s no way I’m going to predict a number
•   Tests up until now have been with very, very good zero suppression, and
    events less than 50 kbyte
•   The event length will increase when we take collision data, so we may have
    to work on event bottlenecks
•   An example of this is the work that Chun and Jamie did this week to “split
    the BB,” by sending the same data to two DCM channels and dividing the
•   The speed tests have also smoked out some FEM’s that have been putting
    out data that can’t be zero suppressed (1/4 of a DC.E FEM that had its busy
    light on all the time)

December 10, 2004                John Haggerty                                9

•   The DAQ is basically back, and almost ready for shift people to use
•   We need work on documentation; I’m starting to like the Wiki I set up: so we can all collaborate on a
    searchable, web-based document
•   Many of the problems of the past have been solved

December 10, 2004                 John Haggerty                               10

Shared By: