DAQ for TestCommissioning at Point 2

Document Sample
DAQ for TestCommissioning at Point 2 Powered By Docstoc
					DAQ for Test&Commissioning
         at Point 2

           Klaus Schossmaier
              CERN PH-AID

     ALICE TPC Collaboration Meeting
          Heidelberg, Germany
           12-13 February 2004
                                        Planning Parameters

                                                       System: DAQ system for
                                                                Test&Commissioning
                                                                (= LHCC milestone)

                                                       Size:               12 DDLs for TPC
                                                                            8 DDLs for others

                                                       Place:              LHC Point 2

                                                       Date:               Q1 2005

                                                       People:             ALICE DAQ team


TPC Collaboration meeting, 12-13 Feb 2004   DAQ Test&Commissioning at Point 2                   2
                               DAQ Installation at Point 2

           SR Hall                                  SX Hall                                           SXL Hall
         (Networking)                                                                       (ALICE sub-detector assembly)




                                            ACR      WR1      WR2

                                                                                                                            DDL
                                              PX24/CR1 (DAQ)
          LAN                                                                                                               LAN
                                              PX24/CR2 (HLT)
                    Access Shaft
                                              PX24/CR3 (DCS)

                                              PX24/CR4 (Misc.)

                                            UX25 (Experimental Area)




TPC Collaboration meeting, 12-13 Feb 2004               DAQ Test&Commissioning at Point 2                                         3
                                   Installation at PX24/CR1
                              PX24/CR2

                                                             1 DSS (DAQ Server Services):
HLT (High
Level Trigger)                                               rackmount PC
                                                             U4 height                          1 KVM switch
                                                             hot-swap SCSI disks
                                                              RunControl, Logger
                                                              On-line Monitoring


 DDL patch                                                                  PDS (Permanent Data Storage)
 panels                                                                        CERN computer center



 20 DDLs
                                            GE switches


                                                                                  1 FC switch



         10 LDCs (Local Data Concentrator):          4 GDCs (Gobal Data Collector):               1 TDS (Transient Data Storage):
         rackmount PCs                               rackmount PCs                                rackmount disk array
         U1, U2, U4 height                           U1 height                                    U2 height
         equipped with 20 D-RORC cards               equipped with FC cards                       IDE of FC disks
          DDL readout, sub-event building            full-event building                         event buffering
          HLT interface
TPC Collaboration meeting, 12-13 Feb 2004          DAQ Test&Commissioning at Point 2                                          4
                                            Installation Issues
                             Hardware/Mechanical issues:
                                     Equipment: rackmount PCs, switches, disk array, …
                                     Rack assembly, DDL installation, cabling, …
                                     Infrastructure: power supply, networking, cooling, …
                             Software/Testing issues:
                                       Operating System: Linux + customizations
                                       DAQ Software: DATE + configuration
                                       Standalone tests
                                       Integration tests


                             Reference System in our AID lab:
                                     small, complete DAQ system with 6 DDLs
                                     to spot problems in advance
                                     to tackle problems during operation

TPC Collaboration meeting, 12-13 Feb 2004         DAQ Test&Commissioning at Point 2          5
                                  Reference Setup – Front

                                                                                L3 rack
            DDLs:
            - to the DDL lab




          4x LDCs:
          - 1U, 2U, 4U height
                                                                                2x GB Ethernet switch:
          - dual Xeon
          - 6x D-RORC cards DDL                                                3COM SuperStack 3

                                                                                KVM switch:
                                                                                Raritan Paragon UMT2161
         2x GDCs:
         - 1U height                                                            Fibre Channel switch:
         - dual Xeon                                                            Broacade SilkWorm 3800
         - Qlogic QLA2310F cards

                                                                                2x disk arrays:
         DSS:                                                                   - Infortrend IFT-6330
         - 4U height                                                            - DotHill SANnet II
         - quad Xeon                                                            - Fibre Channel
         - 3x 36GB SCSI disks


TPC Collaboration meeting, 12-13 Feb 2004   DAQ Test&Commissioning at Point 2                            6
                                    Reference Setup - Rear

                                                                                L3 rack




             Cat5 cables:
             - Ethernet                                                         Mounting rails (!)
             - KVM
             - RJ45 connectors
                                                                                Power
                                                                                distributor



             Fiber Channel cable:
             - 2 Gbit/s multimode
             - LC-LC connectors




TPC Collaboration meeting, 12-13 Feb 2004   DAQ Test&Commissioning at Point 2                        7
                                            DDL & D-RORC

                         DDL link card                      DDL patch panel with D-RORC




      Uniform half-CMC card for both                           Two D-RORC types: 1 DDL link card
       SIU (= Source Interface Unit) and                         plugged in, 2 integrated DDL channels
       DIU (= Destination Interface Unit)
                                                                PCI 64 bit, 66 MHz
      2.125 Gbit/s components (Agilent)
                                                                ALTERA FPGA, IP core from PLDA
       for 200 MB/s bandwidth
                                                                200 MB/s throughput with DATE
      bridge up to 200 meters
TPC Collaboration meeting, 12-13 Feb 2004    DAQ Test&Commissioning at Point 2                       8
                                            Rackmount PCs


         Evaluation/Benchmarking item               LDC                        GDC                     DSS

      Rackmount height                           1U, 2U, 4U                        1U                   4U

      CPU                                     dual Xeon 2.4 GHz        dual Xeon 2.66 GHz      quad Xeon 1.5 GHz
      chipset                                       E7501                    E7501             ServerWorks GC-HE
      motherboard                           Supermicro X5DPE-G2       Supermicro X5DPI-G2          Intel SRSH4

      Main memory                                 2 GB                       2 GB                     4 GB
                                               DDR 266 MHz                DDR 266 MHz              DDR 266 MHz
      Ethernet                              dual Gigabit Ethernet     dual Gigabit Ethernet      Gigabit and Fast
                                                  on-board                  on-board            Ethernet on-board
      Disk                                       40 GB IDE                  40 GB IDE               hot-swap
                                                                                                36 GB SCSI disks
      PCI slots                                 up to 6 PCI-X                1 PCI-X                 6 PCI-X
                                             for D-RORC cards        for Fibre Channel card          2 PCI 5V
      Linux installation                       Red Hat 7.3               Red Hat 7.3        Red Hat 7.3 (rather old)




TPC Collaboration meeting, 12-13 Feb 2004      DAQ Test&Commissioning at Point 2                                         9
                                            Disk Array Systems

                                                                Infortrend IFT-6330-12F2D:
                                                                        12x 300 GB IDE-based disks
      front view                                                        dual 2 Gbit/s Fibre Channel ports
      (IFT-6330)
                                                                        Embedded manager via
                                                                         front panel, RS232C, or IP
                                                                        2x logical volumes: each one has
                                                                        1.1 TB, RAID5, with 5 disks


                                                                DotHill SANnet II 200:
      rear view
      (IFT-6330)                                                        12x 36.7 GB FC-based disks
                                                                        dual 2 Gbit/s Fibre Channel ports
                                                                        RAID configurations via RS232C
TPC Collaboration meeting, 12-13 Feb 2004       DAQ Test&Commissioning at Point 2                            10
                                       DAQ Software: DATE

                   DATE (= Data Acquisition and Test Environment)
                    is the software framework of the ALICE DAQ.

                   Runs on Linux platforms (e.g. Red Hat 7.3)
                   Released (latest is version 4.7) and documented
                   DATE evolves with requirements and technology

                   Features/Packages:
                            Dataflow: DDL readout, event building
                            System configuration and Run control
                            Monitoring: on-line performance, on/off-line data quality
                            Utilities: message logger, control/configure front-end electronics, …


TPC Collaboration meeting, 12-13 Feb 2004     DAQ Test&Commissioning at Point 2                      11
                                            Readout with DATE

                   TPC RCU 3 with SIU
                                                                      Step 0: Hardware is in place
                                                                      Step 1: Run Linux with the
                                                                       physmem and the rorc driver
                                                                      Step 2: Install DATE with a
                                                   FEC               general configuration for the LDC
                                                                      Step 3: Set up configuration file
                                       DDL                            to enter parameters like D-RORC
                                                                      serial number, channel number,
                   D-RORC with DIU                                    page size, checks, etc.
                                                                      Step 4: Start/Stop the acquisition
                                                                      Step 5: Analyse the raw data
                                                                      files, e.g. /tmp/run123.000,
                                                 LDC                             /tmp/run123.001, …

TPC Collaboration meeting, 12-13 Feb 2004      DAQ Test&Commissioning at Point 2                            12
                                            DATE Run Control




TPC Collaboration meeting, 12-13 Feb 2004     DAQ Test&Commissioning at Point 2   13
                              FEE Configuration/Control

                   TPC RCU 3 with SIU                             Step 0: Hardware is in place
                                                                  Step 1: Run Linux with the
                                                                   the rorc driver on the LDC
                                                                  Step 2: Write a script (ASCII file)
                                                 FEC
                                                                  for the FeC2 interpreter
                                                                  Step 3: Run this script, e.g. type
                                       DDL
                                                                  FeC2 123 example.fec2
                                                                           S/N of D-RORC
                                                             # send command, 19 bit information
                   D-RORC with DIU                           write_command 0x10f
                                                             # read status, 19 bit address, 19 bit reply
                                                             read_and_print 0x100 “Register XYZ: 0x%x”
                                                             # download data block, 19 bit address, file
                                                             write_block 0x600 pedestal.hex “%x”
                                                             # verify data block, 19 bit address, file
                                               LDC          read_and_check_block 0x600 pedestal.hex “%x”
                                                             stop_if_failed -1

TPC Collaboration meeting, 12-13 Feb 2004    DAQ Test&Commissioning at Point 2                              14
                                            On-line Monitoring


 MOOD Framework:
        DATE + ROOT environment
        Interfaces to detector code
 MOOD Applications:
        Visualization, e.g. TPC sector
         test data (Hall 167)
        Data integrity
        Detector performance
 Execution:
        DSS machine




TPC Collaboration meeting, 12-13 Feb 2004      DAQ Test&Commissioning at Point 2   15
                                            DAQ Performance

                         ALICE Computing Data Challenge:
                                  1750 MB/s event building without recording, 5 days
                                  280 MB/s event building and recording to CASTOR, 7 days
                         D-RORC readout of one DDL channel with DATE:



                                                                                    206 MB/s readout only


                                                                                    118 MB/s readout with
                                                                                             event building




TPC Collaboration meeting, 12-13 Feb 2004       DAQ Test&Commissioning at Point 2                             16
                                             DAQ Installations


                   DAQ System               Location       DDL        LDC       GDC          Networking           Storage

            Development / Evaluation         AID lab         2         4         4      Gb Ethernet        disk

            DDL Development                 DDL lab          2         1         0      Gb Ethernet        disk

            Detector Groups (e.g. SDD,      Institutes       1         1         0      Fast/Gb Ethernet   Disk
            HMPID, TOF, Muon, TRD)
            TPC Sector Test §                PS T10          2         1         0      Fast Ethernet      disk or CASTOR

            ALICE Data Challenges           CERN IT          0         80        80     Gb Ethernet        TDS + CASTOR

            Reference Setup                  AID lab         6         4         2      Gb Ethernet        TDS

            Test&Commissioning               Point 2        20         10        4      Gb Ethernet        TDS + CASTOR

            Final System                     Point 2       395        185        50     Gb Ethernet        TDS + CASTOR



        §   Given that the RCU 3 is fully functional in time, see Joachim’s presentation.




TPC Collaboration meeting, 12-13 Feb 2004                DAQ Test&Commissioning at Point 2                                  17
                                  Summary + Future Work

                       Objective:
                               DAQ system for Test&Commissioning at Point 2
                               Surface tests of ALICE sub-detectors (e.g. TPC sector)
                               Operational in Q1 2005
                       Installation Layout:
                               20 DDL links between the PX24/CR1 counting room and the
                                SXL mounting hall
                               L3 racks: DDL patch panels, 15 rackmount PCs, 1 disk array,
                                switches for Gigabit Ethernet, Fiber Channel and KVM
                               DAQ Reference System in the AID lab
                       Future work concerning TPC and DAQ:
                               Bergen: Development of RCU 3 firmware for the DDL interface
                               Luciano’s lab: Integration test of RCU 3 with the DDL
                               TPC sector test at T10§: DAQ chain with RCU 3 + DDL + DATE

TPC Collaboration meeting, 12-13 Feb 2004     DAQ Test&Commissioning at Point 2               18
      18

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:4
posted:9/7/2010
language:English
pages:18