ATLAS Lehman Review, Silicon ROD

Document Sample
ATLAS Lehman Review, Silicon ROD Powered By Docstoc
					ATLAS Lehman Review, Silicon ROD


    May 21 to May 23, 2003




   Douglas Ferguson, Richard Jared,
    John Joseph and Lukas Tomasek


              Wisconsin




                                      1
   ATLAS Lehman Review, Silicon ROD




Material Covered
 ROD Schedule and High Level Summary
 Scope
 Major Events
 ROD Overview
 Current Status




                                       2
              ATLAS Lehman Review, Silicon ROD
ROD Schedule and High Level Summary




                                                 3
              ATLAS Lehman Review, Silicon ROD
ROD Schedule and High Level Summary




                                                 4
                       ATLAS Lehman Review, Silicon ROD


High Level Summary

9% of SCT production RODs complete

                      Authorized Funds                          Paid to Date/Committed
W                     Totals($) M&S($)   Labor($)   Travel($)   Totals($) M&S($)    Labor($)   Travel($
1.1.3RODs             1905.5    1486.0   407.5      12.0        167.7      51.5     112.2      4.0

Milestones                               ETC02 Actual Projected
1.1.3.8.1.6   Pixel ROD PRR                6/03         9/03
1.1.3.8.1.8   SCT ROD PRR                 4/03          6/03
1.1.3.8.2     SCT Prod Complete           10/03         11/03
1.1.3.8.3     Pixel Prod Complete          3/04         3/04
1.1.3.8.4     Pur ROD SCT Crates           3/03         9/02 Two crates have been ordered
1.1.3.9.1     Installation complete       2/05           2/05




                                                                                               5
            ATLAS Lehman Review, Silicon ROD




ROD Comparison of Costs


Small call on contingency




The ETC03 had a +48K difference from ETC02 (pixel included). This
is in the noise. The pixel ROD have been added to the baseline.




                                                               6
         ATLAS Lehman Review, Silicon ROD
Scope
 Baseline Scope ATLAS ROD
  Pixel test beam support hardware
  Design to production models SCT and pixel RODs
  Development of ROD test stand
  Fabrication of SCT RODs
  Fabrication of pixel RODs
  Purchase of SCT and pixel ROD crates


Management Contingency
  allocated


                                                   7
               ATLAS Lehman Review, Silicon ROD
Major Events

1. Fabrication and debugging of 9 RevC production model RODs has been
performed.
2. Successful testing of data flow through the RevC production model ROD at
Cambridge.
3. The test stand software has been upgraded to support pixel user evaluation.
4. DSP software has been coded to support user evaluation of SCT and pixel
user evaluation.
5. The ROD passed the US ATLAS FDR.
6. SCT user evaluation has had much effort in the UK. They have configured
modules, readout events and histogrammed the the data.
7. The ATLAS PRR was held on May 13, 2003. The chairman Philippe
Farthouat has said that we passed (report on May 23).
8. Pixel user evaluation has started. Modules have been configured. Testing of
events readout is currently being done .
9. The pixels will require a revised ROD (no changes to VHDL of C code.
FPGAs will be changed to provide more derandomizing space and the DSPs
will be changed to provide more memory space.


                                                                            8
                     ATLAS Lehman Review, Silicon ROD
                           ROD Overview


Back of Crate Card
Pixel or Si Strip
front end electronics   Clock, Trig. and control                  XON/XOFF
                        Event data and                                                ATLAS
                        register contents           1.0             Formatted         Read-out
                                                                    Event Data        Buffers
                                                     ROD
                                                   Preprototype     Event Data,
                                                                    Registers and
      TIM               Clock, Trigger               Card           Status             RCC
                        & event number
                                                                  Configuration
ATLAS                                                                               Local processing
Clock and trigger           BUSY                                  and Control
Distribution




                                                                                                  9
          ATLAS Lehman Review, Silicon ROD
                           1.0 ROD Card

XON/XOFF
                    1.1                    Trans Data
 CLK,                                                             1.2
 Trigger &   Back of Crate                   Rec. CLK
 Control        Card                        Rec. Data
Event data &                                                   Preprototype
             Optical Interface      Bias
Register                                    XON/XOFF           ROD
Contents       1 of up to 16                 R/W Bus
               shown                        ROD data           1 of up to 16
  ROB
  Data                           DC Power                      shown

ATLAS                            Tran CLK Clock, Trigger,
Clk &                                       ids, resets, CAL
Trigger           Tim              VME
  BUSY       Clock and Trigger
                                              BUSY
               Distribution

                                             VME,
                  RCC                        configuration,
 ATLAS                             VME       Control and
 Ether net                                   Monitoring
                 Crate
               Processor




                                                                               10
                                                                    ROD Implementation: FPGA Front End / DSP Back End



                                                                                                         FPGA




                                                                                                                                                                                                          Slink on BOC
                                                                                                                                                                                        Xon/Xoff
                                                                                                                                        Router Halt Output       Router
                                              96       Data    96
                                                                                                       Formatter                        Data Valid               FPGA
                                                                                                        and EFB                                                                       S-link Data
Back of Crate Card / Front End Electronics



                                                                                                                                                                 S-Link
                                                      RCVR                                                                              S-link Data 43
                                                                                                                                                                 & DSP
                                             Serial
                                             Input                                                                                                               DMA
                                             Link                                                                                Control & Status
                                                                                                                                 for Block Xfer/




                                                                                                                                                                DMA 1
                                                                    Token and Dynamic Mask




                                                                                                                                                                              DMA 4
                                                                                                                                 DMA / S-Link
                                                                                                                                                                        ...           DMA




                                                                                             Control, Status, Mask
                                                                                                                     R/W
                                                                     Event Data/Tirg Type
                                               Input    R/W                                                                                                                           Cont.
                                                                                                                     Bus
                                                                                              Dec. Trig Count
                                              Memory    Bus



                                                                                                       Busy
                                              & XCVR                                                                              Host Port                     DSP Event
                                              Control                                                                             Interface                     Trapping,
                                                         R/W                                                                                                    Histograms
                                                         Bus                                                               Boot FPGAs                        (Four DSP Chips)




                                                                                                                                                                                                          VME Bus
                                              BOC                                                                          Program            Data Bus
                                              Setup                                                                                               32         VME Slave
                                               Bus                                                                         Reset
                                                        ROD Controller:                                                    Manager         Cont / Addr        Interface
                                                           Operation,




                                                                                                                                                                                                    Timing Interface
                                             Serial                                                                              ROD Busy
                                             Output     Command, Trigger




                                                                                                                                                                                                        Module
                                              Link                                                                               Trig Data




                                                                                                                                                                                               11
                       ATLAS Lehman Review, Silicon ROD
                               slave DSPs & memories         power supplies   program FLASH




                                                                                      boot
                                                                                      FPGA


                                                                                      ROD bus
       router                                                                         buffers
       FPGAs

       event                                                                             controller
       builder                                                                           FPGA
       FPGA



debug &                                                                                  master DSP
derandomizing                                                                            & memory
memories




        data formatter FPGAs           real time data path
                                                                                              12
           ATLAS Lehman Review, Silicon ROD
Current Status (ROD hardware)
The data and control path in the ROD have been coded and tested. The
SCT VHDL is in maintenance mode (changes implemented as needs are
found). C code is mostly written for module calibration.

EFB FPGA                  VHDL in maintenance mode

Router FPGA              VHDL in maintenance mode

Formatter FPGA              SCT VHDL in maintenance mode
The formatter for the pixel 40MHz links has been written.
The formatter VHDL for the pixel 80Mhz links has to be written.

Controller FPGA           VHDL in maintenance mode

Program Reset Manager     VHDL in maintenance mode




                                                                       13
           ATLAS Lehman Review, Silicon ROD


Current Status (ROD software )
The infrastructure software is written and tested. Many primitives have
been written and tested. The code needed for module calibration has been
written. The code is being upgraded as the user evaluation needs are
defined from use.


The histogramming code will need (for speed) to be executed in machine
language once the full functionality is tested by the users. Queuing of
events in the FIFOs will also be needed to improve the speed.

Future needs:

         histogram fitting,
         occupancy measurement,
         error detection, counting & processing.


                                                                           14
                  ATLAS Lehman Review, Silicon ROD


Current Status (Test Stand Software )
The test stand software is complete and is in maintenance mode. Updates are made as
needed.




Concerns:
The upgrade to the pixel ROD has some risk because the FPGA and DSP pins will
change and the printed circuit card will be rerouted. Three boards will be initially
fabricated to reduce risk.
The time to calibrate the SCT modules is too long. The C code will have to be changed
to improve speed.




                                                                                  15
                       ATLAS Lehman Review, Silicon ROD
US ATLAS FDR (short version)
                      Review Committee Report
       for the SCT & Pixel ROD US-ATLAS FDR of 20-Aug-2002
Introduction:
A Final Design Review of the Read Out Driver (ROD) intended for use by the ATLAS SCT and Pixel subsystems
was held at LBNL on 20-Aug-2002. The primary intent of this review was to evaluate the present status of the
ROD design for use by the SCT and Pixels with a goal of approving the build of 9 pre-production RODs so that
further user evaluation can be obtained prior to a Production Readiness Review (PRR) as early as possible.
While the goal is for the ROD design to be compatible for both SCT and Pixels, it is recognized that there has
been as yet no user evaluation of the design with Pixel readout. Given the schedule constraints of the SCT and
the overall cost constraints of the project, the plan is to evaluate the appropriateness of the present ROD design
for both SCT and Pixels and decide on the pre-production build knowing that further design modifications may
be needed for Pixels after appropriate user evaluation is completed with that readout.

The review committee included:
Christopher Bebek – LBNL
Kevin Einsweiler – LBNL
Alex Grillo – UCSC
Henrik von der Lippe – LBNL
Abe Seiden - UCSC




                                                                                                         16
                       ATLAS Lehman Review, Silicon ROD
US ATLAS FDR (continued)
Recommendations:
The committee was impressed with the amount of work accomplished since the last review. Basic functionality
     of the SCT ROD has been shown along with the first examples of on-board histogramming. While more
     user evaluation is still needed to be sure that the ROD meets all requirements for SCT and Pixels, this is at
     least a good start. The major obstacle to such evaluation is software for a DAQ system and focused
     attention of users running the ROD through all its necessary operations. User evaluation of the Pixel
     version is awaiting a Pixel module to readout.
The committee makes the following recommendations to the ROD design group:
1.   It is clear that the ROD development is ready to proceed to implement all identified design changes into a
     new PCB layout. Also, more ROD units will enhance the chances to obtain more, much needed, user
     evaluation. Therefore, the committee strongly recommends that the design group proceed as quickly as
     possible to complete layout of the next version of the ROD PCB and fabricate sufficient boards to
     eventually load 9 boards. Given the changes to be made to the layout, we recommend that initially only
     two boards be loaded and tested. The committee at first thought that only one should initially be loaded
     for testing but we have accepted the advice of the ROD design team that the initial debug of two boards
     facilitates diagnosis of problems by having two samples to compare. Once basic functionality is
     demonstrated, the remaining 7 boards can be loaded.
      Note: 3 then 7 boards have been fabricated. These boards have been used in user evaluation.
2.   The alternate footprint (BGA456 in addition to BGA676) for the Formatter on pixel boards should not be
     executed. We believe that this creates too big a risk in the layout for an option that may not be needed or
     used. If the planned Formatter FPGA proves to have insufficient performance for the Pixel readout, one
     of the higher performing FPGAs with the same footprint can be utilized (405/EM or 600/E). The added
     cost in that case is a worthwhile risk instead of the risk to the layout now.
      Note: The change was not executed.
                                                                                                          17
                       ATLAS Lehman Review, Silicon ROD
US ATLAS FDR (continued)
3.   The allowable power supply margins of the FPGAs and DSPs should be analyzed vs. the regulation spec of
     the DC-DC converter over the temperature and supply range expected for the board. If it is not
     convincing that a single supply will work reliably for long-term use, the power plane should be split and
     two DC-DC converters should be employed. While splitting the power plane does imply some risk, care at
     this time to avoid errors may be a better risk than possible reliability problems over the lifetime of the
     ROD.
     Note: An analysis was performed and no change was made.

4.   Attempt to optimize the FPGA VHDL code as planned to reduce the utilization. At the time of the PRR,
     the utilization must be reviewed again. If it is still uncomfortably high to accommodate possible needed
     enhancements during commissioning, a decision will have to be made then to upgrade to the high cost
     FPGA.
      Note: FPGA utilization is now Formater 70%, EFB 56% Router 67%, Controller 70% and PRM 50%.
5.   A discussion should be held with the person now re-designing the S-link board for ATLAS reviewing the
     ROD’s S-link interface vs. those of the S-link to confirm that there are no compatibility issues.
      Note: The S-link was reviewed with CERN and has been tested as part of the user evaluation.
6.   There seem to be legitimate concerns about how the detailed phase of the data streams from the on-
     detector electronics will be monitored and timed relative to the TIM-generated crossing clock. Since the
     BOC is the module responsible for this synchronization, it is suggested that this is the correct place to
     implement monitoring of the timing of the “raw” data streams. The SCT/Pixel off-detector design group
     should consider how this may best be implemented in the final system. However, we suggest that the pre-
     production RODs should simply provide a reasonable number of header-based test point, close to the
     FPGAs, for logic-analyzer analysis, but should not attempt to provide multiplexed front-panel access to
     the full set of input data streams.
     Note: The BOC supports passing the raw data to the ROD. A multiplexer has been added to the Formater FPGA
     to allow selection of one stream at a time for monitoring on a front panel connector. Module timing is performed
                                                                                                              18
     by varying the BOC timing and observing the results with the ROD.
                      ATLAS Lehman Review, Silicon ROD

US ATLAS FDR (continued)
7.   Since the current ROD design does not meet the requirement to readout events via VME at 1kHz with a
     full 16 RODs/crate and it would require a major redesign to try to accomplish this, but it appears that one
     ROD/crate would be sufficient for any planned ROD use, the committee recommends that this
     requirement be changed to “VME readout at 1kHz for 1 ROD/crate.”
      Note: The requirement was changed to 100 Hz. Current test indicate that the ROD can be read out at full
     luminosity at 300 Hz.
8.   The SCT community is strongly urged to devote more resources to write the necessary DAQ software so
     that more detailed user evaluation can be performed. To expedite this work, it is recommended that one
     of the existing RODs be sent immediately to Oxford to allow that group to begin learning to use the ROD.
     Note: The SCT community now has a large effort devoted to user evaluation and preparation for the ROD use at
     the macro assembly sites.

The committee wishes to thank the ROD design group for their time preparing for this review and presenting a
     clear status of the project. Those members of committee who are members of the ATLAS collaboration
     would also like to thank Christopher Bebek and Henrik von der Lippe for their contribution to ATLAS by
     spending the time to review this complex development.




                                                                                                        19