Technical, Cost and Schedule Review of the High Intensity

Department of Energy Office of Nuclear Physics Report Scientific Review of the proposed U.S. participation in the Heavy Ion physics research program at the LHC using the Compact Muon Solenoid Detector October 24, 2006 Executive Summary The Large Hadron Collider (LHC), under construction at the European Organization for Nuclear Research (CERN), Switzerland will become the world’s highest energy facility for relativistic heavy ion studies, providing ~ 4 weeks of beam annually for research opportunities. This limited availability would complement and benefit the program of the U.S. Relativistic Heavy Ion Collider (RHIC), a dedicated facility for heavy ion and polarized proton research. The merit of the science accessible at the LHC and the significant opportunities for discoveries were evaluated by the Nuclear Science Advisory Committee (NSAC) in 2004 to be outstanding. Specifically, NSAC recommended participation at the LHC should become a new component of the U.S. Heavy Ion program. The Department of Energy (DOE), Office of Nuclear Physics (NP) completed its Scientific Review of the proposed U.S. participation in the heavy ion physics research program at the LHC using the Compact Muon Solenoid Detector (CMS). A panel of four experts convened at DOE Headquarters on October 24th, 2006 and evaluated the proposal titled “Heavy Ion Physics at the LHC with the Compact Muon Solenoid Detector” submitted by Professor Boleslaw Wyslouch from Massachusetts Institute of Technology (MIT). The scientific merit of the proposed research by U.S. CMS Heavy Ion collaboration (CMS-HI) was deemed to be very high by the panel congruent with the written reports of the mail reviewers. CMS-HI has capabilities for making high quality measurements that could yield significant physics results and possibly new discoveries. The panel is of the view that early measurements at the LHC will undoubtedly provide new information on “global observables” by the ALICE (A Large Ion Collider Experiment at CERN) and CMS experiments. However, the primary purpose of the U.S. effort on the CMS-HI collaboration should be distinguished from the ALICE experiment and aimed from the outset at realizing those measurements that are unique and exploit the strengths of the CMS detector. Thus, the CMS-HI collaboration is encouraged to articulate its mission, goals and milestones which reflect its program of unique measurements in a Research Management Plan (RMP). The RMP should also address the formal mechanisms for U.S. interactions with CMS and CERN. The significance of the proposed research program will depend on the sensitivity which can be reached for key measurements identified in this report. Although, the capabilities of the CMS detector lead to high expectations, the sensitivity expected for such key measures is yet to be fully demonstrated through simulation. The reviewers believe appropriate Monte Carlo simulations should be performed to demonstrate the physics capabilities and sensitivity of the CMS detector for the primary HI physics topics. The funding request comprises Maintenance and Operations (M&O), computing for data analysis in the U.S. and computing equipment for the CMS-HI Trigger. M&O costs are required fees and travel expenses to operate at CERN. The strategies for providing CMS2 HI computing resources need to be developed and presented to DOE ONP, including a fallback scenario in which constant level of effort is assumed. The computing plan may be incorporated in the RMP. The equipment funding is for the expansion of the online HLT farm for the CMS detector at CERN and the purchase of a small High Level Trigger (HLT) development cluster to be located at MIT. No significant technical issues were foreseen since much of the expanded HLT farm relies on Commercial Off-the-Shelf PC purchases. The size of the upgraded HLT farm seems to be driven by the Level 3 dimuon trigger which requires “real time” tracking in the inner silicon detectors. Preliminary studies on establishing the feasibility of tracking within the “time budget” for the level 3 trigger appears encouraging but has yet to reach the “design” goal. That goal should be accomplished as a prerequisite for seeking DOE NP support to expand the HLT farm. The request for a small developmental HLT cluster at MIT is justified as a necessary step towards the development of algorithms needed to execute the CMS-HI program. The Office of Nuclear Physics, Heavy Ion Program intends to support the acquisition of a small developmental HLT cluster at MIT and the completion of the CMS Zero Degree Calorimeter (ZDC) detector by the Kansas University group subject to availability of funds in FY 2007. The CMS HI coordinator should ensure this report’s recommendations are effectuated and a plan is developed for a phased approach to U.S. institutional membership in CMS prior to seeking funding in FY 2008 for the expansion of the CMS HLT farm. The PHOBOS experiment has concluded its operations at RHIC and thus the transitioning of the efforts of a core group of institutions (i.e., MIT, UIUC and Kansas University) to the CMS-HI program is not likely to impact RHIC operations. Therefore, in making a strong scientific case that is aligned with the primary physics topics, these institutions could indicate priority in the RMP to become U.S. members of the CMS-HI collaboration. Institutions with on-going responsibilities at RHIC will require a careful assessment of their roles and responsibilities as formulated in the RMP. Subsequently, institutional proposals may be evaluated by DOE for a decision regarding CMS-HI membership at grant continuation, renewal or supplemental scope change during FY 2007. 3 DOE Recommendations  A 5-year Research Management Plan should be developed with a well articulated, unique science mission, goals and milestones. Within the constraint of constant level of effort, the plan should evaluate the impact on the RHIC program and/or show how joint commitments to RHIC and CMS will be fulfilled. The plan should prioritize the U.S. collaboration’s physics interest, concentrating efforts on the most compelling physics topics where CMS HI has unique capabilities. It should include a strategy for providing adequate computing resources. The RMP should be submitted to DOE by April 13th, 2007. Detailed Monte Carlo simulations should be performed to demonstrate the physics capabilities and sensitivity of the CMS detector on the main HI physics topics. These simulations should include jet reconstruction, sensitivity to jet energy loss, identification of direct gammas, and 0 reconstruction. The proponents should address the comments of the review and submit a formal response prior to seeking funding for the expansion of the HLT PC farm.  4 Executive Summary ...................................................................................................................... 2 DOE Recommendations ............................................................................................................... 4 Introduction ................................................................................................................................... 6 Significance, Merit and Uniqueness of the Proposal’s Scientific Goals ................................... 8 The effectiveness of the proposed collaboration structure to implement and execute the scientific program ......................................................................................................................... 9 The feasibility and completeness of the requested funding and schedule, including workforce availability ................................................................................................................. 10 The status of the technical design, including completeness of technical design and scope, feasibility and merit of technical approach .............................................................................. 10 Other Issues relating to the CMS Heavy Ion Experiment, such as the broader impact on the RHIC program including any proposed re-direction of resources ........................................ 11 Appendix A: Charge Memorandum ......................................................................................... 12 Appendix B: Agenda and List of Reviewers ............................................................................. 13 Excerpts from panel member reports ....................................................................................... 16 Excerpts from written mail reviews .......................................................................................... 26 5 Introduction On October 24th, 2006 the Director of the Research Division of the Office of Nuclear Physics (ONP) completed a Scientific Review of the proposed U.S. participation in the heavy ion physics research program at the LHC using the Compact Muon Solenoid Detector (CMS). A panel of four experts (Professor Andrew R. Baden, University of Maryland; Dr. Timothy Hallman, Brookhaven National Laboratory; Dr. Xin-Nian Wang Lawrence Berkeley National Laboratory and Professor Itzhak Tserruya, Weizmann Institute of Science, Israel) convened at DOE Headquarters, Germantown and evaluated the proposal titled “Heavy Ion Physics at the LHC with the Compact Muon Solenoid Detector” submitted by Professor Boleslaw Wyslouch from Massachusetts Institute of Technology (MIT). Dr. Gulshan Rai, Program Manager for the Heavy Ion Nuclear Physics Program, chaired the review and Dr. Gene Henry, Director for the Research Division was also present. The 2002 Nuclear Science Advisory Committee (NSAC) Long Range Plan (LRP) identified a heavy ion research effort at the Large Hadron Collider (LHC) as an outstanding opportunity and recommended that a “modest” involvement of U.S. nuclear physicists be supported in this effort. A subsequent NSAC review of the U.S. Program in Heavy-Ion Physics (report dated October 7, 2004) recommended certain essential investments be made that included, as a new initiative, participation in the LHC Heavy Ion program. The LHC, under construction at CERN, Switzerland as the world’s premier particle physics accelerator, will also become the world’s highest energy facility for relativistic heavy ion studies, providing ~ 4 weeks of beam annually for research. This limited availability to heavy ion collisions would complement and benefit the program of the U.S. Relativistic Heavy Ion Collider (RHIC), a dedicated facility for heavy ion and polarized proton research. The LHC center-ofmass beam energy exceeds that of RHIC by a factor of ~ 30. As a result, significantly different conditions and changes are expected at the LHC relative to RHIC in key parameters controlling the properties of the Quantum ChromoDynamic (QCD) matter. Currently, there are four major experiments at the LHC. ALICE (A Large Ion Collider Experiment at CERN), the only dedicated heavy ion detector, CMS and ATLAS (A Toroidal LHC Apparatus), the two large particle physics experiments, have planned heavy ion research programs. Three U.S. Nuclear Physics collaborations: ALICE-USA, CMS-HI (Heavy Ions) and ATLASHI, have proposed to initiate research programs focusing on jet and heavy quark flavor probes of ultra dense matter believed to be created in Lead (Pb) beam collisions at the LHC. The purpose of this review was to assess the scientific and technical aspects of the CMS-HI research proposal. The Office of Nuclear Physics needed to understand what scientific progress will be made in advancing the heavy ion program objectives and what resources will be needed to implement the proposed scientific program. In carrying out this charge, each panel member was asked to evaluate and comment on:  The significance, merit, and uniqueness of the proposal’s scientific goals; 6     Effectiveness of the proposed collaboration structure to implement and execute the scientific program; The feasibility and completeness of the requested funding and schedule, including workforce availability; The status of the technical design, including completeness of technical design and scope, feasibility and merit of technical approach; and Other issues relating to the CMS heavy ion experiment, such as the broader impact on the RHIC program including any proposed re-direction of resources. The panelists took into consideration written reports from five mail reviewers that were made available on the day of the review in a suitable form that assured the anonymity of the authors. The review consisted of formal presentations made by the CMS HI Coordinator, Prof. Wyslouch (MIT), and other proponents of the CMS HI proposal. The agenda included a question and answer session with senior “management” and a closed session in the afternoon for panel deliberations. The Heavy Ion Program Manager wrote his précis of concordant opinions in a close-out document. The panelists were also asked to submit their individual evaluations and findings in a “letter report” covering all aspects of the charge letter. The executive summary and the accompanying DOE ONP recommendations are based largely on the information contained in these letters reports. A copy of the charge letter (Appendix A), the agenda (Appendix B) and excerpts from the panel and mail reviews are included. This report is organized according to the five charge elements and adopts the format of findings, comments and recommendations. 7 Significance, Merit and Uniqueness of the Proposal’s Scientific Goals Findings: The LHC will open a new energy frontier in the study of relativistic heavy ion collisions with significant potential for new discoveries. In particular, the matter formed at the LHC might be different from the strongly interacting matter discovered at RHIC. Hard processes are expected to dominate the physics of nuclear collisions at the LHC. Accordingly, the study of jets will play a central role in the heavy ion physics program that could lead to a deeper understanding of the jet quenching phenomena discovered at RHIC. The U.S. CMS-HI collaboration proposes to study a variety of very interesting physics topics with emphasis on global observables in the initial phase and focusing later, when the luminosity is at design level, on the study of high pt phenomena, jet physics, and quarkonia (J/, and upsilon) production. A unique strength of the CMS detector is its large near hermetic acceptance. The large acceptance of the calorimeters and the muon tracking system gives CMS unique capabilities in the detection of di-muons at mid-rapidity and in the energy reach of jets. The large coverage of the CMS detector system is also suitable for the study of jets using gamma+ jet or Z0+jet events which provide better resolution of the initial jet energy determination. Comments: The review panel believes the physics case for the proposed program is strong and the proposal capitalizes on the strongest features of the CMS detector. The electromagnetic and hadronic calorimeters in CMS are considered to be well-suited for the study of high energy jets above 100 GeV. These subsystems provide trigger capability essential to exploit the full LHC luminosity. The study of heavy ion collisions with the CMS detector will have some complementary components to the dedicated heavy ion experiment ALICE, but CMS appears to provide unique capabilities for Z0 and Upsilon detection in the muon decay channel at mid-rapidity. In general, the measurements provided by CMS are viewed as being an important and integral part of the heavy ion program at the LHC for systematic study of a wide array of observables. However, difficulty with 0 identification at high pt will make it extremely difficult to identify high energy gamma + jet events in heavy-ion collisions and to use such events to study jet quenching. Even though jet quenching will increase the gamma/0 ratio as seen at RHIC, the capability for direct gamma detection in the CMS detector, for example using isolation cuts, remains to be demonstrated through Monte Carlo simulations. The review panel thought the capability for direct gamma identification was not demonstrated. Identification of high pt jets has been emphasized as a forte of the CMS-HI proposal. However, some reviewers believe the fluctuations in the underlying event within the jet cone radius will make it difficult to determine the jet energy precisely, limiting the sensitivity to partonic energy loss. Further work is needed to demonstrate the sensitivity of the detector for direct measurement of partonic energy loss using reconstructed high pt jets. 8 Recommendations:  Detailed Monte Carlo simulations should be performed to demonstrate the physics capabilities and sensitivity of the CMS detector on the main HI physics topics. These simulations should include jet reconstruction, sensitivity to jet energy loss, identification of direct gammas, and 0 reconstruction. The proponents should address the comments of the review and submit a formal response prior to seeking funding for the expansion of the HLT PC farm. The effectiveness of the proposed collaboration structure to implement and execute the scientific program Findings: Eleven U.S. institutions are part of the CMS-HI proposal: Colorado, Iowa, UIC, U. of Kansas, Los Alamos, U. of Maryland, Minnesota, MIT, Rice, Vanderbilt, and U.C. Davis. The number of scientists and staff is presently 37, including faculty, research faculty, and post doctoral fellows. These groups have expertise, know-how and an excellent reputation in the relativistic heavy ion community. The lead institution for CMS-HI, MIT, is also leading the PHOBOS experiment. PHOBOS is now in the stages of final analysis, having stopped data taking at RHIC in 2005. MIT’s future plan is to devote full effort on the proposed heavy ion research program at CMS. An organization chart showing the governance and responsibilities within the CMS heavy ion research effort and its relation to CMS was presented. The role of the DOE NP agency representatives to the CMS governance is not yet determined. During the next 3-4 years, the CMS-HI coordinator stated that the priority for participating institutions will shift from RHIC to CMS. Comments: All the institutions participating on the CMS proposal have various levels of continuing near-term responsibilities on the RHIC heavy ion program. Several institutions (Los Alamos National Lab, Rice and Vanderbilt) have major commitments for the construction of detector upgrades on the STAR and PHENIX experiments as part of the RHIC mid-term plan. The organization of the CMSHI collaboration appears adequate for effective management of the proposed program. Some reviewers considered the present size of the U.S. CMS-HI collaboration may be small to properly address the broad physics program outlined in the proposal. Recommendations:  A 5-year Research Management Plan should be developed with a well articulated, unique science mission, goals and milestones. Within the constraint of constant level of effort, the plan should evaluate the impact on the RHIC program and/or show how joint commitments to RHIC and CMS will be fulfilled. The plan should prioritize the U.S. collaboration’s physics interest, concentrating efforts on the most compelling physics topics where CMS HI has unique 9 capabilities. It should include a strategy for providing adequate computing resources. The RMP should be submitted to DOE by April 13th, 2007. The feasibility and completeness of the requested funding and schedule, including workforce availability Findings: The proposal requests a total of $2M for the purchase of 630 PCs. Six hundred of them will be used as part of the CMS High Level Trigger (HLT) and will be purchased as needed, probably at a rate of 200 PCs per year from 2008-2010. Thirty PCs will be purchased in 2007 to form a small cluster located at MIT for the development of trigger algorithms. Beyond funding specifically requested for the HLT, the proposal identifies a need for M&O costs at approximately $600K per year and an additional investment of $400K per year to establish and maintain a dedicated HI computing facility to analyze CMS HI data. Comments: The reviewers believe the requested funding of $2M over three years for the HLT is a modest investment considering the overall cost of the CMS experiment and its potential to contribute significantly to the heavy-ion physics program at the LHC. Recommendations: None The status of the technical design, including completeness of technical design and scope, feasibility and merit of technical approach Findings: For heavy ion running the maximum collision rate is 8 kHz. The full HLT (1500 PCs) is able to process this rate without having to filter the data at Level 1. Algorithms are being developed for the HLT to effectively trigger on rare probes. The size of the HLT farm for the CMS Heavy Ion program is driven by the Level 3 di-muon trigger which requires tracking in the inner silicon detectors. The construction of the ZDC is on schedule for the detectors to be installed by the summer of 2007. The testing of the first ZDC detector with beam was successful. Comments: 10 The group is commended for its efforts in developing a triggering strategy for heavy ion running. Further effort is needed to establish the key di-muon trigger well within the projected time budget of the HLT. The development of algorithms and their integration into the overall CMS DAQ justifies the funding request in FY 2007 for the acquisition of a small off-site prototype HLT system. Recommendations: None Other Issues relating to the CMS Heavy Ion Experiment, such as the broader impact on the RHIC program including any proposed re-direction of resources Findings: The annual operating cost of the proposed Tier 2 analysis facility is approximately $400,000 which covers the cost of recording media and allows for annual updating of the hardware infrastructure. Institutions intending to utilize the farm stated that they will not need additional resources to participate in data analysis beyond their present base funding. The decision on where to locate the Tier 2 facility is yet to be made. Comments: It is important to have concrete discussions with potential host institutions concerning where the HI Tier2 analysis facility should be located with the goal of reducing the associated cost. The proposed HI Tier 2 facility is integral to the Collaboration’s intended physics program; without it, the panel reviewers believe the US leadership of the CMS heavy ion research program may be compromised. In general, the transition of a number of talented individuals currently participating on the RHIC program will have a non-negligible impact on the Collaborations in which they now participate, both as a consequence of intellectual leadership and re-directed resources. The development of the Research Management Plan mentioned above is essential to assess the magnitude of this impact. Recommendations: None 11 Appendix A: Charge Memorandum The Physics Research Division of the Office of Nuclear Physics is organizing a review of the proposed U.S. participation in the Heavy Ion Physics Research at the LHC with the Compact Muon Solenoid Detector (CMS). As you are aware, this review will take place at Germantown, Maryland on October 24th, 2006. The purpose of this review is to assess all aspects of the research proposal. The following main topics will be considered at the review: a. The significance, merit, and uniqueness of the proposal’s scientific goals; b. Effectiveness of the proposed collaboration structure to implement and execute the scientific program; c. The feasibility and completeness of the requested funding and schedule, including workforce availability; d. The status of the technical design, including completeness of technical design and scope, feasibility and merit of technical approach; and e. Other issues relating to the CMS heavy ion experiment, such as the broader impact on the RHIC program including any proposed re-direction of resources. Dr. Gulshan Rai, Program Manager for the Heavy Ion Research, will be chairing the review. He can be contacted at (301) 903-4702, or E-mail: Gulshan.Rai@science.doe.gov. The day’s agenda will consist of presentations by the proponents. The afternoon will include an executive session and individual report writing containing preliminary findings, comments, and recommendations. Your final evaluation report should be submitted within 2 weeks after the review to Dr. Rai. The panel members should contact Christine Izzo at (301) 903-3614 or E-mail: Christine.Izzo@science.doe.gov regarding any logistics questions. I greatly appreciate your efforts in preparing for this review. It is an important process that allows our office to understand the CMS Heavy Ion proposal and its justification to proceed. I look forward to a very informative and stimulating visit. Sincerely, Eugene A. Henry Director Physics Research Division Office of Nuclear Physics 12 Appendix B: Agenda and List of Reviewers Agenda October 24, 2006, DOE/Germantown Tuesday, Room E-401 8:15 am 9:00 am 10:00 am 10:45 am 10:55 am 11:40 am 12:20 am 1:00 pm 2:00 pm 2:30 pm 5:30 pm DOE Executive Session Heavy Ion Physics Program of CMS [40 min] Triggering on heavy-ion collisions in CMS [30 min] Break Zero Degree Calorimeter for CMS [30 min] CMS Heavy Ion offline computing [25 min] Personnel and Resource Management [30 min] Lunch Response to written questions from panel Executive Session & Report Writing Close of Review Mike Murray (Univ. of Kansas) Olga Barannikova, (UIC) Russell Betts (UIC) Bolek Wyslouch (MIT) Gunther Roland (MIT) 13 Review Panel Members Professor Andrew R. Baden Department of Physics University of Maryland 1107 Physics Building College Park, MD 20742-4111 (301) 405-5947 drew@physics.umd.edu Dr. Timothy Hallman Physics Department Brookhaven National Laboratory Building 510A Upton, NY 11973-5000 (631) 344-7420 hallman@bnl.gov Professor Itzhak Tserruya Department of Particle Physics Weizmann Institute of Science Rehovot 76100 Israel 972-8-934-4052 itzhak.tserruya@weizmann.ac.il Dr. Xin-Nian Wang Nuclear Science Division Lawrence Berkeley National Laboratory 1 Cyclotron Road, MS 70R0319 Berkeley, CA 94720-8169 (510) 486-5239 xnwang@lbl.gov DOE Participants Dr. Eugene A. Henry Office of Nuclear Physics U. S. Department of Energy SC-26.1/Germantown Building 1000 Independence Avenue Washington, DC 20585-1290 (301) 903-6093 Gene.Henry@science.doe.gov Dr. Dennis Kovar Office of Nuclear Physics U. S. Department of Energy SC-26/Germantown Building 1000 Independence Avenue Washington, DC 20585-1290 (301) 903-3613 Dennis.Kovar@science.doe.gov Dr. Gulshan Rai Office of Nuclear Physics U. S. Department of Energy SC-26.1/Germantown Building 1000 Independence Avenue Washington, DC 20585-1290 (301) 903-4702 Gulshan.Rai@science.doe.gov 14 Department of Energy Office of Nuclear Physics Report Review Excerpts U.S. participation in the Heavy Ion physics research program at the LHC using the Compact Muon Solenoid Detector October 24, 2006 15 Excerpts from panel member reports Block excerpts from the reports of the review panel members regarding their evaluations are provided below according to the review criteria they were asked to address: The significance, merit, and uniqueness of the proposal’s scientific goals: Panel Reviewer: The high energy frontier for relativistic nucleus-nucleus collisions that will open soon at the LHC has significant discovery potential. It will allow exploration of QCD matter in a system expected to remain significantly above the critical temperature for several fm/c. In addition to providing completely new scientific reach for the study of matter whose energy density as a function of temperature is expected to more closely approximate the Stefan-Boltzmann limit, comparison of LHC results with measurements characterizing the strongly interacting matter discovered at RHIC may further illuminate the nature of the latter as well. The high energy available at the LHC means that once sufficient luminosity has been developed, hard probes (jets, direct photons, quarkonia) will dominate the heavy ion research program. The near-hermetic coverage of the CMS electromagnetic and hadronic calorimeters are well suited in principle to provide trigger capability and energy deposition measurements required for this research. The thrust of the proposed CMS HI research program at LHC startup will be measurement of global observables using the CMS inner silicon tracking detectors. The scientific merit of the proposed research is high. Its significance ultimately will depend on the sensitivity which can be reached for key measurements such as jet energy loss, measurement of direct photons, separation of upsilon family of states, etc. Although the capabilities of the CMS detector and the CMS HI team lead to high expectations, the sensitivity expected for such key measures through simulation is largely yet to be demonstrated. A simple example is the need for a realistic assessment of the effect of the underlying event and its fluctuations on the measurement of the transverse momentum of jets. Although high pt jets should be “obvious by inspection” at LHC energies, the partonic energy loss question underlying much of the interest in jet physics is much less so. The real CMS HI sensitivity to jet energy loss taking fluctuations in the underlying event into account is yet to be simulated. This should be done with a realistic event generator which accounts for the interaction (as opposed to a simple superposition) of the produced jet with the underlying event. Measurement of direct photons is another very challenging aspect of the proposed program whose feasibility/sensitivity is yet to be demonstrated. There is significant value added in having a second HI research effort at the LHC complementary to the dedicated heavy ion experiment, ALICE. Once sufficient luminosity has been developed, the CMS detector will have superior acceptance and trigger capability for jets and their correlations due to its near-hermetic coverage and the availability of hadronic calorimetry. The excellent mass resolution achievable in CMS for quarkonia is expected to allow clean differentiation e.g. of the upsilon family of states, a key to understanding the temperature ultimately reached in the plasma. CMS will have unique capability for measurement of the upsilon family of states (as well as Z0 production) in the muon decay channel at midrapidity. The measurement of global observables with the CMS inner silicon tracking detectors will provide useful information to characterize centrality, measure flow, etc. Significant progress had been made by the time of the review on preliminary simulations to explore the capability of the inner silicon tracking for such measurements, but further work is required to show the real sensitivity of the detector for the full spectrum of PID and non-PID soft physics measurements of interest. Recommendations:  Complete realistic simulations which demonstrate the sensitivity ultimately expected for the highest priority science questions (to be contrasted with first order measurement feasibility studies which are largely complete). Panel Reviewer: The US CMS Heavy ion Collaboration proposes to study relativistic nuclear collisions using the CMS detector at the CERN LHC. The LHC will open a new energy frontier in this field with significant potential for new discoveries. In particular, with the much higher energy available at the LHC the matter formed in these collisions might be different from the strongly interacting matter, the “perfect liquid”, discovered at RHIC and approach the originally sought weakly interacting matter depicted as an ideal gas of free quarks and gluons. The physics of nuclear collisions at the LHC is expected to be dominated by hard processes. Jets will be abundantly produced and their study will likely lead to a deeper understanding and scrutiny of the jet quenching phenomena discovered at RHIC. The CMS detector with its nearly hermetic acceptance is specially suited for the study of jets. In particular, the large coverage of its calorimeters and muon tracking system at mid-rapidity will allow the study of high energy jets. The electromagnetic and hadronic calorimeters will provide trigger capabilities essential to exploit the full LHC luminosity and to extend the energy reach of jets beyond 100 GeV. The calorimeters will also allow the study of direct gamma+jet or Z0+jet coincidences in the cleanest possible way where a direct photon or a Z0 is detected in the calorimeter, providing an unbiased measurement of the initial jet energy. 17 The measurements provided by CMS will be an important and integral part of the heavy-ion program at the LHC. On the one hand they will be complementary to the dedicated heavy ion experiment, ALICE, providing unique capabilities for Z0 and upsilon detection in the muon decay channel at mid-rapidity. On the other hand, they will also provide redundancy and systematic over a broad array of observables. The physics case is very strong and the physics program capitalizes on the strongest features of the CMS detector. However, the proposal lacks details in terms of simulations that support the proposed physics goals and that demonstrate the sensitivity of the detector to the various physics quantities of interest. For example, identification of high energy jets has been emphasized as a strength of the CMS detector. However, fluctuations in the underlying event within the jet cone radius will make it difficult to determine the jet energy precisely, limiting the sensitivity to partonic energy loss. Difficulty with 0 identification at high pT will make it extremely difficult to identify direct gamma + jet events in heavy-ion collisions and to use such events to study jet quenching. Even though jet quenching will increase the direct gamma/0 ratio as seen at RHIC, the capability for direct gamma detection in the CMS detector, for example using isolation cuts, remains to be demonstrated. Detailed physics simulations should be performed that include results of reconstruction of jets embedded in a generated Pb-Pb collision vs centrality and vs jet energy. These simulations are needed to demonstrate (i) the sensitivity of the detector for direct measurement of partonic energy loss using reconstructed high energy jets and (ii) the capability for direct gamma identification. Panel Reviewer: CMS/HI US collaboration proposes to utilize the CMS detector system at LHC to study the formation and properties of dense partonic matter in heavy-ion collisions at the highest energy available in the near future, in which a rather different physics scenario and initial condition is expected that could be qualitatively different from RHIC. A broad range of physics topics are proposed, including quarkonium suppression, jet quenching and transport properties of the medium, thermalization and elliptic flow, initial condition and global observables, and parton distributions in nuclei. The proposal in particular has outlined its plan to utilize the unique capability of CMS detector, e.g., the large coverage of its tracking system, ECAL and muon tracking system to study the mentioned topics. The large coverage of CMS detector system is also suitable for the study of jet quenching with gamma+ jet or Z0+jet events which provide better resolution of the initial jet energy determination. The study of heavy-ion collisions with CMS is necessary, complementary to the single purpose ALICE experiment, and is an important and integral part of the heavy-ion program at LHC for systematic study of a wide array of observables. 18 It is clear from the proposal and the presentations that the strength of CMS/HI is in the high pt and quarkonium area. Even though recent investigation has shown that it is possible to track particle down to very low pt, there is still lack of particle identification in CMS/HI, as compared to ALICE experiment. It is natural for CMS/HI to do some rudimentary global measurements in the first run of heavy-ion beams when high luminosity data are not available. However, it is important for CMS/HI to focus on what it can do best in later years, i.e., high pt jet and quarkonium, given the limited man-power within CMS/HI USA. Because of the lack of particle id at high pt and in particular the pi0 identification, it will be extremely difficult to identify gamma + jet events in heavy-ion collisions and use the events to study jet quenching. Even though jet quenching will enhance gamma/pi0 ratio as seen at RHIC already, one still has to find out at which pt such ratio is becoming much larger than 1, and only then one might be able to use statistic method to identify gamma + jet events. This still has to be investigated through Monte Carlo simulation or theoretical study. Reconstruction and Identification of large pt jet have been emphasized as the strength of CMS/HI and in heavy-ion collisions at LHC in general. However, the underlying background fluctuation makes it difficult to determine the jet energy precisely. One still needs to investigate how the energy fluctuation is compared to the typical expected energy loss and whether such an approach can be used to study the effect of parton energy loss on the resulting spectra. Maybe at the end of the study, one will find that study of nuclear modification of single and dihadron spectra is still or the only relevant measurement for jet quenching study, as at RHIC. Recommendations:  I think DOE should consider fund a modest CMS/HI US program, given the importance of the physics and the importance of US involvement. However, careful consideration should be given to balance between such a modest involvement and the RHIC (current and future) program. Effectiveness of the proposed collaboration structure to implement and execute the scientific program: Panel Reviewer: Eleven US institutions are part of the CMS heavy ion proposal. The lead institution is MIT. The total number of “heads” is 37 at present, including faculty, research faculty, and postdoctoral fellows. Collectively, the institutions involved bring significant experience from experimental work on the BRAHMS, PHOBOS, PHENIX, and STAR experiments at RHIC. 19 The groups involved in CMS HI are accomplished, well-respected groups in relativistic heavy ion physics with the experience, knowledge, and talent to carry forward a significant program of heavy ion research at the LHC. The organizational structure proposed for the Collaboration appears sound and adequate to carry out the program. Several of the groups participating on the CMS HI program (LANL, Rice, and Vanderbilt) have major commitments for hardware upgrades to STAR and PHENIX as part of the RHIC mid-term plan. For these groups, any transition to the LHC program must include a robust plan for meeting existing commitments including construction, commissioning, and first data taking with the RHIC upgrade detectors to which they have already committed. Despite the strength of the CMS HI team, the size of the present Collaboration is likely too small to address the broad spectrum of physics measurements proposed. The Collaboration should prioritize its physics interests and concentrate on compelling topics where CMS has unique or superior capabilities. Recommendations:  The CMS HI spokesperson should make a realistic assessment of the manpower needed versus that available as a function of time. This assessment should properly account for manpower that will remain dedicated to RHIC until ongoing commitments to RHIC hardware upgrades have been met. The physics goals of the CMS HI program should be prioritized taking machine considerations (e.g. projected luminosity versus time) and available manpower into account. In the event of insufficient manpower to cover the full spectrum of measurements proposed, care should be taken to insure the highest priority goals will be adequately covered.  Panel Reviewer: The U.S. CMS HI collaboration has a very broad physics program. It proposes to study a variety of topics with emphasis on global observables in the initial phase and plans to shift the focus later, when the luminosity is at design level, to other topics, mainly the study of high pT phenomena, jet physics, and quarkonia (J/, and upsilon) production. This program seems very ambitious given the present size of the collaboration (see below). The proponents should prioritize their physics interest and concentrate their efforts in the most compelling physics topics where CMS has unique capabilities. The US CMS HI collaboration presently consists of a total of eleven US institutions (Colorado, Iowa, UIC, U. of Kansas, Los Alamos, U. of Maryland, Minnesota, MIT, Rice, Vanderbilt, and U.C. Davis) and 37 scientists including faculty, research faculty and post-doctoral fellows. The groups have expertise, know-how and excellent reputation in the relativistic heavy-ion community. The US groups provide the leadership in the CMS Heavy Ion effort. Without US 20 participation, the CMS Heavy Ion program might be compromised. The lead institution, MIT, is also leading the PHOBOS experiment. However, PHOBOS is now in the stages of final analysis, having stopped data taking at RHIC in 2005 and MIT’s future plan is to devote full effort on the proposed heavy ion research program at CMS. The situation is more ambiguous for other participating institutions. Several of these institutions have at some level, a continuing participation in the RHIC heavy ion program. Fewer institutions, Los Alamos National Lab., Rice and Vanderbilt, have major commitments for the construction of upgrades for STAR and PHENIX as part of the RHIC mid-term plan. On the other hand, the CMS HI coordinator stated that the priority for the participating institutions will gradually shift from RHIC to CMS during the next 3-4 years. This seems to create a conflict with, and could have a serious impact on, the RHIC program. The transition of a number of talented individuals currently participating in the RHIC program may have a non-negligible impact on the Collaborations in which they now participate, both as a consequence of intellectual leadership and re-directed resources. Within the constraint of constant level of effort, a 5-year research plan should be developed. This plan will be essential to assess the magnitude of this impact and should indicate how commitments to RHIC and CMS can both be fulfilled. The proponents presented an organization chart showing the governance and responsibilities within the CMS heavy ion research effort and its relation to CMS. The chart appears adequate for the effective management of the proposed program. The feasibility and completeness of the requested funding and schedule, including workforce availability: Panel Reviewer: The proposal requests dedicated funding of $2M over 3-4 years for PCs which would be incorporated into the CMS High Level Trigger. A small cluster of thirty PC’s is proposed for purchase in 2007 for algorithm development. Beyond funding to purchase PCs, the need for $600k per year in additional M&O costs are anticipated as well as an additional investment of $400k per year to establish and maintain a dedicated CMS HI computing facility The funding requested ($2M over 3-4 years) for PCs which would become part of the CMS High Level Trigger represents a modest investment. 21 It appears that US leadership of the CMS HI collaboration is tied fundamentally to the existence of the proposed dedicated computing facility. It also appears that without US leadership the success of the proposed CMS HI research program may be seriously at risk. The proposed computing facility therefore appears to be central to the success of the proposed program, and the associated costs are not “optional” should the program be approved, even though they were presented as “off-budget” in the present proposal. While modest on the scale of the RHIC operating budget, the recurring M&O costs of approximately $1M per year over the life of the experiment is a significant commitment on the scale of total LHC investment recommended by the Barnes Panel and the last Long Range Plan. Panel Reviewer: The proposal requests a total of $2M for the purchase of 630 PCs. Six hundred of them will be used as part of the CMS High Level Trigger (HLT) and will be purchased as needed, probably at a rate of 200 PCs per year from 2008-2010. Thirty PCs will be purchased in 2007 to form a small cluster located at MIT for the development of trigger algorithms. The requested funding of $2M over three years for the HLT is a modest investment considering the overall cost of the CMS detector and the potential to contribute significantly to the heavy-ion physics at the LHC. Beyond the capital investment for the HLT, the proposal identifies a need for M&O costs at approximately $600K per year and an additional investment of $400K per year to establish and maintain a Tier-2 computing facility dedicated to the storage and analysis of CMS HI data. Panel Reviewer: The requested funding of $2M for three years is relatively a small investment in an experiment that has the potential to contribute significantly to the heavy-ion physics at LHC. The requested funding is mainly for building the HLT trigger system which is essential to take advantage of the full luminosity of the heavy-ion beam. This is an important part of the proposal for the study of rare signals such as high pt and quarkonium suppression. Even with the current members of the CMS/HI US collaboration, this is a rather small group, especially given the wide array of physics topics that are proposed. Even though they focus on a few topics that have high impact factor, some issues regarding work force and man power still have to be resolved. Quite few physics groups beside MIT and UIC are still active involved in current STAR and PHENIX physics analysis and have major responsibility in the RHIC upgrade. It is difficult to imagine these groups can release their responsibility and focus their efforts on CMS/HI, especially given that the time scale of RHIC upgrade and LHC 22 running is about the same. With no other resources, the CMS/HI really has to refocus their effort and try to do what they think they do the best and have the highest impact. The status of the technical design, including completeness of technical design and scope, feasibility and merit of technical approach: Panel Reviewer: The scope of the HLT farm for the CMS HI proposal is driven by the Level 3 di-muon trigger which requires “real time” tracking in the inner silicon detectors. The feasibility of tracking in the inner silicon detectors within the time budget required for the level 3 trigger is yet to be fully demonstrated although preliminary simulations appear encouraging. Recommendation  A full demonstration with realistic simulations of the feasibility of tracking in the inner silicon detectors within the time budget required by the CMS Level 3 trigger should be accomplished before the full scope of the proposed US contributed PC contribution to the CMS HLT is approved. The proposed purchase of a small cluster (30 PCs) in 2007 to demonstrate, with algorithms under development, that the performance required by the L3 trigger can be achieved appears well justified.  Panel Reviewer: For heavy ion running the maximum collision rate is 8 kHz. The HLT farm with a projected total of 1500 PCs is able to process this rate without having to filter the data at Level 1. The size of the HLT farm for the CMS Heavy Ion program is driven by the Level 3 di-muon trigger which requires tracking in the inner silicon detectors. The US CMS Collaboration is already engaged in the development of HLT algorithms to effectively trigger on rare probes and exploit the full LHC luminosity. It is also working on establishing the key di-muon trigger well within the projected time budget of the HLT. The Collaboration is commended for its efforts in this direction. The early development of algorithms appropriate for heavy-ion running and their integration into the overall CMS DAQ system is necessary and justifies the funding request in FY 2007 for the acquisition of a small off-site prototype HLT system. The CMS computing model is a distributed computing model, like that of the other LHC experiments. Within this model, the US-CMS Collaboration is expected to provide computing 23 resources adequate for the storage and analysis of the heavy ion data expected to be collected each year. A Tier-2 site is envisioned for that with an annual budget of $400,000. This budget includes the purchase of storage tape and CPU and does not include infrastructure costs which are expected to be covered by the participating institutions. The decision on where to locate the Tier 2 facility is yet to be made. Other issues relating to the CMS heavy ion experiment, such as the broader impact on the RHIC program including any proposed re-direction of resources: Panel Reviewer: The proposed research is of high scientific merit. The project appears feasible, although further demonstration of the sensitivity that will be achieved for the highest priority science questions should be forthcoming before full approval. It is possible that one reason (beyond technical challenges related to event generators) that this has not already been accomplished is ongoing near-term commitments to the RHIC program by members of the Collaboration. The CMS HI Spokesperson has indicated this priority will shift towards the LHC over the next 3-4 years. It is essential that existing commitments to RHIC, especially by institutions involved in hardware upgrades for STAR and PHENIX be met. The manpower to meet those RHIC commitments should be properly accounted for in the manpower projected to be available by the CMS HI Spokesperson. The scope of the resources required for the proposed research is modest but significant on the scale of overall LHC investment recommended by the Barnes Panel. The question of whether this fits within the envelope of overall U.S. LHC investment [is] considered appropriate/feasible for the U.S. Nuclear Physics program is important but beyond the scope of this review. It is clear however, if approved and if successful, the proposed program of research will yield significant heavy ion physics results and possible new discoveries, some of which may be unique to CMS, all of which will be forthcoming within an international HI research effort unquestionably under US-leadership if the proposed scope is approved. If approved the greatest impact of the proposed research effort will be the loss of some portion of the intellectual leadership of the US-based heavy ion research program. This will not be a complete loss since comparison of the matter produced at the LHC with the new insights provided by a dedicated RHIC II facility will illuminate the most promising directions to be explored in both programs. Panel Reviewer: To summarize, the physics case is strong. There is a great opportunity for a significant US contribution to the LHC Heavy Ion physics program for a relatively small investment. Given its 24 present size, the collaboration should prioritize its physics interest and concentrate its efforts in the most compelling physics topics where CMS has unique capabilities. Detailed simulations should be performed to demonstrate the detector capabilities in, and its sensitivity to, the selected physics topics of interest. The participation in the LHC of groups which are presently involved in the RHIC program might have an impact on RHIC. The Collaboration should develop a detailed plan to assess the magnitude of this impact. The plan should clearly indicate how commitments to RHIC and CMS can both be fulfilled. Although serious questions remain to be answered prior of granting approval to this proposal, the funding request for the purchase in 2007 of a small cluster consisting of 30 PCs for the development of trigger algorithms is fully justified and should be approved. Delaying this purchase may compromise the effectiveness of the US participation in the CMS heavy ion effort. 25 Excerpts from written mail reviews Mail Reviewers’ Comments on the Proposal Titled “Heavy Ion Physics at the LHC with the Compact Muon Solenoid Detector” Reviewer The proposal describes the planned technical and scientific work of the US CMS heavy ion group collaboration over next five years. The proposal is generally of very high quality, reflecting the outstanding credentials of the senior members of the US CMS heavy ion group (Wyslouch, Betts, Bonner, Busza, Cebra, Greene, Mignerey, Roland, and Velkovska) and the superb scientific prospects of the LHC heavy ion program. In my view, there is no question that this proposal should receive substantial support by the DOE. Since I am not a detector or data analysis expert, I cannot comment on the question whether the amount of requested funding is necessary and appropriate. Turning to the physics motivation, the main reason for the US science community to participate at a substantial level in the LHC heavy ion program is that it will offer complementary opportunities to the continuing RHIC physics program. The great kinematic range of various observables (especially jets and heavy quark hadrons) will make it possible to test the scientific concepts for the properties of hot QCD matter developed at RHIC. Insights to be gained from the experiments possible at the LHC will also benefit the RHIC heavy ion program by either confirming, or questioning, the theoretical approaches used to understand the RHIC data. It is of great importance to understand whether the quark-gluon plasma, initially formed at higher energy densities at LHC than at RHIC, still exhibits the strong couplings features observed in the RHIC experiments. The LHC (in the p+A mode) will also be able to put the concept of a gluon saturated initial state, which was developed under leadership of the BNL theory group and has been quite successful in describing certain global features of the RHIC data, to a serious test. Thus, a strong participation in the LHC heavy ion program is of great importance to the whole US heavy ion community as it enters the second phase of explorations of the quark-gluon plasma at RHIC. The CMS detector is ideally suited to exploit the novel aspects offered by the higher energies at LHC. Its exceptionally large acceptance for jets and single charged particles overall, and its high mass resolution for heavy quark mesons will make it possible to study those questions which cannot be similarly addressed at RHIC: the energy distribution within fully resolved, medium modified jets, medium modifications of Upsilon states, and gluon saturation effects in a heavy nucleus at very small Bjorken-x. Having made these general remarks, I disagree with some specific statements made in the scientific part of the proposal. For example, the authors state in the second sentence of section 26 2.1 that the ultimate goal of relativistic heavy ion physics is the exploration of “the structure of the vacuum under varying conditions.” This is an important goal, but I would hold that the quest for an understanding of the phase diagram of nuclear, strongly interacting matter is at least as important and central to the RHIC science program. In Section 2.2, the authors quote predictions of energy densities at LHC being “20 times higher than at RHIC.” While this may ultimately turn out to be true, it is not the currently preferred scenario. Most recent predictions of multiplicities at LHC, based on insights from the RHIC experiments (especially from the PHOBOS data, so the authors should be well aware of it!) is that the charge particle multiplicity density dN/dy at central rapidity at LHC will only be about three times that measured at RHIC. Since dN/dy grows like the square of the parton saturation scale, Qs2, and the initial energy density grows as Qs4, this implies a tenfold, not 20-fold, increase in the energy density at the initial equilibration time. The authors valiantly ignore the extensive work members of the Brookhaven theory group and others have done on this question in recent years. Also in section 2.2.1, the authors state that investigations of the  family are only possible at LHC. This is incorrect. As careful studies of quarkonium yields at RHIC by the RHIC collaborations in the context of the RHIC II planning have shown, RHIC will provide generous data sets of  yields after the luminosity upgrade. In section 2.2.2, it is not true that the transverse momentum of the parton is equal and opposite to that of the Z0 boson at “tree level.” This is only true at lowest order in the coupling constant s. At higher orders of s, but still at tree level, radiation of an additional gluon can destroy this momentum balance. Also, the statement made in the last sentence of 2.2.2, that “different descriptions of parton energy loss, which are indistinguishable at RHIC, will diverge at high pT at LHC” is likely an oversimplification. Improved and more differential measurements at RHIC will, in all likelihood, be able to discriminate between many different descriptions. One may call such words of criticism related to theoretical issues “nitpicking” of an experimental proposal. However, the US-CMS effort will be mostly focused on data acquisition and analysis, and a major advantage of the US team within CMS will be its extensive experience in the RHIC program. It is precisely for this reason that I think it will be important for the collaboration to be aware of the current status of insights from the RHIC physics program. The almost exclusive adherence to the views of a narrow group of theorists they mostly seem to rely on the work of Gyulassy et al. (GLV) on jet quenching, but appear oblivious to the extensive research by Wiedemann and Salgado, as well as Wang and Majumder, Renk, and others in recent years, which differs qualitatively and quantitatively from the GLV results - makes one wonder how effective the know-how transfer from RHIC to LHC by the members of the US-CMS collaboration will be. Two additional notes: I am gratified that six senior members of the 36-member collaboration are women. On the other hand, I am concerned about the lack of clarification of the split of 27 several persons of the proposal, who are currently active members of STAR or PHENIX, between their duties in CMS and possible continuing duties at RHIC. Having noted these concerns about specific aspects of the proposal, I hasten to reiterate that the proposed research is of exceptionally high quality and should be assigned a high priority within the US-DOE relativistic heavy ion physics research program. In my view, this is one of those rare “must fund” proposals. Reviewer: I have read the application for research support entitled "Heavy Ion Physics at the LHC with the Compact Muon Solenoid Detector" submitted to the Office of Nuclear Physics by Boleslaw Wyslouch of Massachusetts Institute of Technology. Although I believe that the Office of Nuclear Physics should support US participation in the heavy ion nuclear physics program of the CMS detector at the LHC, and the MIT group in particular should be supported in this effort, I do not find that this application, that mostly proposes to purchase 630 PCs for High Level Trigger processors, justifies support. With its large acceptance for charged particle tracking, calorimetry measurements, and muon identification, the Compact Muon Solenoid Detector at the LHC has unique measurement capabilities for the study of heavy ion collisions at the LHC. These capabilities complement those of the ALICE detector, which is the detector at the LHC dedicated to the study of heavy ion nuclear collisions. In particular, CMS will have the capability to study jets and jet correlations over a large kinematic range in order to investigate the jet suppression phenomena first observed at RHIC. The large CMS acceptance will provide large yields for the study of jet-jet and gamma-jet correlations. The tracking capability of CMS will allow detailed study of the modification of jet fragmentation in central Pb+Pb collisions, although investigation of the particle content of the jet will be limited within CMS due to its limited particle identification capabilities. CMS also has unique capabilities to study the forward rapidity region and to study ultra-peripheral nuclear collisions. The US CMS heavy ion group has led the effort to investigate and demonstrate the capabilities of the CMS detector to study heavy ion collisions at the LHC and has significant responsibilities within CMS for this effort. In particular, Bolek Wyslouch of MIT is the coordinator of the Heavy Ion Physics Reconstruction and Simulation group within CMS, one of only four physics groups within CMS. In fact, the heavy ion effort within CMS would be severely weakened, perhaps fatally, if the US effort were not supported. The application document presents the anticipated additional costs for the participation of the current set of US nuclear physics groups in CMS. These costs consist of CMS collaboration fees, ZDC maintenance, and increased travel costs that are estimated to rise from about 359k$ 28 in FY2007 to 576k$ in FY2010 and beyond (as discussed in Appendix B). At the same time, costs for data storage and computing for analysis are anticipated to increase from 300k$ in FY2007 to 400k$ in FY2008 and beyond (as discussed in Appendix C). These additional costs to the Office of Nuclear Science are not part of this application, and therefore are not the topic of this review. These costs, rising from 659k$ in FY2007 to more than 900k$ per year thereafter, presumably will be part of the DOE grant applications of the individual groups. This application requests funds for 630 commercial PCs to be used as part of the CMS High Level Trigger (HLT) and includes a small amount of technical support for a total cost of 2M$. These 630 PCs would comprise a portion of the 1536 PCs of the CMS HLT. It is anticipated that the HLT can be staged as the LHC luminosity increases. For p+p collisions the CMS Level 1 trigger selection, based on the calorimeters and muon tracker, must provide a factor of 400 in event rate reduction from the 40Mhz crossing rate (with an average of 25 p+p collisions per crossing) to 100kHz for input to the HLT. The HLT must provide an additional factor of ~660 in rate reduction to reach the data archiving rate limit of less than 150 Hz (225 MByte/s). For heavy ion collisions the archiving rate limit will be 10100Hz, depending on average event sizes (currently uncertain) and therefore a trigger rate reduction of 40-400 will be required for a 4000Hz Pb+Pb min bias collision rate. It is proposed that this rejection for Pb+Pb collisions can be attained entirely using HLT algorithms, and it is argued that the full HLT processor farm is needed for this task. However, I find this argument unsupported. Clearly, since the Pb+Pb collision rate is so low, all events can be passed to the HLT and essentially all of the necessary rate reduction can be accomplished in the HLT, as presented in the proposal. At a minimum, the Level 1 algorithms for jet finding and muon tracking can be implemented and used in the HLT to provide more precise control of the trigger implementation. It was demonstrated that these algorithms have been implemented already and use 5-15% and 5%, respectively, of the full HLT capacity. The necessary jet trigger rejection can be attained by appropriate adjustment of the jet energy threshold. Similarly for the high energy prompt photon trigger. The dimuon trigger is estimated to give a factor of 30100 rate reduction. This rate reduction may be sufficient depending on the event size. These constitute the full suite of triggers currently planned for heavy ion running, as presented in the proposal. In the proposal, it is argued to use the silicon tracking on those events with two or more muons to obtain an improved momentum measurement and to make a pair mass cut for further event rejection. However, the present HLT algorithms to do this require an order of magnitude more CPU power than the full HLT. Presumably additional event rejection for the dimuon triggers could be accomplished with muon momentum cuts, or loose mass cuts, based on the muon tracker momentum measurement, with little additional compute time, but this possibility is not discussed. Thus one needs only ~20% of the HLT capability if the silicon tracking is not used, or ten times the full HLT capacity if the silicon tracking is used. Since it is not demonstrated that the silicon tracking algorithms are necessary on the one hand, or feasible on the other hand, one must conclude that the silicon tracking algorithms are either unnecessary, or precluded. Therefore, the ~60% of the full HLT that would be available 29 without the processors of this proposal is more than adequate for the needs of the heavy ion program. It should also be noted that it is stated that the HLT processors of this proposal will be purchased (or replaced if purchased as part of this proposal) at a later date as part of the HEP program. Without demonstration of the physics that is enabled by the additional HLT processors I see no justification for support of this proposal. The proposed plan to use the CMS HLT for event rejection for heavy ion running is a sound decision and the group has already made significant progress to participate in the HLT and to develop algorithms for heavy ion operation. The proposal requests to install a small cluster of 30 processors at MIT in FY2007 with technical staff in support of this cluster for further effort on the HLT. The justification for this small cluster is unclear. Most of the effort needed by the CMS heavy ion group is to develop, test, and refine algorithms appropriate for heavy ion running. This can be done on a dedicated HLT processor with the appropriate hardware and software environment located anywhere. It certainly does not require a dedicated cluster. Clearly there are major HLT efforts necessary to develop and study system issues of data transfer and networking that require a representative cluster of HLT processors embedded in the CMS DAQ environment. However, it is not apparent that this should be a major component of the effort of the heavy ion group or that it could be or should be implemented at MIT, or anywhere outside the existing CMS DAQ effort. In conclusion, it is recommended that the three-year DOE Grant entitled "Heavy Ion Physics at the LHC with the Compact Muon Solenoid Detector" submitted to the Office of Nuclear Physics by Boleslaw Wyslouch of Massachusetts Institute of Technology, specifically requesting the purchase of 630 High Level Trigger processors, not be funded. Reviewer: The significance and merits of the proposed research and also of past research by the individuals involved; The groups listed on this application have all long track records in the field of high energy heavy ion collisions involving experiments at CERN SPS, BNL AGS and BNL RHIC. Most of the groups have made substantial contributions to the progress of the field and several of the listed people have had major responsibilities in the different experiments. Many of the groups have been essential for the operation of the detector system in the experiments where they participated and their involvement in the CMS heavy ion program will be very important. The groups have a very good mix of hardware and software experience and they will in my opinion continue to make large contribution to the future activities in the field. The proposed research is based on sound scientific grounds with a well defined physics program that will contribute to the understanding of the underlying physics of high energy heavy ion collisions. The physics issues the group is focusing on are some of the best signals for understanding the creation of hot and dense nuclear matter. The proposed participation in the LHC heavy ion 30 program through the US-CMS activities is a natural continuation of the present RHIC program. Their experiences in commissioning and operating large detector systems will be of great importance for the proposed CMS heavy ion program. The people in the US-CMS collaboration are the major players of the CMS heavy ion program and the PI of this application is the leader of the heavy ion efforts in the CMS experiment. The proposed program will largely enhance the overall heavy ion program at LHC. The competence and future promise of the investigator The competence and experience of the investigators is very high and they have made impressive contribution to both hardware and software development in the experiments they are and have been involved in. Therefore, I expect the groups to have strong impact on the development of the field for several years to come through their participation in the CMS experiment. The feasibility of the plans for carrying out the proposed research The plan defined in the application of carrying through the proposed program is well founded. The described hardware and software plans for achieving the goals are very relevant. The main commitments will on the hardware side be to contribute to the computer power necessary for the data taking, event reconstruction, data storage and analysis. The groups are very competitive and I do not see any reason not to expect them to continue along the same lines keeping their strong position in the field. The scope and size of the requested budget Carrying through the proposed program requires a critical mass of people. I find the numbers given to be adequate to effectively do this. The requested funds of $2M for computing power, distributed over 4 years, and the plan for the purchase seems reasonable. Approval of this application, for which there are strong arguments, will also imply additional later costs in terms of M&O and travel money. The M&O costs are fixed based on the number of PhDs or equivalent, while the indicated additional travel money could be subject for discussion at a later stage. The resources and interest of the sponsoring institution I am not completely familiar with all the internal University policies for infrastructure support to externally funded research projects. The information given in the application of faculty support concerning manpower and infrastructure seems, however, adequate for a project of this size. 31 Reviewer: The present proposal by the 10 U.S. heavy-ion CMS institutions is to assemble a 600 PC processor farm for the High Level Trigger (HLT) of the CMS experiment at the LHC and to develop and test event selection software for the heavy ion (Pb+Pb) runs. Given the architecture of the CMS DAQ-Trigger system the proposal appears to present the most optimum, cost-effective plan for enabling heavy-ion research at CMS. The 600 processors augment the approximately 900 processors of the CMS HLT and are to be used to do realtime analysis of up to 8K Pb+Pb events per second, selecting from 10 - 100/sec. for data storage and later offline analysis. This design requires each processor to analyze the calorimeter and muon tracking data for about 10 events/sec to avoid dead-time in the data flow. With present PC speeds that design parameter may be hard to achieve. However, I expect the plan will work due to the anticipated increase in processor speed available by the time the full Pb+Pb luminosity is reached. The plan presented here appears very well thought out and the institutions and faculty-staff have all the necessary credentials to carry out the program. I therefore give the proposal an unqualified endorsement. But I am also compelled to comment on the overall plan for the U.S. heavy-ion program at CMS including the total cost, the proposed physics program, and its potential impact on RHIC. The proposed U.S. heavy ion research plan for the CMS experiment includes: (1) the construction and maintenance of two zero-degree calorimeters (ZDC), (2) hardware and trigger algorithm development for the CMS high level trigger (HLT) for selecting Pb+Pb events with hard-scattering processes (this proposal), (3) construction and maintenance of a large, offline computing farm for reconstruction and physics analysis, and (4) physics analysis of data aimed at studying the hard-scattering processes (jets) and the production and survival probability of heavy-flavor quarkonium in the medium. In addition to the costs associated with the above each of the 10 institutions will require additional funds for travel to Europe and for the CERN membership fees. The latter two costs are estimated in the proposal to start at $350K/yr and increase to almost $600K/yr. The data analysis farm is estimated to cost about $400K/yr. The present proposal for 600 processors for the HLT plus test facility is $2000K. The total US CMS heavy ion program cost (from reading the present proposal) appears to be somewhat over $2M (HLT processors + ZDCs) in construction costs and about $1M/yr for operations. Given the hefty price tag, the physics case must be highly compelling. Of the physics analysis topics listed in the proposal the ones which I find most compelling, both as studies which may shed light on the nature of the medium and for which the capabilities of CMS greatly exceed that of ALICE, include: 32 1) Heavy flavor production and quenching, including the J/Psi, Psi', Upsilon, Upsilon' and Upsilon" via the muon decay channel which exploits the large muon acceptance and high momentum resolution of CMS which far exceeds the capabilities of ALICE. Measurement of heavy quarkonium rates may provide indirect estimates of the "temperature" and Debye screening length in the early stages of the collision. 2) Jet attenuation, very high pt angular correlations, energy loss for tagged jets (qg -> gamma q), quark flavor dependent fragmentation, medium effects on hard parton fragmentation measuring parton energy loss in the medium with potentially more accuracy and energy reach than at ALICE due to the large hadronic calorimeter coverage. 3) Studies at very low momentum fraction x and at very forward pseudorapidity which enables unprecedented studies of parton structure at very low x which may be a result of gluon saturation and the possible Color-Glass-Condensate (CGC) state of QCD. These studies are planned for p+p, p+Pb and Pb+Pb collisions. This program exploits the almost 4pi calorimeter coverage of CMS. In my opinion the two main strengths of the US CMS Heavy-Ion proposal are: (1) the fact that the hard-scattering related physics program outlined in the proposal and summarized above provides important constraints on the properties of the medium which cannot be achieved at ALICE, and (2) CMS would be the only collaboration doing ultra relativistic heavy ion physics in which the membership of the collaboration is dominated by scientist who are not part of the heavy ion community. I believe the latter is an advantage because the influence from the high energy component of the CMS membership will raise the standards of scientific rigor for the heavy ion analyses and publications coming out of CMS and the other heavy-ion experiments at the LHC. The synergy between the large HEP component and the smaller heavy ion nuclear physics membership would be valuable for both groups, especially the latter. The weaknesses of the US CMS proposal in my opinion include the following: 1) Hard-scattering measurements at best only provide indirect estimates of the properties of the medium. For example quarkonium suppression is inevitably a competition between color deconfinement length scales and conventional collision induced dissociation mechanisms. The results from SPS and RHIC for J/Psi suppression lead to ambiguous interpretation and remain controversial. 2) Parton energy loss can be used to estimate gluon density but not the nature of the gluon fields. For instance, if the pQCD energy loss mechanism implies a gluon density of order a few thousand per unit rapidity, the measurements will not differentiate between a dense gluon field confined in many color-singlet objects, or one in which the gluons are in a completely deconfined liquid state or dense gluonic gaseous state etc. 33 3) CMS cannot observe the soft, bulk medium for pt < 1 GeV/c due to the very strong 4 Tesla magnetic field which will cause the soft momentum particles < 1 GeV/c to curl-up inside the inner tracking region, producing multiple spirals and probably reeking havoc with the tracking software for the inner silicon layers. The present proposal indicates that the tracking capability falls-off dramatically below 1 GeV/c. Observing the interplay between the hard scattered partons, their fragmentation products and the soft medium, with respect to the momentum, velocity, net charge, net flavor, etc. correlation structures, will ultimately prove invaluable for determining the nature of the medium produced in these collisions. Already at RHIC various estimates suggest that from about 1/4 to 1/3 of the bulk particle production is due to hard scattering processes. At the LHC this fraction is expected to be even larger. Particles from hard scattering - fragmentation processes may totally dominate the particle production. The "medium" at the LHC may therefore be qualitatively different than at RHIC energies. A robust physics program at the LHC cannot ultimately produce definitive results if the experiment is blind to the bulk particle production from the medium, which remains the principal object of interest in relativistic heavy ion studies. Another concern with this proposal is the timing of the shift in manpower and funding from RHIC to LHC. Clearly, effort should go where the physics potential is greatest. However, before reducing effort at RHIC, which appears to be ideally situated with respect to its available collision energy range to study the onset of a strongly interacting, deconfined QCD state of matter (loosely termed the sQGP), the transition from dense hadronic matter to sQGP needs to be much better understood. That QCD transition will not be the object of interest at the LHC which apparently lies well beyond the energies of the transition. It would be a shame to undermine the unfolding opportunities at RHIC afforded by its fortuitous energy range in order to leap to a vastly higher energy. This is not to say that the higher energy region available at the LHC should not also be vigorously studied, just that the remaining, unique opportunities at RHIC should not be jeopardized. Potential conflicts exist for some members of this proposal with respect to the timing of the ramp-up for the CMS heavy ion program and ongoing experiments at STAR. The affected institutions and approximate number of active researchers at STAR which would have to make some tough choices about their respective levels of effort at STAR and CMS are: UC Davis - 1 involved in STAR's heavy flavor physics group U. Illinois-Chicago - 1 involved in STAR's spectra physics effort Rice U. - 2 with major commitments to the STAR Time-of-Flight upgrade project. I expect that LANL and Vanderbilt together have about 5 members who are in similar situations at PHENIX. Therefore about 1/3 of the permanent research personnel in this 34 proposal may have significant conflict of commitments at RHIC and CMS if the US heavy ion CMS program goes forward. This situation would require realistic planning and careful monitoring. I know many of the collaborators on this proposal and can strongly vouch for their strong abilities and aggressive approach to getting projects done in a timely manner and of the highest quality. The institutions are all supportive of heavy ion research, particularly MIT where the HLT test farm will be assembled. The former members of the PHOBOS experiment who are now participants in this proposal are absolutely first rate and will definitely deliver a sound HLT for heavy ions at CMS. But I have concerns about the ultimate impact of their physics program without having access to the soft component of the particle spectrum and with the timing conflicts between the ongoing RHIC program and CMS. Reviewer: This proposal describes the heavy ion program of CMS and the US participation in that program. The funding requested is specifically for additions to the high level trigger (HLT) farm needed for the physics goals of the heavy ion program. Additional aspects that require additional funding are described but are not included in this funding request. So I will make some general comments about the overall proposal but most comments will be directed toward the HLT farm corresponding to the requested funding, and assume that the other aspects requiring additional funds are reviewed elsewhere. In general, the proposers make a credible case that because the CMS detector was designed for a high-luminosity p+p environment, with many overlapping collisions in the same event, that it also functions well for the lower luminosity Pb+Pb collisions, especially for leptons at high transverse momentum. The optimizations for heavy ion running are only in triggering and algorithms but no modifications to the detector itself. In this regard, the zero degree calorimeter which is planned as a contribution from the US heavy ion groups will be used for both heavy ion and p+p running. The results of a variety of simulations are presented that indicate that the tracking efficiency, momentum resolution and mass resolution for reconstructing J/Psi and Y (from muons) and jets are quite good, for tracks with pT > 1 GeV/c. It is mentioned that the large acceptance of CMS will allow high-precision measurements of a variety of event characterization variables, including dNch/dy, ET, dET/dy, azimuthal anisotropy, reaction plane and energy for neutral spectators. However, the resolution in these variables was not presented. And considering that a significant fraction of the soft particle spectra is expected to fall below pT of 1 GeV/c it warrants more description of the effect of the low pT tracking cut-off and the effect on the sensitivity to the event characterization. The combination of large acceptance and high resolution calorimetry and tracking for high pT in CMS enables probes of the dense medium and initial state properties with high statistics unique in the LHC heavy ion program. This is clearly an important and vital aspect of the 35 overall physics program which should not be missed. However, the claims of high-precision measurements of event characterization observables should be backed up will simulation results before the program is too far underway. The scientists involved in this proposal have a long history of very competent research in the field of heavy ions from the Bevalac, the AGS, the SPS and RHIC, and make a very strong scientific team to carry out the proposed work, in collaboration with the rest of the international CMS heavy ion program. Given the lower luminosity of AA compared to pp collisions at the LHC, and the greater complexity of AA events, the choice of passing a higher fraction of events through to the high level trigger is the appropriate one for CMS heavy ions. This allows triggering and event recording decisions to be made with more complete event information, compared to pp, and fits within the bandwidth limitations of the DAQ system, as long as the commensurate increase in CPU power of the HLT is available. The HLT component of the CMS DAQ/Trigger system is already designed and planned for the pp program and this proposal is to expand the very modular PC farm components to have enough CPU power, and therefore does not add any risk to the design of the DAQ/Trigger system. The planned schedule, starting with a development system for the HLT at MIT in 2007, followed by a ramped buildup of the HLT farms at CERN during 2008 – 2010 is appropriate for the expected beam schedule with a low luminosity heavy ion run late in 2008 and the first full luminosity heavy ion run in late 2009. The initial algorithms in the HLT for the 2009 run will be simpler than those employed later so that expanding the HLT after the 2009 run and before the 2010 run is appropriate. The development system at MIT is very well justified because the HLT at CMS will be occupied with running the detector most of the time after the LHC turn on in late 2007/early 2008 and will have very little availability for development testing. The funding requested is for equipment cost and installation of the HLT development cluster at MIT, and also for the equipment cost to expand the HTL on the detector at CERN. The expenses at MIT for facilities cost hosing the development cluster and operational support are not included in this proposed budget and so are intended to come from other sources. The funding for the HLT installation at CERN is entirely for the computer hardware with all aspects of installation and operations costs coming from elsewhere. The anticipated price/performance for the PCs to be used at MIT and CERN is reasonable and typical. The total size needed for the HLT farm is not, and can not be, precisely defined. Not all of the necessary algorithms are yet developed. A couple algorithms (jet finding and L2 muons) have been benchmarked and fall within the proposed CPU budget. Another algorithm (L3 muons) which has not yet been optimized exceeds the proposed CPU budget, and additional trigger 36 algorithms have yet to be developed. So it is safe to say that the proposed expansion of the HLT farm is not too large but should be sufficient for the initial running through 2010. It is likely that additional CPU power for new algorithms will be desired in the future, and if this will fit within the expected CERN maintenance & replacement program for the HLT or will require additional funding is something that won’t be known without some experience with data taking and analysis. Essentially all of the groups participating in the US CMS heavy ion program are partnering with groups at the same institutions involved with the HEP CMS program. This helps very much with keeping the heavy ion program and the pp programs well integrated and helps assure strong institutional support for those groups. Summary Given that the budget proposed here covers only part of the overall US CMS heavy ion program it is not possible to give an overall rating for the proposal. However, a few summary statements can be made. This will make an important and vital contribution to the LHC heavy ion program. The size and cost of the HLT farm expansion is reasonable and any reduction of the proposed size will likely have consequences on the physics capabilities for CMS for heavy ions. The collaboration is certainly well qualified and capable to carry out the program. The only caveat is to encourage a quantitative analysis of the effect of the low pT cutoff on the event characterization variables. 37