March Professor Robert S Orr Canadian Spokesperson for the by eminems


									March 25, 2006

Professor Robert S. Orr
Canadian Spokesperson for the Atlas Project
Department of Physics
University of Toronto
Toronto, Ontario

Dear Professor Orr:

This is in response to your letter to the National Initiatives Committee (NIC) that is
responsible for organizing the proposal to the Canadian Foundation for Innovation (CFI)
National Platforms Fund for financial support to acquire infrastructure for High
performance Computing. The Committee had requested clarification concerning the
ability and willingness of the ATLAS Community to work within a shared computing
environment in order to meet its’ needs for Tier 2 processing. Your memo entitled
“ATLAS computing request to the CFI National Platforms Fund” and dated March 6 has
satisfied the Committee that the ATLAS Community will be able to work within an
environment in which all of the infrastructure that is financed by the NPF is widely
shared by the Canadian national research community. Thank you for your willingness to
help the Committee better understand ATLAS needs.

Yours sincerely

Ken Hewitt
On behalf of the NIC
Memo to: National Platforms Fund NIC committee
Re:      ATLAS Computing Request to the CFI National Platforms Fund

The suitability of the ATLAS project for inclusion in the National Platforms Fund
application has recently been brought into question. This memo to the NIC is to clarify
both the nature of ATLAS project and the request that we have made to the NPF. The
ATLAS-Canada computing model has been well thought-out, and is designed to make
efficient use of valuable computing resources. It combines a dedicated facility necessary
for large scale data reduction, with shared resources used for analysis work, where the
access patterns to the data make this more appropriate.

1) Particle Physics Experimental Programs and Analysis
   Particle physics experiments, like ATLAS, are large international collaborations of
   researchers with diverse scientific interests. The international ATLAS project is
   designed to exploit the new Large Hadron Collider at CERN. It will address many of
   the outstanding issues in particle physics. It is a $400 million project involving around
   1500 international participants. It will have a lifetime of more than a decade, and
   should be regarded as an experimental facility, rather than an “experiment” in the
   traditional usage of the term. In Canada, 22 University Faculty members participate in
   ATLAS. This will grow to 40 by the end of the decade, when ATLAS-Canada will
   comprise about 50% of the Canadian particle physics community. The project
   comprises the majority of the medium-term computing needs of the particle physics
   discipline. The ATLAS detector is a complex facility composed of multiple detector
   systems with thousands or millions of readout channels. Particle physics experiments
   are statistical in nature. Many millions of individual events are examined for statistical
   trends. This puts a premium on both computational power and data storage.

   Extracting scientific results from the data proceeds in two stages.
     1. Calibrating the detector systems, and processing the raw data to produce
              “analysis objects.”
              This task requires expertise from the physicists working on each detector
              system, and is a continuous operation requiring dedicated computing
              facilities. The computational power in this stage is defined by the amount
              of data produced by the accelerator/experiment complex. The amount of
              data accumulated at the LHC will imply a 24x7 “industrial computing”
              facility, part of which will be in Canada. To meet this need, ATLAS
              Canada has requested a dedicated facility known as a “Tier-1” centre from
              the CFI Exceptional Opportunities Fund.
     2. Studies of the “analysis objects” to extract scientific results.
              This process is often known as “physics analysis”. At this stage individual
              researchers pursue independent physics studies, not unlike the “data-
              mining” that goes on in other fields such as bioinformaticians studying the
              human genome, astrophysicists performing CMB studies with satellite
              data, or astronomers analyzing telescope images. Canadian physicists
              alone will produce many different scientific results extracted from the
              ATLAS data. These individual analyses follow the usual pattern in other
              research fields. Physicists will produce statistical analyses on large data
              sets, and then spend time thinking about the results, and what to do next.
              The nature of this process leads to many peaks and valleys in resource
              needs, and so fits naturally into the environment of shared-use computing
              facilities. Several current particle physics experiments already make
              productive large scale use of existing CFI-funded shared computing
              facilities in Canada. Experience has shown that this stage of the analysis
              is indeed ideally suited to shared facilities.

  Most of these studies involve comparing real data to computer simulated data.
  Detector simulation is a CPU-intensive task. The production of simulated data sets is
  stimulated by what physicists learn during the analysis process. So it also shows large
  fluctuations in the resources needed. ATLAS-Canada has requested NPF resources
  directed at both the analysis and simulation efforts.

2) Shared-Use Infrastructure
   We have discussed above the excellent match between the particle physics analysis
   cycle and the use of shared resources. We appreciate that the NIC has recognized this
   fact, and that the corresponding requested CPU resources fit well into the NPF model.
   However, we are requesting resources for both CPU and disk storage for these
   activities, from the CFI National Platforms Fund, and we would like to examine the
   issue of disk usage.

  Particle physics analysis requires a larger storage/CPU ratio than many fields. This is
  due to the statistical nature of our studies, as described already. While storage
  infrastructure is sharable, the storage volumes themselves are intrinsically more
  difficult to share than CPU. However, the ATLAS use of storage is not qualitatively
  different from other projects based on the analysis of large databases. For example,
  the human genome and satellite data will occupy disk space at shared NPF facilities on
  a long-term basis. In contrast, the ATLAS disk usage at shared facilities will not be
  permanent. The major raw data, and simulated data, repository will be at the
  dedicated Tier 1 centre. These data will be periodically reprocessed at the Tier 1
  centre, and the “analysis object” data at the shared facilities will be renewed on a few-
  week to few-month timescale. At the shared facility level, the amount of disk space
  occupied by ATLAS data can be rather flexible, and can be optimized to meet the
  needs of ATLAS Canada and other users of the systems.

  The large storage/CPU ratio does effectively increase the overall cost of computing
  systems that can be used by the particle physics community. However many of the
  components of such systems are relatively inexpensive and cost effective, as we use
  serial Linux clusters with modest memory and interconnection requirements. We
  believe that in order to serve the needs of significant parts of the HPC community, the
  NPF should fund diverse types of computing infrastructure. Different fields have
  somewhat different computing environments, and a national program should meet
  these different needs. In this context we note that the overall cost of the CPU+disk
  system that we have proposed is very competitive in cost with other architectures
  which are accepted as part of the NPF proposal.
3) Technical and Scientific Justification of Requests
   ATLAS Canada has documented the computing requirements needed to achieve our
   scientific goals in our Computing Model document. This is publicly available at It
   would be very useful if each part of the Canadian community seeking NPF resources
   would produce such a document detailing their request. This would provide the NIC
   with the information it needs to compare the various requests, and to understand the
   best allocation of financial resources in order to achieve a balanced and diverse
   computing infrastructure across Canada. To realize, indeed, an infrastructure base able
   to serve the widest possible community. It appears that the fact that we have
   presented a detailed justification for our request has focused the debate on our
   proposal. We urge the NIC to request a similar level of detail from other groups in
   order to be able to base its allocations on a consistent level of information. Such
   submissions should include not only details on the technical requirements for each
   group, but also information on the sizes of the communities served, and on the
   intellectual and scientific output expected from the proposed investment of resources.

4) Process and Communication
   ATLAS was highlighted in the Canadian computing long range plan exercise as a
   flagship use-case, and has solid scientific goals tied to firm computing needs.
   However, there remain misconceptions on the project itself, and on the nature of the
   resources being requested. We believe that having the opportunity to address
   questions, or impressions, from the NIC directly face-to-face would be beneficial and
   help to save time. We would be more than happy to have a representative of our
   group present the ATLAS-Canada computing strategy to the NIC in person. We
   would at least request that ATLAS-Canada be represented when ATLAS computing
   issues are being discussed. This would avoid the NIC making decisions based on
   incomplete information.

In summary, we hope that we have dispelled the incorrect impression that ATLAS is a
“single experiment”. The ATLAS dataset will yield results which answer many of the
fundamental questions in subatomic physics, and perhaps cosmology. We have explained
why ATLAS should be considered as an experimental facility rather than a “single
experiment”. ATLAS will serve the needs of up to half of the Canadian particle physics
community, and this community is a substantial part of the basic physics research
community in Canada. The use ATLAS will make of disk resources is not unlike other
groups in the NPF, and is really an issue of scale. Furthermore, the total cost of the
CPU/disk systems we are requesting is competitive with other NPF architectures.

Professor Robert S. Orr, University of Toronto
Spokesperson of the ATLAS Canada Collaboration

Cc: Professor Michel Vetterli, Simon Fraser University
    ATLAS Canada computing coordinator

    Associate Professor Robert McPherson, University of Victoria
    ATLAS Canada deputy spokesperson

To top