7
file://///ntserver/wwwroot/memoria/04-html/Scientific%20Activities/P...
7. Summary of Activities of the Port d'Informació Científica (PIC) 2004
The activities developed by PIC in 2004 can be classified mainly in the following areas: 1. 2. 3. 4. Deployment of the GRID Infrastructure of the LCG Project in Spain Startup of the EU project Enabling Grids for E-Science in Europe (EGEE) Participation in the Data Challenges of the LHC experiments Collaboration with UDIAT/Parc Taulí in the area of Medical Image Data
1. Deployment of the GRID Infrastructure of the LCG Project in Spain The first certified version of the software of the LHC Computing Grid (LCG) appeared on 1st September 2003. This version, LCG-1_0_0 was installed immediately at PIC. A few days later this installation was certified by the “LCG Grid Deployment” team at CERN. This way, the first deployed version of the LCG infrastructure had a node in Spain right from the moment that it became operative. Two working groups were created in April 2003 within the realm of the LCG project: “LCG-ES Deployment and Implementation” (LCGES-DI) and “LCG-ES Operations” (LCGES-OP). The first group is meant to coordinate the activities related to the initial deployment and development of future versions of LCG and the second concentrates on the coordination of operational activities that are run on the stable LCG infrastructure that is “in production” at any given moment. The LCGES-DI working group has been coordinated from PIC since its creation. Therefore every two weeks a videoconference is organized to which representatives of every Spanish LCG center connect. The minutes of these meetings are available online (http://cvu.rediris.es/pub/bscw.cgi/0/427950) as well as a list of the actions that have been initiated as a result of the group activities. PIC, as the Spanish Tier-1, has dealt with the coordination of the LCG deployment in the Spanish Centers, supporting directly the installation, maintenance and problem analysis via telephone and email. Between October 17th and November 12th all the LCG-ES Centers (IFIC, CIEMAT, USC, UAM, UB, IFCA) were incorporated in a controlled way and in coordination with CERN. Spain has been one of the countries where the LCG deployment has been realized very fast therefore achieving a good visibility within LCG. By mid January 2004 the deployment of the second version of LCG, LCG-2 was initiated. In this new version important changes of the middleware were introduced which were not compatible with earlier versions. For this reason during the migration from LCG-1 to LCG-2 the two infrastructures coexisted in parallel for some time. Initially seven LCG-2 core sites (CERN/CH, CNAF/I, FNAL/USA, FZK/D, NIKHEF/NL, PIC/E and RAL/GB) were chosen, where the first LCG-2 version would be installed in a controlled manner. The experiments could migrate data registered in the LCG-1 catalogues to the new version. After a few weeks an updated version of LCG-2 was published, in which the problems encountered during the trial period in the core sites had been corrected. This new version was the one that was deployed massively in all the nodes that up to that moment had belonged to LCG-1. In Spain, the deployment of LCG-2 in all the LCG-ES centers was coordinated by PIC through a mailing list and periodical meeting of the LCGES-DI working group and was finalized within one month, between March and April 2004. Since that moment all the LCG-ES centers (USC, IFIC, CIEMAT, UB, IFCA, UAM and IFAE) have been operative in LCG-2 and their resources have been used in an intensive way for the Data Challenges of the LHC experiments. 2. Startup of the EU project Enabling Grids for E-Science in Europe (EGEE) Since the beginning of 2002, Prof. Manuel Delfino has been involved in the design and the negotiations
1 of 5 02/05/2005 15:08
7
file://///ntserver/wwwroot/memoria/04-html/Scientific%20Activities/P...
of the European project Enabling Grids for E-Science in Europe (EGEE). In October 2003 the EGEE project was approved and started in April 2004 with 70 participant European, Russian and USA institutions, coordinated with similar projects in USA, Japan and other countries. EGEE deploys and operates in production the first International Grid Infrastructure on a large scale based on a federated scheme of Regional Operation Centers. PIC administrates the Operations Center Southwest (Spain and Portugal) which is coordinated by Dr. Andreu Pacheco. From June 2004 onwards, the GRID infrastructure of LCG was integrated into EGEE as is shown in Fig 1. Also shown is the integration of centers with specialisations other than High Energy Physics, as for example the National Center of Biotecnology (CNB) and the Astrobiology Center (CAB).
Figure 1: Map of the centers which are connected to EGEE in September 2004
3. Participation in the Data Challenges of the LHC experiments During 2004, the four LHC experiments ran a series of tests, the so-called “Data Challenges” (DC), whose purpose is to test the computing infrastructure that will process and analyze the data generated by the detectors. One of the aims of the DC in 2004 was to demonstrate that with the current infrastructure and computation models 10% of the foreseen LHC data flow in 2008 can be absorbed. The 2004 DCs represent a test bench of fundamental importance for the development of LCG. PIC participated in the DCs of the three experiments in which Spain is involved: ATLAS, CMS and LHCb. The DC of the CMS detector was the first to be carried through and didn’t overlap in time with any of the others. This circumstance facilitated the operation of the LCG node at PIC as far as it wasn’t necessary to share resources with other experiments. The two most outstanding milestones that were reached were: · Loading of the network with massive data transfers: 760 GB of data were replicated simultaneously from CERN to PIC, INFN (Bologna/Italy) and FNAL (Chicago/USA). The transfer to PIC was the fastest and the total data volume was transferred in 10 hours. This was in spite of PIC having the
2 of 5
02/05/2005 15:08
7
file://///ntserver/wwwroot/memoria/04-html/Scientific%20Activities/P...
slowest network, showing the importance of global optimization of the data transport system. The average transfer was 25 MB/s with peaks of 50 MB/s limited by the bandwidth between PIC and the academic network (400 Mbps). . · Data analysis in real time in the Tier-1: Monte Carlo data was generated at CERN which, as soon as it was reconstructed, was replicated to the Tier-1 and analyzed in quasi real-time. The relative contribution of PIC to the DCs of ATLAS and LHCb, in terms of computing power and in consideration of the total which is contributed by LCG, was around 9% and 7% respectively. These tests provided a realistic and intensive CPU cycle-competition environment that has been very useful to test different configurations of the GRID “Computing Element” (CE) Service and find the one which grants the largest flexibility and efficiency when sharing computing power. It is worth mentioning that the computing power supplied by PIC during these tests comes from one single cluster of computers which provides CPU cycles both to the DCs, through the LCG node, as well as to the rest of PIC users. The latter cause an analysis work load whose temporal and data access pattern is “chaotic”. This experience has therefore been very useful, because it enabled engineers at PIC to exercise simultaneously the resources for activities which represent work load patterns that are so different as the analysis on final users and the systematic production of Monte Carlo. One of PICs fundamental roles within the LCG-ES architecture is to provide its users with quasi-online storage and access to large amounts of data. This service has been constructed based on the CERN CASTOR software (http://www.cern.ch/castor), which allows to virtualize the space on magnetic tapes provided by an automated storage system and to present this to the user as a file system. As a part of the LCG deployment, a series of GridFTP servers have been installed at PIC (http://www.globus.org/datagrid/gridftp.html) which provide secure and quick access to the data stored at PIC from any LCG node in Spain and the rest of the world. This service has been used intensively by the three experiments for their storage and also to access the data afterwards for analysis. In December 2004, 35 TB of data are stored in CASTOR at PIC (15 TB for ATLAS, and 10 TB for each of CMS and LHCb).
Fig. 2. Tape Library at PIC
During 2004, PIC participated not only in the Data Challenges as LCG-2 Resource Center but also
3 of 5
02/05/2005 15:08
7
file://///ntserver/wwwroot/memoria/04-html/Scientific%20Activities/P...
contributed in the Combined Test Beam (CTB) (http://atlas.web.cern.ch/Atlas/GROUPS/GENERAL/TESTBEAM/testbeam.html) of Atlas, where for the first time data representing a complete section of the detector (including all the subdetectors) was operated in a test beam. PIC was responsible for the offline processing of the CTB data and plans, once this will be finalized, to get involved in supporting the analysis activities of these data in the GRID, in collaboration with IFAE. 4. Collaboration with UDIAT/Parc Taulí in the area of Medical Image Data Hospitals and clinical centers in general accumulate an ever increasing amount of medical image data, corresponding to mammographies, radiographies and tomographies, amongst others. This leads to a large demand of storage capacity and collaterally increases the time necessary to access the data. The requirements are twofold: the clinical staff need to access the medical histories of the patients, and researchers need to realize data mining processes and comparative image analysis. Emilio Hernández, Visiting Professor of the Universidad Simón Bolívar/Venezuela (financed through the PIC budget), at PIC since March 2004, has been leading a specific pilot project with UDIAT-CD, a company belonging to the Corporación Sanitaria Parc Taulí at Sabadell, which is in charge of the storage and management of the Medical Images of the Hospital Parc Taulí. UDIAT is connected to the Catalan academic data network (“Anella Científica”), a circumstance that turns it into an ideal partner for such a pilot project. Another advantage to start with UDIAT is that this hospital operates already a rather modern software and hardware infrastructure. In particular they have developed an image management software, called RAIM, which enables the real time access by doctors to medical image data. The pilot project will study the offering a stable data service with confidentiality guarantee and dimensioned to cover the data consumption requirements of the Hospital Parc Taulí. In addition, the training effect of this experience will allow to develop the guidelines necessary for the extension of this service to other clinical centers, particularly in Catalonia. Moreover, this experience will establish the basis for the definition of a data access service with the intention of doing research in the area of the processing of medical image and the analysis of these images with different purposes, as for example the automatic diagnosis of diseases or previous conditions to the development of a pathology. The achievements of the project Parc Taulí are so far: · The design of a transfer and storage platform for medical images at PIC. Under this platform, UDIAT transfers anonymized images to PIC, where they are stored transitionally in a caché area and afterwards are kept in the tape robot in volumes. This pattern is the result of a series of tests that allowed us to optimize the transfer mechanisms, having detected that the storage of individual images made rather inefficient use of the robot. · As an outcome of this development, a volume based (mainly ISO files) storage and data retrieval pattern has been designed, in order to make the data management transparent to the end user. Thus, UDIAT users for example, send and recover individual images, whereas internally in PIC everything is stored in volumes of a much greater size. This pattern is extensible to other requirements which involve storage of small files, also beyond this specific application field. Aside from the clinical requirements, which consist normally of recovering images for a given patient, a system for the specification and creation of data sets for research purposes, that is the massive processing of medical images, has been partially created with the aid of a student working on her engineering thesis at UAB. This system allows to select images by a specific criterion, i. e. by a certain type (mammographies, tomographies, etc.), a certain type of patient (by sex, age), or by any other criterion included in the data base. This development is about 60% completed.
·
A first draft of a project plan to re-implement the whole medical image system using Grid Infrastructure techniques has been developed, aiming for implementation in 2005.
4 of 5
02/05/2005 15:08