UK Plans for LHC Grid

UK Plans for LHC Grid John Gordon HEP-CCC, Bologna June 2001 LHC Computing Model Uni x USA Brookhaven USA FermiLab Lab m UK Uni a Lab a Physics Department Desktop Tier2 Tier 1 Italy France CERN Uni n ………. Germany  Lab b   Uni y NL Lab c Uni b John Gordon Tier1&2 Plans - Bologna 2 UK Grid not just LHC UK John Gordon Tier1&2 Plans - Bologna 3 US experiments - Grid Plans John Gordon Tier1&2 Plans - Bologna 4 BaBar • • • • • • • • 8x80cpu farms 10 sites with 12TB disk and Suns simulation data mirroring from SLAC - Kanga, Objectiity data movement and mirroring across UK data location discovery across UK - mySQL remote job submission - Globus and PBS common usernames across UK - GSI gridmapfiles • Find data - submit job to data - register output • BaBar planning a distributed computing model – TierA centres John Gordon Tier1&2 Plans - Bologna 5 CDF • Similar model to BaBar with disk and cpu resources at RAL and universities plus farm for simulation. • Development of Grid access to CDF databases lead by UK. • Data replication from FNAL to UK and around UK • Data Location Discovery through metadata John Gordon Tier1&2 Plans - Bologna 6 D0 • • • • Large data centre at Lancaster ship data from FNAL to UK simulation in UK and ship data back to FNAL Gridify SAM access to data – data at FNAL and Lancaster John Gordon Tier1&2 Plans - Bologna 7 GridPP History Collaboration formed by all UK PP Experimental Groups in 1999 to submit £5.9M JIF bid for Prototype Tier-1 centre at RAL (later withdrawn) Added some Tier-2 support to become part of PPARC LTSR “The LHC Computing Challenge”  Input to SR2000 Formed GridPP in Dec 2000 included CERN, CLRC and UK PP Theory Groups John Gordon From Jan 2001 handling PPARC’s commitment to EU DataGrid UK 8 Tier1&2 Plans - Bologna UK Strengths Wish to build on UK strengths • Information Services • Networking - world leaders in monitoring • Security • Mass Storage UK Major Grid Leadership roles • Lead DataGrid Architecture Task Force (Steve Fisher) • Lead DataGrid WP3 Information Services (Robin Middleton) • Lead DataGrid WP5 Mass Storage (John Gordon) •Strong Networking Role in WP7 (Peter Clarke, Robin Tasker) • ATLAS Software Coordinator (Norman McCubbin) • LHCb Grid Coordinator (Frank Harris) Strong UK Collaboration with Globus • Globus people gave 2 day tutorial at RAL to PP community • Carl Kesselman attended UK Grid Technical meeting • 3 UK people visited Globus at Argonne Natural UK Collaboration with US PPDG and GriPhyN John Gordon Tier1&2 Plans - Bologna 10 Proposal Summary • • • • £40M 3-Year Programme LHC Computing Challenge = Grid Technology Five Components: • Integrated with EU DataGrid, PPDG and GriPhyN Facilities at CERN, RAL and up to four UK Tier-2 sites Centres = Dissemination LHC developments integrated into current programme (BaBar, CDF, D0, ...) Robust Management Structure Deliverables in March 2002, 2003, 2004 • • • • • – – – – – • Emphasis on Grid Services and Core Middleware Foundation Production Middleware Exploitation Value-added Exploitation Approved in principle, May 2001 Financial details to be confirmed John Gordon Tier1&2 Plans - Bologna 11 GridPP Workgroups Technical work broken down into several workgroups - broad overlap with EU DataGrid A - Workload Management Provision of software that schedule application processing requests amongst resources F - Networking Network fabric provision through to integration of network services into middleware B - Information Services and Data Management Provision of software tools to provide flexible transparent and reliable access to the data G - Prototype Grid Implementation of a UK Grid prototype tying together new and existing facilities C - Monitoring Services All aspects of monitoring Grid services H - Software Support Provide services to enable the development, testing and deployment of middleware and applications at institutes D - Fabric Management and Mass Storage Integration of heterogeneous resources into common Grid framework I - Experimental Objectives Responsible for ensuring development of GridPP is driven by needs of UK PP experiments E - Security Security mechanisms from Certification Authorities to low level components John Gordon J - Dissemination Ensure good dissemination of developments arising from GridPP into other communities and vice versa 12 Tier1&2 Plans - Bologna Major Deliverables Prototype I - March 2002 • Performance and scalability testing of components • Testing of the job scheduling and data replication software from the first DataGrid release. Prototype II - March 2003 • Prototyping of the integrated local computing fabric, with emphasis on scaling, reliability and resilience to errors. • Performance testing of LHC applications. Distributed HEP and other science application models using the second DataGrid release. Prototype III - March 2004 • Full scale testing of the LHC computing model with fabric management and Grid management software for Tier-0 and Tier-1 centres, with some Tier-2 components. John Gordon Tier1&2 Plans - Bologna 13 GridPP Collaboration Meeting 1st GridPP Collaboration Meeting - Coseners House - May 24/25 2001 John Gordon Tier1&2 Plans - Bologna 16 Current UK testbed John Gordon Tier1&2 Plans - Bologna 17 UK-Sites Glasgow Edinburgh Durham Lancaster Clusters – Scotland – North West – Midlands – London Liverpool Dublin Manchester Testbed Site Sheffield Birmingham (integrated) Oxford RAL Cambridge QMUL,UCL,IC,Brunel,RHU Bristol John Gordon Tier1&2 Plans - Bologna 18 Globus MDS Explorer John Gordon Tier1&2 Plans - Bologna 19 External Resources External Funds (additional to PPARC Grants and central facilities) have provided computing equipment for several experiments and institutes BaBar (Birmingham, Bristol, Brunel, Edinburgh,) 12TB disk, 10 Suns, (Imperial, Liverpool, Manchester, QMUL, RAL, RHUL) 8 Linux farms MAP (Liverpool ) 300 node farm ScotGrid (Edinburgh, Glasgow) farm, disk, tape D0 (Lancaster) 200 node farm, 30-200TB tape Dark Matter (Sheffield) Tape CDF/Minos (Glasgow, Liverpool, Oxford, UCL) Disk, servers and farm CMS (Imperial) Farm ALICE (Birmingham) Farm Total £5.4M All these Resources will contribute directly to GridPP Many Particle Physics Groups are involved in large SRIF bids in collaboration with other disciplines mostly to form e-Science centres. The amount of resource available to GridPP from this SRIF round could be several £M John Gordon Tier1&2 Plans - Bologna 20 GridPP Organisation Hardware development organised around a number Regional Centres • Likely Tier-2 Regional Centres • Focus for Dissemination and Collaboration with other disciplines and Industry • Clear mapping onto Core Regional e-Science Centres Software development organised around a number of Workgroups John Gordon Tier1&2 Plans - Bologna 21 Tier1&2 Plans • RAL already has 300 cpus, 10TB disk, and STK tape silo which can hold 330TB • Install significant capacity at RAL this year to meet BaBar TierA Centre requirements • Integrate with worldwide BaBar work • Integrate with DataGrid testbed • Integrate Tier1 and 2 within GridPP • Upgrade Tier2 centres through SRIF (UK university funding programme) John Gordon Tier1&2 Plans - Bologna 22 Tier1 Resources 80 70 60 50 40 30 20 10 0 2001 2002 2003 kSI95 disk TB tape TB/10 John Gordon Tier1&2 Plans - Bologna 23 Tier1 Integrated Resources 120 100 80 60 40 20 0 2001 2002 2003 kSI95 disk TB tape TB/10 John Gordon Tier1&2 Plans - Bologna 24 Liverpool • MAP - 300 cpus + several TB of disk – delivered simulation for LHCb and others for several years • Upgrades of cpus and storage planned for 2001 and 2002 – currently adding Globus – develop to allow analysis work also John Gordon Tier1&2 Plans - Bologna 25 Imperial College • Currently – 180 cpus – 4TB disk • 2002 – adding new cluster in 2002 – shared with Computational Engineering – 850 nodes – 20TB disk – 24TB tape • CMS, BaBar, D0 John Gordon Tier1&2 Plans - Bologna 26 Lancaster Worker Worker Worker Switch 500 GB Bulkserver 500 GB Bulkserver 100 MB/s Ethernet 1000 MB/s Ethernet Fiber 196 Worker CPUs Switch Controller Node Controller Node Tape Library Capacity ~ 30 TB Finalizing Installation of Mass Storage System ~ 2 Months John Gordon Tier1&2 Plans - Bologna k£11/30 TB Not Fully Installed 27 Lancaster • Currently D0 – analysis data from FNAL for UK – simulation • Future – upgrades planned – Tier2 RC – Atlas-specific John Gordon Tier1&2 Plans - Bologna 28 ScotGrid • • • • • • Tendering now 128 CPU at Glasgow 5 TB Datastore + server at Edinburgh ATLAS/LHCb Plans for future upgrades to 2006 Linked with UK Grid National Centre John Gordon Tier1&2 Plans - Bologna 29 Wider UK Grid • Prof Tony Hey leading Core Grid Programme • UK National Grid – National Centre – 9 Regional Centres • Computer Science lead • includes many sites with PP links – Grid Support Centre (CLRC) – Grid Starter Kit • vesion 1 based on Globus, Condor, ...... • Common software • • • • e-science Institute Grid Network Team Strong Industrial Links All Research Areas have their own e-science plans Tier1&2 Plans - Bologna 30 John Gordon Network • UK Academic Network, SuperJANET entered phase 4 in 2001 • 2.5GB backbone, December 2000-April 2001 • 622Mbit to RAL, April 2001 • Most MANs have plans for 2.5GB on their backbones • Peering with GEANT planned at 2.5GB John Gordon Tier1&2 Plans - Bologna 31 John Gordon Tier1&2 Plans - Bologna 32 John Gordon Tier1&2 Plans - Bologna 33 John Gordon Tier1&2 Plans - Bologna 34 TEN-155  GEANT • • • John Gordon 2.5 Gbps to 10 Gbs & double every year for 4 years Consolidated Global Connectivity Geographic Expansion Bandwidth, QoS, • Managed Tier1&2 Plans - Bologna VPN 35 Summary • UK has plans for a national grid for particle physics – to deliver the computing for several virtual organisations (LHC and non-LHC) • Collaboration established, proposal approved, plan in place • Will deliver – UK commitment to DataGrid, – prototype Tier1 and 2 – UK commitment to US experiments • Work closely with other disciplines John Gordon Tier1&2 Plans - Bologna 36