hpc-india-6.doc - CDAC has devel

Document Sample
hpc-india-6.doc - CDAC has devel Powered By Docstoc
					1. Introduction

The history of scientific computing in India dates back to the 1950’s, right from the time
India started its research activities as an independent country. The growth of computing
as a discipline in India has been somewhat slower in the first 3 – 4 decades and had its
ups-down. Dependent on the western counties for the technology and the prohibitively
expensive cost of computing systems has been the major factor in this slow growth.
Nonetheless, the growth was steady and matured to the right level where the researchers
can not only exploit the new technology but also innovate and enable them. A clear
evidence of this is the dramatic growth in high performance computing witnessed in last
15 – 20 years in India. From the state of longing for a single mainframe high
performance system for the entire country, the nation has moved to state where it is
capable of building high performance computing systems (with both off-the-shelf
components exported from other countries and indigeneously developed components,
specifically for network subsystem). The expertise to effectively use the high
performance computing systems has been growing steadily since the days of induction of
parallel computing systems in India in the early 1990s.

The Indian government has been instrumental in enabling this change. Starting from the
early initiative which encouraged the design of the TIFRAC system (see discussion on
this in Section XX), to the founding of the Centre for the Development of Advance
Computing for the design of parallel computer systems, to the most recent Grid Garuda
initiative, the government has significantly funded major research initiatives in the high
performance computing system. The support given by the government were fully
supplemented by the efforts of various research organizations and their researchers to
make sure they lead to growth and competence in the high performance computing area.
Today, researchers in India work on various aspects of high performance computing ---
system architecture, performance evaluation, software system, and applications --- to
advance the frontiers of science and engineering through high performance computing..

In this report we present the facilities available at some of the organizations/institutions
and their research efforts. A comprehensive report on either of these would be near
impossible and we do not attempt it here. Instead, we try to present a flavor of both
these in the report and possible future directions the HPC research in India might take.

In the next section, we give a brief history of HPC in India. We then list and give a brief
profile of the HPC centers we have considered and their HPC facilities. In addition to the
isolated HPC centers, the Indian Grid initiative called Garuda has established a federation
of multiple HPC facilities located at distributed sites to form a grid system. We describe
the Garuda grid framework in detail. We then describe the applications that are executed
in these HPC centers, with focus on three major application areas, namely, climate
modeling, bioinformatics and computational physics. We then present the system
software development work for HPC systems. The HPC capability of India is tracked by
a project called Top Supercomputers-India. This project lists the top powerful
supercomputers in India twice a year and is the Indian equivalent of the Top500 project
that lists the 500 powerful supercomputers in the world. We also describe the project and
its current status. The HPC activities in India are also nurtured by some major
conferences and workshops that have specific focus of HPC in India. We also give a
description of these conferences, and events. Finally, we present the future roadmap of
HPC in India.

2. History of High Performance Computing in India

2.1 Computing Facilities

India’s research in using computing systems for scientific calculation started way back in
the 1960’s. Indian researchers used some of early mainframes available at select
institutions in the country. The IBM 1620 and IBM 7044 (at the Indian Institute of
Technology, Kanpur), the IBM 1401 (at the Indian Statistical Institute, Calcutta), the
CDC 3600 (at the Tata Institute of Fundamental Research, Bombay), and the IBM
360/44 (at the Indian Institute of Science, Bangalore) were some of the early mainframe
systems. Subsequently other mainframes such as the IBM 370/165 (at the Indian
Institute of Technology, Madras), the DEC 1077 (at the Tata Institute of Fundamental
Research, Bombay) and the DEC 1090 systems (at the Indian Institute of Science,
Bangalore and the Indian Institute of Technology, Kanpur) were installed and used
extensively by researchers in 1970’s. Computing applications were mainly in the areas of
finite element analysis, computational fluid dynamics, protein structure, power system
analysis, and structural chemistry and solid state physics.

Simultaneously research and graduate programs in computer science started in India in
mid 1960’s. Research groups grew at the Tata Institute of Fundamental Research,
Bombay, the Indian Institute of Science, Bangalore, the five Indian Institutes of
Technology, Indian Statistical Institute, Calcutta, Jadavpur University, and the Birla
Institute of Technology and Science, for example. Early focus was on numerical
computing, theory of algorithms, complexity, program verification and formal techniques
for programming, systems software and programming, and artificial intelligence.

Research in high performance computing started in India in the mid 1980’s. Several
academic institutions /research organizations aspired to acquire supercomputing systems
in order to meet the demands of their computational scientists/engineers. At the Indian
Institute of Science, Bangalore, an initiative to set up a national supercomputing facility
was started by the Department of Electronics and Ministry of Human Resource
Development, Government of India in mid 1980’s. This fructified as the Supercomputer
Education and Research Centre (SERC) which hosts a number of high performance
computing systems in the country. Disheartened by delays in procuring a Cray XMP
system, SERC hosted a Cyber 992 system and a VAX 8800 system in the late 1980’s and
early 1990’s. Also, SERC set up one of the first workstation farm of IBM headless
workstations connected by a network. SERC also deployed a PARAM 8000 machine,
indigenously developed in India by the Centre for Development of Advanced Computing
(C-DAC). By 1993/94, SERC hosted a variety of distributed high performance
computing systems such as IBM’s 32-node Scalable Parallel (SP) system, SGI’s 6-
processor Power Challenge system as well as an 8-processor DEC-8400 system in
addition to a large number of IBM and Sun workstations.

Around the same time, a national initiative in supercomputing in the form of a time
bound mission to design, develop, and deliver a supercomputer in the Gigaflop range was
launched. This fueled HPC systems research in research organizations (outside an
academic institution). Organizations such as Centre for Development of Advanced
Computing (C-DAC), Centre for the Develop of Telematics (C-DoT), Bhaba Atomic
Research Centre (BARC), Advanced Numerical Research Group (ANURAG), and
National Aerospace Laboratories (NAL) initiated several complementary projects to
develop high performance computing systems. While C-DAC came us with the first
indigenous supercomputing system, called PARAM 8000 series, which used Inmos 800
and 805 Transputers to achieve a theoretic peak performance of 1 Gigaflop. The
sustained speedup achieved was 100-200 MegaFLOPS. Other research organizations
joined the race with parallel systems ranging from 8-nodes to 64-nodes have been
designed and developed independently. More details on the history of parallel computing
can be found in [KahanerReport96].

India’s first Top500 system came from CDAC, Bangalore, in June 2003 with the Param
Padma supercomputer. It featured in the Top500 list at rank number 169. It is the first
supercomputer to find place in the Top500 list which is indigenously designed, developed
and commissioned in India. It used a cluster of 62 4-way, IBM pSeries P630 nodes
interconnected through a high performance System Area Network, PARAMNet-II,
designed and developed by C-DAC and achieved a sustained performance of 532

Another supercomputing development at an academic environment was carried out at the
Institute of Mathematical Sciences, Chennai (IMSC). This linux cluster, named Kabru,
with 144 nodes (Intel Dual Xeon 2.4 GHz) interconnected by Dolphin 3D SCI network.
The cluster achieved a sustained performance of 959 GigaFLOPS and was ranked 264 in
the June 2004 Top500 list. This was the first Top500 system from an academic
institution in India. The system was built to perform large scale numerical simulations in
the area of Lattice Gauge Theory. Eka (the Sanskrit name for number one), developed by
Computational Research Laboratories (CRL), Pune, with technical assistance and
hardware provided by Hewlett-Packard, was the first to break into the top 10 ranking in
the Nov. 2007 Top500 list. This system has 1780 computing nodes (3560 Intel
QuadCore Xeon 3GHz) interconnected by a CLOS network with Infiniband
interconnect technologies. It achieved a sustained performance of 132.8 TeraFLOPS and
was ranked 4 in the Top500 list (Nov. 2007).

In the last 5 years a number of (10-15) high performance computing systems have been
developed/acquired with performance ranging from 4 TeraFLOPS – 20 TeraFLOPS at
various academic institutes (mostly clusters). More importantly there are a large number
of smaller clusters with performance ranging from 500 GigaFLOPS to 1 TeraFLOPS
which are used for developing high performance application in various disciplines in
science and engineering.

2.2 Computer Systems Research in India

India’s first effort in building a large digital computer started way back in 1954, when a
team of scientists at the Tata Institute of Fundamental Research, Bombay, embarked on
the design of TIFRAC (TIFR Automatic Calculator). It was fashioned on the general
principles enunciated in the classic von Neumann report. The basic components of this
machine were the arithmetic unit, the memory unit, the input-output unit, and of course,
the control unit. TIFRAC used an innovative carry bypass adder, textual and graphical
output on a visual display on a CRT screen, and 1024 words of the then relatively new
ferrite core memory. When completed in 1960, TIFRAC had 2700 vacuum tubes, 1700
germanium diodes and consumed a total power of 18 Kilowatt. [TIFR-AALec]. TIFRAC
also resulted in the first attempts at system programming in India.

In 1966, the country took a quantum leap as the second-generation computer, first
transistor-driven computer, was indigenously developed. The computer was the product
of a joint venture between the scientists of the Indian Statistical Institute, Calcutta, and
those of Jadavpur university and was fondly nicknamed `ISIJU'.

Academic research in mid 1980s focused on Fifth Generation Systems which resulted in
several research efforts on dataflow computing systems. Research work on Knowledge
Based Computer Systems (KBCS) project, funded by Department of Electronics with the
assistance of the United Nations Development Program (UNDP), focused on parallel
processing, including parallel processing machines, logic programming environments,
and the engineering of parallel AI workstations.

The national initiative to design, develop, and deliver a supercomputer and the founding
of the C-DAC in the late 1980’s, triggered a series of research efforts in building
supercomputing systems. Other research organizations such as C-DoT, BARC,
ANURAG, and NAL were involved in the design of parallel computer systems and
associated software. These efforts resulted in a series of parallel systems during the
1990s. C-DAC’s sustained efforts fructified as a series of PARAM systems (PARAM
10000, PARAM Padma, and more recently the PARAM Yuva). In addition to
development of these system, CDAC als developed the software stack for the PARAM
systems and developed the PARAMNet-II interconnection network, which is a significant
HPC system design and development work in India.

3. High Performance Centers and Facilities
The Computational and Research Lab (CRL), Pune, a wholly owned subsidiary of
Tata Sons Limited (a group holding company), has been specially set up to achieve
global leadership in the field of High Performance Computing (HPC). CRL’s interests
include computer system architecture, parallel system design, parallel systems software,
hardware design and interconnection networks, parallel scientific, engineering, digital
media, communication and business analytics software libraries and applications. CRL
hosts the fastest the HPC system in India, , called EKA, currently ranked at No. 13 (Nov.
2008 list) with a sustained performance of 132.8 TeraFLOPS and a peak performance of
172.6 TeraFLOPS. It uses infiniband interconnect technology and supports a parallel
cluster file system for data storage. It was the fastest system in Asia and 4th fastest
system in the November 2007 Top500 list. The system is extensively used for various
Computational Fluid Dynamic applications, graphics animation, and weather and climate
modeling applications. The HPC system research focus of CRL has been on the
development of Interconnect systems based on Infiniband hardware and the development
of FPGA based accelerators for various integer applications. CRL is also working on
system level network topology simulation tools for study of future large system behavior.

                            EKA Supercomputer in CRL, Pune

The Centre for Development of Advanced Computing (C-DAC), established in March
1988, as a Scientific Society of the Department of Information Technology (formerly,
Dept. of Electronics), Ministry of Communications and Information Technology
(formerly, Ministry of Information Technology), Government of India, is primarily an R
& D institution involved in the design, development and deployment of advanced
Information Technology (IT) based solutions. CDAC is engaged in the design,
development and deployment of High Performance Computing (HPC) Systems and
Over the past two decades, C-DAC has delivered a series of PARAM supercomputers
and has built national capabilities to make use of supercomputing technologies for human
enrichment. The National Param Supercomputer Facility at Pune and TeraScale
Supercomputer Facility at Bangalore are CDAC’s primary HPC centers. CDAC’s Param
Padma in 2003 was one of the first Indian systems to enter Top 500, a periodically
updated list that maintains the list of top supercomputers of the world. PARAM Yuva is a
latest addition to the PARAM series. This system with 288 nodes, sustained performance
of 37.80 TFs and peak performance of 54.01 TFs has ranked 68th in the coveted list of
Top500 supercomputers of the world and is the 2nd powerful system in India according to
TopSupercomputers-India. Para Yuva uses several indigenously designed system
components including ParamNet System Area Network, accelerator cards, application
and system software suite.

                              Param YUVA in CDAC, Pune

The C-DAC Bangalore, houses the Param Padma TeraFLOP cluster, the first Top500
system from India, which is a 248 CPU cluster with IBM Power 4 processors. As a part
of the Grid initiative, the centre augmented the Param Padma with a TeraFLOP of
computing power with an additional 168 cores of IBM Power 5 processors. More
recently the centre has added a 4 TeraFLOP (peak performance) 320-core Intel Xeon
based cluster to its computational facility. (RG: I have included some words about Param
Padma in the previous para. There is some redundancy here).
                            Param PADMA in CDAC, Bangalore

The Center for Modeling, Simulation and Design (CMSD), University of Hyderabad,
is an established approved by Union Grants Commission (UGC), and is supported by
Department of Science and Technology (DST) under its FIST (Fund for Improvement of
S&T Infrastructure in Higher Educational Institutions) programme. CMSD is a state-of-
the-art High Performance Computing (HPC) center and focuses on computer based
simulations of scientific applications and realizes the importance of scientific research,
based on modeling, simulation and design. The Center for Modeling, Simulation and
Design (CMSD), University of Hyderabad, has a 2.0 TeraFlop consisting of 6 SMP
systems and 4 Tera Bytes of storage, and a CDAC Param cluster with 16 nodes.

Indian National Centre for Ocean Information Services (INCOIS) is an autonomous
organization under Ministry of Earth Sciences (MoES), Govt. of India. The mission of
INCOIS is to provide information and advisory services to society, industry, government
agencies and scientific community through sustained ocean observations and constant
improvements through systematic and focussed research. INCOIS is involved in various
activities including implementing potential fishing zone advisories, developing and
maintaining Argo profiling floats for measurement of temperature and salinity through
the upper 2000 metres of the open Indian Ocean in real time, and R&D in area of ocean-
atmosphere modeling focusing on the ocean predictability and climate variability. It is
also the organization responsible for providing early warning for tsunami and storm
surges. INCOIS was designated as the National Oceanographic Data Centre under the
International Oceanographic Data Exchange programme of the Intergovernmental
Oceanographic Commission of UNESCO. Indian National Centre for Ocean Information
Services (INCOIS) possesses a 80 processor SGI cluster and a 100 TeraByte storage
system to meet its specific requirements of running coastal, regional and global/regional
ocean models in operational, data assimilation and other future computational
requirements for handling large amounts of data.
The Tata Institute of Fundamental Research (TIFR), is an autonomous Institute
established in 1945 under the umbrella of the Department of Atomic Energy of the
Government of India. TIFR does basic research in physics, chemistry, biology,
mathematics and computer science. TIFR has campuses in Mumbai (Bombay), Pune and
Bangalore and research facilities in various other places in India. TIFR was one of the
few academic institutions to host a computer system in the early 1960s. TIFR is also
known for its pioneering work in developing the first indigenous computing system
using vacuum tubes, called TIFRAC. The computer centre at TIFR, Bombay, presently
hosts a CRAY X1 (acquired in 2004) and a Blue Gene P (acquired later in 2008). The
Cray has 16 vector processors, each 12.8 GFLOPS (total of nearly 200 GigaFLOPS).
The BlueGene/P has 1024 processors (4096 cores) which achieved a sustained
performance of 11.32 TeraFLOPS on HP Linpack code. Major HPC applications include
Quantum chromodynamics, quantum field theory, condensed matter and statistical

Supercomputing Facility for Bioinformatics & Computational Biology (SCFBIO) is
a HPC center at Indian Institute of Technology (IIT) Delhi. Its mission is to develop
highly efficient algorithms for genome analysis, protein structure prediction and active
site directed drug design. The facility is committed towards providing bioinformatics and
computational biology tools and software freely accessible to bioinformatics community.
SCFBIO possesses a 104 core AMD Opteron cluster and a 70 core Sun Fire Ultra Sparc
III cluster with a 4 TeraByte SAN storage system and a combined performance of 0.7
TeraFlops. It will soon install a 600 core AMD Opteron cluster with 12 TeraBytes of
Direct Attached Storage (DAS) for storing the data, and a combined performance of 4

Jawaharlal Nehru Centre for Advanced Scientific Research (JNCASR) is located in
the north of Bangalore. The Centre for Computational Materials Science (CCMS) is a
unit of JNCASR. CCMS, JNCASR, has a 512 core HP Prolian cluster with 32 TeraByte
aggregate storage capacity. The cluster is ranked 8th in the latest list of Top

The National Centre for Biological Sciences (NCBS), Bangalore, is part of the Tata
Institute of Fundamental Research and is located in Bangalore. The mandate of NCBS is
basic research in the frontier areas of biology. The research interests of the faculty are in
four broad areas ranging from the study of single molecules to systems biology. NCBS
possesses 3 AMD Opteron based clusters with a total of 420 cores.

Supercomputer Education and Research Centre (SERC) is a department in Indian
Institute of Science, Bangalore. It is conceived of as a functionally distributed
supercomputing environment, housing leading-edge computing systems, with
sophisticated software packages, and connected by a powerful high-speed fibre-optic
network. The center possesses multiple high performance clusters including a 8192-core
IBM BlueGene/L cluster, 256-core IBM Power5 based Linux cluster, three 32-core IBM
Power4+ processor Regatta systems, a 32-core Intel Itanium2 SGI Altix 3700 system,
two 16-core Intel Itanium2 SGI Altix 350 system, a 16-core IBM RS/6000 SP3, and a six
4-core COMPAQ AlphaServer ES40 with a total of 8648 cores and a combined capacity
of about 25 TFlops. Its IBM BleGene/L system is the 3rd powerful system in India
according to Top Supercomputers-India and is ranked 213th in the latest Top500 list.

                         IBM BlueGene/L in SERC, IISc, Bangalore

[RG: You may want to move SERC up the order]

While we have covered a few major computational facilities in India, the list is by no
means comprehensive. We observe that both the number of the HPC facilities and the
performance capabilities has been on the increase in the last 5 years. The advent of
multicore architectures has enabled quite a few HPC systems in India which have more
than 128 cores and a peak performance in excess of 1 TeraFLOP. We anticipate that the
number of such systems at present to be at least 50 or more. These are predominantly
used for various high performance applications. Not surprisingly, only a few (less than 5)
of these systems are used for HPC system research.

4. Garuda – Indian Grid Initiative
The National Grid Computing Initiative GARUDA is a collaboration of science
researchers and experimenters on a nation wide grid of computational nodes, mass
storage and scientific instruments that aims to provide the technological advances
required to enable data and compute intensive science for the 21st century. A
computational grid is an aggregation of heterogeneous and geographically distributed
resources such as computing, storage and special equipments. The aim of the grid is to
dynamically share resources to facilitate optimal utilization. Building a commanding
position in Grid computing is crucial for India. Allowing anyone, anywhere, anytime to
access supercomputer level processing power and knowledge resources, grids will
underpin progress in Indian science, engineering and business.

The Department of Information Technology (DIT), Government of India has funded the
Centre for Development of Advanced Computing (C-DAC) to deploy the nation-wide
computational grid GARUDA. To achieve its objective, GARUDA brings together a
critical mass of well-established researchers, from 45† research laboratories and
academic institutions of India, who have constructed an ambitious program of activities.

In Proof of Concept (PoC) phase of GARUDA, 45 R&D and academic institutes from 17
cities across the country were connected with an aim to bring “Grid” networked
computing to research labs and industry. From April 2008 the Foundation Phase of
GARUDA is in progress with an aim to include more users’ applications, providing
Service Oriented architecture, improving network stability and upgrading grid resources.

                    GARUDA – The Indian National Grid Computing Imitative

Currently GARUDA has aggregated HPC resources from CDAC centers and Partner
institutes totally to over 1500 CPUs and 60TB of storage. GARUDA has also laid the
foundation for the next generation grids by addressing long term research issues in the
strategic areas of knowledge and data management, programming models, architectures,
grid management and monitoring, problem solving environments, tools and grid services.
To ensure progressive evolution and durable integration, GARUDA is managed by C-
DAC with the Grid monitoring and Management Centre at Bangalore. The GARUDA
network is a Layer 2/3 MPLS Virtual Private Network (VPN) connecting select
institutions at 10/100 Mbps with stringent quality and Service Level Agreements. This
Grid is a pre-cursor to the next generation nationwide Gigabit Network with high
performance computing resources and scientific instruments for seamless collaborative
research and experiments.

The key deliverable of this project include grid tools and services to provide an integrated
infrastructure to applications and higher-level layers, a pan-Indian communication fabric
to provide seamless and high-speed access to resources, aggregation of resources
including compute clusters, storage and scientific instruments, creation of a consortium to
collaborate on grid computing and contribute towards the aggregation of resources and
grid enablement and deployment of select applications of national importance requiring
aggregation of distributed resources

5. HPC Systems Research
Procurement of HPC systems has to be augmented with strong systems research for
sustenance, maintenance and enhancement of HPC capabilities, development of
indigenous solutions, and attainment of cost-effectiveness. Realizing this, various Indian
HPC centers and academic institutions conduct HPC system-oriented research to support
high performance of applications. The areas of research include hardware systems,
namely, development of high performance network stack or software, co-processors,
switches, reconfigurable systems, multithreaded and distributed shared memory
architectures, and programming models for general purpose architectures, software
solutions, namely, message passing libraries, debugging environments and monitoring
tools, and grid middleware, namely, and grid middleware, namely scheduling and fault-

5.1 HPC Hardware Systems Research
Centre for Development of Advanced Computing (CDAC) has built different system
software products including network stack, monitoring and debugging frameworks and
reconfigurable computing systems (RCS). CDAC has developed its own system area
network called ParamNet. The latest version, ParamNet-3 can provide up to 10 Gbps
bandwidth for high performance communications. PARAMNet-3 has tightly integrated
hardware and software components. The hardware components consist of Network
Interface Cards (NIC) based on Communication Co-Processor “GEMINI”, and modular
48-port Packet Routing Switch. The software component “KSHIPRA” is a lightweight
protocol stack designed to exploit capabilities of hardware and to provide industry
standard interfaces to the applications. KSHIPRA communication substrate designed to
support low latency and high bandwidth is the key to the high level of aggregate system
performance and scalability of C-DAC HPCC software.
C-DAC has also pioneered the RCS technology for HPC in the country, through its state-
of-the-art design of hardware, system software and hardware libraries called Avatars.
These Avatars are dynamically changeable circuits, corresponding to the compute
intensive routines of the application codes. A great advantage that C-DAC’s RCS offers
compared to other technologies for application acceleration is in terms of tremendous
saving on power and space at the same time increasing application performance by many

Research at academia in India has been focusing on issues relating to architecture design,
performance evaluation of HPC systems. Here we report the research activities of two
groups at the Indian Institute of Science (IISc), Bangalore. The High Performance
Computing group at the Supercomputer Education and Research Centre (SERC) worked
on the design of simultaneous multithreaded architectures and software distributed shared
memory architectures on distributed memory machines. More recently their interest are
on the design of high performance systems using off-the-shelf general purpose
accelerators such as the Graphics Processing Units (GPUs), the Cell Broadband Engine,
RapidMind, ClearSpeed, Imagine, etc. These accelerators support tens to 100s of cores
in a processor and a complex memory hierarchy largely optimized for higher bandwidth.
However, these architectures limited support for general purpose programming models.
A major challenge in using accelerator architectures is programming and exploiting
performance from them. The non-existence of standard toolchains and frameworks also
makes programming on these processors makes this problem extremely difficult and
challenging. The HPC group at SERC has been working on developing programming
models and compiling techniques for HPC systems with accelerators.

The Computer Aided Design Laboratory (CAD Lab) at SERC has been working on
reconfigurable architectures and accelerator architecture designs based on reconfigurable
architectures. The supercomputing platform envisaged with such a design would be
“future proof” both in terms of hardware and software, scalable, and runtime
reconfigurable. The goal of this exercise is to build a prototype general purpose
supercomputing platform that can manifest as application specific hardware accelerators
for every application running on it. The current focus is on SoC Design Methodologies,
Micro-architectural optimizations for Cache hierarchy in Chip Multiprocessors (CMPs)
and Multiprocessor SoCs (MP-SoCs). Research is also underway on supporting multiple
computing paradigms (simultaneously) on MP-SoC platforms and Dynamically
Reconfigurable SoC Architectures.

5.2 HPC System Software Research
C-DAC High Performance Computing and Communication (HPCC) software effectively
addresses the performance and usability challenges of clusters through a high
performance flexible software environment. The HPCC software suite of products
includes high performance compilers, parallel debuggers, data visualizers and
performance profilers. High performance communication protocols and efficient MPI
implementation provide the correct framework to extract maximum performance. In this
direction, CDAC has also developed its own implementation of Message Passing
Interface (MPI) standard, C-MPI, for parallel computations on Cluster of Multi
Processors (CLUMPS). CDAC’s DiViA is a complete debugging environment for
enterprises that need to develop parallel application using message-passing and provides
portability, execution visualization and performance analysis. The PARMON tool of
CDAC is a comprehensive cluster monitoring system and is designed using state of the
art Java technology and offers portable, flexible, and comprehensive environment for
monitoring of large clusters. It follows client-server methodology and provides
transparent access to all nodes to be monitored from a monitoring machine.
MetricAdvisor is a software engineering toolset for developing high quality programs.


5.3 Grid Middleware

The National Grid projet, Garuda, has developed grid middleware components for
seamless solution of large-scale applications on the distributed high performance

                                   Garuda Components

The Service Oriented Architecture (SOA) is a component based model for building an
application architecture within which all functions are defined as independent number of
discrete services with well defined invokable interfaces, that can be called in a sequence
to form a larger application. Grid middleware is now moving towards Web Standards
based on OGSI (Open Grid Standards Architecture) and WSRF (Web Services Resources
Framework). A service oriented architecture is enabled on GARUDA using the Globus
Toolkit 4.0.3 as the grid middleware.

The GARUDA Portal is the single point entry to Garuda resources and implemented
services. The Garuda Grid Portal provides a web-based environment for users to submit
and track jobs in the GARUDA. The GARUDA portal which provides the user interface
to the Grid resources hides the complexity of the Grid from the users. It allows
submission of both sequential and parallel jobs and also provides job accounting
facilities. Gridway open source scheduler for grid has been customized and deployed in
selected resources of GARUDA and configured in a peer-to-peer mode to support high
availability of compute resources. A QoS framework supporting resource reservation has
been enabled. Program Development Environment including a Grid Integrated
Development Environment and a Grid Program Debugger have been developed. Data
grid solution using Storage Resource Manager (SRM), iRODS and Grid File System

                               GARUDA Job Submission Portal

To enable data oriented applications, GARUDA provides an integrated but distributed
data storage architecture by deploying the Storage Resource Broker (SRB) from Nirvana.
SRB creates and maintains a Global Namespace across multiple heterogeneous and
distributed storage systems in the Grid. The SRB provides advanced services including
transparent data load and retrieval, data replication, persistent migration, data backup and
restore, and secure queries. GARUDA has adopted a pragmatic approach for using
existing Grid infrastructure and Web Services technologies. The resource management
and scheduling in GARUDA is based on a deployment of industry grade schedulers in a
hierarchical architecture. At the cluster level, scheduling is achieved through Load
Leveler for AIX platforms and Torque for Linux clusters. At the Grid level, the Moab
scheduler from Cluster Resources interfaces with the various cluster level schedulers to
transparently map user requests onto available resources in the Grid.

The Indian Grid Certification Authority (IGCA) accredited by the APGridPMA has been
setup in C-DAC Bangalore. The IGCA will help scientists, users and collaborative
community in India and neighboring countries, to obtain an internationally recognized
digital certificate to interoperate with state-of-the-art grids worldwide. The IGCA was
inaugurated on 14 Jan 09, by Dr. R Chidambaram PSA to Govt. of India and presided by
Secretary DIT. This is the First CA in India for the purpose of Grid research. IGCA is
fully operational and issuing digital user and host Certificates to support the secure
environment in Grid.

GARUDA grid is monitored using an in-house developed comprehensive Grid
Monitoring Tool called Paryavekshanam. The tool monitors all the compute-storage-
instrument resources, network and grid middleware & software. The Alert and
Notification System of escalates the errors and warning to concerned administrators for
speedy rectification. A dedicated Grid monitoring and management centre at C-DAC,
Bangalore, with state-of-the-art display walls helps in managing and monitoring all the
components in the Grid.

                Display wall at GMMC                              Paryavekshanam Tool

The Grid Applications Research Lab (GARL) in SERC, IISc, conducts grid
middleware research for high performance computing. It has developed optimized MPI
collective communication algorithm for one-many and many-many communications for
grids. It has built performance modeling strategies for automatically characterizing the
behavior of parallel application executions on Grids. It has also developed scheduling
algorithms and rescheduling strategies for parallel applications on grid systems. The work
on rescheduling strategies is the first work on handling both application and resource
dynamics for parallel applications on grid resources. GARL also conducts research on
and has developed software for checkpointing, fault-tolerance and migration of parallel
applications on heterogeneous and dynamic grid resources. With these robust grid
middleware techniques, GARL aims for large-scale deployment and widespread use of
parallel applications on computational grids by the scientific community. GARL has
developed a grid middleware framework called MerITA (Middleware for Performance
Improvement of Tightly-Coupled Parallel Applications on Grids) encompassing the
performance modeling, scheduling and rescheduling techniques for efficient execution of
long-running scientific applications on multi-cluster grids consisting of dedicated or non-
dedicated, interactive or batch systems.

The Department of Computer Science and Enginnering, IIT Kanpur, conducts research
and has developed solutions for checkpointing parallel applications on grid resources,
automatic program parallelization tools, workflow development framework and
scheduling methodologies for composing applications on grids, modeling and verifying
complicated workflows, frameworks for fault-tolerance and hierarchical dynamical
scheduling, performance modeling and energy-aware scheduling for grids.
[RG: Janakiram’s page is not opening up.]

6. HPC Applications
India’s HPC procurements are predomiantly geared towards supporting the high
performance application needs of the country. The different HPC centers of India
specialize in various cutting-edge scientific applications. Many applications are
developed in-house by application groups to enhance the scientific potential in their
respective domains. Some applications are built from mostly off-the-shelf open-source
components for various scientific domain to help the domain scientists perform studies
related to the Indian context. In this section, we describe the activities related to three
primary areas, namely, climate modeling and weather forecasting, bioinformatics, and
computational chemistry.

6.1 Climate Modeling and Weather Forecasting
Climate research in India is focussed towards understanding and forecasting the Indian
summer monsoons. A widely accepted view is that simulation and forecast of the Indian
summer monsoon is one biggest challenges facing atmospheric/climate modellers
worldwide. This research is being conducted by Indian researchers using state-of-the-art
models for the climate system. They also use global and regional atmospheric models for
forecasting and research. Oceanographers in India have used ocean models and climate
system models to study the circulation processes in the Indian Ocean. Indian researchers
were one of the earliest in the world to use parallel processing techniques for climate
modelling and have always been innovative in the use of upcoming technologies to study
monsoon and climate.

Forecasting of Weather/climate is done by India Meteorological Department (IMD) and
National Centre for Medium Range Weather Forecasting, Noida (NCWRF). Ocean state
forecasting is being conducted at Indian National Centre for Ocean Information Systems,
Hyderabad (INCOIS). Major centres for climate research are CDAC, CSIR Centre for
Mathematical Modelling and Computer Simulation (CMMACS), Indian Institue of
Tropical Meteorology, Pune (IITM), Indian Institute of Technology, Delhi (IIT-D),
Indian Institute of Science, Bangalore, National Aerosopace Laboratories, Bangalore
(NAL) and Space Application Centre, Ahmedabad (SAC).

CDAC has successfully implemented many atmospheric and oceanographic application
on its Param series of supercomputers. The models implemented include WRF, the
NCMRWF model and the Community Climate System Model (CCSM). Work is also
underway to develop a regional climate model using Regional Ocean Modelling System
(ROMS) and Regional Climate Model (RegCM). An end-to-end forecasting product
(based primarily onWRF) called `Anuman', has been developed at CDAC-Pune. Ocean
model of Insitute of Numerical Mathematics of Russia has been succesfully implemented
on their computing platforms. Studies on parallel implementation and scalability have
also been conducted here. Application of grid to climate studies is also being peformed
on the Garuda Grid. CDAC was a major player in the country's first atmospheric
modelling intercomparison project called SPIM (Seasonal Prediction of Indian
Monsoon). This involved successful implementation and seasonal scale ensemble
simulations of five AGCMs belonging to various organizations. This exercise was carried
out by CDAC on its Param-Padma supercomputer. Community Climate System Model
(CCSM) of NCAR, Climate Forecast System (CFS) and Seasonal Forecast Model (SFM)
and UKMO atmosphere/ocean model are some of the prominent applications available on
PARAM Padma computing systems.

INCOIS is involved in the mathematical modeling of the climate system (individually /
coupled) to obtain better understanding over the less observed regions, since the present
network of observations in the ocean is not sufficient to understand the ocean dynamics
and thermodynamics completely. Some of the objectives of the INCOIS’s HPC system
include ocean modeling for providing description of past, present and future state of
ocean at appropriate spatial and temporal resolutions, assimilation of in-situ and remotely
sensed data (Argo Profiling Floats, Moored Buoys, Ship Observations, Satellite
Measurements, etc) with the help of suitable ocean general circulation models, providing
boundary forcing for atmospheric models by performing forecast runs of ocean models,
understanding variability of Ocean and Marine environment, simulation experiment to
optimize the observation system and serving as a National Computational Facility for
ocean models. INCOIS uses various high performance ocean models including MOM,
ROMS, CUPOM and HYCOM, WAM, WAVEWATCH-III and Tsunami N2 model.

NCMRWF (www.ncmrwf.gov.in) is entrusted with the task of generating medium range
forecasts (3-15 days).. Global weather models at a resolution of about 50 km are being
used here. Regional weather models with even higher resolutions are also being used for
shorter range forecasts upto 3 days.

IMD (www.imd.gov.in) Delhi generates short-range forecasts upto 3 days while IMD
Pune generates seasonal forecasts upto 90 days using numerical models.

SAC (www.sac.isro.gov.in) : The emphasis at SAC is on the use of satellite data for
weather/climate studies. Both regional and global models are used for study on the
impact of ingesting satellite products on weather forecasting and climate studies.
Seasonal forecasts are also generated here in collaboration with NCMRWF.

IITM (www.tropmet.res.in) conducts research on tropical meteorology (and
oceanography) with emphasis on Indian summer monsoons. Studies are also being
conducted on the likely impact of anthropogenic climate change on the Indian Summer
monsoon. Regional weather models, global atmospheric models (AGCMs) and coupled
ocean atmosphere models are being used at this centre.

CMMACS (www.cmmacs.ernet.in): research is conducted here both in meteorology and
oceanography. Meteorological studies includes research on seasonal forecasting of
monsoons and likely impact of anthropogenic aerosols on the strength of the monsoons.
Oceanographical studies involve studies related to circulation in the Indian Ocean, bio-
geo-chemical processes in the ocean and data assimilation.

At the Indian Institute of Science (IISc), researchers in the Centre for Atmospheric and
Oceanic Scinces, researchers work on ocean, atmospheric and coupled systems. The
emphasis is on understanding mechanisms that govern various processes that affect the
Indian summer monsoon. AGCMs, OGCMs and coupled climate system models and
regional weather models are used for these studies. Issues such as the impact of
anthropogenic (black carbon) aerosols on the Indian Summer monsoon, impact of
atmosphere on oceanic parameters over the Indian region, study of the Indian Ocean
Dipole, impact of African and Himalayan orography on monsoons etc have been carried
out using these models.

India's first parallel implementation of a climate model was conducted by IISc in
collaboration with NAL and the first major climate system (coupled model) simulation
conducted in association with CDAC. Studies have successfully conducted to increase
scalability of atmospheric models using message-compression techniques. A new load
balancing algorithm has been been developed and successfully implemented in a climate
system models (one of the few such exercises conducted worldwide on a coupled climate
system model). Currently work is underway to grid-enable a climate system model.
This involves not only modifying the cimate system model but also development of
related middleware to facilitate concurrent execution of subsystems at multiple sites (.

NAL (www.nal.res.gov.in) has been developing its own parallel computing system,
Flosolver, for over two decades. Both hardware and related software are developed here.
The hardware and software design has been done keeping meteorological applications
such as the spectral global prediction models in mind. The novelty of the communication
switch developed here (called the Floswitch) is that global operations (such as global
sums, max, min etc) are conducted on the switch itself and the resultant information is
sent to the computing elements. This significantly reduces bandwidth demands.
Atmospheric models have been re-engineered and re-coded here. Starting with NCEP
global weather forecast model, a new model called VARSHA, has been developed. The
latest version under development uses object-oriented programming concepts. Research
mode forecasts are being generated on medium range and seasonal scales using these

IIT-Delhi (www.iitd.ac.in) : AGCMs and regional weather models are used at this centre
for study of monsoons and intense weather systems over the Indian region. A global
spectral model has been successfully implemented based on longitudnal decomposition
instead of the conventional latitudinal decomposition. Scalability studies for such a
decomposition have also been conducted. Using these models issues such as impact of
Eurasian snowcover on the Indian monsoon have been addressed. Ocean models are also
being used to study coastal and large-scale circulation of the Indian Ocean.

Future Directions

 The emphasis for Indian climate researchers will continue to be on monsoons. Even
today, most climate system models have major problems in simulating the Indian summer
monsoon. The looming worry of anthropogenic climate change, specifically its impact
on the Indian summer monsoon is a matter of added concern. These factors imply that
models need to be improved further. Researchers would be interested in including more
sophisticated implementation of various processes such as the sub-grid scale cumulus
parameterization and boundary layer parameterization, improved numerical techniques,
incorporation of additional tracers in the atmosphere (such as aerosols) and ofcourse
higher spatial resolution. These would increase demands on computational resources.
HPC researchers (specifically in the field of climate studies) would need to look at
innovative techniques such as grid computing (modest beginnings have already been
made) and use of GPUs and cell-processors. Grid-computing could help agglomerate
resources that may not be available at a single site. GPUs and cell-processors could be
effectively used by applying multi-threading techniques to hive off computationally
intensive sub-components to these processors. The GCMs software being developed
using Object-Oriented techniques would ease the job of researchers in implementing new
ideas in their models.

More automatic weather stations are likely to be installed all over the country. New
satellites such as the Indo-French Megha-Tropique are to be launched to study monsoons.
A network of Doppler radars is being setup to monitor intense weather events. An array
of deep-sea sensors are in the process of being deployed for tsunami and ocean-state
forecasting. All these would need to integrated into an instrument grid which in turn
would need to be interfaced to a datagrid for effective utilization of these new resources.

6.2 Computational Chemistry
Research problems pursued in the computational chemistry/physics area can be broadly
classified into two; (i) Study of electronic structure,and (ii) Dynamics of complex
systems. In the former, traditionally the focus has been on crystalline solids and of late on
nanostructured materials, primarily by practitioners of computational physics using
approaches such as tight binding theory, muffin tin orbitals, many body calculations and
so forth. Another independent stream of workers interested in electronic structure are
quantum chemists, whose primary interest used to be small molecules studied using semi-
empirical, Hartree-Fock (HF) or post HF methods. The advent of density functional
theory (DFT) with extremely robust exchange-correlation functionals has energised these
two distinct communities and have led them to investigate non-traditional atomic and
molecular architectures in the areas of nanomaterials, supramolecular systems and so

The second direction of research in computational chemistry/physics is the study of
complex systems. Until a decade ago, many research groups used to study the structure
and dynamics of simple liquids, both atomic as well as molecular ones, using primarily
the technique of molecular dynamics (MD) or alternately using Monte Carlo (MC). These
methods employ an empirical interatomic potential of interaction and the time evolution
of the system was studied. In the recent past, this category of researchers have been
increasingly focussing on phase behavior, mesoscale structure (morphology) and slow
dynamics in complex systems. This development has been made possible due to many
factors such as (i) faster algorithms (such as FFTs for calculating Coulomb interactions),
(ii) better single processor performance, (iii) easier availability of MPI based public
domain codes, and (iv) commoditization of HPC hardware and support software (such as
compilers, libraries etc..).

Some of the activities in these research areas are presented in brief below.

State-of-the-art multireferenced Coupled-Cluster calculation of accurate electronic
structure of atomic systems are important for spectroscopic studies, lifetimes of energy
levels, transition probabilities, polarizabilities and to probe particle and nuclear physics
by combining with experimental measurements. Coupled cluster at Singles and Doubles
(CCSD) level for both relativistic and non-relativistic methods have been ported and
executed on PARAM Yuva cluster. Extensive ab initio calculations on nanomaterials
using density functional pseudopotential plane wave method on a variety of Al-Au
clusters have shown exceptional stability of 58 valence electron (leaving the 5d electrons
of Au) clusters such as an empty cage doubly charged icosahedral fullerene Al12Au20 and
an endohedral Al12Au21 - fullerene anion in which a gold atom lies at an off-center
position inside the Al12Au20 cage. Formation of such caged structures by metal atoms is
very unusual and novel. Further detailed studies on different endohedral dopings as well
as other caged and small clusters of Al-Au and Al-transition metals have been carried
out. Stabilities of Al-Au clusters having 18, 20, and 40 valence electrons corresponding
to magic behavior in a jellium model, have been explored. Studies on many other systems
in bulk and nanoforms are also being conducted.

CMSD, Hyderabad, conducts research in various aspects of electronic structures
including physics of low dimensional systems, topological defects in restricted
geometries, critical phenomena, quantum chemistry, molecular modeling and design of
new materials.

The cluster at CCMS, JNCASR, is exclusively devoted to carrying out computations
related to materials science. Typical application software include CPMD, LAMMPS,
Quantum Espresso, SIESTA, VASP, and ADF and many home brew codes. Broadly they
fall into two categories: those based on quantum mechanical approaches and those which
are not – an example of the latter being LAMMPS. All codes have been found to scale
rather well, provided the problem size is large enough. CPMD, SIESTA, and Espresso
are based on density functional theory and are used to study the details of electronic and
atomic structure and dynamics of various advanced materials, including metal-DNA
complexes, multiferroics, surfaces, nanocatalysts, ionic liquids etc. LAMMPS is used to
model supercritical carbon dioxide, room temperature ionic liquids and various glass
forming liquids using empirical potential molecular dynamics. VASP is used to study
many magnetic and other materials possessing interesting electronic structure, while ADF
which is based on quantum chemical approaches is used to study molecular complexes.
CPMD and LAMMPS have exhibited linear scaling for large problems. An important
component of the activity at JNCASR is computational materials design in which novel
atomic and molecular architectures are crafted and their properties studied using the
techniques described above. It is then a challenge to experimental researchers to come up
with strategies of synthesizing such compounds in the laboratory.
At the Indian Institute of Technology, Kanpur, the HPC facilities are extensively used to
carry out computational studies on chemical dynamics in condensed phases, surface
adsorption and related kinetic processes, chemical reaction dynamics in gas phase,
structure and energetics of biologically relevant molecules and many other key problems
in chemical sciences and related areas. While Gaussian03 has been used for high level
quantum chemical calculations of molecules and clusters at zero temperature, CPMD,
VASP and Quantum Espresso are used for finite temperature simulations clusters and
condensed phases using full many-body quantum potentials. These later packages are
also used for many of the static pseudopotential based quantum calculations of
chemically interesting systems which appeared to be too large for explicit all-electron
calculations. For very large systems, such as interfaces, solutions of complex solutes etc,
AMBER, DLPOLY or GROMOS are used for empirical potential based dynamical
simulations. An interface of GROMOS-CPMD is also used combined classical-quantum
simulations of many of the chemical systems of interest such as proton transfer through
narrow hydrophobic pores. Up to 24 processors for a single job for quantum were used
and combined classical-quantum simulations and the scalability has been found to be
very good in this domain. In addition to the above software, homemade application
codes are also heavily used for various computational studies.

Future Trends

As alluded to in the Introduction, in the area of electronic structure theory and
computation, synergistic interactions between traditional computational physics and
chemistry groups are taking place. This could eventually lead to new methods being
developed. Another significant development has been the ability to carry out ab initio MD
simulations of many chemical systems. One can expect computational studies of
enzymatic reactions using realistic QM/MM approaches in the near future.

Although a significant amount of work has been accomplished by Indian researchers in
the area of computational chemistry (both in terms of development of methods and in
applications), contributions towards the development of algorithms and large codes have
been lacking. One way to tackle this lacuna is probably to co-opt with the famed Indian
software industry. Efforts in this direction are paramount if Indian computational
chemistry research has to make a global impact, in a qualitative manner.

On the whole, one can expect major contributions in computational chemistry research
from India in the near future.
    FIGURE 1: Large scale coarse grained MD simulations have led to the accurate
 characterization of the phase of a room temperature ionic liquid to be lamellar in nature
                     (S. Balasubramanian et al, JNCASR, Bangalore)

  FIGURE 2: Accurate quantum chemical calculations have been able to explain the
variation of Raman signals from two-dimensional graphene in the presence of donor and
               acceptor molecules (S.K. Pati et al, JNCASR, Bangalore)

6.3 Bioinformatics, BioScience and Genomics
Computational-science studies carried out in the biological sciences area at the Indian
Institute of Science include molecular-dynamics (MD) and electronic-structure
calculations of proteins and nucleic acids, systems-biology investigations such as
simulations of virtual cells, drug discovery and rational-vaccine design via virtual
screening- ligand docking, ab-initio ligand design, and epitope design and detection in
pathogenic genomes, the elucidation of the relation between sequence, structure, and
function has been carried out by genome-sequence analysis, structure prediction, protein-
protein interaction, phylogenetic profiling, biological image analysis, feature detection.
Some specific examples of the studies carried out are given below: the dynamics of the
top three sidechain clusters of Met-tRNA Synthetase, which has 550 amino-acid residues,
was simulated for 10 ns; MD simulations have been carried out for a variety of DNA
sequences (about 32 base pairs) to study their intrinsic curvature in solution; hydrogen
bonds between two amino acids have also been studied by such simulations: virtual
screening, for drug discovery and design has been carried out for flexible ligand with
sizes from10-50 atoms (flexible) and (rigid) proteins with sizes from 2000-5000 atoms
(1 docking run requires roughly 106 energy evaluations, and so far only a modest ligand
database of about 100 compounds has been used with rigid, not flexible, proteins);
similarly rational vaccine design has been carried out by using a T-cell epitope prediction
algorithm; phylogenetic profiling and molecular taxonomy has been done for 400
bacterial genomes; a computer-intensive approach has been developed for the sensitive
detection of extremely remote protein homologues; databases on protein structures,
distant homology detection, domain architectures, multiple profiles of protein families
and protein kinases are updated regularly.

CDAC’c Param supercomputers also perform Multiple Expectation maximization for
Motif Elicitation (MEME). MEME is an important application code for researchers
working in the area of comparative and functional genomics, especially, gene regulation
and microarrays. Study of Motif has implications in detection of transcription factor
binding sites, gene-regulatory network, drug-target prioritization and drug-design.

The National Center for Biological Sciences (NCBS) performs experiments in the
applications of computational neuroscience and systems biology in its HPC clusters. It
also conducts research in homotopy continuation searches for roots of systems of
nonlinear equations, and in data mining and protein structure analysis related to

The Center for Modeling, Simulation and Design (CMSD), University of Hyderabad,
uses it HPC clusters to conduct research in various bioinformatics applications including
biosequencing analysis, structural and functional genomics to map protein coding
regimes and sequencing of DNA, protein folding, computational design of specific drug
molecules based on the drug-receptor interactions.

SCFBIO has developed a genome analysis software suite called Chem Genome, a protein
structure prediction software called Bhageerath, and an In-Silico drug design software
called Sanjeevani. Chemgenome is based on the hypothesis that both the structure of the
DNA and its interactions with regulatory proteins and polymerases decide the function of
a DNA sequence. The ChemGenome could distinguish genes from non-genes in more
than 900 bacterial genomes with > 90% accuracy. Bhageerath is an energy based
computer software suite for narrowing down the search space of tertiary structures of
small globular proteins. The protocol comprises eight different computational modules
that form an automated pipeline. It combines physics based potentials with biophysical
filters to arrive at 5 plausible candidate structures starting from sequence. The
methodology has been validated here on 50 small globular proteins consisting of 2–3
helices and strands with known tertiary structures. Bhageerath, the protein structure
prediction software developed in-house at IIT Delhi could run on a 1024 processor IBM
Blue Gene. A three helical system which took around 5-6 hours on a 32 processor cluster
took just 726 second (~12 minutes) on a 1024 processor Blue Gene machine. SCFBIO
has also developed an atomic level binding free energy based methodology for active site
directed lead molecule design (Sanjeevini). The various modules of this suite are
designed to ensure reliability and generality. The software is currently being optimized
on Linux and Sun clusters for faster and better results. The Sanjeevini protocols could
sort drugs from non-drugs for a few drug targets helping in addressing both affinity and
specificity issues in drug design. The web-enabled versions of the software suite
developed by SCFBIO are available for free usage at www.scfbio-iitd.res.in. Besides
these in-house packages SCFBIO also uses free available high performance software
including Amber, Gaussian, Gromacs, Gamess.

                  Study of Relationships between Bistable Chemical Systems
                                      NCBS, Bangalore

6.4 Grid Applications

The Indian National Grid Middleware project, Garuda, has developed Problem Solving
Environment (PSE) in the domains of Bio-informatics, and Community Atmospheric
Model to support the entire cycle of problem solving for the specific domains by
supporting problem formulation, algorithm selection, numerical simulation and solution
visualization. Applications of national importance that require aggregation of
geographically distributed resources have been developed and deployed on the GARUDA
Grid. A Disaster Management application for Flood Assessment, a Bioinformatics
application on Molecular Docking using OpenEye, E-Learning, Computer Aided
Engineering, and applications in Atomic Physics have benefited from grid enablement in
terms of improved performance and increased collaboration. C-DAC in association with a
partner research institution has mined data from a network of sensors deployed over vast
disaster prone regions and upload it to GARUDA as input to forecast models appropriate
for various stages of Natural Disaster Management. This will enable timely dissemination
of disaster information to user agencies for effective mitigation and relief measures. C-
DAC’s Bio-informatics Resources & Applications Facility on their PARAM
Supercomputing facility, is accessible for the Bioinformatics community involved in
insilico molecule identification and new drug discovery. The enormity of data and
complexity of algorithms in bio-informatics demand tremendous computational cycles
necessitating effective use of grid resources beyond those available at any single location.

               Disaster Management                                  Molecular Docking

The Grid Applications Research Lab (GARL), SERC, IISc, develops solutions for the
effective use of grids to solve large-scale scientific problems. GARL’s current research is
on developing grid-based solutions for two applications, namely, climate modeling and
phylogenetic predictions in bioinformatics. The research on climate modeling deals with
various challenges associated with executing components of a coupled climate model on
different grid sites and shows the benefits of grids for solving applications related to
climate modeling. The research has developed a novel 3-level load-balancing strategy
and has also conducted a novel study using a robust simulator for analyzing the potential
benefits of executing the climate model on grids. GARL is also developing a general grid
framework for efficient execution of multi-component applications like coupled climate
models on grids. In the area of phylogenetic predictions in bioinformatics, GARL uses
Grid resources to analyze neighbor-dependent mutations on different branches of a
phylogenetic or evolutionary tree and use the analysis to predict future DNA sequences.
6.5 Other HPC Applications
Disaster Management

CDAC’s DMSAR is a RADAR based Airborne earth imaging system through which the
real time situations of any natural and man induced disasters can be mapped even in
rough weather conditions. Synthetic Aperture RADAR (SAR) fitted beneath aircraft
captures data of flood region. DMSAR scans the terrain and capture the data in different
resolution modes The minimum amount of data captured in a single run is 9GB of raw
data. Data is processed by CDAC’S HPC centers to expedite decision making.

Winglet Optimization

The efficiency of an aircraft can be increased by decreasing the drag force of an aircraft.
One of the ways to reduce induced drag is to have winglets at the tip of the wing. The
design of the winglet requires the use of an optimization algorithm known as Genetic
Algorithm. This algorithm automatically generates a huge amount of random winglet
shapes. The performance of these winglet shapes and their effect on the overall aircraft
design is then analyzed using Computational Fluid Dynamics (CFD) and Computational
Structural Mechanics (CSM) software. This whole framework, named Winglet
Optimization System (WOS), which is the property of Zeus Numerix Pvt. Ltd. requires
huge state-of-art super-computing facilities which is provided by CDAC. This has
enabled us to execute the Winglet design work at an accelerated pace.

Machine Learning

CMSD, Hyderabad uses its HPC clusters to develop products and conduct research in
cognitive neuroscience, computational intelligence, and natural language understanding.

7. HPC Industry in India
The HPC industry have grown mature in India in the last 2-3 decades.
Multinationals such as IBM, Cray, DEC, NEC, and SGI also had strong
influence on the HPC market in the early 1990s. Indian companies, e.g.,
Hindustan Computers Limited (HCL), Wipro Technologies, Hindtitron
Systems, Computer Maintenance Corporation (CMC), Tata ELXSI, were early
players in the HPC market segment. However, in the early years, many of the Indian
companies as well as the Indian outfits of the Multi-national companies were primarily
enablers for the sale and after-sale services of HPC systems. However, all of these Indian
companies or Indian outfits of the multinationals had a strong group on systems software.
In fact, systems software has been a major strength of the Indian industry in the 1980s.

In the last decade, with the advent of cluster computing, the scenario
somewhat changed. Indian companies were getting involved in integration of cluster
architectures. One of the notable successes in this front is the IMSC’s Kabru cluster
where Summation Enterprises and Netweb Technologies were instrumental in the
integration of cluster. Today, a number of cluster of various sizes have been integrated
by various Indian companies as well as the multinationals in India. Organizations such as
CDAC have also in-house expertise in cluster integration.

Recently few companies including Yahoo! India and Amazon Development Center,
India, have created strong research groups for research and development on cloud
computing. Cloud computing involves creation of “clouds” of services that can be
invoked remotely by customers for solutions. By virtualizing the remote, complex,
distributed network of hardware and software resources, cloud computing offers cost-
effective solutions for small and medium-scale companies to make use of the already
created solutions for various applications, including, data-mining, web-search and
retrieval of large-data sets. Cloud computing has huge potential for high performance
computing wherein, high performance applications can be created as cloud services and
these services can be invoked remotely on large high performance centers and data

8. Top Supercomputers-India
Top Supercomputers-India (http://topsupercomputers-india.iisc.ernet.in), developed in
and by Supercomputer Education and Research Centre, Indian Institute of Science,
Bangalore, is an effort to list the most powerful supercomputers in India. This effort is
complementary to the Top500 project that lists the top 500 supercomputers in the world.
The aim of Top Supercomputers-India, similar to Top500, will help create and promote
healthy competition among the supercomputing initiatives in India and can substantially
lead to significant supercomputing advancement in the nation. LinPack benchmark and
the same submission and evaluation procedures in Top500 is used for ranking the
systems. The number of machines in the TopSupercomputers-India list is based on
minimum performance threshold. The project was announced in September and
submissions were collected in the beginning of October 2008. After verification, the first
list was released on December 18 in the Plenary Industrial Panel session of the
International Conference on High Performance Computing (HiPC), 2008, held in
    Bangalore. Certificates containing the signatures of Dr. Jack Dongarra were given during
    the event. The first list contained 11 entries and the current and the second list contains
    13 entries. The top entry is the EKA supercomputer of Computational Research
    Laboratories Ltd (CRL), a wholly owned subsidiary of TATA Sons Ltd. The first four in
    the list are also in the Top500 list. The combined performance of the machines in the list
    is 243.71 TeraFlops. Five of the systems are from Centre for Development of Advance
    Computing (CDAC), proving its status as a leading high performance computing center
    in the nation.

                          Top Supercomputers-India: June 2009 List
                                                   Cores/Processor Rmax     Rpeak
Rank Site                      System
                                                   Sockets/Nodes   (TFlops) (TFlops)
                                 HP Cluster Platform
                                 3000 BL460c (Dual
                                 Intel Xeon 3 GHz
1                                quad core E5365         14400/3600/1800 132.80        172.60
          Laboratories Ltd,
                                 w/Infiniband 4X
          Center for             PARAMcluster (Intel
          Development of         Xeon (Tigerton) 2.93
2         Advanced               GHz quad core quad 4608/1152/288           37.80      54.01
          Computing (C-DAC)      processor X73xx
          India                  nodes w/Infiniband)
                                 IBM eServer Blue
                                 Gene Solution,
                                 BlueGene/L (IBM
          Indian Institute of
3                                PowerPC 700 MHz         8196/4096/4096     17.81      22.94
          Science, Bangalore
                                 440x5 processors
                                BL460c (Xeon 54xx
          Paprikaas Interactive
4                               3.0GHz                   1920/ /            13.16      23.04
          Services India
                                 IBM Blue Gene
          Tata Institute of
                                 BlueGene/P (IBM
5                                PowerPC 850 MHz         4092/1024/1024     11.32      13.5
          Research (TIFR),
                                 450 processors
          Indian Institute of HP 256 Intel Xeon
6                                                        1024/256/128       8.62       12.29
          Technology Madras, Processor E5472 (3
     Chennai                GHz w/Infiniband)
     Indian Institute of HP 256 Intel Xeon
7    Technology Madras, Processor E5472 (3         1024/256/128   4.55    8.64
     Chennai             GHz w/Gigabit)
                            HP Proliant DL140
     Jawaharlal Nehru
                            G3 (dual processor
     Centre for Advanced
                            dual core Intel Xeon
8    Scientific Research                           512/256/128    3.86    6.00
                            3GHz 5160 nodes
                            w/Infiniband 4X
     Institute of           HP CP3000 (576
     Genomics and           Intel Xeon Processor
9                                                576/576/288      3.06    4.08
     Integrative Biology,   X3.6GHz/800-2MB
     Delhi                  w/Infiniband)
     Centre for             HP Cluster (Dual
     Development of         Intel Quad Core
10   Advanced               Xeon EM64              320/80/40      2.999   3.943
     Computing (CDAC),      processor
     Chennai                w/Infiniband)
     Centre for             HP Cluster (Dual
     Development of         Intel Quad Core
11   Advanced               Xeon EM64              320/80/40      2.976   4.044
     Computing (CDAC),      processor
     Bangalore              w/Infiniband)
     Centre for             HP Cluster (Dual
     Development of         Intel Quad Core
12   Advanced               Xeon EM64              320/80/40      2.614   3.94
     Computing (CDAC),      processor
     Hyderabad              w/Infiniband)
                       HP-DL580-G5 (Intel
     Centre for        Xeon (Tigerton)
     Development of    2.933 GHz quad core
13   Advanced          quad processor      256/64/16              2.165   3.00
     Computing (CDAC), X7350 nodes
     Pune              w/10Gbps
9. Conferences and Events
Several international conferences, workshops, and user fora have been held in India in the
last 2 decades. Of these, the International Conference on High Performance Computing
(HiPC), started in 1994 as a parallel processing workshop, and the International
Conference on Advanced Computing and Communication (ADCOM), started in 1993,
are annual conferences held in India. These conferences focus on an aspects of high
performance computing and communications, and their scientific, engineering, and
commercial applications. The HiPC conference features typically 50 research papers, 4-6
workshops, and an equal number of tutorials. In the recent years, the conference has
attracted more than 300 submissions from 30 countries. . The conference is well
attended with 250-300 participants, of which a significant number of them (more than
100) are from other countries. The ADCOM conference features typically 120 papers
and 4-6 tutuorials and a 2-4 workshops every years. It attracts about of 400 submissions
each year and has an attendance of 150-200 participants, of which about 75 or more are
International participants.

In addition to the above annual events, the 6th International Conference on High
Performance Computing in Asia-Pacific Region (HPC-Asia) was held in Bangalore in the
year 2002. The HPC User Forum Conference was held in New Delhi and Bangalore
from Feb. 28 – March 2, 2007. Several computational science researchers from India
have participated in this conference and presented their work. The forum also had
industry participation and presentation.
Overall these conferences have brough greater awareness on high performance
computing in India. They have exposed the Indian graduate and undergraduate students
to recent research work in HPC. They have served as a fora for the Indian HPC
researchers to present their research and network with their peers.

10. Future Roadmap
Various organizations plan to expand their HPC capabilities in the near future. CDAC
aims to build a PetaFlop HPC facility by 2012. CMSD, Hyderabad, is currently
augmenting its cumulative performance to 15 TeraFLOPS. INCOIS, CCMS at JNCASR,
SCFBIO, and many of the other data centers that were surveyed in this report are
planning to enhance their compute capability to a few 10s of TeraFLOPS. Their projected
investment over the next 2-3 years would be ranging from 1 – 3 Million US$. We
anticipate to see a 3-5 major centers which would host facilities with a few hundred
TeraFLOPS to may be in the PetaFLOP range. At the same time, we anticipate a fairly
large number of (may be 100 or so) smaller clusters of a few 10’s of TeraFLOPS. While
the smaller clusters are like to be utilized for their full capability, with applications
scaling to the number of cores in the clusters, the major challenge will be on the
scalability of application on larger clusters/systems. We anticipate these systems will be
used more for capacity (large number of jobs running on a few hundreds of cores) than
for capability (jobs running on a few thousands of cores). This challenge needs to be
addressed through training and specialized course programmes in computational
science. Institutions like the Indian Institute of Science are offering Masters programs in
computational science as an inter-disciplinary program. These programs are at their initial
stages and are expected to yield fruits in the next 5 to 10 years.

The government funding for HPC is expected to grow and support the above plans. .
The Grid Garuda project is expected to go into the main phase with significant support
from the government. The Grid project is expected to consolidate the resources across
the country. The Grid communication fabric connecting various organizations will be
enhanced to 10-100’s of Gigabits per second connectivity. The Garuda communication
fabric and the Knowledge Network connectivity to various organizations will play a
major role in ensuring a high-speed connectivity across the country. The emergence of
Cloud computing and presence of international players will ensure cloud services being
offered to research organizations on demand. It is not clear to what extent this (taking
cloud services) will pick up in India, where research organization typically have an
investment plan rather than going for a pay-per-use model.

From the applications perspective, it is expected that more focus would be on
computational aspects of seismic analysis, leading to earth quake and tsunami prediction,
and climate change. In the computational biology area, the focus would be on drug and
vaccine discovery through virtual screening for tropical and infectious diseases that are
locally relevant. Sequence and structural analysis at genomic scale is likely to yield a
holistic view of the functioning of a cell. Further, the simulation studies would move
from macromolecules-level to macromolecular assemblies. In addition to the traditional
computational problems, problems in the areas image and video analytics, applying
machine learning techniques, applications enabling e-governance activities in various
government functionaries, processing of information collected from distributed sensors
are expected to draw more attention

   a. Adding more in introduction with good general perspectives. – RG [DONE!]
   b. Section 5 - Some pictures??
   c. Section 9 – Writing a good future roadmap section with good general
      perspectives. – RG [DONE!]