Science Gateways and their treme

Document Sample
Science Gateways and their treme Powered By Docstoc
					          CyberGIS and
TeraGrid Science Gateways update

              Nancy Wilkins-Diehr
  TeraGrid Area Director for Science Gateways

            TeraGrid Quarterly, December 7, 2010

•SGW funds a Cyberinfrastructure in GIS workshop in
conjunction with the UCGIS meeting in DC in February, 2010
 – Co-led by Shaowen Wang at NCSA and Nancy
 –Approved by UCGIS board after a lengthy voting process
 – Expected outcome of the workshop – all are happening!
   •Increased visibility
   •New partnerships for TeraGrid, UCGIS, and other pertinent organizations
   •Workshop report
   •Interesting collaborative proposal ideas
   •Potential future publications
 – 50 attendees from throughout the US, Sweden and Australia
 – Workshop attendees also attended UCGIS meeting, including briefing
   from high-ranking administration officials

                        TeraGrid Quarterly, December 7, 2010
Shaowen and Nancy represent TeraGrid
Congressional Reading Room, Library of Congress

            TeraGrid Quarterly, December 7, 2010
   Administrative officials in attendance
•Karen Siderelis
 – GIO Department of Interior,
   Acting Chair Federal Geographic
   Data Committee
•Raphael Bostic
 – Assistant Secretary for Policy
   Development and Research,
•Derek Douglas
 – Special Assistant to the
   President for Urban Affairs
•Jerry Johnston
•Stephen Lowe
 – GIO, Dept of Agriculture Quarterly, December 7, 2010
               Workshop brings results
•Scientific Software Integration (SSI) award for CyberGIS
 – Shaowen Wang, PI
 – Largest SSI awarded in FY10
   •$4.4M, 10/1/10-9/30/15
 – Academia, government, industry, international partnerships
   •Arizona State University
   •The Computer Network Information Center of the Chinese Academy of Sciences
   •Environmental Systems Research Institute (ESRI)
   •Georgia Institute of Technology
   •Oak Ridge National Laboratory
   •University College London Centre for Advanced Spatial Analysis (England)
   •University Consortium for Geographic Information Science
   •University of California-San Diego
   •University of California-Santa Barbara
   •University of Illinois at Urbana-Champaign
   •University of Washington
   •U.S. Geological Survey
                              TeraGrid Quarterly, December 7, 2010
   •Victorian Partnership for Advanced Computing (Australia)
                 Goals and objectives
•Establish CyberGIS as a fundamentally new software
 – Integration of CI, GIS, and spatial analysis/modeling
 – Widespread scientific breakthroughs, broad societal impacts
•Participatory evolution of CyberGIS community requirements
•CyberGIS software integration roadmap
•High performance and scalable CyberGIS
•Online CyberGIS gateway
•CyberGIS testing and integration with national and
international CI
•Community-based and application-driven evaluation of
                       TeraGrid Quarterly, December 7, 2010
                               CyberGIS vision

Wang, S. 2010. "A CyberGIS Framework for the Synthesis of Cyberinfrastructure, GIS, and Spatial Analysis.“
Annals of the Association of American Geographers, 100(3): 535-557.
                                 TeraGrid Quarterly, December 7, 2010
               Initial software components
 – Large-scale spatial analysis and modeling (SAM), web interface
 – Spatial data analysis: exploratory data analysis (EDA), exploratory spatial data analysis
   (ESDA), maximum likelihood (ML) spatial regression
 – Open source library for spatial analysis: weights, computational geometry, ESDA, spatial
   econometrics, clustering and spatial dynamics
 – Community access to high-resolution, Earth science-oriented, topography data, and
   related tools and resources
 – On-line tools for expanding public participation in transportation improvement
 – GRASS (Geographic Resources Analysis Support System)
    • Geospatial data management and analysis, image processing, graphics/maps production, spatial
      modeling, and visualization
 – pd-GRASS
    • Shell scripts that distribute display functions of GRASS GIS through the network among several
      physical monitors in a synchronized manner
                                   TeraGrid Quarterly, December 7, 2010
              Early application targets
•Emergency management
 – Fire, flood, disease, earthquake
 – Managing during the incident and post incident, tracking victims, etc.
 –Real-time integration of data from multiple sources
 – Combining distinct analysis tools via workflows, incorporating HPC
   when warranted (“computational intensity maps”)
•Distributed analysis support, collaboration and participation
tools through an online service
•Proof of concept research

                       TeraGrid Quarterly, December 7, 2010
  What have the gateways been up to?
•Record high 896 users of community accounts on TG
 – 31% of all users charging jobs
 – 645 users of CIPRES gateway
   • Cited in at least 35 publications including Nature, PNAS, and Cell
   • 77% of all jobs have been submitted from the US, including top-tier institutions such as Harvard, Yale, and
   • Jobs received regularly from 17 EPSCOR states
   • Job submissions from 34 countries on 5 continents
   • At least 5 undergraduate classes known to use the portal routinely. This is likely an underestimate (based on Web
     log patterns).

•International representation
 –Matthew Woitaszek gives keynote address at the International
   Workshop on Science Gateways (IWSG) in Catania, Italy
 – Rion Dooley also in attendance
 –This deployment is the final piece needed to support attribute-based
 – Much testing by gateway staff
   •Special kudos to Suresh Marru, Raminder Singh, David Carver, Stu Martin,
                         TeraGrid Quarterly, December 7, 2010
2.5M CPU hours of Q3 gateway use
                     Source: Dave Hart

   By Resource                                   By Community User

                 abe.ncsa.teragrid                              Cipres Community User

                 frost.ncar.teragrid                            Gridamp Community
                 ranger.tacc.teragrid                           Gisolve Community
                 cobalt.ncsa.teragrid                           Gridchem Chemistry
                                                                Community User
                 steele.purdue.teragrid                         ccsmuser Community
                 condor.purdue.teragrid                         Sidgrid Community
                 kraken.nics.teragrid                           Robetta Community
                 lonestar.tacc.teragrid                         DES Community User

                 queenbee.loni-                                 DESDM Community
                 lsu.teragrid                                   User
                 bigred.iu.teragrid                             Tera3D Community
                 lincoln.ncsa.teragrid                          OGCE Community User

                 TeraGrid Quarterly, December 7, 2010
       Much progress on standardized
     treatment of community accounts
•Victor Hazlewood, Matthew Woitaszek, Jim Marsteller
•Nancy’s goal is to provide gateway developers a menu of
what to expect at TG sites
•Fold all paper agreements into single, existing TG user
responsibility form

                   TeraGrid Quarterly, December 7, 2010
    Nancy’s “end of program” vision for
           community accounts
•Access to resources via gsi-ssh and GRAM
•If direct logins to community accounts are restricted:
 – Allow identified developers to “su in”
•If execution directories are restricted:
 – Provide developer controls through commsh

                        gsi-ssh       GRAM             su      commsh

              Site A        X             X
              Site B        X                           X        X
              Site C                      X             X
              Site D        X                                    Xh

                        TeraGrid Quarterly, December 7, 2010
•Uintah and PET
 – Gateway support staff and documentation are so good, need for
   advanced support nearly eliminated for large scale CFD gateway
   •Much to our surprise, Uintah developer presents at TG10 on nearly finished
    gateway after 20 minute phone conversation with Matthew Woitaszek and
    pointers to documentation!
   •New work outlined on Population-Environment-Technology (PET) model

                        TeraGrid Quarterly, December 7, 2010
           Analytical Ultracentrifugation
Emerging computational tool for the study of proteins

•Samples from researchers all over
 the world
 – Some (Germany, Australia) have
                                          The Center for Analytical Ultracentrifugation of
   their own ultracentrifuges and use Macromolecular Assemblies, UT Health Sciences
   only the analysis capabilities, others
   send samples to UT to spin
•Spin the samples at high speeds,
 learn about macromolecule
•Monte Carlo simulations
•Observations are electronically
 digitized and stored for further
 mathematical analysis

                              TeraGrid Quarterly, December 7, 2010
   Source: Suresh Marru, IU
  Comprehensive data analysis environment
•Management of analytical ultracentrifugation data for single
users or entire facilities
•Support for storage, editing, sharing and analysis of data
 – HPC facilities used for 2-D spectrum analysis and genetic algorithm
   •TeraGrid (~2M CPU hours used)
   •Technische University of Munich
   •Juelich Supercomputing Center
•Portable graphical user interface
•MySQL database backend for data management
•Over 30 active institutions

                             TeraGrid Quarterly, December 7, 2010
  Source: Suresh Marru, IU
               Gateway and ASTA support
                             a growing trend
•TeraGrid advanced support
 – Fault tolerance
 – Workflows
 –Use of multiple TG resources (using Lonestar, expanding to QueenBee
   and Ranger, using Quarry for test server, waiting for GRAM5 on
 – Community account implementation
 – Remote steering
 – Improved UI (no manual specification of CPU time)
 – Applying lessons learned from GridChem, LEAD, incorporating new
   features into OGCE
   •LEAD is portlet-based, Gridchem is java swing client side app, Ultrascan is
    php and perl-based gateway, all can use OGCE
   •Big MPI app that forks off many independent runs, improvements here will
    be tackled by TG's advanced support team
                             TeraGrid Quarterly, December 7, 2010
  Source: Suresh Marru, IU
           New advanced support work
•Ocean Land Atmosphere Simulation (OLAS) group
 – PI Craig Mattocks, UNC
 – Simulation of flooding and inundation
 –Using TG’s gateway hosting service to set up high availability, real-time
   data server to ingest live data from NCAR's LDM, OPeNDAP and
   THREDDS services
 – Gateway work includes workflows to execute OLAS coupled models
   based on triggers from events in LDM data streams
   •Building off similar work in LEAD

                         TeraGrid Quarterly, December 7, 2010
   Emerging gateway through DataONE
•Proposed MOU would have
 DataONE appear to TG as data
 oriented science gateway and TG
 appear to DataONE as a set of
 Member Nodes
•Combining distributed and diverse
 data sets to create new scientific
 insight from new syntheses of data
 – Observational integrated
   with environmental observational
   data such as NASA’s MODIS data
   from the ORNL DAAC to generate
   predictions of bird species migration
 – TG10 paper by Daniel Fink

                          TeraGrid Quarterly, December 7, 2010
                Gateway software listing
•Populate TeraGrid’s
information service with
gateway software
 – A search for computational
   chemistry packages should turn
   up both commandline software
   and packages accessible
   through a gateway
 – Web services and programmatic
   generation of package listings
   •So the RENCI science portal folks
    don’t have to hand-enter 140

                           TeraGrid Quarterly, December 7, 2010
 New gateway activities in the extension year

•Helpdesk support expanded
 – From .2 FTE in PY5 to 1.7 in Extension
   •Helpdesk and Condor support, new GIS communities, SimpleGrid extensions
 – Improved views for gateways now that we have attributes
•Community accounts
 – Continued work toward improved standardization
•Prebuilt VMs with gateway software
 – OGCE, SimpleGrid
•Online tutorials with CI Tutor and the EOT team
 – OGCE, SimpleGrid
•More example-based documentation
 – Less talk, more action, short videos, based on user feedback
•Remote vis for gateways – contract delays

                          TeraGrid Quarterly, December 7, 2010
     Targeted Support in the Extension
   All staff available for assignments as new projects come in

 – Meet the needs of several groups with large TG allocations
•GridChem, PolarGrid, Ultrascan
 –Scheduling, vis, Matlab processing, processing of centrifuge data for
  large international project
 – Continuing work to combine capabilities
•OpenSocial for gateways
•Condor and cloud support

                       TeraGrid Quarterly, December 7, 2010
      TG to XD transitions for gateways

•Stu, Nancy and XD gateway leads
 to conduct focused discussions
 with each gateway, pending
 architecture definition
 – Update use case description
 – Add/subtract gateways as
 – Ask about impact of transition
   (depends on the architecture)
 – Ask about entries in software
 – Ask about attribute-based
   authentication (also depends on
•Assessment and transition
of gateway advanced
support projects too
                   TeraGrid Quarterly, December 7, 2010
             Gateway Sustainability Study
                       Small, non-TG, EAGER grant
•Characteristics of short funding cycles
 – Build exciting prototypes with input from
 – Work with early adopters to extend
 – Tools are publicized, more scientists
 – Funding ends
 – Scientists who invested their time to use
   new tools are disillusioned
    • Less likely to try something new again
 – Start again on new short-term project
•Need to break this cycle
•EAGER grant to look at characteristics
 of successful gateways and domain                      4 focus group meetings over 2 years
 areas where a gateway could have a                           First 2 held June, 2010
 big impact

                                TeraGrid Quarterly, December 7, 2010

TeraGrid Quarterly, December 7, 2010

Shared By: