Get documents about "Project Execution Plan Samples
W
Description
Project Execution Plan Samples document sample
Document Sample
Project Execution Plan
November 2010
Version 1.1
Pending review and approval by NSF and NASA
The Virtual Astronomical Observatory (VAO) is
managed by the VAO, LLC, a non-profit
company chartered in the District of Columbia
and established as a partnership of the
Associated Universities, Inc. and the
Association of Universities for Research in
Astronomy, Inc. The VAO is sponsored by the
National Science Foundation and the National
Aeronautics and Space Administration.
Submitted to the National Science Foundation
Pursuant to Cooperative Agreement
AST-0834235
November 4, 2010
Collaborating Organizations
Association of Universities for Research in Astronomy, Inc.
Associated Universities, Inc.
California Institute of Technology
High Energy Astrophysics Science Archive Research
Center
Infrared Processing and Analysis Center, Jet Propulsion
Laboratory, California Institute of Technology
National Optical Astronomy Observatory
National Radio Astronomy Observatory
Smithsonian Astrophysical Observatory
Space Telescope Science Institute
The Johns Hopkins University
National Center for Supercomputing Applications
VAO Project Execution Plan 1 Collaborating Organizations
Document Change History
Date Version Changes
November 4, 2010 1.1 1.1 Changed default document font to Helvetica to ensure
interoperability.
1.2 Corrected name of Center for Astrophysics; Project
Baseline: Building and Analyzing SEDS (second list
item)
1.3 Modified statement of delivery for Time Series Data;
Project Baseline: Time Series Astronomy
1.4 Corrected FTE levels for NCSA and CACR; Project
Baseline: Staffing
1.5 Updated description and function of software test teams;
Operations: Software Test Teams
1.6 Added Product Development’s role to Configuration
Control Board; Operations: Configuration Control Board
1.7 Added support for IVOA Web support and document
curation; User Support: IVOA Web Site and Document
Repository
1.8 Removed NOAO’s role in scientific validation of services;
User Support: The Testing Process
1.9 Changed Research and Development to Research and
Prototyping; User Support: Research and Prototyping
1.10 Added JIRA as a documentation source for test plans
and results; User Support: Testing Tools and Policies
1.11 Updated definition of test processes in Table 4-2; also
changed ―Test‖ to ―Quality Control;‖ User Support:
Quality Control Activities
1.12 Changed ―Integrated Testing‖ to just ―Testing‖ and
identified types of tests conducted; Product
Development: Project Life Cycle Overview
1.13 Updated Development Stage to clarify where candidate
releases are built, and by whom; Product Development:
Development Stage
1.14 Changed ―Integrated Testing Stage‖ to ―Testing Stage‖
and identified team responsible for this stage; Product
Development: The Testing Stage
1.15 Updated table showing work area leads; Management
August 13, 2010 1.0 Initial version
VAO Project Execution Plan 2 Document Change History
Contents
Collaborating Organizations ____________________________________________ 1
Document Change History ______________________________________________ 2
Signature Page _______________________________________________________ 5
Mission Statement _____________________________________________________ 6
Executive Summary ___________________________________________________ 7
Introduction _________________________________________________________ 11
Section 1. Project Baseline _____________________________________________ 12
VAO Project Outcomes ___________________________________________ 12
Scientific Initiatives_______________________________________________ 12
Science Deliverables for Year One __________________________________ 21
Community Involvement in Science Deliverables _______________________ 28
Study Initiatives for Year One ______________________________________ 28
Science Deliverables for Years Two and Three ________________________ 30
Facility Milestones and Deliverables _________________________________ 32
Work Breakdown Structure ________________________________________ 34
Staffing ________________________________________________________ 36
Metrics ________________________________________________________ 37
Schedule ______________________________________________________ 38
Section 2. Management ________________________________________________ 40
Governance ____________________________________________________ 40
Business and Financial Management ________________________________ 46
Budget ________________________________________________________ 48
Safety, Environment and Health ____________________________________ 49
Risk Management _______________________________________________ 50
Contingency Management _________________________________________ 55
Section 3. Operations _________________________________________________ 56
Deploying and Monitoring Services __________________________________ 56
VAO Support Services ____________________________________________ 58
Quality Assurance _______________________________________________ 61
Configuration Management ________________________________________ 63
Section 4. User Support _______________________________________________ 82
Goals _________________________________________________________ 82
Quality Assurance and Testing _____________________________________ 82
Documentation __________________________________________________ 82
IVOA Web Site and Document Repository ____________________________ 82
Training and Advocacy ___________________________________________ 83
VAO Web Site __________________________________________________ 84
Help Desk and Ticket System ______________________________________ 85
VAO Project Execution Plan 3 Contents
Quality Assurance and Testing _____________________________________ 86
Staffing Roles and Responsibilities __________________________________ 95
Section 5. Product Development ________________________________________ 97
Goals _________________________________________________________ 97
Evolution of the Development Plan __________________________________ 98
The Project Life Cycle ____________________________________________ 99
Tools and Policies ______________________________________________ 101
The Development Process________________________________________ 105
Section 6. Standards and Protocols ____________________________________ 111
Challenges ____________________________________________________ 111
Interactions With the IVOA________________________________________ 112
Types of Deliverables ___________________________________________ 113
The VAO Standards Development Process___________________________ 114
Development of an IVOA Standard _________________________________ 116
Priorities for Year One ___________________________________________ 116
Section 7. Data Preservation and Curation _______________________________ 120
Introduction and Context _________________________________________ 120
Data Preservation ______________________________________________ 121
Cross-Repository Linking _________________________________________ 125
Preservation and Curation Standards _______________________________ 126
Work Plan ____________________________________________________ 127
Roles and Responsibilities ________________________________________ 128
Goals ________________________________________________________ 128
Deliverables ___________________________________________________ 130
Section 8. Technology Assessment ____________________________________ 132
Sources of Requests ____________________________________________ 133
Roles and Responsibilities ________________________________________ 133
Deliverables and Exit Criteria______________________________________ 134
Example Areas of Interest ________________________________________ 136
Section 9. Education and Public Outreach _______________________________ 138
Broader Impacts of the VAO EPO Plan ______________________________ 140
Year One _____________________________________________________ 140
Year Two _____________________________________________________ 141
Year Three ____________________________________________________ 142
Year Four _____________________________________________________ 143
Year Five _____________________________________________________ 144
Acronyms, Abbreviations and Glossary _________________________________ 146
List of Tables and Figures ____________________________________________ 158
Appendix A: Traceability Matrix _______________________________________ 159
VAO Project Execution Plan 4 Contents
Signature Page
Dr. Robert Hanisch Dr. Ethan J. Schreier
VAO Director Principal Investigator, VAO and
President, Associated Universities, Inc.
(AUI)
Dr. G. Bruce Berriman Dr. John S. Gallagher, III
VAO Program Manager Chairman, VAO LLC Board Of Directors
Dr. Giuseppina Fabbiano Dr. William S. Smith
Chair of VAO Science Council President, Association of Universities for
Research in Astronomy, Inc. (AURA)
VAO Project Execution Plan 5 Signature Page
Mission Statement
The VAO, building on the success of prior infrastructure developments, is now
poised to realize the benefits long-promised to the astronomical research
community, and more generally, educators and the general public. Through a
close and ongoing dialog with the community, facilitated by an advisory Science
Council and focused partnerships with active research collaborations, the VAO
will provide the underpinnings for new and more efficient ways to correlate and
integrate data from large surveys, heterogeneous sets of pointed observations,
and theoretical models and simulations. The VAO will provide robust, general
purpose applications that can be accessed through a Web browser, or as
appropriate, can be downloaded for local installation. The VAO will also provide
the components, libraries, and templates that allow users to craft their own VO-
enabled applications for research problems of their particular interest. Through
this combination of VAO-provided tools and astronomer-created applications, the
virtual observatory will permeate the astronomical research community.
VAO Project Execution Plan 6 Mission Statement
Executive Summary
This Project Execution Plan (PEP) describes the long-term vision and the near-
term (1-2 year) implementation, deployment, operations, public outreach, and
support plans for the Virtual Astronomical Observatory (VAO).
The intent of the PEP is to:
Establish a common understanding with the funding agencies—NSF and
NASA—of how the VAO intends to meet their expectations for an operational
virtual observatory that provides significant improvements to the research
capability of the astronomical research community.
Provide maximum transparency into the levels and types of effort required to
meet the above goals.
Inform and hold accountable the team members of the VAO collaboration in the
various areas of scientific, technical, and operational responsibilities.
The PEP will be updated annually to reflect the changing nature of astronomical
research, the changing nature of the computational environment in which
astronomers do their work, and the actual performance of the VAO collaboration.
The PEP takes into account advice from the VAO Science Council and the
experience gained from working with the user community. It is responsive to the
expectations identified in the Cooperative Agreement. The PEP is reviewed and
approved by the VAO Science Council, VAO Board of Directors, and the
NSF/NASA VAO Oversight Group.
The VAO, building on the success of prior infrastructure developments, is now
poised to provide the tools and services previously promised to the astronomical
research community, and more generally, to educators and the general public.
Through a close and ongoing dialog with the community, facilitated by an advisory
Science Council and focused partnerships with active research collaborations, the
VAO will provide the underpinnings for new and more efficient ways to correlate,
integrate, and analyze data from large surveys, heterogeneous sets of pointed
observations, and theoretical models and simulations.
In order to begin the provision of new and productive VO tools and services to the
research community, we have defined seven primary science initiatives that will
support a range of research capabilities that are essential to astronomy in the
digital age. Our funding levels do not allow for simultaneous progress in all seven
areas, therefore four have been selected for immediate development and three
will begin in a Study and Requirements phase for a year, followed by a more
concerted development effort. These seven initiatives are as follows:
VAO Project Execution Plan 7 Executive Summary
Year One Development
1. A new VAO data discovery and data access portal
2. General and expanded cross-matching
3. Spectral energy distribution building, editing, and analysis
4. Time series data exploration and analysis tools
Year One Study, Year Two Development
5. Advanced linking between journal articles and data
6. Data mining and statistical analysis tools
7. Integrated desktop tools
Initiative 4, the support of time series data, also includes integrating the VOEvent
system for global distribution of transient event notices into the VAO core.
VOEvent has other support at this time through a project called SkyAlert, thus our
effort here will be coordinated so that as SkyAlert ramps down, VAO support will
ramp up. Approximately 30% of the effort in the VAO is dedicated to development
of the above products.
To help guide these efforts and assure their implementation meets the needs of
the research community, the VAO will work with two research groups on two
astronomically diverse programs that will rely on VO infrastructure.
1. In order to understand the three-dimensional structure of the Small Magellanic
Cloud (SMC), its interactions with the Large Magellanic Cloud, and more
generally, to understand the dynamics of interactions among dwarf galaxies
and the impact of such interactions on galaxy evolution, we will generate a
cross-matched multi-wavelength directory of objects with a 10-degree radius
of the SMC. This will rely on new cross-matching algorithms that can
accommodate multiplicities of matches and different spatial resolutions in
different bandpasses. These algorithms will be of general utility to the
community in dealing with current and next generation survey data. B.
Madore (Carnegie Observatories) will be our principle liaison to this research
effort.
2. Understanding the evolution of galaxies and black holes from z~8 to z~1.5 is
the goal of the Hubble Space Telescope (HST) Multi-Cycle Treasury program,
the Cosmic Assembly/Near-infrared Deep Extragalactic Legacy Survey
(CANDELS). This effort is led by S. Faber (University of California, Santa
Cruz) and H. Ferguson (STScI). Like the SMC study, CANDELS will rely on
robust cross-matching across multi-wavelength observations, from X-ray to
radio, using a wealth of deep observations from HST and many ground-based
VAO Project Execution Plan 8 Executive Summary
observatories. In addition, new tools for building and analyzing spectral energy
distributions (SEDs), and combining SEDs with cross-matching to find
samples uncontaminated by confusion, will be a critical component of these
studies. Other VAO-enabled research will include multi-wavelength
morphological classification, dynamic image cut-out services (registered and
consistently sampled across bandpasses), and tools to make virtual
observations of cosmological simulations. The primary liaison is A. Koekemoer
of the Space Telescope Science Institute (STScI), who is the lead data
scientist for CANDELS.
Close collaboration with these research efforts will help assure that generic VAO
functions truly serve the needs of the community, and the community will see
VAO-enabled science results in concert with the release of the first research
papers from these projects. As we move forward in the VAO, we will seek
additional collaborations to help us respond to the research needs of the
community in other areas, such as data mining.
Although it is of primary importance that the VAO activities are extremely relevant
to the scientific community, it is equally important that the VAO research
capabilities be deployed in a stable, tested, and well-documented environment.
Thus, major work areas also include Operations (15% of total effort) and User
Support (13% of total effort). The VAO is inherently distributed, relying on
services provided by data archive centers and major observatories worldwide. Our
Operations team will monitor services for availability and adherence to VO
standards, provide failover configurations for the services we provide ourselves,
and maintain our software repository and revision control system. Our User
Support team will provide an online Help Desk and ticket tracking system,
maintain our Web site, produce and distribute newsletters, create an online forum
for users to easily communicate with each other and the VAO team, and organize
professional outreach programs such as summer schools, tutorial sessions,
American Astronomical Society (AAS) exhibits, and other Web-based
communication and social networking facilities.
While the National Virtual Observatory (NVO) and the International Virtual
Observatory Alliance (IVOA) have established a rich suite of standards, it has
been understood since the beginning of the VO development efforts that these
standards would need to continue to evolve. We have allocated 14% of our effort
to ongoing work on standards. These standards are not developed without a clear
and pressing scientific motivation. For example, the spectrum access protocol
needs to be modified somewhat to become a time series access protocol.
Standards activities also include reference implementations and common libraries
that make it easier for data centers to provide access to their holdings through VO
protocols. It is anticipated that the VAO will continue to work with the IVOA in the
development of these new standards.
VAO Project Execution Plan 9 Executive Summary
Another 4% of our effort will go into data curation and preservation. An immediate
focus is on the highly processed data sets that are published in graphical form
(pictures, line plots) in the journals, but where the underlying digital data is not
systematically preserved. We will work in collaboration with the American
Astronomical Society, the Astrophysics Data System (ADS), the NASA/IPAC
Extragalactic Database (NED), and the Centre de donneés astronomiques de
Strasbourg (CDS) to establish a process whereby digital data associated with
publications can be captured and fully annotated as part of the manuscript
submission process. We are also collaborating with the National Science
Foundation’s DataNet program through one of the funded projects, the Data
Conservancy (S. Choudhury, Johns Hopkins University, Principal Investigator).
We are dedicating a small effort, 2%, to targeted technology assessment studies.
Such studies will be undertaken in order to understand if a new approach to areas
such as Web services, or software development, or management of distributed
data storage is of sufficient maturity and stability—and of sufficient benefit to the
VAO—to incorporate into our infrastructure. Technology Assessment is also an
ongoing element of software development and standards development, but this
effort is distributed over the many people involved in these activities.
Some 5% of our effort is in an Education and Public Outreach program.
Finally, about 16% of the program resources go into management-related
activities, including the Director (0.8 full-time equivalent, or FTE), Program
Manager (0.5 FTE), Project Scientist (0.25 FTE), and Business Manager (1.0
FTE). We include in the management costs the fractional efforts of the work area
leaders and deputies and the effort of the Program Council members.
The above facilities and operational systems will be developed in a structured
manner using industry-standard software development processes, a thorough
testing program, and a program-wide configuration management system.
VAO Project Execution Plan 10 Executive Summary
Introduction
This document is the Project Execution Plan (PEP) for the Virtual Astronomical
Observatory (VAO). The VAO is operated for the National Science Foundation
(NSF) and the National Aeronautics and Space Administration (NASA) by the
Virtual Astronomical Observatory Limited Liability Company (VAO, LLC), a non-
profit business entity established by the Association of Universities for Research
in Astronomy (AURA, Inc.) and the Associated Universities, Inc. (AUI) and
chartered in the District of Columbia.
The VAO is an operational facility whose services will enable astronomers to
discover, access, integrate and analyze the vast quantities of astronomy data
available electronically. The plan responds to the requirements of the cooperative
agreement AST-0834235 between the National Science Foundation and the VAO,
LLC, and to the recommendations of the Science Council.
This agreement calls for the VAO to:
Develop robust services, built to meet community needs and developed
according to industry standards, backed by responsive user support and
training, and professional outreach programs.
Provide leadership in international cooperative efforts, coordinated by the
International Virtual Observatory Alliance (IVOA) to standardize data access
and discovery protocols.
Develop innovative Education and Public Outreach (EPO) programs that
convey to educators and students the science content of the range of data
accessible through the VAO.
Version 1.0 of the plan assumes that all subawards will be in place by Oct. 1,
2010, and we will use this date as a starting date for project development.
The plan will be updated annually according to the recommendations of the VAO
Science Council, with the approval of the VAO Board of Directors and the
acceptance of the Virtual Observatory Oversight Group (VOG) convened by NSF
and NASA.
VAO Project Execution Plan 11 Introduction
Section 1. Project Baseline
VAO Project Outcomes
Astronomy is being transformed by the vast quantities of data, models and
simulations that are becoming available to astronomers at an ever-accelerating
rate. The VAO will become the astronomer’s principal resource for discovery and
access to these data in the science initiative areas identified in this Project
Execution Plan. The VAO’s services and libraries, developed to respond to the
growing scale and complexity of modern data sets, will be indispensable tools for
astronomers integrating data sets and creating new data sets. It will therefore
accelerate archive-based scientific research (an area pioneered by the NASA
data centers), and this will be reflected in rapidly growing citations in the peer-
reviewed literature.
The VAO will collaborate and cooperate with missions, observatories and new
projects, who will be able to routinely integrate VAO libraries into their processing
environments to simplify and accelerate the development and dissemination of
new data products.
The VAO will be a pioneer in the development of the ―interactive journal paper of
the future,‖ in which self-documenting data sets reported by the authors are
curated and made accessible to readers through simple links.
The VAO will share its expertise with data archiving practitioners in other fields
and be seen as a valuable resource in advising them on developing powerful tools
and services that respond to their requirements.
The VAO will collaborate with ―citizen-science‖ and ―crowd-sourcing‖ programs
with large user bases to develop innovative outreach programs that engage
people in all walks of life, including underserved populations. The VAO will
support national curriculum standards by providing classroom materials that
address the theme of information technology.
Scientific Initiatives
The VAO will deliver science applications, tools and libraries in seven science
areas. These areas have been endorsed by the VAO Science Council, and are
described in detail below.
VAO Project Execution Plan 12 Project Baseline
VAO Portal
The VAO Portal is the central element of the VAO Web site that initially exposes
VAO users to the data discovery, data access, and other advanced research
capabilities of the VAO. Surrounding the portal are other navigation links to
important information about the VAO: documentation, how-to guides, FAQs,
calendar events, education and public outreach materials, and so on. The portal
elements are active, either taking the user to a Web-based application or invoking
that application directly. The NVO search portal, which served as a prototype, was
designed to mimic an iPhone, with icons representing various Web-based
applications (e.g., DataScope, Inventory, Directory, Open SkyQuery, VO-CLI).
But even with brief text descriptions of these functions, it has not been clear to
users just where to start or how the functions interoperate. The goal of the VAO
Portal is to provide a highly effective Web-based search facility for the
astronomical research scientist.
Every Internet user is now familiar with the ―one box‖ interface pioneered by
Google for Web searches. Interviews with NVO users indicated great interest in
there being a simple one-box interface for data discovery. Bringing this paradigm
to the VO is desirable. Google focuses primarily on text, making interpretation of
the user’s input straightforward. However, Google’s one-box interpreter is much
more sophisticated. It recognizes ―1+1‖ as an arithmetic expression and evaluates
it for you. It recognizes ―map Washington DC metro‖ as an interest in an actual
map, not a document about maps, and returns a link into the maps.google.com
service. A search on ―flights Baltimore Chicago‖ provides a link to Expedia and
automatically runs the search with the origin and destination cities inserted. All of
this also features text completion, showing the user possible complete queries
based on just a few characters having been entered. Further typing restricts the
possible queries. And, of course, Google ranks the pages it returns, showing what
it believes is most relevant to the user first.
While we have experimented with some of these technologies in the NVO, we did
not bring any of them into production. A priority for the VAO Portal is to bring
Google-like capabilities into action. Why do we not simply rely on Google? Much
of the information indexed in the VO is not in the form of documents or services
visible to Google. The availability of a collection of HST images in the archive, for
example, can only be determined by properly formatting a query to the HST
archive interface, or to its VO-compliant Simple Image Access Protocol (SIAP)
service. But much like how Google understands that ―flights Baltimore Chicago‖
maps into an Expedia query, we should be able to understand in the VO context
that a string ―images M51 HST‖ should map into a SIAP request to the Hubble
Space Telescope archive.
VAO Project Execution Plan 13 Project Baseline
Ranking search results is more problematic, as most astronomical images do not
contain sufficient metadata for an end-user to decide a priori which is better than
another for the current purposes. However, were we to expose the most
frequently used selection criteria—the extent to which the images equal the region
of interest for the user, the spatial resolution, the image extent, the bandpass, and
perhaps most important, some objective measure of image quality—then users
could easily sort and filter the results table and winnow down the selections to the
most relevant observations.
These are the capabilities we will implement in a new version of the VAO Portal.
We will move the supporting infrastructure of cone searches, SIAP requests and
registry records fully behind the scenes where they belong. Astronomers will
pose queries in terms natural to them (e.g., ―infrared spectra coma cluster‖) and
find relevant results immediately, rather than having to build a query chain (i.e.,
get a list of Coma Cluster galaxies, find sites having infrared spectra, submit list of
galaxies and find matching observations, select observations for download and
inspect). The VAO Portal will leverage modern semantic search and visualization
technologies, layered on the VO-compliant services at astronomical data centers,
to bring powerful search, discovery, and integration tools to the research
community. Portal development will include regular and frequent feedback from
end-users to assure that research expectations are being met.
Scalable Cross-Matching Facilities
The analysis and interpretation of multi-wavelength data in astronomy, especially
from surveys, relies on the comparisons of source properties (flux, morphology)
from different bandpasses, instruments and telescopes. The first step in such
analysis is to perform a spatial cross-match between source lists or catalog
entries based on position and a region of interest, such as a position uncertainty
ellipse, or a PSF size. This approach is far from straightforward, however. Unless
the source catalogs and object lists are based on data that are of very similar
spatial resolution, similar sensitivity, and fully overlapping spatial coverage, it is
difficult to assess whether an apparent match is of physical significance
(originating from the same astrophysical entity) and whether a lack of a match is
also significant (rather than a sensitivity mismatch or a region lacking coverage in
one or another catalog). Furthermore, astrophysical objects manifest themselves
in profoundly different ways in different parts of the spectrum. A radio galaxy may
show bright, extended lobes of emission with faint/unremarkable nuclear
emission, but be physically associated with a distant elliptical galaxy. Matching the
radio lobes to optical objects would give spurious results. And as deep
optical/infrared surveys have shown, the light of galaxies at increasing redshifts
eventually moves out of the optical bandpasses. This is a significant effect for
VAO Project Execution Plan 14 Project Baseline
inferring the distances of the galaxies and indicates that negative cross-matches
are as important as positive ones.
Despite these challenges, spatial cross-matching is a necessary tool of the trade
and is in high-demand by the research community. The National Virtual
Observatory (NVO) prototype service, Open SkyQuery, provided multi-band
cross-matching across catalogs using a chi-square confidence estimator that took
into account an estimate of positional uncertainties and signal-to-noise levels.
However, owing to the difficulty of estimating the computational resources needed
to compute large, multi-catalog matches (and avoiding a monopolization of CPUs
by one or a few users), we had to limit result sets to 5,000 rows. Open SkyQuery
was the most frequently used NVO application—and also most frustrating
application—because this limit constrained astronomers to studying very small
regions of the sky. An NVO pilot project to investigate cross-matching between
the Sloan Digital Sky Survey (SDSS) and the Two Micron All Sky Survey
(2MASS) catalogs showed the power of this capability, for it yielded the first
measurement of space density of T-type brown dwarfs.
It is thus a priority for the VAO to bring generalized, high-performance, large-scale
(all-sky) cross-matching into the research astronomer’s toolkit. Moreover, this
development, which is computationally more tenable now owing to advances in
database configuration pioneered within our collaboration, will be guided by
collaborations with external research teams (the Small Magellanic Cloud study,
and the CANDELS project).
Building and Analyzing Spectral Energy Distributions
Spectral energy distributions (SEDs) are one of the most fundamental tools for
understanding the physical nature of astronomical objects. SEDs are typically
constructed by assembling disparate photometric and spectroscopic
measurements from the literature and online catalogs and databases. The
NASA/IPAC Extragalactic Database (NED) constructs SEDs for extragalactic
objects through systematic scanning of the astronomical literature for new
measurements, entering those measurements and their associated metadata
(filter or bandpass, aperture, units of measurement) into the database, and then
converting the measurements to a common set of units. It is difficult, however, to
automatically correct for differences in aperture, and there are always problems
arising from errors in the published metadata. Thus it is not uncommon to see
SEDs with flux measurements in the same bandpass that are apparently
inconsistent. Are these the result of aperture differences, metadata errors, or—
more interesting astrophysically—time-varying phenomena?
VAO Project Execution Plan 15 Project Baseline
The first steps in generalizing the kinds of capabilities developed by NED are to
expand the scope to galactic astronomy and to allow for dynamic collection of
photometric and spectroscopic observations using VO protocols, potentially
including dynamic aperture photometry measurements on suitably calibrated and
documented image collections. Second, the collected data need to be presented
to the user in a dynamic, graphical form that enables exploration of the
provenance of the measurements. For example, mousing over a particular data
point should show the origin of the measurement, the aperture, bandpass, and
date of observation. Clicking on the point will take the user to the original paper
and show how the measurement was interpreted and converted to standard units.
The user could then decide to retain the point or reject it.
An extensive library of fitting functions will complete the SED toolkit, allowing
users to compare model SEDs with observations and measure the temperatures,
densities, power law exponents, etc. SEDs can also be corrected for reddening
and converted to restframe wavelengths, as appropriate. Fortunately, a
substantial body of software for various aspects of SED assembly and analysis
already exists within the VAO collaboration and with our international VO partners.
We will build on this base.
Time Domain Astronomy
The time domain is often referred to as the last frontier in astrophysics. Although
much is known about time-variable phenomena, from solar and stellar pulsations
to gamma-ray bursts, our sampling of the data in the time domain is terribly
incomplete. A number of new time domain surveys are now in progress or being
planned, most notably with the Panoramic Survey Telescope and Rapid
Response System (Pan-STARRS) and the Large Synoptic Survey Telescope
(LSST). These programs will undoubtedly give rise to discoveries of previously
unknown time-variable phenomenon, and will enrich our understanding of already-
known phenomenon by increasing our sample sizes by orders of magnitude.
The VAO connection to time domain studies is primarily in two areas: distribution
and management of transient event notices (VOEvent) and discovery, access,
and analysis of time series data. Our VOEvent infrastructure is well developed
and based on an IVOA standard. Dozens of observatories worldwide are
participating in the VOEventNet network, with transient event notices
automatically streaming onto the Internet and VOEvent subscribers listening in for
events that interest them. SkyAlert.org (a project funded by NSF and led by VO
colleagues at Caltech and NOAO) provides a clearinghouse of VOEvent notices
and allows individuals to subscribe to and/or monitor the events coming from
various event streams and feeds. As SkyAlert will be funded for another two
years, and the personnel involved with SkyAlert are also involved with VAO, we
VAO Project Execution Plan 16 Project Baseline
will not devote significant effort to transient event support in the first year of the
VAO program. We will develop a plan for integration of VOEvent support
capabilities into the main body of the VAO so that as SkyAlert funding concludes,
ongoing operations will be sustained.
There are several significant repositories of time series data, such as the Time
Series Center (TSC) at Harvard, the NASA/IPAC/NExScI Star and Exoplanet
Database (NStED), the Kepler mission archive at STScI/MAST, and the COROT
mission database from the European Space Agency. Working with our national
and international collaborators, we will assure the interoperability of these data
collections and will adapt the existing VO protocols for spectral data to time series
data to support distributed discovery and access. A rich body of time series
analysis software will be brought into a VO-compliant environment; some methods
may be developed by VAO staff, others adapted from astronomical resources like
the HTSC, and others from external code sources. Coupled with the event alerting
services, astronomers will have unprecedented access to and analysis
capabilities for the data associated with time variable phenomena in astrophysics.
In addition, several members of the VAO collaboration are involved with the
Science Organizing Committee for the International Astronomical Union
Symposium New Horizons in Time Domain Astronomy, to be held in Oxford, UK in
September 2011. This will be a prime forum for exposing the research community
to the capabilities of the VO for understanding the time domain.
Data Linking and Semantic Discovery
There is no application in astronomy today that is able to combine a bibliographic
search with a data search based on observational parameters (such as
instrument, wavelength, object classification, etc.) and semantic information about
astronomical objects into a user-friendly interface. Searches such as ―find all
papers describing UV imaging observations of clusters of galaxies,‖ or ―find all
fields of view described in Jones et al. (2010) that have been observed by HST
and Chandra‖ lead to maximum re-use of existing data, better efficiency in
allocation of rare resources (new observing time), and maximum credibility of the
research record itself. But for such a capability to be useful, it must be low latency
in the face of huge amounts of metadata so the user can work in an exploratory
mode, and when done, switch to a high bandwidth mode to be able to make
downloads automatically from the VAO infrastructure.
In the coming era of data-intensive science, it will be increasingly important to be
able to move transparently between scientific results, the data used to publish
them, and the processes used to produce them. Furthermore, we will need to be
able to establish the provenance of data sources and processes operating on this
data so that research may be repeated, variations on the research may be carried
VAO Project Execution Plan 17 Project Baseline
out, and new research on existing objects and data sets may be made easy.
Thus, any search system we design to address this challenge must work with
metadata-rich resources that have unambiguous naming, and must utilize and
formalize the link structure between resources that we, as a community, and
others in fields such as digital libraries have been developing.
A critical corollary to a system that provides full interlinking between the literature,
data, and the semantics of astronomical objects is that we need to assure the
data described in the literature is itself curated and preserved. While the
astronomy community now has a well-established system of data centers and
national observatory archives, the highly-processed data that astronomers
describe in their research publications rarely finds its way back into one of these
established archives. We need to integrate data capture, curation (validation of
content and the descriptive metadata), and preservation into the astronomical
research infrastructure.
The VAO will work with the AAS, ADS, NED, and major international data centers
such as the CDS to develop a process for data capture and validation as part of
the publication process, and for long-term storage of the digital data objects
appearing in graphical form in journal articles. Through a collaboration with the
NSF DataNet-funded Data Conservancy project, in which VAO team members are
participants, this process could be extended to the arXiv (astro-ph, which is
hosted at the Cornell University Library, and is participating in the Data
Conservancy as well). Our goal is to have the digital data represented in scholarly
publications captured, curated, and preserved, linked from the journals, and
independently discoverable and downloadable through VO interfaces.
Desktop Tool Integration
The astronomical research community utilizes a variety of desktop computer
environments and tools to do their data reduction and analysis. There are large,
general purpose systems that originated within and are supported by
organizations within astronomy, such as Image Reduction and Analysis Facility
(IRAF), Astronomical Image Processing System (AIPS), Common Astronomy
Software Applications (CASA), CIAO, and ESO-MIDAS. There are also
commercial packages, most notably IDL, and public domain environments like
Python and R, that have wide or growing use, owing to the ease of prototyping
and development. (Python is also becoming an integral part of environments such
as IRAF and CASA.) And there are a myriad of more special-purpose
applications, such as DAOPhot and SExtractor for photometry, that are in wide
use and are generally accepted as tried-and-true tools.
VAO Project Execution Plan 18 Project Baseline
These applications and environments are each a sort of private domain. You
cannot do some processing in IRAF and then pass it over to Python without
writing out a file, leaving one environment and entering the other. There are
already international studies in progress to formulate a software development
framework that would allow components from different environments like this to be
combined and interoperate, and the VO initiatives worldwide are involved in these.
But of even more immediate concern is to bring VO-based capabilities into the
environments and applications that astronomers already know, and making it as
easy as possible to share information among these tools.
There is already an IVOA standard for basic applications interoperability called
SAMP, the Simple Applications Message Protocol. SAMP allows several
independent applications running on the user’s desktop to communicate with each
other, so that, for example, a selection of objects in an image display application,
such as Aladin, automatically updates a scatter plot in a graphics application,
such as TOPCAT. We will accelerate incorporation of SAMP into astronomy
applications. In the VAO, we have emphasized Web-based applications over
download-and-install applications. Web-based applications present challenges for
the SAMP technology, although some progress has already been made. We will
focus on developing a standard approach to integrating Web-based applications
with SAMP, in collaboration with our international VO partners.
We will also work with the organizations supporting legacy applications to
integrate access to VO services directly into these systems. Data discovery,
access to VO data access protocols, and utilization of network-based storage
through, for example, the VOSpace interface definition, can all become built-in
functions. And with Web-based access methods, desktop invocations of remote
cross-matching or SED-building services can be called from within familiar
environments. Ultimately, many legacy applications and environments will gain
the capability to access remote data and services transparently and will become
more interoperable for direct use or scripting from within the desktop research
environment.
Our near-term goal is to bring the VO to astronomers, not to require astronomers
to come to the VO and think they need to learn an entirely new way of working.
Our longer-term goal is to create an environment of ―seamless astronomy,‖ from
desktop to Web, from legacy application to VAO service, from private data and
code to commodity and network-based data and computational services.
Data Mining, Statistical Analysis, and Visualization
Data mining, multivariate statistical analysis, and the emerging field of
astroinformatics are all aimed at distilling information from large, multi-
VAO Project Execution Plan 19 Project Baseline
dimensional, heterogeneous data sets. This includes identifying unique or unusual
classes of objects, estimating correlations, computing the statistical significance of
a fit to a model in the presence of missing data or data with upper limits. The
computational challenges can be enormous, but so can the barriers to use and
understanding the tools of the trade. The more sophisticated statistical methods
are not often used in astronomy, and determining which statistical test to use for
what situation requires learning a complex jargon.
An essential aspect of data mining is for the researcher to have the ability to
define samples of large data sets that meet some selection criteria and then to
preserve these samples (which may require computationally intensive database
queries to generate in the first place) in a storage space close to the data and
algorithms that will be used for further analysis. The technology for this now
exists, and the VAO will deploy VOSpace storage facilities at a number of its
organizations.
Visualization of multi-dimensional data sets will also be essential. For example,
among a million objects, of which has 50 measured parameters, how does the
astronomer get a visual sense of the significant correlations? Such tools are
available in the informatics arena but need to be introduced to and adapted for
use in the astronomy community.
Similarly, sophisticated algorithms (discriminant analysis, neural networks,
decision trees, etc.) are in wide use for automated and semi-automated (trained)
classifiers.
Because of the wide variety of applications and algorithms already available, the
VAO will first engage in a Study and Requirements phase to determine where
adaptation and integration of data mining, statistical analysis , and visualization
tools will be most effective. We will consult with groups with established expertise,
such as the Center for Astrostatistics at Penn State University and the newly
established Knowledge Discovery in Databases (KDD) group of the IVOA, and
work to bring already functioning tools into the VO framework. As these tools are
deployed in the VAO, they will be presented in the language of astronomy and
couched in terms of the practical problems astronomers routinely face. We will
provide test data sets so users can experiment with known results before applying
applications to data sets with unknown properties. We will provide guidance to
users on what to expect as the size of data sets grow, and to the extent we can
leverage computational resources within our organizations and gain access to
large-scale computational facilities on behalf of the community, provide the
capabilities for analysis of substantial subsets of major surveys.
VAO Project Execution Plan 20 Project Baseline
Science Deliverables for Year One
In year one, the VAO will deliver user applications in four of the seven science
initiative areas. These deliverables have been chosen for three reasons:
They respond to the science priorities recommended by the Science Council.
The VAO team members have very strong scientific and technical expertise in
all four initiatives.
These applications can take advantage of mature libraries, tools and data
access protocols that will support rapid development
All four applications will be accessible through common Web browsers, and will
not require software downloads or custom system configurations. The cross-
matching, SED building and time series applications will be accessible through
existing archive portals and through the VAO Web page itself.
A Development Lead will be identified for each effort before the VAO team
meeting on Sept. 29, 2010.
Finally, we will complete the integration of access to VO data into the IRAF
package. This project has been ongoing for several years without formal funding,
and has now reached the Test phase. The VAO will support testing and
preparation on user documentation.
Table 1-1 summarizes the full-time equivalents (FTEs) that will be devoted to
each application.
VAO Project Execution Plan 21 Project Baseline
Table 1-1: Summary of FTEs For the Four Science Deliverables
User Standards
Product Total
Application Support and Operations
Development FTEs
(Testing) Protocols
Portal 1.5 0.5 0.7 0.1 2.8
Cross-Matching 1.5 0.5 0.5 0.1 2.6
Building and
Analyzing 2.0 0.5 0.7 0.1 3.3
SEDs
Time Domain
1.0 0.5 0.5 0.1 2.1
Astronomy
Desktop Tools 0 0.4 0 0.1 0.5
Total FTEs 6.0 2.4 2.4 0.5 11.3
Portal
DELIVERABLE (JUNE 2011)
A new VAO search portal will be deployed within the framework of the VAO Web
site. The first portal release will provide the science user with basic search and
retrieval capability through a gateway to VAO-enabled applications and services.
The portal capability will leverage existing connectivity to VO-enabled search and
visualization applications in addition to key scientific research services for finding
data collections and reference data sources. The portal will serve as a user-
friendly VO gateway for the first time user as well as for experienced clientele. It
will offer links to visualization through tools that astronomers already use day-to-
day, such as Aladin and DS9.
Users will provide input as free-form text, and the portal will parse the text into
scientifically meaningful results; this process will be transparent to users. Users
will be channeled by the portal to specialized applications for retrieval of data and
metadata, and to relevant technical documentation. For example, a source name
or set of coordinates input to the service will return an appropriate set of data
provider services that contain the source data. Input on a scientific topic such as
―binary star‖ or ―high redshift‖ will return references to data collections containing
the topic. Finally, the portal will support fast access to image data collections
over wide areas.
VAO Project Execution Plan 22 Project Baseline
TECHNICAL BACKGROUND
The VAO team has many years experience in developing user portals and user
interfaces. This expertise covers all aspects of portal development. This includes
development of client-side capabilities, including user interfaces and capabilities
for filtering and organizing results, and server-side capabilities that underpin the
portal, such as organization of data for fast searches, development of standards
to access distributed data, and development of command-line tools that access
VO data sets. The team’s expertise derives from many years of supporting
archive and data centers and the NVO.
The VAO Portal will take advantage of existing technologies developed by the
NVO and the IVOA. These include existing VO image, spectral and source
catalog access services that are in operational use, and technologies developed
by the NVO for organization, filtering and management of large return data sets,
and for providing fast access to large data collections.
SCOPE OF WORK
The portal will undergo a detailed design phase that will identify the components
needed to support the portal. We anticipate substantial re-use of existing
components. The bulk of the effort will be in the development and extensive
testing of the portal translation service, which will likely be built from the ground
up.
Scalable Cross-Matching Facility
DELIVERABLE (AUGUST 2011)
The VAO will deliver a service that supports pair-wise positional cross-matching
between very large catalogs, containing more than 1 billion records. This service
will be accessible both through Web forms and a program-friendly interface. It will
allow end-users to upload their own catalog of sources to cross-match with
archived catalogs.
The service recognizes the distinction between cross-matching and cross-
identification of catalog sources. A cross-identification (cross-ID) is a reliable and
physically valid correspondence between two or more observations of the same
astrophysical source, often at different frequencies and epochs. The
establishment of cross-IDs requires scientific and statistical analysis beyond
spatial proximity, including consideration of differing spatial resolutions,
astrometric uncertainties, source confusion (in dense areas), projected sizes (for
VAO Project Execution Plan 23 Project Baseline
resolved sources), redshift/distance measurements, and proper motions (where
available and applicable).
A cross-match (or cross-comparison) is a candidate cross-ID based solely on
positional proximity on the sky. The VAO service will return lists of cross-matches
between catalogs with the option of including cross-match parameters (e.g.
distance) in the output.
The architecture will be designed to accommodate algorithms that perform cross-
identifications and the ability to scale to much larger catalogs, containing tens of
billions of records.
TECHNICAL BACKGROUND
Johns Hopkins University (JHU) and the Infrared Processing and Analysis Center
(IPAC) each brings over 10 years experience in this field. They routinely manage
catalogs and databases containing billions of sources, and pioneered
mechanisms for efficient querying against them. IPAC has investigated the use of
open source frameworks to build prototype cross-match products between the
USNO-B and 2MASS catalogs. IPAC was the technical lead on an NVO project
that cross-matched the SDSS and 2MASS catalogs and which led to the first
measurement of the space densities of T-type brown dwarfs. JHU developed the
prototype NVO cross-matching service, hosted several large all-sky cross-
identification products, including 2MASS vs. SDSS, SDSS vs. GALEX and SDSS
vs. USNO-B, and is currently working on cross-identifying the Hubble Legacy
Archive data with other data sets.
The cross-match service requires access to distributed tables and access to an
area for staging the results of the cross-match. The service will take advantage of
the Table Access Protocol (TAP), which specifies how to access distributed tables
through a common program interface, and VOSpace, an interface to a secure
area for staging data for use by distributed data access services.
SCOPE OF WORK
The following tasks will be performed:
Design architecture for scalable cross-matching and test to prove the
architecture.
Develop cross-matching engine.
Specify and implement a TAP-compliant interface that can serve a cross-
comparison service.
VAO Project Execution Plan 24 Project Baseline
Implement VOSpace, an interface to a secure, distributed staging area for
cross-matched data sets.
Define and deploy REST-based Web service that supports cross-comparison
between user tables and catalogs, quite likely as an extension to TAP.
Develop user interface that accesses the cross-comparison engine through or
in close conjunction with TAP interfaces.
Deploy pair-wise pre-computed cross-matches. Commonly used cross-
matches (such as SDSS vs. 2MASS) need only be performed once and the
results served as catalog products in their own right.
Building and Analyzing SEDs
DELIVERABLES (JULY 2011)
The VAO will enhance the interoperability and capabilities of existing SEDs,
visualization and spectral fitting tools. There will be three science deliverables:
1. SED Interaction and Analysis: The NASA/IPAC Extragalactic Database
(NED) creates spectral energy distributions (SEDs) from a fusion of
photometric measurements spanning radio to gamma ray frequencies (as data
are available), underpinned by robust and continuously improving source
cross-identifications from over 75,000 journal articles and sky survey catalogs.
Original published units are preserved, and they are converted to common flux
density units with extensive metadata to provide a dynamic SED building
service. The SEDs are necessarily constructed from observations utilizing
widely differing telescopes, detectors, filters, apertures, etc. The VAO will
enable users to utilize all this information to perform the next steps required for
scientific analysis, by making the SED building process interactive and
dynamic, and by facilitating SED comparison and fitting capabilities. Users will
click on points in an SED and be presented with their known properties,
attributes, and provenance. Measurements that do not appear to be physically
consistent with others (i.e., due to differing aperture or beam sizes) can be
omitted from the SED, and the physical units of the data can be changed. The
tool will make it easy to fold new measurements into the analysis to integrate
with published data from NED. Users will also be able to compare the SEDS
of different galaxies and active galactic nuclei (AGN), and compare observed
SEDs with a library of population synthesis models.
2. SED Fitting: The Harvard-Smithsonian Center for Astrophysics developed
Sherpa, a powerful general-purpose spectral fitting toolkit, developed originally
for the Chandra mission but applicable over a broad wavelength range. It can
fit several data sets in a single SED simultaneously, for Poisson and for
Gaussian statistics. The VAO will make Sherpa seamlessly accessible to
applications within the astronomical community and as a first implementation
will support spectral fitting of SEDs accessible through NED.
VAO Project Execution Plan 25 Project Baseline
3. SED Visualization: STScI has developed and maintains Specview, a
powerful tool for interactive visualization of 1-D spectra. It supports data
quality handling, flexible spectral units conversions, custom plotting attributes,
plot annotations, tiled plots, hardcopy to JPEG files and PostScript file or
printer, etc. It also supports quick measurements on spectra. As with Sherpa,
VAO will make Specview seamlessly accessible to applications within the
astronomical community.
Interoperability between the fitting functions of Sherpa, the visualization and
analysis capabilities of Specview, and the extensive SED content of NED will
deliver powerful new capabilities that directly support the VAO science initiatives.
TECHNICAL BACKGROUND
The Chandra mission maintains the Sherpa toolkit as part of its operational
activities, and NED constructs, validates and serves SEDs for extragalactic
objects. STScI maintains the Specview Java applet. Specview SED analysis and
fitting tasks require a IVOA Spectrum/SED Data Model and function library,
currently under development by SAO in cooperation with the VAO.
SCOPE OF WORK
The following tasks will be performed:
1. Complete the development of the IVOA Spectrum/SED Data Model and
function library.
2. VO-enable SEDs in NED using the IVOA Spectrum/SED Data Model.
3. Deploy Sherpa through the IVOA Spectrum/SED Data Model.
4. Evaluate existing Web-based and desktop-based applications suitable for
SED visualization and fitting, such as Specview (STScI) and SPLAT (UK).
5. Develop the interactive capabilities with SEDs, initially with Specview.
6. Develop the ability to compare galaxy SEDs and synthesis models.
7. Support seamless spectral fitting of SEDs from NED utilizing Sherpa.
Time Series Astronomy
DELIVERABLE (SEPTEMBER 2011)
The VAO will deliver Web services that will provide seamless access to all the
light curves from time series data sets housed at NStED, Harvard Time Series
VAO Project Execution Plan 26 Project Baseline
Center, and MAST. These time series data include the public data released by
the active missions Kepler and COROT. These missions are transforming the
fields of exoplanet discovery and stellar variability. The service will enable users
to compute periodograms of time series data sets and discover light curves with
given characteristics via an automated algorithm.
TECHNICAL BACKGROUND
Since 2006, NStED is NASA’s archive of record for exoplanet data and their host
stars, and for time series data sets designed to discover transiting planets. It is the
U.S. portal for the COROT mission, and its holdings include a copy of the public
Kepler data sets. It has developed and maintains the periodogram algorithm. The
Time Series Center (TSC) at Smithsonian Astrophysical Observatory (SAO) is an
interdisciplinary effort that hosts the world's largest collection of time series data
sets and develops algorithms to understand various aspects of those time series.
It has deployed and maintains the classification algorithm. Between them, NStED
and TSC serve over 2.5 million light curves.
The deliverable service involves making interoperable the two data analysis
applications described above.
SCOPE OF WORK
Development of a VO time series data access protocol, to support queries to
the two light curve data sources in a uniform fashion.
Make NStED and TSC time series tools mutually interoperable.
Deploy a user interface for end-users, accessible from the VAO, NStED, and
TSC Web pages.
Desktop Tool Integration
DELIVERABLE (AUGUST 2011)
The VOA will provide seamless access to VO-data in the IRAF data reduction
package.
TECHNICAL BACKGROUND
This development effort, begun several years ago without formal funding, has
produced a collection of command-line tools that support access to VO-data.
These tools are designed for integration into existing software packages.
VAO Project Execution Plan 27 Project Baseline
SCOPE OF WORK
Test the integration of VO-access tools into IRAF.
Deliver user documentation.
Community Involvement in Science Deliverables
To help guide the science deliverables and assure their implementation meets the
needs of the research community, the VAO will be proactive in engaging the
community to evaluate services and support science validation of results. We
have already engaged two research groups having diverse research programs
that will rely on VO infrastructure:
In order to understand the three-dimensional structure of the Small Magellanic
Cloud (SMC), its interactions with the Large Magellanic Cloud, and more
generally, understand the dynamics of interactions in dwarf galaxies and their
impact on galaxy evolution, we will generate a cross-matched multi-wavelength
directory of objects with a 10-degree radius of the SMC. This will rely on new
cross-matching algorithms that can accommodate multiplicities of matches and
different spatial resolutions in different bandpasses. These algorithms will be of
general utility to the community in dealing with current and next generation
survey data. B. Madore (Carnegie Observatories) will be our principle liaison to
this research effort.
Understanding the evolution of galaxies and black holes from z~8 to z~1.5 is
the goal of the Hubble Space Telescope (HST) Multi-Cycle Treasury program,
CANDELS, led by S. Faber (University of California, Santa Cruz) and H.
Ferguson (STScI). Like the SMC study, CANDELS will rely on robust cross-
matching across multi-wavelength observations, from X-ray to radio, using a
wealth of deep observations from HST and many ground-based observatories.
In addition, new tools for building and analyzing spectral energy distributions
(SEDs), and combining SEDs with cross-matching to find samples
uncontaminated by confusion, will be a critical component of these studies.
Other VAO-enabled research will include multi-wavelength morphological
classification, dynamic image cut-out services (registered and consistently
sampled across bandpasses).
Study Initiatives for Year One
For three science initiative areas, we will perform studies that will identify science
applications to be developed in years two and beyond, that have high science
return and yet can be developed within the budget of the VAO. The reports will be
presented to the Science Council to inform their recommendations on science
priorities.
VAO Project Execution Plan 28 Project Baseline
1. Data Mining (September 2011)
We will study the components necessary for generating and analyzing sample
sets with an emphasis on applying data mining and statistical techniques. We will
specifically engage experts in the community for input into this study. This study
will include:
Designing and prototyping a portal-based tool for aggregating data on sources
and selecting them via scientific constraints.
Identifying data mining applications that can be readily packaged for broad use
by the community (e.g. a galaxy classifier).
Identifying and testing existing statistical and data mining tools that can be
leveraged to apply data mining and analysis techniques to VO-accessible data.
2. Data Curation and Preservation (September 2011)
In a collaborative study and pilot project between SAO (ADS), IPAC (NED), AAS,
ADEC members, and NSF Archives, we will:
Generate requirements for data publishing tools that can be integrated into a
journal publishing workflow.
Create a prototype repository system addressing nomenclature, publishing,
linking, and preservation of datasets (both user-supplied and archive-curated).
Collaborate with the Data Conservancy project and arXiv as that data capture
project matures.
We will investigate topic-based literature search and faceted browsing of results
through views on objects, object types, data sets, and wavelengths. We will
investigate two types of searches:
Search the literature by topic, refine results based on object types, data
products and wavelengths.
Search the sky by object name(s) or position, filter related literature by
keyword(s).
3. Time Series Astronomy (Transient Sources) (September 2011)
The transient sources community is converging on using VOEvent, the VO-
defined protocol, for representing, transmitting, publishing, and archiving the
discovery of a transient celestial event. It also wants robust and reliable tools to
enable production of and subscription to these packets. SkyAlert is a good
VAO Project Execution Plan 29 Project Baseline
prototype of the type of requested broker service: users use a Web interface to
define tasks - messaging, fetching data from archives, running a computation, etc.
- that are performed when an appropriate event is received. SkyAlert is sponsored
independently of the VAO until Fall 2011. We intend to integrate SkyAlert into the
VAO but we first need to understand how to best place it on a sound, continual
basis, aligned specifically with VAO requirements.
We will undertake a comprehensive review of the existing SkyAlert code base;
primarily, to determine where re-engineering is required to conform with VAO
coding standards and practices—appropriate test suites, for example—but also to
recommend enhancements to improve and extend functionality, etc. We will
analyze SkyAlert's existing Operations Plan to ensure it is both compatible with
the VAO Operations Plan and also manageable by the VAO Operations team; for
example, understanding what level of effort is necessary to meet specific quality-
of-service levels. The result of this study phase will be a report detailing the
results of these analyses and a proposed schedule, together with level of effort
estimates, for integrating SkyAlert into the VAO. The report will also include a risk
assessment for the integration activity and a contingency plan.
Science Deliverables for Years Two and Three
The description of science deliverables in the years beyond the first cannot be as
detailed or well defined as those projects in the immediate future of the VAO.
However, it is possible to outline likely areas of emphasis and activity in the years
immediately following the startup of the VAO.
1. Continued Development of Ongoing Projects
Foremost among these is the further development of the VAO Portal. This is a key
project for the VAO because it is the principal interface between the VAO and the
astronomy research community. Hence, it is essential the portal remain relevant,
stable, and easy to use. The first-year portal efforts described in the previous
section do not have sufficient resources to develop the scientific capability of the
portal fully, and thus work will continue on this project in years two and three. In
addition to the realization of planned development of the portal, it is very likely that
continuing experience with, and feedback from, the astronomy community will
strongly influence the direction of future evolution of the VAO Portal. It is foreseen
that this project will continue to have high priority within the VAO in years two and
three.
Other ongoing projects which are likely to require further development in years
two and three, due to their complexity and to the support they receive from the
VAO Project Execution Plan 30 Project Baseline
research community, are the general and expanded cross-matching project and
the time series data exploration. The cross-match capability will be under
increasing pressure to expand as more and more data become available from
current and planned large-scale survey projects. Fast and very large-scale cross-
matching will become a much more commonly used tool in the near future, and
there is likely to be significant demand for further development of this VAO
capability. The time series tools will gain more support as this newly emerging
area of research activity grows, and, in addition, there is the currently planned
increase of VAO support in this area as the SkyAlert project ramps down.
2. Evolution of Projects Under Study in Year One
There are three projects currently planned for study in year one of the VAO
operations: (a) linking between journal articles and data, (b) data mining and
statistical tools, and (c) integrated desktop tools. The first of these already has
initiatives underway in other areas, and as these move to maturation there will be
some demand for additional VAO effort in this area.
Data mining has recently become a formal area of inquiry through the formation of
an IVOA Interest Group, and it is very likely that this area of research will become
better defined as this international group begins its work. This area is in need of
such further definition, and as this occurs, possible roles for fruitful VAO
involvement will also become more clear. The general area of integrated desktop
tools is also, at present, not well defined in terms of specific goals and objectives.
However, the initial explorations of this topic appear very promising, and it may
well be that this will become a very popular, productive and rapidly growing area
of astronomical data manipulation. If so, significant VAO involvement will be
essential.
3. Integration of Theory and Observational Data – The Virtual Telescope
Progress in theoretical astrophysics and in the quality of astronomical data have
now reached a level of sophistication that significant comparisons can be made
between observations and theory, to the extent that definitive distinctions can be
made among various astrophysical processes. These advances are not just
limited to massive, large-scale numerical simulations, but also include new and
sophisticated techniques for calculating the evolution of stellar interiors, the
interstellar medium, and the formation of planetary systems. This synergy
between theory and observation will continue to grow as VO-related tools and
applications become more capable, and the VAO will need to provide leadership
in this area.
VAO Project Execution Plan 31 Project Baseline
One example of this fruitful synthesis of theory and observation is the ―virtual
telescope‖ that will permit ―observations‖ of the observable signals emerging from
theory calculations to be made with a virtual telescope that employs current
telescope and instrument characteristics. This concept is just one of many that
could evolve from the coming interaction between large-scale theory and
observational databases, and such initiatives would naturally benefit from
interaction and collaboration with the VAO.
4. New Initiatives from Community Input
A major source of advice and direction for new initiatives for the VAO in years two
and three will come from the astronomy community. In particular, it is anticipated
that the VAO Science Council will continue to provide guidance on VO science
priorities during this period.
A particularly fruitful source of direction for new initiatives may come from specific
scientific collaborations between the VAO and the community. Two of these that
will begin in year one have been previously described, and it is anticipated that
additional collaborations will be formed in the years following the first. These in-
depth collaborations may prove to be very valuable proving grounds for new VO
tools, and they will also serve to identify areas where additional tools and
capabilities are clearly needed.
Finally, we will solicit the community on further developments of desktop tool
integration.
Finally, access to VO data in IRAF will act as a proving ground for integration of
VO access into desktop tools, and we will solicit the community on new
capabilities and on integration of VO access into extant tools. In particular, we will
investigate the value of providing VO-access to the ―R‖ statistical analysis
package, at the request of the Science Council.
Facility Milestones and Deliverables
This section lists facility milestones and deliverables, that is, those milestones and
deliverables that support the building of the VAO as a facility, that support
professional outreach, and that support development of EPO services. The
milestones are organized by WBS area. Some of these milestones are roll-ups of
several related deliverables, and individual sections describe them in more detail,
as appropriate.
VAO Project Execution Plan 32 Project Baseline
Management
Science Council Meeting 2010 March 2010
Technical Response to Science Council April 2010
Meeting
Project Execution Plan (this document) August 2010
First Quarterly Report January 2011
Second Quarterly Report March 2011
Third Quarterly Report June 2011
Annual Report September 2011
Science Council Meeting September 2011
VAO Oversight Group October 2011
User Support
Develop first release of Web pages December 2010
Publish standards and guidelines for December 2010
testing and documentation
Support VAO exhibit at AAS, Seattle January 2011
Deploy Operational Help Desk and Ticket
March 2011
Tracking System
Establish User Forum, including
May 2011
guidebooks to the VAO
Operations
Monitoring Service May 2011
Deliver Configuration Management March 2011
System
Plan for mirroring and backup July 2011
VAO Project Execution Plan 33 Project Baseline
Standards and Protocols
Represent VAO at Winter IVOA Meeting December 2010
Deliver protocols, reference May 2011
implementations and data models to
support deliverables: Time Series
Protocol, TAP Parameter Query, Tap
reference implementation, SED Data
Model
Represent VAO at Spring IVOA Meeting May 2010
Technology Assessment
Reports delivered as commissioned
Education and Public Outreach
Develop work plan in collaboration with January 2011
partners
Deploy EPO Web site May 2011
Curriculum support tool for the K-12 September 2011
community to tells the VAO technology
story
Work Breakdown Structure
The VAO science services will be built and operated as part of a managed facility
that will deliver robust services, responsive user support, and will engage the
scientific community in designing, maintaining and improving its services.
The work of building and operating a facility will be performed under a Work
Breakdown Structure (WBS) that contains eight task areas. The task areas will be
managed by a Lead and Deputy who will report to the Program Manager, and
through the Program Manager to the Director.
1. Management
2. Operations
VAO Project Execution Plan 34 Project Baseline
3. User Support
4. Product Development
5. Standards and Protocols
6. Data Preservation and Curation
7. Technology Assessment
8. Education and Public Outreach
VAO Project Execution Plan 35 Project Baseline
Staffing
Table 1-2 below summarizes the staffing levels of each work area, organized by
participating organizations.
Table 1-2: Work Breakdown Structure Staffing Levels
Organization
HEASARC
VAO (AUI)
WBS
Percent
NOAO
NRAO
CACR
NCSA
STScI
Total
IPAC
SAO
JHU
1. Management .30 .30 .60 .10 .20 .35 .10 .20 1.25 1.0 4.40 16%
2. Operations .20 1.15 .30 .50 .20 .20 .40 .40 .40 0 3.95 15%
3. User
.40 .15 0 0 0 1.80 .40 .40 .30 0 3.55 13%
Support
4. Product
.90 .80 1.70 1.50 .60 .40 .50 .70 1.20 0 8.20 30%
Development
5. Standards &
0.4 .10 0 0 .50 .20 1.40 1.0 .20 0 3.70 14%
Protocols
6. Data
0 0 .50 0 0 0 0 .70 0 0 1.20 4%
Preservation
7. Technology
.20 0 0 .20 0.2 0 0 0 0 0 .50 2%
Assessment
8. Education &
Public 0 .25 0 .20 0 0 0 .10 .85 0 1.40 5%
Outreach
Total 2.40 2.75 3.10 2.50 1.70 2.95 2.80 3.50 4.20 1.0 26.90 84%
Percent 9% 10% 12% 9% 6% 11% 10% 13% 16% 4% 100%
Notes: STScI figures include both NSF- and NASA-funded effort.
HEASARC and IPAC effort is NASA-funded.
VAO Project Execution Plan 36 Project Baseline
Metrics
The VAO will record metrics that measure the growth in scientific usage of its
services, responsiveness to users, and quality of its services. The metrics will be
reported to NSF and NASA as part of the Quarterly and Annual Reports.
The following metrics will be collected quarterly, and each quarter’s records will be
added to cumulative records that will be kept over the life of the VAO:
Number of citations to the VAO or VAO services in peer-reviewed literature
Number of acknowledgements of the use of the VAO in Web pages or
documents reporting new data products
Number of queries to VAO services, broken down by service
Fraction of time that VAO services are operational, broken down by service
Volume of data downloaded from queries to VAO services
The number of tickets submitted to the Help Desk, broken down by defects and
requests for assistance
Length of time to close tickets, broken down by defects and requests for
assistance
Number of defects found in testing each VAO service, and the number
corrected. Report the trend in these values with release number
Number of milestones delivered and delivery date, compared with plan
The VAO will maintain records of missions and projects that have integrated VO
tools and services into their processing environments to support efficient
production and dissemination of products. Where possible, the VAO will work with
these projects to assess gains in efficiency and performance and cost savings
that accrued through the use of VO tools.
The VAO will conduct periodic online surveys to assess user satisfaction. The
results will be reported to the NSF and NASA.
The VAO will measure the efficiency of its processes as the basis for developing
more efficient development practices. Measurements will include items such as:
completion times for individual tasks vs. expected times; time spent on each
phase of life cycle vs. expected times; and time to prepare for reviews.
VAO Project Execution Plan 37 Project Baseline
Schedule
Figure 1-1, on the next page, summarizes the schedule of deliverables and
milestones described in this section. The schedule assumes a project start date of
Oct. 1, 2010.
VAO Project Execution Plan 38 Project Baseline
Figure 1-1: VAO Schedule of Deliverables and Milestones
VAO Project Execution Plan 39 Project Baseline
Section 2. Management
Governance
The functions of the Board, Directorate, Program Council, and Science Council
are described in this section of the document.
The management structure of the VAO is shown in Figure 2-1, below.
Figure 2-1: VAO Management Structure
VAO Project Execution Plan 40 Management
The seven primary work areas have been designated with a Lead and a Deputy,
as described in the following table below. Technology Assessment and Education
and Public Outreach only have lead positions in accord with the level of effort
available to work in these areas. The EPO Lead will be hired as soon as possible
at STScI.
Table 2-1: VAO Work Areas and Designated Leads
Focus Lead Deputy
Operations T. McGlynn, A. Thakar, JHU
HEASARC
E. Stobie, NOAO M. Nieto-Santisteban,
User Support
STScI
Product Development R. Plante, NCSA G. Greene, STScI
Standards and Protocols M. Graham, Caltech D. Tody, NRAO
Data Preservation and J. Mazzarella,
A. Rots, SAO
Curation IPAC/NED
Technology Assessment A. Mahabal, Caltech N/A
Education and Public TBD N/A
Outreach
Management Structure
Direct management oversight of the VAO is provided by the Board of Directors of
a limited liability company, Virtual Astronomical Observatory, LLC, which we have
established solely for the purpose of managing and operating the VAO. This
company has been established by the Associated Universities, Inc. (AUI) and the
Associated Universities for Research in Astronomy, Inc. (AURA).
AUI, responsible for operating the National Radio Astronomy Observatory (NRAO)
and for the North American interests in the Atacama Large Millimeter Array
(ALMA), and AURA, responsible for operating the National Optical Astronomy
Observatory (NOAO), the U.S. interest in the Gemini Observatory, the National
Solar Observatory (NSO), and the Space Telescope Science Institute (STScI),
together support the research interests of many U.S. astronomers and each has
over 50 years of experience in managing national astronomical facilities for the
VAO Project Execution Plan 41 Management
community on behalf of NSF and/or NASA. Most of the institutions in our proposal
team have ties to AURA and/or AUI.
AUI and AURA will fully vest oversight responsibility for the VAO in its own Board.
Governance matters of the VAO, LLC are documented at
http://www.aui.edu/vao.php.
Roles and Responsibilities
The Board of Directors has seven members. AUI and AURA each appoint three
members, including the presidents of AUI (Dr. Ethan Schreier) and AURA (Dr.
William Smith) who serve ex officio. Prof. Jay Gallagher (University of Wisconsin)
has been appointed as the first chair; subsequent chairs will be selected by the
other six board members.
VAO, LLC Board of Directors
Dr. Jay Gallagher (Chair) University of Wisconsin
Dr. Ethan Schreier (ex officio) Associated Universities, Inc. (AUI)
Associated Universities for Research in
Dr. William Smith (ex officio)
Astronomy, Inc. (AURA)
Dr. Bruce Margon University of California, Santa Cruz
Dr. Catherine Pilachowski Indiana University
Dr. Roscoe Giles Boston University
Dr. Scott Kirkpatrick Hebrew University, Jerusalem
Board members serve three-year terms, and the terms of the initial members are
staggered so the Board does not turn over completely at one time. The Board's
primary responsibility is to assure the overall scientific merit of the facility, and,
through the review and approval of an annual Project Execution Plan prepared by
the Director (i.e., this document), assure that resources are being used effectively
and efficiently.
The Board appoints a full-time Director of the facility. AURA and AUI have agreed
that Robert Hanisch (STScI) will serve as the first Director. The first term of the
Director is three years. Subsequent terms will be five years and are renewable.
The Director is responsible for producing an Annual Operating Plan (i.e., this
document) for Board approval, for the appointment of all senior personnel, and for
all matters related to the conduct of the facility. The Director is the primary, though
not exclusive, contact with international VO development and support projects.
VAO Project Execution Plan 42 Management
Senior Management and Staff
The VAO senior management team comprises the Director’s office staff (Director,
Program Manager, Project Scientist, Technology Advisor, Business Manager) and
Leads of the work areas. The senior management team meets by weekly telecon.
The status of all work areas is reviewed regularly, with weekly status highlights
and monthly reviews focusing on progress against the annual Project Execution
Plan and monitoring of work against organizational deliverables. The Program
Manager and work area leads support the Director in preparing quarterly and
annual reports. These reports cover all VAO-related activities, both NSF and
NASA funded, and are intended to be responsive to both agencies. Reports are
submitted through NSF's FastLane system.
Business administration services and general administrative support are provided
through a subaward to AUI. There is a dedicated Business Manager who is
responsible for the placement, monitoring, and reporting for all subawards and all
financial dealings with NSF. The VAO, LLC holds the funds that support the
governance and management of the program (Board, Science Council, and
Program Council meetings, professional training and outreach events, etc.).
The staff in each work area also meet regularly by telecons convened by the Work
Area Leads. Discussions and decisions are documented. All work areas maintain
work plans and schedules that are visible throughout the facility, and long-term
tasks are broken into smaller, traceable components. Software products having
distributed components or that are required to interoperate with other applications
or components have well-defined interfaces. Work groups and project teams
(which may cross work area boundaries) communicate by email and telecon
discussions as needed, and conduct face-to-face meetings and work sessions.
Program Council
The VAO engages its collaborating organizations at two levels. The Program
Council (PC) is composed of those senior managers at each core organization
who have management and financial responsibility for the VAO work being done
at that organization. The members of the PC work with the Program Manager and
Director to define work packages and allocate budgets for the annual Project
Execution Plan. If the leaders of the various technical work areas are not satisfied
with the quality or quantity of work being done by an organization, they come to
the Program Manager, who then works with the PC to resolve the problem. Some
members of the PC may also be themselves directly responsible for a VAO work
area, but since most work areas involve several organizations, the PC member
must be concerned with all tasks being done by his or her organization.
VAO Project Execution Plan 43 Management
Team Meetings and Communications
The VAO facility team has face-to-face meetings at least twice per year. The
purpose of these meetings is to assure program-wide awareness of progress and
problems. These meetings are coordinated with meetings of the Science Council,
Program Council, and Board of Directors in order to economize on time and
travel.
Effective management of this distributed facility requires efficient and innovative
methods of communication. We make extensive use of telecons and a program-
wide wiki site. We will explore and adopt new communications technologies, such
as blogs, instant messaging, Internet-based videoconferences, and a VAO-
dedicated chat room.
External Collaborations
The VAO collaborating organizations include the major data archive centers for
NASA’s space astronomy missions and the major federally funded (NSF) ground-
based national observatories, as well as prominent research universities and a
national supercomputer facility. We also have institutional associates with whom
we have agreed to collaborate and to include in activities such as team meetings,
requirements analysis, prototyping, and evaluation and testing. These include the
National Astronomy and Ionosphere Center/Arecibo Observatory, the Gemini
Observatory, the Panoramic Survey Telescope And Rapid Response System
(Pan-STARRS), and the Large Synoptic Survey Telescope (LSST). We will share
information and experiences with VO-like initiatives in related fields, such as the
NASA Planetary Data System and the NASA Virtual Solar-Terrestrial
Observatory. The National Center for Supercomputing Applications (NCSA) at the
University of Illinois is a core collaborator in the VAO, but also an external partner
in terms of access to large-scale computational and storage resources. Microsoft
Research is an ongoing external collaborator, with VO data access protocols fully
integrated into the WorldWide Telescope (WWT) application. Through the VAO
core organizations we have links to facilities under construction, such as the
Expanded Very Large Array (EVLA) and Atacama Large Millimeter Array (ALMA),
and we will make contacts with other major development programs to discuss how
they can benefit from take-up of VO standards and libraries and how the
community can benefit by their provision of VO-compatible data discovery and
access protocols.
A few individuals are primary contacts and have provided assurances of some
level of in-kind support for collaborations with the VAO, including Alyssa
Goodman (Harvard), Alberto Accomazzi (Director, Astrophysics Data System),
Pavlos Protopapas (Time Series Center), and Jonathan Fay (Microsoft Research).
VAO Project Execution Plan 44 Management
VAO Oversight Group
As stated in the cooperative agreement, the NSF Division of Astronomical
Sciences (AST) will establish a VAO Oversight Group (VOG) to provide oversight
of the VAO and act as a point-of-contact NSF and NASA (the Agencies) and the
Project Director. The VOG will provide guidance in defining and refining the
structure of the Project Execution Plan and will approve annual updates to the
document.
The VOG may conduct a review of the project at least once per year. During the
first year of funding, the VOG will carry out an in-depth review of project direction,
management, cost and schedule, with possible assistance from an external panel
of experts. The VAO should be prepared to show a successful change of
emphasis towards user support. The review may result in recommending the
redirection of funds, alteration of future-year project spending profiles, and/or
changes in levels of further support. The terms, conditions, and schedule of this
review will be mutually agreed between the VOG and the Project Office.
Additional reviews may be carried out if deemed necessary by the VOG. All
reviews will be held on mutually agreed-upon dates and locations.
The VOG will review the performance of institutions with subawards from the
Project Office, if any, via regularly scheduled reports and possible site visits.
However, because the project has established its own governance structure, the
VOG may choose at its discretion to rely -- to a greater or lesser extent -- on those
mechanisms for any of its review requirements. The purpose of the reviews is to
gauge progress towards a working astronomical observatory supporting research
and education.
Should additional oversight be required, the VAO Board, in consultation, may
appoint a Visiting Committee to assess progress against the PEP and to evaluate
the overall effectiveness of the organization.
Science Council
A Science Council (SC), lead by Giuseppina Fabbiano of Smithsonian
Astrophysical Observatory (SAO), provides guidance to the Director and the
management team on matters of scientific priority. The SC is advisory to the
Director, who determines to what extent the SC’s inputs can be addressed given
available resources.
The SC will meet at least once annually and have regular teleconferences and
email communications, providing both formal and informal input. The SC Chair
provides input and feedback on the annual Project Execution Plan in consultation
VAO Project Execution Plan 45 Management
with the SC. The VAO senior management team provides the SC with metrics
showing the use and impact of the VAO for astronomical research.
Reporting
Quarterly reports will be delivered to the NSF via FastLane. The Fourth Quarterly
Report will be submitted through the FastLane Annual Progress Report module,
and will conform to NSF requirements on annual reports. Reports will be
submitted 30 days following the end of the reporting period. Ad hoc reports will be
delivered as requested by NSF and NASA.
Business and Financial Management
The Virtual Astronomical Observatory, LLC (Limited Liability Company) is a not-
for-profit 501(c)(3) organization registered in the District of Columbia on March 20,
2008. AUI and AURA are equal partners in the VAO, LLC and have filed
performance guarantee letters with NSF that assure "(a) the full and prompt
payment and performance of all obligations, accrued and executory, which
Contractor presently or hereafter may have to the Government under the
Contract, and (b) the full and prompt payment and performance by Contractor of
all other obligations and liabilities of Contractor to the Government, fixed or
contingent, due or to become due, direct or indirect, now existing or hereafter and
howsoever arising or incurred under the Contract." The full text of these letters is
on file with the NSF Program Executive.
AUI and AURA have a formal Operating Agreement, signed by the presidents of
these organizations on March 20, 2008, that defines the management and
governance structure of the VAO (described in Management section of this
document).
The principal place of business of the VAO, LLC is 1400 16th Street NW, Suite
730, Washington, DC 20036.
The officers of the VAO, LLC are:
Robert J. Hanisch, Director (STScI)
Cynthia Allen, Secretary (AUI)
Deborah Narcisso, Treasurer (AURA)
VAO Project Execution Plan 46 Management
The financial and business operations of the VAO, LLC are managed through a
subaward to AUI, who employ a Business Manager for this purpose. Specifically,
the Business Manager is responsible for:
Monitoring revenues and expenses of the VAO, LLC account and reviewing the
monthly general ledger.
Managing daily operations of the accounts payable function and monitoring
accounts receivable.
Assisting in development of budgets and forecasts.
Providing financial reports and analyses.
Assisting with annual financial statement audits and A-133 compliance audits
and tax filings.
Managing all financial aspects of the Cooperative Agreement and subaward
administration.
Ensuring that NSF terms and conditions for the Cooperative Agreement are
followed.
Analyzing costs listed on subaward invoices and determining allow-ability
based on OMB A-122, and resolving invoice problems.
The VAO general ledger is maintained within JD Edwards Enterprise One General
Ledger software on a secure server operated by AUI's National Radio Astronomy
Observatory (NRAO). The VAO general ledger is set up as an entirely separate
business entity.
The VAO, LLC bank account is with Wells Fargo Bank in Washington, D.C.
The VAO, LLC maintains a Financial and Administrative Policy and Procedures
manual that documents all business and administrative operations. These policies
and procedures are designed to assure that the VAO, LLC is operated in full
compliance with all relevant federal and state laws and with OMB Circulars A-110
(Uniform Administration Requirements for Grants and Agreements), A-122 (Cost
Principles for Non-Profit Organizations) and A-133 (Audit of State and Local
Governments and Other Non-Profit Institutions).
The policies and procedures defined in the manual include: cash management,
cash tracking, state and federal tax reporting, preparation and support for external
audit, monthly financial statements, accounts payable, accounts receivable,
general ledger, review of allowable and unallowable expenses, annual budget
preparation, draw-down process, sub-recipient monitoring, and internal controls.
VAO Project Execution Plan 47 Management
As the VAO, LLC is a partnership between AUI and AURA, the VAO, LLC
financial and administrative policies and procedures are based on same from
these organizations and represent practices long recognized by both NSF and
NASA.
Budget
The VAO facility is funded jointly by NSF and NASA. NSF funding is provided
through a Cooperative Agreement to the VAO, LLC ($4M/year, or 73% of the
total). NASA provides funding ($1.5M/year, or 27% of the total) through direct
awards to three of the organizations within the collaboration (GSFC/HEASARC,
JPL/IPAC, and STScI). The VAO, LLC has no direct employees; rather, all VAO
staff are paid through subawards from the LLC to the partner organizations or
awards from NASA headquarters.
The budget is distributed among 11 entities: nine organizations comprising the
VAO technical, scientific, and management team, the AUI corporate office, and
the VAO, LLC itself. Budgets are based on the work that each organization has
agreed to take on, which is in turn based on the extant level of VO expertise within
each organization and on each organization’s expertise in areas essential to the
new VAO. Although the funding comes from two agencies and via separate paths,
for project and budget planning purposes we take an integrated view.
The budget allocations for Project Year One are shown in the Table 2-2 and 2-3.
Table 2-2: Funded by NSF Cooperative Agreement
Associated Universities $149,432
California Institute of Technology 457,929
Johns Hopkins University 465,015
National Optical Astronomy Observatory 566,274
National Radio Astronomy Observatory 457,892
Smithsonian Astrophysical Observatory 553,383
Space Telescope Science Institute 649,926
University of Illinois 361,840
Total Subawards $3,661,691
VAO, LLC direct costs $338,309
VAO Project Execution Plan 48 Management
Total $4,000,000
Table 2-3: Funded by NASA
GSFC/HEASARC $474,865
JPL/IPAC 777,177
Space Telescope Science Institute 247,958
Total $1,500,000
The primary budget item overall is for labor and associated benefits and
institutional overheads on labor. Each organization carries a budget for computer
equipment and project-related travel.
Expenses related to the governance of the overall program are carried in the
VAO, LLC budget. These include travel and logistical costs associated with Board,
Science Council, and Program Council meetings, as well as Participant Costs for
professional training programs and a small fund ($65,000) for development costs
where the needed expertise is not available within the collaboration. There is no
explicit contingency budget. Other costs for corporate functions (office space
rental, office expenses, audit and legal services, and insurance) are also
budgeted as VAO, LLC direct costs.
Rates of spending will be monitored and allow-ability of costs will be verified by
the VAO Business Manager. Each subaward will have an agreed-upon schedule
for submission of invoices (no more frequently than monthly nor less frequently
than quarterly) and processing of payments. Significant deviations from steady-
state expenses will be investigated and resolved by the Business Manager and
the cognizant financial official at each organization.
The budgets in Years 2-5 are currently fixed at Year One levels. As the Project
Execution Plan is updated annually, the balance of effort among the organizations
will be re-evaluated, taking into account updates to priorities in development and
support efforts, and actual performance.
Safety, Environment and Health
The VAO, LLC has no employees and has no physical facilities of its own. All
safety, environmental, and health policies and practices are governed by the
organizations within the VAO collaboration. All relevant and required terms and
conditions in the NSF's Cooperative Agreement—Financial and Administrative
VAO Project Execution Plan 49 Management
Terms and Conditions (CA-FATC) document are included in the subaward
agreements to the NSF-funded organizations. The NASA-funded organizations
are similarly bound by the relevant terms and conditions of the existing contractual
instruments.
Risk Management
This plan governs how technical, cost and schedule risks are to be identified,
assessed, managed and communicated.
Roles and Responsibilities
The VAO will operate on a strict budget and will be subject to hard milestones.
Special attention must be paid to those risks affecting budget, schedule, product
quality and science return. Risks will be tracked and reported to the sponsor or
oversight committees as required. Consequently, the responsibilities for Risk
Management will be distributed as follows:
The VAO Program Manager is responsible for acting as Risk Mitigation Officer
(RMO) for the project. This person is responsible for identifying and tracking all
risks, developing risk mitigation strategies and overseeing them.
The VAO Director will act as Deputy Risk Mitigation Officer (DPMO), and assist
the Risk Mitigation Officer in identifying and tracking risks, and developing risk
mitigation strategies.
The RMO will provide detailed reports, to be updated monthly, of all risks and
their impact on budget, schedule, product quality and science return. The
reports will be available to all members of the VAO project. The RMO will
approve all strategies for mitigating risks.
The RMO will include the status of risks in the quarterly reports to the
sponsors.
The VAO team will be jointly responsible for identifying risks and developing
strategies for minimizing them.
Task Area Leads will be responsible for reporting risks and progress in
mitigating them to the VAO Program Manager.
VAO Project Execution Plan 50 Management
Risk Management Process
The VAO risk management approach consists of a process with the six steps
shown in Figure 2-2. The approach is sequential and iterative, with the last five
steps reinitiated throughout VAO life cycle.
VAO Project Execution Plan 51 Management
Figure 2-2: VAO Risk Management Process
VAO Project Execution Plan 52 Management
Risk Identification
VAO personnel review technical performance, schedule progress, and cost data
to identify candidate program risk items. Candidate risk items will be documented
in the VAO wiki.
Risk Analysis
A common strategy will be used across project elements during the initial risk
assessment and for subsequent reviews and updates.
Every two weeks at project meetings, the team will be responsible for:
Proposing and assessing risks, and developing a risk mitigation plan for each
risk.
Assessing progress in mitigating existing risks.
Reviewing the prioritization of existing and new risks (low, medium, high).
Risk Handling
Individuals responsible for handling risk will formulate an action plan for all
moderate and high priority risks. An action plan addresses how the risks will be
handled and includes the specifics on what should be done, when the action plan
needs to be implemented, specifies the responsible individual, determines an
associated cost for coping with the risk item, and defines a schedule. The four
major options available for handling risks are avoidance, mitigation, transfer, or
assumption.
Risk Tracking
The Risk Mitigation Officer will update the documentation of existing risks, add
new risks to the table, and reorder the risks according to changes in their priority.
This will be done biweekly.
Risk Control
The heart of the VAO risk management process is the Control step, where risk
management decisions can be made responsibly with information collected from
the four prior steps. Risk identification, analysis and handling provide concrete risk
descriptions and plans to address each risk.
VAO Project Execution Plan 53 Management
In addition, as risks are tracked, the risk control process ensures the analysis for
each risk is still valid and that appropriate actions are taking place as planned.
Risks will be reviewed, extended and reprioritized every month.
Risk Communication
The results of the VAO risk management process will be documented on the VAO
wiki. The documentation will consist of a table listing the following information:
Description of risk
Date open
Level of risk (high/medium/low)
Mitigation plan and progress
Date closed
The risks will be listed in order of priority. A sample (fictitious) risk assessment is
shown as follows:
Figure 2-3: Sample Communication Assessing Risk Item
Deliverables
Monthly risk assessment reports will be delivered to the VAO team. Quarterly and
annual risk and assessment reports will be delivered to the NSF.
VAO Project Execution Plan 54 Management
Contingency Management
Protection of VAO Assets
Assets include software, code documentation, users guides and tutorials,
technical reports and papers, usage logs, defect reports and user submitted
tickets, and technical reports hosted in behalf of the IVOA. The assets will
generally be distributed on servers housed at the collaborating organizations.
Plans for protection of assets are described in the Operations section. The VAO
does not maintain its own hardware, and protection of hardware will follow
institutional plans.
Disaster Recovery
The collaborating organizations will follow their institutional plans for recovery
from natural disasters and from Acts of God.
Budget Contingency
The VAO does not provide contingency for cost overruns. All expenditure is
monitored closely throughout the project, as described in the Budget section.
Schedule Contingency
The VAO considers on-schedule delivery of its services as crucial to attracting
and retaining end-users. The following steps will be taken to protect the schedule:
The software design phase will identify the minimum requirements that must be
satisfied in a delivery. The software development effort will focus on ensuring
these requirements are met.
The detailed schedule for each delivery will include a margin of 50% of the total
life of the project.
Projects will be broken into a series of sub-milestones whose delivery dates will
be closely monitored.
Development processes will be monitored and upgraded to improve efficiency.
Management will reassign staff to support on-time delivery of services, as
required.
VAO Project Execution Plan 55 Management
Section 3. Operations
This section describes the operational activities of the VAO. This area is divided
into three elements: deploying and monitoring VAO services, quality assurance of
VAO resources, and configuration management. It includes the operations and
maintenance and configuration management plans of the VAO.
Deploying and Monitoring Services
Goals
The fundamental goal of the VAO is to provide services to the astronomical
community. This work area addresses the actual deployment of software services
on the VAO Web sites.
Scope
This area addresses the installation, operational testing and monitoring of VAO
services. This includes both science services offered to the community, and
internal VAO resources, e.g., the configuration management system or issue
tracker.
Roles and Responsibilities
Operations Monitor: The Operations Monitor is responsible for ensuring the
automated monitoring and validation tools are running properly and for following
up on issues as they arise.
Site Managers: Each VAO site shall have one or more Site Manager associated
with deployed services. These managers will be contacted whenever an issue is
found at a service at that site. Site Managers are also responsible for routine
activities at the site, e.g., collection of log files.
Service-Responsible Parties: Each deployed VAO service shall include a
responsible party (or parties) associated with the service. These parties will be
contacted (normally automatically) whenever an issue is detected with a given
element of software. The Project Leader for a development project will often
become the responsible party for a service when it is deployed.
VAO Project Execution Plan 56 Operations
VAO Web Presence
The VAO Web presence comprises Web pages currently resident on machines at
the Center for Advanced Computing Research (CACR), Infrared Processing and
Analysis Center (IPAC), Johns Hopkins University (JHU), Space Telescope
Science Institute (STScI), High Energy Astrophysics Science Archive Research
Center (HEASARC), National Optical Astronomy Observatory (NOAO) and
National Center for Supercomputing Applications (NCSA). VAO Operations is
responsible for ensuring that all of these Web sites are operational and working
correctly. Each site is associated with an institution and a responsible party.
In addition to its public presence, VAO Operations shall manage an internal Web
site that provides links to tools and services used by the VAO Operations that are
not directly useful to our science community. This includes documentation of
internal VAO services (e.g., configuration management and logging), and Web
forms to initiate privileged activities (e.g., specifying a downtime alert). VAO
Operations shall automatically monitor each VAO Web site and provide a report
within an hour should any site go down. The VAO Operations Manager and the
responsible party shall be notified automatically by email when any site is
unavailable. An issue shall be opened in the VAO issue tracker for each such
downtime, unless a downtime alert for the given service is found in the downtime
database. Downtime will be recorded as a metric and will be reported to NSF and
NASA as part of the Quarterly and Annual Reports.
Science Services
The VAO Web site shall support a number of science services. These services,
shall be maintained and operated by the VAO in collaboration with VAO host
institutions. Failing services shall be diagnosed, remedial action taken and
services restarted. All such actions shall be logged and reported weekly.
New VAO services and enhancements to existing VAO services shall be released
by VAO Operations in accordance with the VAO product development standards
described in the Product Development section. During the first year of the VAO a
schedule for migrating heritage NVO applications into the VAO will be established.
Such heritage applications will only be deployed in the VAO after they have gone
through the VAO testing and configuration management procedures. Candidate
heritage applications are listed in Table 3-1.
VAO Project Execution Plan 57 Operations
Table 3-1: Candidate Heritage NVO Applications
URL/Domain Service
(preceded with http:// or https://)
nvo.stsci.edu Directory
openskyquery.net Open Sky Query
voservices.net Footprint, spectrum, filter
heasarc.gsfc.nasa.gov/vo DataScope, SimpleQuery Validation,
heasarc.gsfc.nasa.gov/cgi-bin/vo monitor, alert
envoy5.cacr.caltech.edu:8888 Visual Integration and Mining
iraf-nvo.noao.edu VOClient
irsa.ipac.caltech.edu/cgi-bin/VOInventory Inventory services
hachi.ipac.caltech.edu:8080/montage Montage Image Mosaics On Request
voeventnet.caltech.edu VO Event Network
Monitoring VAO Services
All VAO services will be monitored for liveness on an hourly basis. Automated
alerts will be sent to the Operations Monitor, the Operations Lead, the Site
Manager and responsible party for the software element notifying them of any
issues. An ticket shall be created and closed when the service is restored.
Anticipated downtimes may be recorded using the VAO alert service. Additional
sites and services to be monitored may be added by the VAO Operations
Manager at the direction of the VAO management.
VAO Support Services
The VAO supports a number of infrastructure services that support VAO
operations and development.
VAO Project Execution Plan 58 Operations
Configuration Management Operations
The VAO Operations Manager shall designate a Configuration Manager
responsible for the operations of the VAO configuration management system, as
described in the Configuration Management section. The CM system shall be fully
backed up daily and a mirror CM system shall be available at a geographically
distant site from the primary VAO CM repository.
A report detailing the number of updates and retrievals to the CM system shall be
provided monthly. The report shall be broken down by the projects managed in
the CM system.
Issue Tracking System
The VAO shall provide a single-issue tracking system that shall be used to
manage a variety of different requests. These include:
Bug reports submitted by all users of the system, including the public and
members of the VAO team.
Requests for new projects or modifications of existing projects
Document change requests
Software downtime notices
Additional issue streams may be added. For each issue stream, a specified set
(which may include the public) of those eligible to submit requests, those informed
when requests enter the stream or change status, and those able to modify the
status of the request is defined. Table 3-2 describes the existing streams.
The VAO internal Web site shall include forms where new issues can be raised
and existing issues edited.
Table 3-2: Issue Streams
Type Submitters Responsible Party Others Notified
Bug reports and
All, including
enhancement Project Lead Software Manager
public
requests
VAO Project Execution Plan 59 Operations
Type Submitters Responsible Party Others Notified
New project or Configuration Program Manager,
Software Manager
project modification Manager Operations Manager
Document change Other CCB members,
VAO team CCB Chair
requests Program Manager
Automated,
Program Manager,
Problem notifications Operations Operations Manager
responsible party
Manager
Weekly reports on the all issue streams shall be prepared for the operations
reports. The Help Desk system provides tools for preparing reports on a project-
by-project basis.
Other Infrastructure Services
Operations shall support wikis and other distributed documentation facilities as
requested by VAO management. All wikis shall be backed up daily.
VAO Operations shall support the creation and maintenance of mailing lists as
requested by VAO personnel.
The VAO shall provide a account registration system wherein any member of the
VAO team can quickly get appropriate access to all appropriate VAO internal
services.
Logging
Logging of all user requests to VAO services shall be generated and logs shall be
collected to a centralized location. VAO and public users shall be able to query
this database. No user-identifiable information shall be maintained in the central
database.
VAO Web Site Rationalization
During the first year of VAO operations, all VAO services at all sites shall be
associated with the URL of the primary VAO Web site (usvao.org). All full
references (i.e., non-relative URLs) to VAO Web sites shall use these locations.
The VAO primary Web server shall redirect requests to specific VAO host
machines.
VAO Project Execution Plan 60 Operations
Quality Assurance
Goals
The goal of the quality assurance elements of the Operations Work Breakdown
Structure is to ensure that services meet appropriate quality standards and are
robust against hardware failures and other external incidents.
Roles and Responsibilities
The roles in this work area overlap the roles in the previous element.
Automated Quality Checks of Services
SERVICE “SMOKE” TESTS
All deployed VAO science services will include a suite of tests that assure the
services are running properly. These tests will normally be run hourly, but this
may be adjusted if appropriate. An issue shall be opened whenever a failure is
noted. The responsible parties for both the site on which the service is hosted,
and for the service itself, will be automatically notified. Similar tests for VAO
infrastructure services will be developed by the Operations group and run hourly.
BROKEN LINKS
All VAO-managed Web sites shall be checked for broken links weekly. The latest
report on broken links shall be provided on the VAO internal Web page.
SERVICE VALIDATION
All VAO services providing outputs using standard VAO protocols shall be
validated against these protocols, where feasible.
SECTION 508 COMPLIANCE
VAO services shall be checked with tools that assess compliance with Section
508 Web guidelines, which establish the minimum levels of accessibility for users
with disabilities. The latest report shall be provided in the VAO internal Web page.
VAO Project Execution Plan 61 Operations
Monitoring And Validating Non-VAO Services
Since the operation of VAO services depends upon the existence of VO resources
outside the control of the VAO, the VAO shall monitor all known VO sites and
provide mechanisms for interactive and automated assessment of whether they
are operational. Site tests shall typically be done hourly. When a site hosts
multiple services, typically only one service of each general type shall be tested.
On a monthly time scale, all VO services for which there is a validation capability
will be validated, and the results of that validation will be saved in a database.
The operations monitor will periodically send out summaries of issues detected at
VO sites (both VAO and non-VAO) with recommendations for addressing issues.
Software Testing
While the bulk of testing of VAO services is beyond the purview of the Operations
group, new and enhanced services will normally be released first in a test
environment where they can be tested in the context of a full VAO environment.
VAO operations staff may also participate in the load testing of software that has
passed functional testing.
VAO Operations will provide a testing environment in which new or enhanced
services can be tested without interfering with VAO Operations. During the first
year of operations, this environment may not be distributed in the fashion of the
operational environment. A test environment more closely mimicking the
operational systems will be deployed in the second year.
Other QA Operations
BACKUPS
All VAO online services shall be backed up at least weekly to an offsite location
separate from the primary location.
MIRRORS
VAO Web services shall be duplicated in more than one physical location where
feasible. The primary VAO Web site shall be duplicated.
VAO Project Execution Plan 62 Operations
Security
VAO Operations shall monitor the security posture of all VAO Web sites and notify
sites of potential issues.
Configuration Management
Goals
This section describes the configuration management (CM) processes used in the
VAO. The VAO is a distributed organization, with groups from many institutions
contributing to resource development and operations. CM plays a critical role in
ensuring software and other VAO resources of are delivered in a consistent,
reproducible, and timely fashion. This CM plan discusses which elements of VAO
activities are covered by the CM plan, the roles and persons responsible for its
implementation, software tools used for CM, and the structure which shall be used
when VAO resources are included in the configuration management repository.
Configuration management ensures that all elements needed to provide VAO
tools and services are available in a consistent, documented and reliable fashion.
Resources are stored in a repository. Different versions of a resource may be
available. Authorized users may commit new versions. They and other users (who
may have lesser privileges) may retrieve resources. Releases of software or other
resources are available from the repository with a well-defined version of each
element used in the resource. The remainder of this section of the document
describes the procedures by which this is done.
In addition to keeping track of the changes, the configuration management system
tracks why changes were made. Bug reports and software enhancements are
traced to the specific code changes in which the change was made. As VAO
policy changes, changes in related documents are traceable to the policy change.
In the VAO context, there are two primary cases for configuration management:
configuring software, and configuring the documentation of the structure and
processes of the VAO.
In software development efforts, the configuration management system serves as
the conduit though which new and enhanced tools and services flow from the
development teams, through testing and into operational deployment. The
configuration management system provides a backup system for software
developers, a tool which coordinates parallel developments in the same area, a
database which associates changes to software with the specific requests and
bug reports that drove these changes, and an ability to recover earlier software
VAO Project Execution Plan 63 Operations
versions.
For the structural documentation of the VAO, the configuration management
system stores the definitive documents for the goals and processes of the VAO,
along with a complete history of the changes to these documents and the
requests that drove these changes. The configuration management process
ensures that changes to these documents reflect the goals of the VAO.
Scope of Configuration Management
The configuration management system comprises three main elements: a
complete archive of all versions of a configured element (the repository), a
database describing the reasons for each change, and a set of policies and
procedures through which these changes are populated.
All modifiable resources used in VAO Operations or provided by the VAO to
others are subject to configuration control. Controlled resources include, but are
not limited to:
All documents describing the processes and practices of the VAO, including:
o Project data management plans
o Software Development Plan
o Configuration Management Plan (i.e., this document)
o Quality Assurance Plan
o Risk Management Plan
o Operations Plan
Documentation associated with specific software elements of the VAO,
including:
o Test plans
o Functional and system requirements documents
o Science requirements documents
o Design document
Software developed by the VAO, including:
o VAO-developed data products (see below for limitations)
o Web pages
o Code
o Documentation
o Test data sets
External resources integral to VAO elements (see below for limitations),
including:
o Software tools and libraries
VAO Project Execution Plan 64 Operations
o External data sets
Not all documents are subject to configuration control, notably those where
modification is inappropriate (e.g., minutes of meetings are not subject to
modification). Similarly, logs produced in operations are not configured.
Data products developed and used at the VAO, such as FITS images or catalogs,
are subject to configuration control unless they can be regenerated by automated
procedures from other configured elements. For example, if a catalog includes
manual edits then it must be configured. If it can be derived from running a
configured script on another configured base catalog, then the derived catalog
need not be subject to configuration control.
Many external resource are needed to support VAO Operations, including
software, data and documentation developed outside the VAO. Normally a copy of
these resources are included in the VAO repository where feasible. This
requirement may be waived if the external resource is provided outside the VAO
in a well defined version. For example, it may not be legal for VAO to configure a
copy of proprietary software. Keeping a copy of large external databases like the
Sloan Digital Sky Survey (SDSS) or Guide Star Catalogue II (GSC2) would be a
waste of resources since these are easily accessed from well defined locations.
Such external dependencies must be explicitly noted in the CM profile for the
project, including the specific version of the resource on which the VAO depends.
However, when possible, it is recommended that the appropriate version of the
external resource be managed as an element of the projects with which it is used.
This allows repository users to easily create complete, working VAO resources
without having to download needed elements from multiple sites.
Just because an external resource can be used by an operational VAO service
does not mean a copy of the resource should be placed in the VAO configuration
management system. Consider three potential services:
1. A VAO service is built that can access arbitrary databases using Table
Access Protocol (TAP). One of the science drivers for this service uses the
Two Micron All Sky Survey Point Source Catalog (2MASS PSC). Unless the
2MASS PSC is used in the test plan for this service, the catalog need not be
mentioned as a dependency. Even if the catalog is in the test plan, there is no
need to keep a copy of the catalog in the VAO repository since it is accessible
through a well defined external interface.
2. A VAO service is built explicitly to analyze the 2MASS PSC. Here, the
service is explicitly dependent on the catalog and the dependency must be
noted in the CM profile for the service. However the VAO CM system will not
include a copy of the 2MASS PSC.
VAO Project Execution Plan 65 Operations
3. A VAO service is built explicitly to analyze a new test catalog that is not
available publicly. Here, the VAO CM system must provide a mechanism to
retrieve this catalog for anyone wishing to build this service.
All changes to the configured elements within the configuration management
repository must be documented. Changes that affect released versions of
resources go through the configuration control procedures described below.
Changes to code under development should include a comment briefly describing
the change. Individual development projects may place additional requirements
for the documentation of changes during development.
Roles and Responsibilities
The CM system involves the interactions between VAO management, the
Configuration Change Board (described later in this section), the Configuration
Manager (described later in this section, as well as in Product Development, User
Support and Testing and Operations). Figure 3-3 shows the roles of the staff in
the life cycle of a project, and Figure 3-4 show the decision points in the overall
flow of the process.
VAO Project Execution Plan 66 Operations
Figure 3-3: Roles and Responsibilities in the VAO Project Life Cycle
Project
User
VAO Config. Dev.
Phase CCB Leader Support Ops
Mgmt. Manager Team
(testers)
Research
Limited role for configuration management during this phase.
Define
new
project,
specify
leader
Send
request
to CCB to
start
project
Approve
project
Record
Design and Planning
CCB action
Send
project
file to
Config.
Manager
Create CM
area for
project
Develop
project
plan
Submit
plan
Approve
plan
Record
CCB action
VAO Project Execution Plan 67 Operations
(Figure 3-3 continued)
Project
User
VAO Config. Dev.
Phase CCB Leader Support Ops
Mgmt. Manager Team
(testers)
Code,
unit tests,
docs
Tag
release
Request
release to
test
Development and Testing
Approve
release to
test
Record
action
Unit tests
Request
release to
User
Support
Approve
release to
User
Support
Tag release
version
System and
int. test
cases
User Support and
Run system
and int. tests
Testing
Test
artifacts in
CM
VAO Project Execution Plan 68 Operations
(Figure 3-3 continued)
Project
User
VAO Config. Dev.
Phase CCB Leader Support Ops
Mgmt. Manager Team
(testers)
Request
approval
for end-
user eval.
Approval for
end-user
eval.
Record
CCB action
User Support and Testing (continued)
End-user
evaluatio
n
Correct
defects,
regressio
n test
Request
approval
for
release
Approve
release
Record
CCB action
Ensure all
material in
CM
Release
to ops
Installation
Operations
tests
Release
Submit bug reports and enhancement requests.
VAO Project Execution Plan 69 Operations
Figure 3-4: Flow chart showing decision points in the project life cycle
(Figure 3-4, continued)
VAO Project Execution Plan 71 Operations
In practice, the progress is not so linear. For example, an issue may arise in testing that
requires modification of the requirements and test plan. The issue and subsequent
change must be submitted to the Configuration Control Board (CCB) for review and need
further development before a new test release can be brought to the testing group. The
CCB may reject a change request as being vague. VAO management may task a project
with new requirements, initiating a change to the project documentation. However, this
figure gives the basic sense of how a project progresses.
CONFIGURATION MANAGER
The VAO Operations Manager, in consultation with the Program Manager, will
designate a Configuration Manager with overall responsibility for the operations
and procedures of the VAO configuration management system.
The Configuration Manager's responsibilities include:
Setting up and operating the configuration management system in collaboration
with the Operations group.
Defining software projects and the personnel associated with each project,
including:
o Ensuring each project and its CM profile are properly defined
o Ensuring appropriate permissions within the CM system for project
team members
o Managing updates to the profile and team membership
Resolving CM issues as they arise, referring conflicts to the Operations Lead or
PM as appropriate.
Notifying the Test and Operations teams when new releases are made
available.
Ensuring the integrity of the CM system, including backups and mirror sites as
deemed appropriate
Providing periodic reports to VAO management on usage of the CM system
and any current issues.
Managing the documentation change request database and providing reports
as requested to the Configuration Control Board on requested changes.
Recording CCB actions associated with projects.
VAO Project Execution Plan 72 Operations
SOFTWARE DEVELOPMENT TEAMS
Software development in the VAO is divided into projects which develop discrete
deliverables. Projects are defined by VAO management and led by a defined
Project Lead. A new project is formally created by application to the configuration
control board with the name and purpose and leader of the project. Once a project
is defined, the Project Lead shall provide a project profile to the VAO
Configuration Manager. The project profile includes:
Project name
Brief description of the project
Project leader
Other personnel associated with the project (and their roles if the project is
sufficiently complex).
How the CM repository for the project is to be organized (where feasible, this
should be based on the software and documentation organizations suggested
in the next section) .
Any external dependencies in the project.
The project name should be short and easily recognized since, in conjunction with
the release version, it provides a unique identifier for the VAO deliverables of the
VAO.
Each project must develop an individual project plan that includes requirements
and design and test plans. Initial versions of these documents and changes to
them shall be submitted to the CCB. Upon approval, the standard versions of
these documents in the repository are updated.
The project team may create any number of development areas for the project
development within the overall area provided by the Configuration Manager.
Changes made by developers within these development areas shall be described
using the CM facilities, but the team requires no external authorization to make
changes.
SOFTWARE TEST TEAMS
Software test teams are assembled and associated with projects early in the
project development life cycle. These teams advise and ensure requirements are
verifiable, develop testing plans, and conduct final user acceptance testing. At the
end of development (using procedures described in the Quality Assurance and
Testing section), the testing team completes final testing, passes on candidate
VAO Project Execution Plan 73 Operations
releases of products, and reviews the execution of a project’s test plan. From that
review, they can recommend deployment or further development. When
deployment is recommended, operational product releases are tagged.
OPERATIONS GROUP
Once a released version of a software element is available and release has been
approved by the CCB, the Operations group will perform final operations testing
and then release the software using the procedures described in the software
development plan.
VAO MANAGEMENT
The VAO Director and Program Manager are responsible for approving new
software projects. Their approval of software and testing releases will be carried
out through the Configuration Control Board (CCB).
The VAO Manager will be responsible for initiating changes to the Project
Execution Plan (this document). Changes will follow the CM procedures described
here, but will also require the approval of the Board of Directors.
CONFIGURATION CONTROL BOARD
The VAO Program Manager chairs the Configuration Control Board, which
includes the VAO Director, the Product Development, Operations, Testing and
User Support Leads of the VAO or their delegates, and other personnel as
nominated by the Program Manager. Nominally the CCB shall meet bi-weekly, but
it is expected that more frequent meetings may be required. The CCB may, at its
discretion, meet using email or other electronic means that allow for continuous
operations of the CCB.
For changes to documentation, the CCB shall ensure the consistency and clarity
of the VAO documents.
New projects shall be reviewed to ensure the coherence of the VAO effort and to
note boundaries with other efforts within the VAO.
Releases shall be reviewed to ensure testing was adequately performed and that
the software meets the desired goals.
The CCB shall approve or reject all requests. When a request is rejected, reasons
for the rejection shall be given. The response of the CCB shall be sent to the
requestor and to the Configuration Manager, who will include the response in the
VAO Project Execution Plan 74 Operations
CM repository for the project (or in a special area for a rejection of a request for a
new project).
Configuration Management Tools
The primary configuration repository for the VAO shall be a Subversion repository
hosted at the Center for Advanced Computing Research at the California Institute
of Technology. The VAO repository is distinct from the NVO repository, but user
accounts from this earlier system shall be maintained. A mirror of the CM
repository shall be maintained at another site and updated at least daily.
The Operations group will develop Web forms through which any VAO member
can submit document change requests. A database of such requests shall be
maintained by the Configuration Manager.
Projects in the VAO Configuration Management System
Elements in the VAO configuration management system are organized into
projects. Projects are defined by VAO management and each project has a
designated project lead and project members. Only members of the project will
have permission to commit changes to the given project. The Configuration
Manager shall maintain the list of projects and members. A large project may
include sub-projects.
At the initiation of a project, the Configuration Manager will provide the Project
Lead with a CM template – a simple, plain-text file specifying a short name for the
project using standard syntax that specifies the content to be configured as a
hierarchical file structure. Some sample templates are provided for common
examples of a document project and a software project. Templates must be
approved by the Configuration Manager.
Some projects may be developed in environments that have built-in configuration
management facilities, for example, documentation developed using wiki software
or mailing lists maintained using contact management tools. With these
exceptions, all other projects are included in a single VAO repository project
hierarchy.
Note: In the following, italics indicate that a given name/path will be replaced by
some other string, while bold indicates the string will be used as given. Normal
font is used for example names.
VAO Project Execution Plan 75 Operations
TOP-LEVEL STRUCTURE
The projects are organized under in the repository as:
vao/ Top-level directory. All elements of the repository used in VAO
operations are stored under this directory. This includes non-VAO
elements used in the VAO, with some exceptions.
documents/ Projects associated with overall VAO documentation. Projects in this
directory should normally use a template derived from the
documentation template.
software/ Projects that involve programming to provide tools or services. Projects
in this directory should normally use a template derived from the
documentation template.
operations/ Projects that involve the maintenance of databases and resources
used in VAO operations (e.g., maintaining the list of projects and
project teams is itself an operations project).
PROJECT ORGANIZATION: GENERAL
Each project will be placed in a single directory inside the software area. The
Configuration Manager may choose to organize projects with intermediate
directory levels between the software project and the top-level project directory.
The top-level project directory contains:
The project CM template describing the project organization.
A whois file describing the members of the project team.
A CCB directory containing the reports of all CCB actions associated with this
project.
A README file giving a brief overview of the purpose of the project.
PROJECTION ORGANIZATION: SOFTWARE
In addition to the files and directories for the project as a whole, software projects
will normally contains three subdirectories:
devel All commits are made within this directory. This directory is
maintained by the project team developing the project.
test Test releases of the software are made under this directory using tags
of the devel area. This area is maintained by the group responsible
VAO Project Execution Plan 76 Operations
for testing the software.
release Public releases of the software are made under this directory using
tags of the devel area. This area is maintained by the VAO
Operations group. Only this area may be used in the procedures used
for building and maintaining operational VAO tools and services.
The devel area will normally have a trunk and may have branches
subdirectories. The primary development version will be the trunk version.
Subdirectories in the test and release areas will normally be made by tagging the
current trunk.
For many projects, the branches directory may not be needed, but if multiple
users are developing in a project, or if a given user is considering alternative
paths to development, then the branches directory may be used to maintain the
alternate paths. Each subdirectory of branches should contain a complete copy
of the project. The name of this subdirectory should indicate the user or reason for
the branch, and the SVN repository version at which the branch began:
mcglynn_237 or xsltversion_743. Branches should normally be intended to be of
finite lifetime. Either the branch will be merged back into the trunk, or the branch
will split off into a separate project.
The test and release directories will contain directories that are complete
versions of the project. These directories should normally be named as v
versionnumber. The scheme for versioning is described below.
ORGANIZING FILES WITHIN SOFTWARE PROJECTS
The previous section discussed how we might have multiple copies of a given
project within the repository. This section discusses what a single copy of the
project might look like.
A build file should also be in the top. The build file may be either a make Makefile
or a build.xml file or other format appropriate to the project. The build should try
to use standard target names for common actions. These include:
clean Delete any files created by any of the other targets.
compile Compile code, but do not create libraries or jar files.
docs Create any dynamically created documentation.
exe Build executable files.
install Install a working version of the software, including appropriate
documentation.
VAO Project Execution Plan 77 Operations
libs Create any libraries, jars, zips or other aggregate files needed.
test Test the software (normally prior to installation).
Targets may depend on and automatically perform other targets. For example, a
typical sequence might be clean, compile, libs, docs, exe, test, install, but the
user may only need to invoke the install target explicitly to perform all of these.
Not all projects will need all of these targets (e.g., Perl need not be compiled).
Other targets may be appropriate for specific targets. The README should note
these.
The top-level directory in any version included in the test or release areas should
include a release.notes section. This may include text describing a particular
release. It must include a statement enumerating all known tracked issues
partially or fully resolved by the release.
The subdirectories of the project directory include:
src Source code used within the project. This includes not only
compilable code, but also things like HTML, CSS, and other non-
compilable files used in the operations of the project. HTML that is
explicitly documentation may be in the docs directory.
docs Documentation for the project. This may include documentation that
will be used in the help system of the project. Note that if
documentation is dynamically generated (e.g., Java JavaDocs), then
a docs directory may be created by the build file, but need not be in
the repository. If the documentation is not clearly separable from the
source code, then it should be included there (i.e., if the program runs
fine without the documentation files, other than missing some help
text, then the documentation files should be separated out).
data The data directory is used to store data files used within the program
(e.g., XML, FITS files). Images displayed online are included in their
own directory.
conf The conf directory contains configuration files that can be used for
either the compiler or runtime configuration. Configuration files
normally should not be header files or source code files; these should
be under src.
images This directory contains graphics images that will be linked to Web
pages.
Each of the subdirectories in the project directory may themselves
have subdirectories as appropriate. In particular, the src directory
may have subdirectories for each of several languages used in a
mixed language project.
The following list suggests names for given languages’ directories:
VAO Project Execution Plan 78 Operations
c C language
css Cascading Style Sheets
cpp C++
csharp C#
fortran Fortran
html HyperText Markup Language. Note that this directory contains HTML
integral to the project while pure help pages may be placed in docs.
java Java
javascript JavaScript
python Python
scripts Scripts in any of various shell languages.
xsl eXtensible Stylesheet Language
If a project is overwhelmingly in one language, then the language directory may
be omitted. Some languages use multiple types of files which may themselves be
segregated into directories (e.g., C source and header files). All such directories
should be included under the given language directory.
DOCUMENTATION PROJECTS
Documentation projects are usually substantially simpler than software projects,
though they can also involve many discreet files. The top-level organization of the
documentation project is similar to that of software projects, but often only a
release subdirectory is present since the CM system is not required to track
intermediate versions of a document under development.
SPECIFYING A VERSION
Every released version of a VAO document or software service shall have distinct
version. Adding the project name as a prefix provides a unique key to all VAO
deliverables:
Project.Version
A released version of a document or service is given as two integers separated by
a dot, such as 0.3, 12.9 or 16.14. The first number indicates the major version,
which should be incremented whenever there is a major change in the
VAO Project Execution Plan 79 Operations
functionality of the software and must be incremented when a version is
significantly incompatible with the version it replaces. A major version of 0 may be
used to indicate software missing significant functionality prior to initial release. No
guarantees of compatibility are made for different minor versions when the major
version is 0.
The minor version should be incremented for each new release of a software
where the major version is left unchanged. Minor version 10 follows minor version
9, so that version 1.10 is the release after version 1.9. The minor version should
reset to 0 for each new major version. For example, a valid sequence of releases
would be: 1.0, 1.1, 1.2,...1.11, 2.0, 2.1, 3.0
The Subversion repository maintains an integer version number that increases in
increments with each change to any repository element. Test versions of software
(or data or any other element subject to testing) shall begin with the version
number that the test version would be released with if it were to be accepted,
followed by a period (.), and then the repository version number at the time the
repository is tagged to create the test version.
Note that the tagging itself will update the repository version. If we are about to
test version 1.3 of a software system and the repository is currently at version
1342, then the project shall create a test version v1.3.1342. If that software
passes testing it may be released as version v1.3. If there are problems with the
testing and some additional development is needed, a second test version may be
created, perhaps version v1.3.1367, where there were 25 changes to the
repository across all projects between the release of the two test versions.
Year One Deliverables
Automated monitor service checking the liveness of VAO services.
Automated validation service checking the correctness of VAO services.
Automated services checking the liveness and correctness of non-VAO
services.
An operational configuration management system, including software tools,
documented procedures and a full backup.
An operational issue tracking system including software tools, documented
procedures and a full backup. This will support multiple classes of issues.
A rationalized VAO Web site that integrates references to all VAO services into
a common address space.
VAO Project Execution Plan 80 Operations
An interim testing environment enabling testing of VAO services without
disruption of VAO Operations.
A development plan for a full VAO test environment that includes the
distributed nature of the VAO.
Quarterly reports on Operations issues, including:
o CM usage by project and site, including total volumes and number of
committed changes.
o Summaries of the status of all tickets submitted to the VAO issue
tracking system.
o Summaries of usage of all VAO services derived from logs.
o Summaries of the reliability of VAO services.
o Summaries of communications with non-VAO sites regarding issues
detected in routine validations of Web services.
Offsite backups for all essential VAO services
VAO account registration system
VAO Project Execution Plan 81 Operations
Section 4. User Support
Goals
The VAO User Support Group will provide the primary interface between the VAO
and its user community. The group will be responsible for quality assurance and
testing, documentation, training and advocacy, the VAO Web site, and the VAO
Help Desk. The User Support Group also recognizes the need for a user
committee composed of knowledgeable "power" users of the VAO, who will
provide advice and suggestions on ways the VAO project can better serve the
research needs of the general astronomical community. This should be a high
priority in late 2011.
Quality Assurance and Testing
The User Support organization will lead quality control and testing activities.
Although for most testing activities the User Support role will be to act as the
coordinator of the activities and as reviewers, this is not the case for user
acceptance tests (UAT) and Readiness Reviews. User Support, on behalf of the
user, will perform User Acceptance Testing (UAT), which will be used, along with
other tests and quality control reports, to prepare readiness reviews. (See the
Quality Assurance and Test section).
Documentation
User Support scientists will write and/or complete user documentation that helps
scientists have a better understanding of VAO services and applications and how
to use them. Documentation packages may include deployment instructions,
general descriptions, tutorials, cookbooks, etc.
IVOA Web Site and Document Repository
User Support will assume responsibility for maintaining and coordinating the IVOA
Web site and maintaining the IVOA document repository.
VAO Project Execution Plan 82 User Support
Training and Advocacy
The VAO will be promoted through community engagement and advocacy.
Advocacy
User Support will communicate directly to the general astronomical community
through a regular electronic newsletter, online tutorials and related documentation
published on the VAO Web portal. The newsletters will highlight new VAO
features and services and keep our community informed about general VAO
activities and events. The tutorials will be developed for various VAO tools and
services to guide first-time users through sample use cases. For more technically
oriented users, User Support will provide cookbook-style documentation covering
common VAO activities that use multiple VAO and VO tools and services.
The newsletters, tutorials, and recipes will complement the fundamental and
detailed documentation provided by the VAO on the capabilities and use of the
individual tools and services.
In order to have a dynamic VAO that is responsive to the changing needs of the
astronomical community, User Support will actively pursue community feedback
and collect ideas for improvements, ranging from basic improvements to existing
user interfaces, to broader suggestions of new features for existing services and
tools, as well as suggestions for innovative new services and tools. Feedback
from the VAO and general astronomical communities will be woven into User
Support community engagement activities. User Support will host a Community
Forum on the VAO Web site that encourages user interaction.
Training
There will be training at various levels, ranging from new users, power users, and
programmers. The NVO ran many successful summer schools that served a
technically oriented audience. The VAO plans to continue the summer schools
and conduct shorter science workshops and other training events that focus on
the use of the VAO through its interactive Web-based applications.
The winter and summer American Astronomical Society (AAS) meetings provide
an excellent venue for VAO training activities. The VAO will have an active
presence at the semiannual AAS meetings in order to educate and train new
users. The VAO will submit a proposal to the AAS in May 2011 for a special
session dedicated to the VAO, to be held during the 2012 winter meeting in
Austin, Texas.
VAO Project Execution Plan 83 User Support
VAO Web Site
The VAO proposal refers to the VAO Web site as the cornerstone of support for
its growing community of users. Key features provided by the site will be access
to an extensive repository of documentation, FAQ lists, the VAO Help Desk (see
Help Desk and Ticket System), and diverse operational services such as the VAO
Portal. User Support is responsible for maintaining the Web site (for both external
and internal users) and for administering content contributed by User Support, or
(more frequently) by other groups.
Prioritized, short-term (first year) deliverables include:
1. Deploy the basic functionality of a VAO Web site as the next generation of the
current NVO Web site. This includes identifying and deploying near-term
server resources (likely non-scalable) for hosting the VAO site and
implementing maintenance procedures.
2. Provide access to the user Help Desk ticketing system.
3. Provide public and internal access to a repository of the VAO/NVO
documentation that is currently available.
Other short-term and ongoing Web site activities include working with the VAO
Executive Group to define high-priority documentation, such as FAQs, that should
be developed immediately, and assigning these tasks to appropriate personnel
throughout the VAO; working with appropriate groups to establish pragmatic Web
interface/look and feel definitions for the portal, Help Desk, and access to other
current and near-term VAO services; defining (with consultation) and
implementing a minimum set of collaboration tools for internal VAO use, e.g., wiki,
etc. (Other tools, such as source code control, are the responsibility of other
groups such as developers.)
Longer-term (subsequent years) deliverables, in no order of priority, include:
A plan for extending short-term solutions to encompass evolving, long-term
VAO goals.
A process for collecting astronomer feedback through the VAO Web presence.
Adding advanced features, such as text searching, to the documentation
repository.
Remaining relatively current with changing Web technologies.
In general, User Support will continue to seek ways to provide an advanced Web
presence. Many such features are controversial and will require broad discussions
to develop a consensus of vision. Examples include social networking
VAO Project Execution Plan 84 User Support
technologies in support of the development of a robust online community, or,
alternately, hiding all the rich VAO service complexity under a single text box
interface similar to Google.
Help Desk and Ticket System
NOAO uses JIRA (http://www.atlassian.com/software/jira) for its Help Desk
support as well as its issue/bug tracking system. We plan to also use JIRA for the
VAO Help Desk because it meets all VAO User Support specifications.
Handling Trouble Reports
Issue tracking and Help Desk support go hand-in-hand. Users may report an
issue with one of the VAO services or inquire about the usage of a service. They
will contact the VAO Help Desk through a public Web form or email address. A
public issue is created immediately and the user receives an automated response.
The Help Desk Manager will also receive an email notice of the issue creation.
Public issues are visible to everyone through the Help Desk Web interface.
The Help Desk Manager may respond to most questions of a general nature, but
issues involving failed services or usage of services or utilities will result in the
generation of an ―internal‖ issue (ticket). An ―internal‖ issue may only be viewed by
"jira-developers" who login to the JIRA Web interface.
Managing User Tickets
A trouble ticket is commonly called an issue in JIRA. An issue may represent a
software bug, a project task or a Help Desk ticket. Projects will be created so that
issues may be created and assigned. At minimum, there will be a Help Desk
project that has public access as described above. Internally, multiple projects will
be created to represent the major work packages. Each project may have sub-
components. Projects and sub-components will have leaders who are responsible
for managing issues assigned to a project or sub-component.
All issues are stored in a database. Time tracking will allow the monitoring of time
spent to resolve issues.
Statistics and Reports
Reports will be generated to show statistics for projects, versions, and other fields
within issues
VAO Project Execution Plan 85 User Support
Backup and Recovery
Automatic backups of the JIRA database contents will occur regularly. In addition,
native database-specific tools will be used for secure backup.
Quality Assurance and Testing
Scope and Goals
The ultimate goal of Quality Management is to minimize the number of defects a
product contains when it is released. Studies indicate the most effective way to
produce quality products is by establishing a quality process, which in turn not
only improves quality, but also reduces costs and mitigates risks. The VAO
Product Development organization defines a quality development process
describing the product development life cycle, and establishes tools and policies
to be followed by each project (see the Product Development section).
Testing is a fundamental component within an organization aiming to deliver
quality products. The main goals of the testing process are finding defects before
the product is released and verifying that the product meets its specifications. A
testing process committed to these two goals will successfully contribute to the
success of the VAO project as whole.
The Testing Process
The VAO will employ a formal process for testing software products through their
life cycle similar to the product development process. We will adapt the product
development model and define a testing process of six stages where testing
activities are tailored to each specific product. The product test plan evolves as
the product is developed, and both developers and testers gain knowledge of
what the system should do versus what it is currently capable of doing.
The general philosophy we will follow is that testing is an investigative activity, and
the value of a test lies in the information it provides, in so, we do not enforce a
―test all, document all‖ approach. The VAO testing process aims to reduce inertia
within the project development. The less inertia we build into a project, the more
responsive the development group can be to stakeholder requests for changes
(design changes and error fixes). Our goal is to take advantage of serious unit
testing early in the development process, making functional, integration, and
system tests run far more effectively and efficiently. This requires close
coordination and collaboration between testers and developers.
VAO Project Execution Plan 86 User Support
Testing should address all areas of potential customer dissatisfaction, not just
functional errors. Because matters of usability, performance, localizability,
supportability, and security are critical factors in the acceptability of the product,
test groups should become skilled at dealing with them. More specifically, VAO
acceptance tests will focus on usability and scientific validation of services
and applications with detailed attention to how the software responds
to possible/probable error conditions, especially in the case of misuse.
The tester’s job will be to research defects or design weakness well enough and
to report them clearly to the responsible parties such as the development team,
the Product Development leadership, and the Program Manager.
Testing Life Cycle
Similar to the product development process we envision the following stages in
the life cycle of testing a product:
RESEARCH AND PROTOTYPING
This is the exploratory stage. Exploration lies at the core of competent
investigation and is not merely a way to test without specs or scripts. Testers must
be able to test well without authoritative specification. During this stage the tester
gathers insight and information about the product. It is called also the ―Quick &
Dirty‖ stage where smoke tests, configuration and/or variations on a theme may
be executed. Exploratory test results must be documented, and published in the
shared document system (wiki) unless more detailed tests are being reported.
DESIGN AND PLANNING
After the exploratory stage, a test plan describing how the product is going to be
tested, including a schedule with milestones and dates, is prepared. The test plan
should include:
Brief description of the product under test.
Test Objectives. Because different objectives drive you toward different
testing techniques and results reporting method, the plan should state the
scope and objectives of the test. For example, what is the purpose of the test
suite?
o Find important bugs, to get them fixed.
o Assess the quality of the product.
o Help managers make release decisions.
VAO Project Execution Plan 87 User Support
o Block premature product releases.
o Help predict and control costs of product support.
o Check interoperability with other products.
o Find safe scenarios for use of the product.
o Assess conformance to specifications.
o Certify the product meets a particular standard.
o Ensure the testing process meets accountability standards.
o Minimize the risk of safety-related lawsuits.
o Help scientists improve product quality and testability.
o Help scientists improve their work.
Test framework description with information such as operating system
(Windows, Mac OS X, Linux, etc.), browsers (Firefox, Safari, IE, etc.), Web
services containers (Axis, .Net, etc), and any other relevant hardware and
software information where the application is being tested. This description
should be consistent, but not necessarily limited, to the content of the product
development design and planning document.
Testing tools, if any, being used: NUnits, JUnits, custom software, Agitator,
Performance Profilers, etc.
Test type to be performed such as the typical Unit, Component, Integration,
System, Regression, Acceptance, and Beta tests. Other type of tests such as
Installation, Failover, Recovery, Robustness, Load, Performance, Security,
Stress, Usability, and Data Challenges should also be considered depending
on the product specifications.
List of test cases. Test cases should specify, when possible, entry criteria and
expected outcome. Table 4-1 describes what type of information should be
considered for each case.
Testing techniques applied: Black box, Context driven, First-Test then code,
Random testing, etc.
Testing execution considerations: For example, automation should be done
when the cost of running the tests manually is too high and the information
value of the test justifies the expense. On the other hand, not all tests need to
be automatic. Careful consideration should be taken at the time of test design
whether the cost to maintain up-to date testing code, such as regression tests,
is worth the effort. Other considerations might be isolation requirements,
especially when testing is executed on shared hardware, etc.
Reporting method used: Graphs, list of test cases with entry criteria,
expected results, and actual results, detailed procedural documentation, etc.
Any additional information that may help in the understanding of what is
being tested, and provides useful information.
Knowing what needs to be tested at early stages of the product development is
not always possible. We expect test plans are updated as the product
VAO Project Execution Plan 88 User Support
development progresses through design, development, deployment, and
operations.
Table 4-1: Test Case Specification
Test Case Item Description
Unique identifier to reference the item under test, platform,
Identifier test type, test number, and the test iteration.
The goal of the test , including the name of the product
Purpose and/or the feature in the product under test.
Module Version Version of the software under test
Tester Name of tester
Date Date when tests are performed
Input Specification All input and conditions needed to run the test
Output Specification Expected results
Procedure Procedure to follow
Test Framework Machine, OS, etc. on which to run the test
Pass/Fail Criteria Precise description of pass/fail criteria
Test Results Describe test results vs. what was expected
Pass/Fail Pick one, ID problem(s) if Fail
Fail (correct immediately)
Serious (must correct for release)
Defect Severity Workaround (include description)
Cosmetic
Retest Results Results of retest; state new version if updated
DEVELOPMENT
During this stage, tests scripts and/or code are implemented. Depending on the
type of test, technique, and objectives, this implementation may consist of writing
VAO Project Execution Plan 89 User Support
test scripts and/or custom software, which will be automatically or manually
executed. Careful consideration should be taken at the time of test design
whether the cost to maintain up-to date testing code, such as regression tests, is
worth the effort.
DEPLOYMENT
Tests are executed and results are recorded and documented. These results,
along with any other type of feedback, may be collected through the Help Desk or
users committees and serve to elaborate release readiness reviews.
OPERATION
Some of the test cases and scripts developed during the prior stage, such as
regression and system tests, may require being executed many times after the
product has been released due to operating system updates, software patches,
etc.
RETIREMENT
Eventually, tests become obsolete either because the product changes or
because no new, relevant information is gathered. Instead of running the exact
same test suite repeatedly, the test suit may be retired and those resources are
directed to different problems.
Testing Tools and Policies
The testing process previously described does not differ much from the product
development process. Although we do not anticipate having to develop a
significant amount of code, nor should it have the complexity of the products
under testing, it seems reasonable to use similar tools and policies. After all,
software development uses tools and establishes policies to improve the quality of
the software and ease the development process. Why shouldn’t testing take
advantage of the benefits of a quality process as well?
In this light, we plan to use the subversion repository to store test scripts, small
data sets, and/or any other custom software used for testing purposes. The issue
system (JIRA) will be used to manage the testing process, report defects, and
track problem resolutions. Defects may be reported at any time before or after a
product has been released. We will publish test plans and document test results
using JIRA and a sharable document system (wiki), which, combined with
subversion, will serve as our documentation configuration management system.
VAO Project Execution Plan 90 User Support
Different programming languages may be used to implement test cases; the
language may be the same as the product is developed, but not compulsory.
Code builders, unit test frameworks and documentation extractors will be used as
the need arises and seems convenient and practical.
Coding and documentation guidelines may not be enforced as strongly as for
development, but their usage will be equally recommended. Test design reviews
may occur before entering the test development phase to verify we are not
missing any relevant requirement. At this time, test plans will be compared against
the Requirement Traceability Matrix as defined by the Product Development team.
Quality Assurance and Testing Team
The User Support organization will lead quality control and testing activities.
However, given the size of User Support, the responsibility to carry out testing and
quality control activities will have to be shared across the VAO project and
occasionally call for external help from the community when we lack experience or
scientific knowledge within the VAO.
The role of Quality Assurance and Testing (QA&T) is to assure upper
management that the software development work was performed the way it was
supposed to be, and that software has been tested. The QA&T team is not
responsible for producing quality products or making quality plans; these are the
jobs of the development teams. QA&T is responsible for auditing the quality
actions (Quality Control) as established in the VAO Software Development
Process (SDP) [see the Product Development section], perform additional tests as
necessary, and inform of deviations and impact on product quality.
Quality Assurance is most effective when individuals performing the testing and
quality control tasks are not directly involved in the design and implementation of
a product. After a project is approved to enter the Design and Planning Stage in
coordination with the Program Manager and Development Product leadership, a
small Quality Assurance and Testing team (two to four people with most likely a
representative of User Support as the Lead) will be assembled. The team will
share responsibility for testing and verifying that quality procedures are being
followed, and that software is being properly documented and tested. Although
members of this team must not directly design and implement the product, this
does not ban them from participating in design or/and code discussions. As a
matter of fact, they should be encouraged to participate in such discussions, and
especially attend product development formal reviews. This approach has three
main benefits. First, once in a while everyone has an opportunity to become
familiar with what other teams are developing. Second, the QA&T team is
involved early and continuously in the development process, improving the
VAO Project Execution Plan 91 User Support
likelihood to find defects sooner, which in turns reduces cost and mitigates risk.
Given that we aim for a lightweight product development process, this early and
continuous involvement may well be the best opportunity for testers and
documenters to gather information. Third, and most important, everybody has the
chance and privilege of supporting and improving the quality of others’ products.
Additionally, the QA&T Lead may contact users (of various levels of
expertise) to test products further that have already passed QA&T testing, and to
evaluate the quality of the documentation. This solicitation may happen before,
especially when we lack expertise within the VAO project, or after the product has
been released. Final user involvement is mutually beneficial. VAO tools are
exposed to the community, and the community gets involved in the product
development. The VAO project has a better chance to deploy quality products that
meet users’ expectations.
Quality Control Activities
The Product Development section describes a six-stage life cycle for each VAO
project. Each stage produces deliverables and defines transition requirements,
such as formal reviews. A list of tasks and documents that QA&T will verify exist
and review has been included in Table 4-2. The content of this table may change
as the VAO software development process and testing process evolve and we
gather experience.
Table 4-2: Definition of Quality Control Processes
Task When
Verify the Product Definition Document includes a Stage 1: End of Research and
description of functionality, and initial schedule with Development Stage
milestones
Verify the Project Definition Document has been End of Research and Development
reviewed. Stage
Verify the Project Definition Document is under End of Research and Development
configuration control. stage
VAO Project Execution Plan 92 User Support
Task When
Verify the Project Development Plan includes: Stage 2: End of Design and Planning
specific project goals and deliverables, product or Stage
enhancements design summary and planned
development, including programming languages,
leveraged tools, IVOA standards and dependencies,
list of tasks, assigned staff and delivery schedule.
Verify the Project Development Document has been End of Design and Planning Stage
reviewed.
Verify the Project Development Document is under End of Design and Planning Stage
configuration control.
Verify creation and review of test plans End of Design and Planning Stage
Verify test plans are under configuration control End of Design and Planning Stage
Verify code has been checked in the repository. Stage 3: During Development Stage,
when the product enters Deployment,
and especially before it enters the
Operation Stage.
Verify code follows coding guidelines. During Development Stage, when the
product enters Deployment, and
especially before it enters the Operation
Stage.
Verify creation and execution of test plans. During Development Stage, when the
product enters Deployment, and
especially before it enters the Operation
Stage.
Verify that test plans and test results are under During Development Stage, when the
configuration control. product enters Deployment, and
especially before it enters the Operation
Stage.
VAO Project Execution Plan 93 User Support
Task When
Verify that test code and test data are tracked under During Test Development Stage, when
configuration control. the product enters Test and
Deployment, and especially before it
enters the Operation Stage.
Verify that code documentation has been generated, During Development Stage, when the
is under configuration control, follows documentation product enters Deployment, and
guidelines, and is adequate for development especially before it enters the Operation
purposes. Stage.
Verify that deployment documentation, such as Stage 4: When the product enters
platform descriptions and installation procedures, has Deployment, and especially before it
been generated, is under configuration control, enters the Operation Stage.
follows documentation guidelines, and is adequate for
deployment purposes.
Verify that user documentation has been generated, When the product deployment, and
is under configuration control, follows documentation especially before the product enters the
guidelines, and is adequate for the intended Operation Stage.
audience.
Verify that quality control reviews have been Stage 5: Through the full development
generated, and are under configuration control. process and especially before readiness
reviews when the product enters the
Operation Stage.
Verify that readiness reviews have been generated Stage 6: Before the product enters the
and are under configuration control. Operation Stage.
Quality Assurance and Testing Deliverables
As the development of a product progresses, the Quality Assurance and Testing
team will publish in the shareable document system:
Exploratory tests reports
Test plans
Test results reports
Quality control reviews
Readiness reviews
VAO Project Execution Plan 94 User Support
Before a product is deployed, a Readiness Review Report will be prepared with
recommendations based on test results reports, quality control reviews,
documentation, and external feedback from the Help Desk and user committees.
The report will be presented to the Product Development leadership and the
Program Manager, who shall decide whether the product should be released.
Staffing Roles and Responsibilities
USER SUPPORT LEAD
Provides oversight of all User Support activities within the VAO.
Ensures that such activities remain on schedule and within allocation limits.
Reports the status of User Support activities regularly to the PC, PM and
Director, and to other groups as necessary.
Liaises with Task Leads, the PC, the PM, the Director and others as necessary
regarding the scheduling, provisioning, performance, and completion of User
Support activities.
Reviews and signs off on acceptance test plans and final reports.
Ensures all relevant user documentation is made available and appropriately
archived.
Oversees User Support's presence at the AAS and other relevant meetings.
Disseminates information regarding VAO tools and services to potential users
using the most effective avenues, including the VAO User Forum, AAS
meetings, newsletters, publications in conference proceedings, and tutorials.
Assists in the formation of a VAO User Committee and support its meetings.
QUALITY ASSURANCE AND TESTING ENGINEER
Leads quality assurance and testing activities.
Verifies that software development quality procedures have been followed.
Reviews test results.
Reviews development and deployment documentation.
Conducts user acceptance testing and documents results.
VAO Project Execution Plan 95 User Support
Validates scientific use of service or tool.
Prepares Readiness Reviews.
DOCUMENTATION SPECIALIST
Writes scientific documentation for VAO products.
TRAINING AND ADVOCACY SPECIALISTS
Maintains user forum on VAO Web site.
Supports training activities at the AAS and other astronomical meetings.
Publishes electronic VAO newsletters regularly.
Creates tutorials for new users and cookbooks with recipes for experienced
users.
Supports the VAO summer school.
WEB MASTER
Evolves and supports the VAO Web site.
Provides convenient access to the VAO documentation.
HELP DESK MANAGER
Provides access to the User Help Desk.
Maintains the Help Desk software.
Manages and assigns the Help Desk tickets as appropriate.
VAO Project Execution Plan 96 User Support
Section 5. Product Development
Goals
This chapter describes the development model and process that will be used to
deliver the VAO software products identified in this Project Execution Plan.
Products, in this context, refer to all the Web-based tools and services, service
toolkits, and desktop tools the VAO will deliver. They include:
Science services for end-users
Tools to support the Data Curation and Preservation work area
Desktop integration tools
Research and development efforts, as approved by the VAO Program
Manager, such as those needed to support Technology Assessment
These software products will be developed simultaneously by several teams
within the VAO. The goals of the product development plan are as follows:
Provide a rigorous process that drives the production of reliable software
products, yet is flexible enough to evolve according to the needs of the product,
its science goals, and changes in technology.
Minimize duplication of effort across product development teams.
Provide mechanisms to release software ―early and often.‖
Coordinate the development with User Support (Testing), Operations, and
Standards and Protocols.
Ensure maximum efficiency by regular reviews of the effectiveness of the plan.
Ensure intellectual co-ownership of the process by the development team.
We recognize that VAO products will be deployed into an overall VO ecosystem
that includes tools and services developed by other VO projects or individual
users, and that must interact with each other. In many cases, the VAO will
leverage many of those tools and services. Some of those "outside" tools may be
produced by people involved with the VAO. In that light, we note that the focus of
the process described here is specifically on products that the VAO invests effort
to develop or maintain. The VAO may, depending on that level of investment,
place requirements on how the product is developed, particularly in terms of how
much of our process must be applied to the product. For example, the VAO
VAO Project Execution Plan 97 Product Development
investment in a product may be as little as hosting the code repository and
associated product pages. At the other end of the spectrum, a product which is
produced solely by the VAO will adhere rigorously to the VAO process.
Evolution of the Development Plan
This document defines a baseline Product Development Plan. We plan an update
to this baseline plan in December 2010. This update will include two changes to
the baseline:
It will reflect comments on the baseline plan by the development team, once
assembled. This step is essential in ensuring intellectual co-ownership of the
development process by all its stakeholders, and will be an immediate priority
at project start-up.
It will make final decisions on the tools needed to support product development
(see the subsection Tools and Policies).
The new plan will be captured in a document called the VAO Software
Development Plan. It will be prepared by the VAO Product Development (PD)
Task Lead and VAO Product Development Deputy Task Lead, and approved by
the VOA Project Manager. The VAO Software Development Plan will be a
versioned document that is subject to VAO configuration management practices.
Thereafter, revisions are expected to made quarterly in year one, semi-annually
in year two, and annually thereafter.
The process for producing the software development plan is as follows:
1. Inputs can be gathered throughout the year from the developer team in the
form of brief change proposals. Approximately two months before the end of
the year, a call for change recommendations will go out to the team. Change
proposals are made available to the entire PD team.
2. Approximately one month before the end of the year, the proposed changes
are reviewed and discussed by the PD team with the goal of achieving
consensus on which proposals to adopt and how they might be modified.
3. Incorporate such changes to the process as mandated or recommended by
VAO Management, the VAO Board of Directors, or the NSF or NASA, through
the VAO Oversight Group.
4. With assistance from the Deputy Lead, the PD Lead makes the final selection
of changes and produces an updated version of the SDP which includes a
summary of the changes. This version is made available to the Program
Council for review and approval.
VAO Project Execution Plan 98 Product Development
5. The VAO Program Manager approves the revision.
The Project Life Cycle
VAO software deliverables will be developed by several teams simultaneously.
The Task Lead and Deputy Task Lead are responsible for ensuring the software
is delivered on schedule and meets their requirements. It is highly likely these
efforts will need a common set of tools and libraries for performing common tasks,
such as querying for data, formatting and presenting results. Thus, the software
products will use a component-based architecture, whereby components can be
plugged together to support building new components. These common
components, which we will call products, will be identified during the design phase
and at the Task Lead’s discretion. They may be developed independently of the
deliverable products. These components will be treated as deliverables in their
own right and will be developed by the same life cycle as the software products
that will use them. Services and applications that are created by using these
components and by generating their own custom code will be called projects.
Choice of Life Cycle Development Model
The VAO considers on-schedule delivery of its products and projects as crucial in
attracting and retaining end-users. We have therefore adopted the ―design-to-
schedule‖ development model for releases of all products and projects except
research and development efforts.
The ―design-to-schedule‖ model operates by identifying the highest priority
requirements that must be delivered in a release, and then ensuring they pass
through the stages in the development cycle. If there is time available in the
schedule for that release, then the next highest priority requirements are satisfied,
and so on until the deadline is reached. This model is most effectively employed
in new teams with hard deadlines, and provides the option of early release of
important features desired (―release early and often‖). The model provides
stakeholders very good visibility of progress, but at the price of limited ability to
make midcourse corrections. This risk will be mitigated by ensuring that
requirements are accurately specified.
The primary risk to meeting the schedule lies in the inefficiencies in distributed
development. The following steps will be taken to protect the schedule:
The detailed schedule for each phase of development will include a margin of
50% of the estimated time for completion.
VAO Project Execution Plan 99 Product Development
Projects will be broken into a series of sub-milestones whose delivery dates will
be closely monitored.
The sub-milestones will be developed by named personnel who be held
responsible for their completion.
Development processes will be monitored and upgraded to improve efficiency
and improve scheduling of future releases.
Management will reassign staff to support on-time delivery of services, as
required.
Research and development efforts will use the evolutionary prototyping model,
which is efficacious when requirements are poorly known or change rapidly and
delivery is not driven by schedule. The model involves designing and
implementing the most prominent parts of the program in a first prototype, and
then refining. The development will require requirements analysis, design, etc.,
but in much smaller increments than in the design-to-schedule approach.
Project Life Cycle Overview
A product will generally have multiple to satisfy all its requirements. There are five
phases in the life cycle of each release of a VAO software product:
1. Requirements Gathering, Research and Prototyping: An exploratory phase
aimed at gathering knowledge, experience, identifying tools and components
that can be re-used, and exploring algorithms that can be applied to the
project.
2. Design and Planning: A stage that creates a Project Plan that refines
product requirements and outlines how the product will be developed,
deployed, and operated.
3. Development: The stage where the Project Plan is carried out and the
product is developed, unit tested, and documented.
4. Testing: Candidate releases are built in a test environment for evaluation by
the Testing team. Candidate releases of the product are delivered to the
Quality Assurance and Testing team for testing. The following types of testing
will be performed: system testing (including interface and integration testing),
acceptance testing, and end-user evaluation.
5. Deployment: Final releases of the products are created and submitted to the
configuration repository (see the Operations section). Service products are
deployed to production systems by the Operations team.
The project will then return to the requirements specification to begin the process
for the subsequent release.
VAO Project Execution Plan 100 Product Development
Tools and Policies
We will use various development tools (e.g., a code repository) to track the
evolution of a project as it passes through the six stages of its life cycle. Similarly,
there will be policies (e.g. reporting, coding guidelines) that will be applied
throughout the life cycle. This section summarizes the role of key tools and
policies that will be part of our development process and enumerates
requirements and recommendations that should guide the choices we make when
working out the detailed SDP. The next section describes the process in life cycle
order, and makes reference to these tools and policies. We note that what is
presented here is primarily an outline with many omitted details to be determined
in the version of the SDP (see Process Evolution, above).
Tools
CODE REPOSITORY
A code repository (such as Subversion, currently used by the NVO) allows a
distributed development team to share software source code in an up-to-date and
coherent manner. An important feature of a code repository is that it "versions" the
individual source code files automatically: each version has an associated
identifier, any version can be extracted, and differences between versions can be
readily computed and displayed. It can be used effectively to manage code within
coherent products and projects. as well as manage different releases of a product.
As all code developed with support from the VAO is expected to be open (i.e.
freely accessible and distributable), the code repository needs to be open to read-
only access to the public. (Where local institutional policies for intellectual property
or, when applicable, agreements with industrial partners conflict with this practice,
an alternate repository may be used.) Further, all code developed with support
from the VAO is expected to be maintained in the VAO repository (with exceptions
to be documented in the appropriate Project Development Plan; see The Project
Life Cycle, below ). Recognizing that our user community can be a great source
for software updates, bug fixes, or other more substantial coding contributions,
our repository plans must also address possible update by developers outside the
VAO project. That is, there needs to be a mechanism for enabling write access for
"external" developers, along with a corresponding policy granting such access.
A uniform organization of code within the repository for a product will use a
consistent organization as much as appropriate for the kind of product. This will
make maintenance easier internally as well as make its use clearer to users.
VAO Project Execution Plan 101 Product Development
ISSUE SYSTEM
An issue system (also referred to as a ticket system) is a software tool that allows
us to identify, describe, and track units of work assigned to specific people. There
are three types of software issues we need to track: (1) planned development
tasks drawn from a Project Development Plan, (2) significant development
milestones (e.g. product releases), and (3) bug reports and minor enhancement
requests. Like the code repository, it must work naturally with a distributed group
of developers; this is typically accommodated by the commonly used issue
systems through a Web interface.
The VAO will, in the first instance, use the JIRA ticket system that is maintained
as part of User Support, and evaluate its capabilities as a user support and defect
tracking system at the end of year one.
SHARABLE DOCUMENT SYSTEM (WIKI)
The development process will rely on documents authored by developers which
will evolve with though a project life cycle. These include design documents,
product release information, and general end-user documentation. Our
experience in the NVO project showed wikis provide a convenient, portable, and
lightweight way to create and update group-authored documents in a distributed
environment. The usefulness is greatly increased when it is integrated with the
code repository and the issue system (as it is with the NVO's Trac system): it
makes it simple for document authors to refer and link to specific development
tasks or bits of code.
PROGRAMMING LANGUAGES
As a whole, our community of tool developers desire reusable, VO-supporting
code in a variety of software languages. As an ecosystem, the VO is not
intrinsically tied to any particular software language. Our users want access to VO
capabilities from any language. We as a project must, in principle, support any
language. The choice of programming language or languages to be used in
implementing a VAO software product will need to be decided on a project-by-
project basis during the design and planning stage. The choice will be made
based on issues of performance, availability of useful libraries, and maximizing
take-up by the targeted community. Our support tools (e.g. code repository, issue
system) will need to be similarly language-agnostic. The SDP may, however,
recommend specific versions of languages or compilers to maximize the number
of supportable platforms and provide consistency across products in both
behavior and ability to build them.
VAO Project Execution Plan 102 Product Development
CODE BUILDERS (MAKE SYSTEMS)
While many software-building tools (e.g. make, ant, scons) provide multi-language
support, certain builders work best with tools written in a specific language. For
example, "ant" is considered the best tool for building Java products efficiently,
while "make" works well with compiled languages (such as C, C++, and
FORTRAN). An important factor in choosing a builder tool is what is already
typically familiar to users (which depends on the product language). We expect to
endorse multiple builder tools, but we will minimize this list so as to maximize
consistency across tools; that is, we would like to be able to build a product using
one of only a few possible ways.
UNIT TESTING FRAMEWORK
Unit tests are a set of tests that verify the behavior of a software tool on sub-tool,
functionality-oriented level. Like the code builder, the best framework to use to
create unit tests depends on the language used to implement the product. The
SDP will recommend the unit framework to use for each language in use in the
VAO. The framework must be integrable with the systems used by the Quality
Assurance and Testing team and meet all of their requirements.
CONTINUOUS INTEGRATION SYSTEM
A continuous integration system (CIS) is one that periodically and automatically
builds a software tool and, possibly, runs its unit tests. Such a system can be
used to check when code updates "break" a software product. This kind of
automated testing is a development tool—one intended to inform developers as to
how their day-to-day check-ins are affecting the health of the product’s main-line
code, as well as its integration with other products. (This is in contrast with the
final integration testing done by the QA&T group and the VO health testing done
by the Operations group. When integrated with the code repository, a CIS can
report the broken code and the responsible developer. The SDP may specify the
use of such a system.
CODE DOCUMENTATION EXTRACTORS
Several languages (including Java, Python, and C#) feature built-in code
documentation extraction systems. These systems usually define special
documentation markup that a developer uses to associate documentation with
specific parts of the code, in particular, software interfaces. A tool can then extract
the documentation into convenient, browse-able documents for developers who
will use that code. For compiled languages (e.g. C, C++, FORTRAN), the open
VAO Project Execution Plan 103 Product Development
source package Doxygen provides this capability. We expect to recommend the
use of these tools in conjunction with specific documentation guidelines (see
Coding and Documentation Guidelines below).
Policies
CODING AND DOCUMENTATION GUIDELINES
Coding guidelines attempt to establish a common style for composing code. The
purposes of such guidelines include encouraging good coding practices that avoid
common errors. They also serve to encourage consistency and clarity that makes
the code easier to read. This is important in a development team that may evolve
over time in which the developer who originally writes the code is not the one who
maintains it, or is otherwise called on to add additional functionality.
Documentation guidelines help ensure the software is sufficiently documented.
Consistency in the documentation can not only improve clarity, but also the use of
specialized markup allows the creation of well-organized, browse-able interface
documentation.
As we will be developing tools in a variety of languages, it will be difficult (and
perhaps too expensive) to establish detailed coding and documentation guidelines
for all of the languages we leverage. However, we expect in the first year to set
down broad recommendations with examples in one or two languages. Similarly,
broad documentation guidelines may also be set, with some minimum
requirements for passing quality assurance.
PROJECT REVIEWS
We will employ reviews of each development project during strategic points in the
project life cycle to help ensure the resulting products deliver the intended
capabilities and are robust and effective.
The set of expected reviews include:
Project Definition Review: Based on a written description of a project and its
deliverables and requirements in terms of user capabilities, this review approves
the definition of a project so that it can have resources allocated to it.
Design Review: Based on a written description of a product design and
implementation plan, this review approves a projects transition from the Design
and Planning Stage to the Development Stage.
VAO Project Execution Plan 104 Product Development
Final Product Review: This review approves a product for release to and/or
deployment for use by users in a production environment. It is based on the
results of a quality assurance assessment, extensive testing, and verification of
adherence to any required coding and documentation guidelines.
The details for how these reviews will be conducted will be spelled out in the
Software Development Process document; however, below in the Project Life
Cycle subsection, we describe how these reviews fit into the evolution of a project.
RESOLVING PRODUCT ISSUES (BUG FIXING)
In collaboration with the teams for Quality Assurance and Testing (QA&T) and
User Support, we will develop for the Software Development Process a flow
control for handling the resolution of issues—i.e., bugs—against released
products. In general, this will involve formally assigning the issue to a developer
for resolution. The QA&T team will test the release (possibly involving the
reporting user), and the developer will create a new release of the product (see
also Bug Fix Releases).
We recognize we must strike a balance between ensuring robust software with a
practice of delivering "early and often." To that end, we look for ways to keep the
first two types of reviews as lightweight as possible while still ensuring the
products deliver the intended capabilities. Documentation is at the core of these
reviews; keeping these documents concise and based on a well-defined template
can be the key to keeping the process lightweight.
STATUS REPORTS
The PD developers will submit brief but regular reports to the Product
Development Lead documenting progress on project milestones. The exact form
of the reports and the interval between them will be specified in the first version of
the SDP. The Product Development Lead will use these reports as input for the
VAO quarterly reports.
The Development Process
This section summarizes the development process as a whole, in which a project
moves through the different stages of development. As summarized in the
Introduction of the Product Development section, the six stages represent a cycle.
A new project can, in principle, enter the cycle at any stage, and a project can
cycle back to a previous stage as a means of adding new functionality to a
product or products. We note that the Configuration Management Plan, described
VAO Project Execution Plan 105 Product Development
in the Operations section, plays a key role in the evolution of a project, and Table
3-4 summarizes the decision points in the life cycle. Below, we describe how
configuration management enters into the project life cycle.
Project Definition
SCIENCE-DRIVEN PROJECTS
The VAO Program Council, in consultation with the Science Council, develops a
set of current science initiatives defined in terms of a set of broad capabilities that
allow a user to carry out some type of astronomical research activity. The VAO
Director and Project Scientist select and prioritize the capabilities to be worked on,
highlighting those to be delivered in the coming year. The Program Development
team, led by the PD Lead and Deputy, organize the functionality needed to deliver
the capabilities into projects. As different science initiatives may require common
functionality, so may projects serve multiple initiatives. Each project is assigned a
Project Lead; this lead is responsible for creating a Project Definition Document
(PDD)—a brief description of the project with the functionality to be delivered—
and a broad, initial schedule of milestones.
The PDD is submitted to the Configuration Manager as a proposal.
The PDD is subject to a Project Definition Review by the Configuration Control
Board (CCB) or their designates. Based on the review, the VAO Program
Manager gives final approval to the project, and resources are allocated to pursue
the project goals.
Typically, a new project will enter into the life cycle either though the Research
and Prototyping Stage or the Design and Planning Stage, depending on how well
the project is defined.
EXPLORATORY PROJECTS
Exploratory projects may be proposed by anyone either inside or outside the
VAO. Such projects provide an opportunity to explore an innovative or risky idea
for improving the effectiveness of the VO environment. Such a project is proposed
be creating a Project Definition Document and submitting it to the CCB for a
Project Definition Review. If approved, the proposer would typically be the Project
Lead. Given the speculative nature of such projects, they would typically enter the
life cycle through the Research and Prototyping Stage.
VAO Project Execution Plan 106 Product Development
Requirements Gathering, Research and Prototyping Stage
A project entering into this stage may only have a rough concept of the
capabilities it intends to deliver. The purpose of this stage is to clarify the project
concept. The primary deliverable from this stage should be a more specific Project
Definition Document (PDD) that can detail the specific functionality to be delivered
in the form of a list of requirements. If the exploration fails, a simple report of the
findings will be delivered instead. The requirements specification will recommend
an order of priority for their implementation.
To move to the next stage, an updated PDD is be submitted to the Configuration
Manager and is then subject to a Product Definition Review by the CCB. The
results of the review, including recommendations, are appended to the PDD. The
VAO Program Manager must give final approval in order for the project to move to
the next stage. The VAO Director and VAO Project Scientist must approve the
recommended implementation priority.
Design and Planning Stage
The main deliverable of this stage is a Project Development Plan (PDP).
Following a set format, it should spell out the specific project goals and
deliverables in terms of delivered software products or enhancements to existing
products. In particular, it should spell out a refined list of requirements that can be
used to assess whether the products meet the project goals. The document
should summarize the design of the products (or enhancements). It should also
describe the planned implementation, specifying the programming languages
used, existing tools leveraged. It should describe any use of existing IVOA
standards as well as dependencies on new IVOA standards to be developed.
Finally, it should provide a list of tasks to be assigned to specifically named
people and a schedule for completing them.
A Project Development Plan may also include an enumeration of the exceptions
to be made against the normal software development process. For example, if the
project involves a collaboration with an industrial partner and integration with a
commercial product, it may request that a different code repository be used. Each
exception should be justified.
To move to the Development Stage, the Project Development Plan must undergo
a design review commissioned by the Product Development Lead. In this review,
the design and implementation should be assessed to ensure it will deliver the
stated capabilities, complies with the software development process and
requirements, that its exceptions to the process are justified, and that it can be
VAO Project Execution Plan 107 Product Development
practically operated and maintained after deployment. The reviewed Project
Development Plan is submitted to the Configuration Manager.
Development Stage
In this stage, the assigned tasks and schedules are integrated with the overall
VAO project schedule and work is begun. The developers will be required to
regularly report progress on the assigned tasks to the Project Development Lead.
All code is continuously checked into the project code repository. Development
includes the creation of required unit tests and documentation.
When all development tasks are complete, a product should be ready for final
testing. Candidate releases of the products updated by the project are created
(these are the ―test versions‖ described in the Configuration Management Plan in
the Operations section). With the approval of the PD Lead or Deputy, the
candidate releases are built in the test environment and made available to the
Quality Assurance and Test team.
The Testing Stage
The Quality Assurance and Test (QA&T) team handles the integrated testing. This
team is responsible for system testing (including interface and integration testing),
acceptance testing (verifying that the code satisfies its requirements) and end-
user evaluation. The QA&T team interacts with the project team by reporting
problems and issues through the issue tracking system. Resolution of the issues
(bug fixes) would result in new candidate releases which are passed back to the
QA&T team. That team concludes their work with a brief report summarizing the
testing done (see Quality Assurance) and their recommendations: either to
approve the products for final release or recommend returning the project to the
Development Stage to address remaining problems. Even if the team approves a
product for deployment, it may recommend further development usability of the
product or add a necessary but previously unforeseen feature.
For projects that produce releases for multiple products, approval may be
recommended on either a per-product or per-project basis, depending on what is
most appropriate for the project (taking into account, for instance, software
dependency issues). The CCB gives final approval to products ready for release
and deployment.
VAO Project Execution Plan 108 Product Development
Deployment Stage
Once a project’s products have been approved for release, the project team
creates a final release for the product and submits it to the Configuration Manager
(see Specifying a Version) to be put into the Configuration Repository. The
Operations team then handles deployment of the product.
Some products—such as libraries and user tools—are "deployed" with an
announcement of a release with an assigned version identifier (e.g. 1.0.0). For
such products, the lead developer is responsible for creating a user-oriented
product description page viewable over the Web. This should provide links to
packaged releases of the product along with pointers to instructions on how to
install and use it. Other products are intended to be deployed as a service
maintained and operated by the VAO. The code for these products should also be
available as download-able bundles, just like a library or tool product. In addition,
the development team works with the Operations team to install the service,
connects it with other VAO services (e.g. the VAO Portal), and sets it up for
continuous monitoring.
Other Life Cycle Considerations
LOOPING BACK IN THE CYCLE
Under a "release early and often" strategy, it is expected that a Project
Development Plan will include a schedule with multiple phases, each delivering
additional functionality. In this case, once such a project’s products go into
operation, it effectively returns to the Development Stage for work on the next
round of features. If the Product Development Plan needs to be updated, say, in
response to recommendations from the Integrated Testing phase, the project
would re-enter a (possibly brief) Design and Planning Stage; the updated PDP
would need to be subsequently re-reviewed.
We note that all projects are subject to re-evaluation and prioritization as part of
the yearly update to the Program Execution Plan and the scientific initiatives. As
part of this process, ongoing projects might be scaled back or retired projects may
be revived to deliver additional capabilities.
BUG FIX RELEASES
A portion of every developer’s time will be allocated to addressing bugs that are
reported through the issue system against production releases of products. The
User Support team can assign an issue to a product developer based on
information from the Configuration Repository. (The developer may reassign the
VAO Project Execution Plan 109 Product Development
issue to another developer as appropriate.) When the developer has checked in to
the code repository a fix for the product, a candidate release is created and
submitted to the QA&T team for validation. Upon approval from the team, the
developer can issue a final release. This release is referred to a ―bug release;‖ the
process for its creation can be compressed compared to the planned releases.
When the final release for product used in a deployed service is submitted to the
Configuration Manager, it is at the discretion of the CM as to whether and when
the service is redeployed with the updated product. For example, the CM may
wish to wait for several fixes to accumulate before updating a deployed service.
ROLES AND RESPONSIBILITIES
Figure 3-3 summarizes the roles and responsibilities of staff supporting this task.
VAO Project Execution Plan 110 Product Development
Section 6. Standards and Protocols
The VO is not a single, integrated system, but rather a federation of distributed
components (portal applications, desktop client applications and tools, data
services, information services, computational services, etc.) that run at widely
distributed sites and connected via the Internet. These components can be very
diverse in terms of implementation and origin. While critical elements of the
facilities delivered by the VAO may be implemented directly by the VAO itself,
others will come from international partners or from the broader community of
users, application writers, or data providers. Often existing legacy software or
facilities must be retrofitted to integrate with the VO.
The key to integrating such a diverse collection of components into a useful
system is to define the interaction between the components in terms of open
standards and protocols, each defined independently of the underlying
implementation. These standards and protocols define much of the structure and
functionality of the distributed VAO system, as well as the global virtual
observatory itself.
Challenges
The challenge for the VAO is to provide a useful system that delivers robust,
science-driven functionality to the astronomical community, while at the same time
working with international partners to define the standards required to provide
uniform access to major astronomical data collections and other related resources
from around the world. These goals are often in conflict: delivering advanced
capabilities on a reasonable time scale to a specific community requires a well
organized project-oriented approach with dedicated resources, whereas a global
standards-based solution may often require lengthy negotiations with international
partners who may have different priorities and schedules and who all have to
deliver to their own user communities.
To succeed on a global scale, the goals of all major international partners must be
addressed. What is required is a classical scientific collaboration, with
international partners working together in a collegial fashion to forge a common
solution, while respecting the possibly differing interests and requirements of each
other's projects.
In practical terms, achieving both of these goals will often require a two-track
approach that allows project schedules to be met while ultimately converging to a
common standard. In the case of the VAO, standards development must be done
VAO Project Execution Plan 111 Standards and Protocols
within the context of the global IVOA collaboration. Draft standards sufficient to
support VAO development can be produced within the VAO while collaborating
with IVOA partners. These draft standards should be sufficient for initial
implementation of real VAO facilities, but will effectively be prototype
implementations from the broader IVOA perspective. Other IVOA partners may
produce comparable prototype implementations in parallel with those being
developed within the VAO. These working standards and their associated
prototype implementations will help inform and speed development of the final
IVOA standard. Once a stable IVOA recommendation is achieved, the VAO
implementations will need to be updated to be standards-compliant, however,
such software evolution is already a normal part of the development of any large
system.
Interactions With the IVOA
The VAO is an active partner in the IVOA. This international collaboration aims to
provide uniform, standards-based access to data globally, and to enable sharing
and interoperability of software. Hence, for example, tools developed in Europe
such as TOPCAT (for table analysis) and Aladin (for image and catalog
visualization) are heavily used within the U.S. for analysis of data obtained using
the VO. Likewise, many tools from the U.S. will be useful elsewhere, for example
the SED and time series tools being developed by the VAO as well as VO-
enabled legacy software such as IRAF and IDL. Collaboration within the IVOA
framework makes it possible to develop and promote the standards needed for
such software sharing and interoperability, as well as to enable uniform global
data access.
Many of the current VAO team members, as part of the former NVO, were
instrumental in forming the IVOA in 2002 and have assumed leadership roles in
succeeding years. Bob Hanisch served as the first IVOA Chair starting in 2003,
with the Chair subsequently rotating through successive heads of the national VO
projects. Dave DeYoung is the current Chair of the IVOA Standing Committee on
Science Priorities. Senior members of the VAO team have chaired or co-chaired
many of the IVOA working and interest groups since the formation of the IVOA,
and have been heavily involved in developing the IVOA standards and current VO
technical infrastructure.
The IVOA committee on science priorities, formed from senior members of the
national and regional VO projects, identifies the top priority science initiatives for
the IVOA, used to define priorities for ongoing IVOA standards development. The
IVOA working groups define the standards; for a given standard, the design team
will be international, representing the major IVOA partners involved and bringing
together the top scientific and technical experts in the field. VAO team members
VAO Project Execution Plan 112 Standards and Protocols
participate in and often lead individual IVOA standards efforts, and hence are in a
good position to help coordinate VAO and IVOA developments.
Types of Deliverables
Standards are required to define the high-level structure and interfaces of large
federated systems, e.g., protocols, document formats, data and metadata
standards, and so forth. Standards development requires some or all of the
following types of deliverables:
Specifications
Reference implementations
Supporting deliverables, as needed, such as verification and compliance test
tools
The heart of any standard is the specification, a versioned document specifying
the interface, protocol, data model, etc., defined by the standard. In the case of
the IVOA, a process is already in place for producing and publishing formal
standards documents; this is patterned after the World Wide Web Consortium
(W3C) standards process. VAO will comply with these existing standards
processes when producing such a broader international standard.
Some standards may be internal to the VAO, in which case the specification will
be an internal VAO technical document. So far as VAO development is
concerned, there is no functional difference between internal VAO standards and
broader IVOA standards. Standards developed within a national project such as
VAO will necessarily often go beyond what is defined by an international
standards body such as the IVOA, and will often inspire future international
standards development.
Complex software standards are not likely to be realistic and reasonable to
implement unless they are accompanied by one or more reference
implementations. A reference implementation is an implementation produced in
concert with the specification document and illustrates how to implement the type
of software referenced in the specification correctly. Both the IVOA and FITS
standards processes require (or at least strongly encourage) interoperable
reference implementations.
In the case of the VAO, a VAO-produced reference implementation of a standard
may also be used to implement VAO facilities, or made available to the user
community as a software product. For example, the DALServer service framework
provides a reference implementation of the VO data access services to help
VAO Project Execution Plan 113 Standards and Protocols
inform IVOA standards development, but it is also a software product used to
construct data services.
Supporting deliverables associated with a standard may include, as needed, a
verifier (a tool used to verify the correctness of an implementation), demonstration
applications, user documentation, etc.
The VAO Standards Development Process
Standards development is an iterative process. Priorities and requirements for
standards and other technical infrastructure flow from specific science use cases
as well as the more general scientific usage anticipated. Useful standards and
related infrastructure are produced and used to build end-user applications or
other facilities. The process then repeats to enhance the delivered capabilities
based on user feedback and needs. In the case of (IVOA) standards, an
additional development track proceeds in parallel with that of the VAO, with the
two tracks converging as the standard becomes stable and is finalized.
Roles and responsibilities. Any VAO team member may be involved in
standards development. The team assembled to produce a standard may include
not only the authors of the standard, but also scientific consultants or application
developers who will have to use the standard. The Standards and Protocols Lead
and Deputy are responsible for coordinating standards development within the
VAO, including identifying areas where new standards are required, composing
standards definition teams, and reviewing and signing off on the final product. The
Standards and Protocols Lead and Deputy have primary responsibility for
planning and coordinating VAO technical standards development with the IVOA.
Prioritization. A VAO-wide process looks across all planned VAO development
(e.g., the primary science use cases and related facility applications) and
determine where standards and standard infrastructure are required. This will be
done as part of the program planning process each year, although the activity will
continue at a lower level throughout the year. As part of the yearly program
planning process, the Standards and Protocols Lead and Deputy, working with the
Program Council, will analyze the requirements of the development plan for the
upcoming year and update the priorities for standards development accordingly.
IVOA priorities and commitments must also be respected when determining the
final priorities for VAO standards development.
Interface design and specification. Once priorities have been established and
work on a standard is ready to go forward, the next step is to design and specify a
working draft standard. This is done by the team formed to develop the standard,
and begins with analysis of the design reference use cases. Once a working draft
VAO Project Execution Plan 114 Standards and Protocols
is complete, it is made available for review and comment within the VAO, and in
particular by any initiatives within the VAO that depend on the standard. The exit
criteria for this phase is a working draft standard sufficient to support prototyping
and reference implementations within the VAO.
Prototyping and reference implementations. A reference implementation and
testing and verification tools (used to test implementations against the standard)
will be produced for any major standard. Prototyping of applications that use the
standard may also occur; in the case of VAO applications these might be actual
user applications or other VAO facility software. In general, both prototyping and
reference implementations are required for any major standard before a final
standard is produced. In simpler cases of standards internal to the VAO,
prototyping may be sufficient.
Community interaction and review. While standards for technical infrastructure
required only to produce end-user applications might be produced entirely within
the VAO or broader technical communities, in many cases it will be necessary to
get the science user community directly involved before a standard can be
finalized. This will typically be necessary where science data or scientific data
analysis is involved and much or most of the applications development is
expected to be done by the user community, outside the VAO per se. For
example, before the standard interface can be finalized for image data cube
access (SIAV2). It will be necessary to involve the radio community to ensure the
necessary capabilities are provided for describing, accessing, and visualizing
radio data cubes. Similarly, analysis topics such as time series analysis and
spectral/SED analysis will require the engagement of scientific experts and
developers from the broader community.
Delivery and iteration. Development of standards and the applications that use
them is done in cycles, "closing the loop" in each cycle. For standards
development this means a stable, often working draft standard is produced,
development goes forward to the point of delivering applications to end-users and
getting feedback, and then the next cycle of development begins.
In the case of VAO, standards development is part of the overall science initiative
or product development cycle, hence this cycle repeats yearly beginning with
updating the program plan for the next year of VAO development. Hence the
process described here repeats yearly for most standards development,
beginning with prioritization. An exception occurs when development of a specific
science initiative through to a completed phase 1 end-user science product may
exceed the yearly development cycle (perhaps due to limited resources causing
development to be stretched out), in which case some stages of the process
described here could still be scheduled and completed within a single
development cycle.
VAO Project Execution Plan 115 Standards and Protocols
Development of an IVOA Standard
If a two-track approach is required to develop a standard to support internal VAO
development, the result of the VAO standards development process outlined in
the previous section is not normally an IVOA standard. Some additional amount
of time (often 1-2 years, sometimes even longer) is typically required to develop a
major IVOA standard. From the IVOA perspective, the result of the VAO
standards development process is either a working draft for the eventual IVOA
standard, or merely an IVOA Note to be used as input to the IVOA standards
development process. Updates to the VAO working draft version of the standard
may be required as IVOA standards development proceeds. When the process
ultimately converges, the VAO standard and the IVOA Recommendation are one
and the same.
Priorities for Year One
As noted in the VAO Standards Development Process above, the priorities for
VAO standards development are determined yearly, as part of the program
planning process, based on what is required to support the VAO science
initiatives or other high priority development planned for the upcoming year.
These priorities flow from the VAO priorities for science capabilities to be
delivered to users. Yearly priorities are driven primarily by the short-term schedule
(what can be delivered in the development and planning cycle that is defined by
the time scale to update the yearly Program Execution Plan), but priorities for
longer-term initiatives must also be taken into account, as well as IVOA priorities
and commitments made to international partners.
The highest priority items for VAO standards and infrastructure development in
year one, identified by the process described above, are summarized below.
Much of what is needed to support the year one VAO science initiatives is already
available as the result of ongoing IVOA standards development, however
additional work is needed in many cases. For example, IVOA already supports
access to spectrophotometric data in the form of 1-D spectra, but the data model
and access interface need to be extended to support SED and time series data,
which are closely related but not identical. Likewise, IVOA already defines a
protocol for querying table data, but additional work is needed to support querying
tables of observational data to support global data discovery in the VAO portal, or
to support large-scale distributed cross-matches.
The following standards development tasks are required to support the VAO
science initiatives or other high priority VAO/IVOA development in year one:
VAO Project Execution Plan 116 Standards and Protocols
1. Working draft SED data model and access protocol, and photometry
data model. Required to support the SED science initiative.
2. Working draft time series data model and access protocol. Required to
support the time domain science initiative.
3. Table query-related standards. Required to support the portal (global data
discovery and access) and cross-match science initiatives, however, only the
initial standards development and prototyping is expected to be completed in
year one.
4. Simple Image Access version 2 (e.g., cube data). Not tied to any specific
year one VAO science initiative, but required to support cube data from
modern radio instruments and surveys now coming online (e.g., ALMA), as
well as modern IFU and event counting instruments. Only a working draft
standard sufficient to support 2-D prototyping and review is expected to be
completed in year one.
5. SED data model toolkit. Required to support the SED science initiative, to
develop SED data services and client analysis applications.
6. Time series data model toolkit. Required to support the time series science
initiative, to develop time series data services and client analysis applications.
7. DALServer reference implementations of new service protocols.
Required to provide data service support for the SED, time series, portal, and
cross-match science initiatives, as well as to provide reference
implementations of the new services as part of standards development.
8. DALValidate support for verification of selected new services. Required
to verify that data provider-implemented services function correctly.
9. Event notification enhancements. Required to meet the VOEvent standard
(VOEvent 2.0) to support the time series science and portal initiatives.
VAO Project Execution Plan 117 Standards and Protocols
Table 6-1: Year One Priorities
Task
Number Deliverables Status
Working draft data model and Extension of existing IVOA standard
access protocol specifications, 1D spectrum interface and access
1&2
sufficient to support VAO protocol. Some work already done
development on SED and Photometry models.
IVOA ObsTAP specification nearly
Completed draft IVOA ObsTAP
complete. Parameter query draft
specification. Draft parameter
3 exists, but need to add support for
query specification for simple
ObsTAP queries once the data
queries of observation tables.
model is completed.
Working draft SIAV2 specification Draft exists, but not yet ready for
sufficient to support 2D prototyping prototyping and reviews. Some
4
and review of advanced data cube research done on cube access
access capabilities. capabilities.
Already have version supporting 1D
5 SED data model Java classes.
spectra.
Time series data model Java
6 Same as for SED/spectra.
classes.
DALServer framework supporting
Reference implementations for
existing standards already available.
7 table, SED, and time series data
Need to add support for new
access services.
standards.
DALValidate support for new DALValidate framework supporting
8
services. existing standards already in use.
Some ongoing standards development work is also expected to be done on the
following, though this work is not a priority for completion in year one:
VAO Portal/application standards (data interchange, table manipulation and
viewing, populations). Required to support the VAO Portal science initiative.
VO Grid and Web Services standards for per-user network data storage
(VOSpace 2.0), asynchronous job execution (UWS), and high performance
data transport (VOPipe). Required to support advanced capabilities of the VO
science initiatives in the out years, as well as ongoing IVOA standards
development. The large scale cross-match capability for example will
ultimately require all of these standards.
Registry enhancements. Some enhancements to the registry standards are
required to support the new data services as well as the science initiatives.
VAO Project Execution Plan 118 Standards and Protocols
Footprint standards and data access. A footprint and region arithmetic
standard is needed to more precisely define the spatial coverage of a data
collection, and speed data discovery queries. Prototyping of much of the core
functionality has already been completed (JHU, SAO, STScI).
Space-Time Coordinate (STC) data model toolkit. Required to support
general spatial region queries, spatial footprint manipulation and usage, and
robust space-time coordinates.
Client-side standards (applications messaging, task execution, parameter
management, container, packaging). Some research and planning is required
to support the desktop integration science initiative planned to go forward in
year two.
Most of this ongoing standards development is required to go forward by at least a
minimal level to support ongoing IVOA standards development work in which VAO
team members are involved. All of this technology is expected to be required to
support VAO science initiatives in the out years, as more advanced capabilities
are added.
VAO Project Execution Plan 119 Standards and Protocols
Section 7. Data Preservation and Curation
Introduction and Context
While the bulk of primary data from most of the premier astronomical
observatories resides in well-established archives, there are many data sets that
do not. Among them are data from many smaller observatories, but also,
significantly, the higher-level data products (such as images, spectra, time series,
and simulations) that are published in the literature, but are not stored in digital
form in any systematic way. These higher-level products have often been
processed with custom software and form the basis for the analyses and
interpretations that appear in the literature. Without these data, the scholarly
research record is incomplete; analysis cannot be verified, and results cannot be
readily compared with new information.
We have been working with the American Astronomical Society and their
publishing partners, and with university research libraries, to provide a system in
which, at the time of manuscript submission, the digital data associated with
publications can be captured, annotated, indexed, and made available to the
research community for comparison with other data and for further analysis. The
VAO will contribute to the data curation and preservation services as part of this
collaboration and work actively with the journals, the AAS, and the ADS to
promote community buy-in. We intend to facilitate the process of integrating digital
data associated with journal articles into existing repositories using VO standards.
Although new repositories may be needed for data sets not currently
accommodated, such as theoretical simulations, we will leverage (not duplicate)
existing repositories such as those maintained by NED and the NASA mission
archives (e.g., IRSA, MAST, HEASARC, Chandra).
When astronomers make use of the VAO, they must be able to trace the
provenance of their data and understand its scientific strengths and limitations.
They must also be able to locate data using standard keywords and nomenclature
(e.g., for photometric passbands and spectral lines) that are shared in common
between distributed repositories. Therefore, the VAO has a responsibility to
promote and facilitate standard methods for data repositories to curate and
annotate their data holdings, and ensure archive quality assurance and registry
integrity. The Data Curation and Preservation activity will coordinate these
standards and monitor their implementation, while providing a link between
Astronomy and the Digital Library community, whose efforts we can build on.
As an overview, the long-term outcome of this work will include:
VAO Project Execution Plan 120 Data Preservation and Curation
(a) Comprehensive capture of unique and valuable machine-readable object lists,
tables, images, spectra, and other digital astronomical data objects associated
with journal publications with the assistance of authors at the time of publication.
(b) Rapid and efficient integration of data and metadata into object databases and
repositories of digital astronomical data objects, such as images and spectra.
(c) New data mining capabilities that utilize ontologies and extensive metadata
including observation attributes, astrophysical object attributes, and a rich set of
topical keywords to facilitate powerful science queries. Year one is basically a
research and development phase for this task area, and the primary deliverables
will be a design, deployment plan, and prototype user interfaces for data capture
and faceted searches of the literature, although the latter project will be somewhat
further along thanks to generous in-kind contributions of resources.
Data Preservation
We will coordinate and support the preservation of valuable data sets published
by individuals and organizations unable to maintain their own persistent
repositories. As a pivotal component of this effort, we will operate a pilot
repository for high-level scientific data building upon the experience of the
Astronomy Digital Image Library (ADIL) at NCSA (U. Illinois) and on the
NASA/IPAC Extragalactic Database (NED) at IPAC (Caltech). This will be a
distributed but unified repository, partly funded through other programs, focusing
on preserving highly processed images and spectra that traditionally appear in the
peer-reviewed journals only as plots. By integrating data capture into the journal
article publication and referee process, we will vastly expand the number of
astronomical data objects, such as images and spectra, available for further
research in existing repositories. In later years, this task will extend the
panchromatic image and spectral database developed by NED to include images
and spectra for Galactic objects, solar system observations, survey fields, and
theoretical simulations.
A key challenge is to engage authors of journal articles in the preservation of data
and metadata, with automation using easy-to-use Web-based forms or wiki
technology, during the refereeing and publication process. VAO and NED have
initiated discussions on this topic with the AAS publications office and the Institute
of Physics, publisher of the Astrophysical Journal and the Astronomical Journal. It
is important to emphasize that there will be no scanning of figures and tables from
paper. We will engage authors of new journal articles during the publication and
editorial process with a series of Web-based tools that simplify the collection and
upload of unique, highly processed digital images, spectra, tables, object lists,
etc., and that facilitate the collection of standardized metadata. This new
VAO Project Execution Plan 121 Data Preservation and Curation
approach will foster adherence to VO data models and standards, and lead to a
vast increase in the amount of unique data and metadata available for science
queries through the VAO portal and through established partner services, such as
the ADS and NED. Figure 7-1 presents a conceptual overview of the workflow we
envision.
VAO Project Execution Plan 122 Data Preservation and Curation
Figure 7-1: Overview of the Components and Capabilities of a Data Capture
Workflow for Data in the Astrophysics Literature
This work also directly addresses a public policy mandate summarized in a recent
report by the U.S. National Academies entitled Ensuring the Integrity,
Accessibility, and Stewardship of Research Data in the Digital Age:
―Research data and other information integral to publicly reported
results should be publicly accessible. Although legitimate reasons
may exist for keeping some data private or delaying their release,
the default assumption must e that research data and the
information needed to interpret them will be publicly accessible in
a timely manner to allow verification of findings and facilitate future
discoveries.‖ 1
1
Science 24 July 2009: Vol.325. no. 5939, p. 368; DOI: 10.1126/science.1178927
(http://www.sciencemag.org/cgi/content/full/325/5939/368)
VAO Project Execution Plan 123 Data Preservation and Curation
We will approach the problem of obtaining buy-in from the community at multiple
levels. In coordinating with the Director of Publishing and editors for the AAS
journals, we will raise awareness of the responsibility of the research community
to publish the highly processed digital data from which their results were derived,
to achieve the many advantages summarized in the National Academies report.
As mentioned before, we will develop a suite of Web-based interface tools for
authors to make the process of data and metadata capture as simple and efficient
as possible. Further, we will demonstrate an increase in the speed at which
metadata and data from the literature become available for science queries
throughout the VAO. Finally, we will remind authors of the potential for an
increase in citations that is likely to result from greater visibility and accessibility of
their data for follow-up analysis by the professional research community and
citizen scientists.
The data preservation tasks will tackle, over time, a number of problems and
challenges inherent in the manner in which information is traditionally published in
journal articles that create impediments to the automation of extraction and
analysis of data, and therefore severely limit the efficiency of data integration into
NED and the wider VAO. The fundamental problem is that every journal article
presents data details differently. Table 7-1 presents a summary of challenges and
potential solutions we plan to work out further in collaboration between the VAO,
AAS journal publishers, and authors of journal articles.
Table 7-1: Data capture from the Astrophysics Literature: Challenges and
Solutions
Challenge Potential Solution
Multiple methods are used to describe the Application of uniform column descriptors.
same photometric filters in data tables.
Crucial metadata are scattered throughout Introduce data tags that describe
the text in an unstructured manner. observations in a standard way.
Include a table of relevant astronomical
Source names or coordinates are often
object names (identifiers) with each
truncated or ambiguous, making cross-
journal article and cross-check against
identification with prior observations very
NED, NStED and SIMBAD, as well as the
difficult or impossible.
IAU Clearinghouse.
VAO Project Execution Plan 124 Data Preservation and Curation
Challenge Potential Solution
Data ―published‖ on personal URLs listed
Deposit all data discussed in journal
in article footnotes are rarely refereed to
articles into established data center
meet VO standards, and they often
repositories.
disappear (unstable).
Develop and make available Web-based
Data “behind the plots” are not
tools that simplify upload of unique,
part of traditional publications,
highly processed digital astronomical
greatly reducing reproducibility
data objects, analyze FITS headers, and
and the ability to build on
apply VO standards for data and
published results.
metadata storage and retrieval.
Data preservation includes an obligation to migrate data from one storage
medium to another, apace with advances in capacities, I/O performance, and
decreasing cost. For the most part this obligation is met by the data centers,
observatory archives, and supercomputer centers who are the primary data
providers to the VAO. Our approach to data preservation will be based on an
infrastructure that is media-independent. We will pursue the use of tools such as
the open source Fedora or iRODS distributed data management system (which
we will use in collaboration with our partners) that provide a level of abstraction for
digital objects that enables easy transition of the underlying hardware and
operating system.
Cross-Repository Linking
We will extend the current developments in links between different repositories,
like the data set identifiers joining ADS entries with the data archives. Long-term
preservation of the linking tags—not just the data sets—will become crucial. We
will strengthen the integration of ADS metadata with observational data and
provide semantic Web services to enable the discovery and linking of VAO
resources. The ADS has been a model for how astronomical resources have been
connected and integrated into a searchable, metadata-rich environment. These
links directly support the discovery and access of data products related to
observations from the major missions and observatories. For example, many
archives now create bibliographies linking specific observations with journal
articles in a many-to-many relationship (a single observation can be used by more
than one paper, and a single article can use more than one observation). The
ADS collects and integrates these bibliographies into a unified, indexed system of
VAO Project Execution Plan 125 Data Preservation and Curation
links to data (currently accounting for more than 50,000 articles). By adopting
emerging semantic computing standards and technologies, we propose to
formalize and publish these relationships to facilitate new types of searches, for
example, the retrieval of all papers that worked on a certain list of data sets and
that discuss a particular research topic. These activities are not part of the ADS’s
regular responsibilities, but are crucial to the VAO since they will enable a tighter
integration between ADS's bibliographic metadata and other VAO resources (e.g.,
registry records), which can lead to the creation of more advanced discovery tools
such as the conceptual user interface shown in Figure 7-2.
Figure 7-2: Concept Diagram of a Three-Pane (Data, Publication, and
Astronomical Object) Browsing Interface
Preservation and Curation Standards
We will participate in the development of general standards for certifying archival
repositories. There is a substantial effort ongoing in the Digital Library community
to develop such standards and a draft—Criteria and Checklist for TRAC:
VAO Project Execution Plan 126 Data Preservation and Curation
Trustworthy Repositories Audit and Certification—is currently available.2 We
intend to remain informed about the developments and participate in the process,
representing the VAO, in order to safeguard its interests and provide support for
astronomical data centers in implementing the standards, if and when needed.
In addition, we propose to devote attention to a number of activities that require
very modest resources on our part. We will:
Collaborate with Standards and Protocols in developing registry and curation
standards, and evaluate their effectiveness.
Collaborate with Standards and Protocols on standards for provenance and
calibration metadata, without which astronomers cannot trust the data.
Consult with projects that work on the digitization of non-digital collections, so
that they will be compliant with IVOA standards. This is a relatively new
development and it is important that these projects (such as the digitization of
the Harvard plate stacks) be integrated from the start.
Develop guidelines for providing and propagating provenance information when
astronomical data and services are used by other (non-astronomical)
applications (commercial or otherwise), to ensure proper attribution and credit.
Work Plan
Technical efforts will include integrating ADS and VAO through semantic
computing and semantic publishing technologies; participating in the definition
and curation of IVOA Dictionaries and Ontologies; modeling and exposing ADS's
bibliographic metadata as semantic relationships via W3C and IVOA standards
(e.g., RDF); and integrating ADS and VAO resources with Digital Library
standards, in particular within the OAI-ORE effort. Data preservation efforts will
focus directly on actual curation of images and spectra contributed by authors of
peer-reviewed journal articles; this will include development of Web-based tools to
simplify data capture, communication with authors to validate the required
metadata and data and developing data quality assurance tools and procedures.
2
Developed and published under the auspices of the Research Libraries Group (RLG)
and National Archives and Records Administration (NARA), in collaboration with, among
others, Digital Curation Centre (DCC), Center for Research Libraries (CRL), and Online
Computer Library Center (OCLC), as well as representatives from the astronomical
community.
VAO Project Execution Plan 127 Data Preservation and Curation
Roles and Responsibilities
The Center for Astrophysics will bear primary responsibility for the data linking
and ontology-related project, and IPAC will lead the data preservation effort.
Details are contained in the Data Curation and Preservation sections of the
corresponding organizational work packages.
Goals
The long-range goals are:
Enable and promote linking and annotation of data products, objects, and
facilities in the published literature (and possibly arXiv)
Facilitate the submission of high-level data products by authors to digital
repositories
Enable effective, topic-based discovery of data referenced in the literature
We will enable the correlation of astrophysical topics (e.g., Young Stellar Objects)
with astronomical sources based on their appearance in the literature, and
integrate semantic discovery into a unified data discovery facility in the VAO
Portal ("Seamless Search"). This is why ontologies are going to be essential for
this effort. There is no application in astronomy today that does what has been
called ―Seamless Search:‖ combining a bibliographic search with a ―data‖ search
based on observational parameters (such as instrument, wavelength, object
classification, etc) and information about astronomical objects into a user-friendly
interface. ―Seamless Search‖ needs to be low latency in the face of huge amounts
of metadata, so that users can work in an exploratory mode, and when done
exploring, switch to a high bandwidth mode to be able to automatically make
downloads from VAO infrastructure. See Figure 7-2 for a conceptual view of such
a tool.
In the first year, we intend to initiate the following:
In a collaborative study and pilot project between SAO (ADS), IPAC (NED),
AAS, ADEC members3, and NSF Archives, we will:
o Generate requirements for data publishing tool that can be
integrated into a journal publishing workflow. A conceptual overview
of the workflow and data capture tools is given in Figure 7-1.
3
Astrophysics Datacenters Executive Council, a collaboration framework of the NASA-
funded datacenters.
VAO Project Execution Plan 128 Data Preservation and Curation
o Create a prototype repository system addressing nomenclature,
publishing, linking, and preservation of data sets (both user-supplied
and archive-curated).
o Collaborate with the Data Conservancy project and arXiv as that
data capture project matures.
Topic-based literature search and faceted browsing of results through views on
objects, object types, datasets, and wavelengths
o Search literature by topic, refine results based on object types, data
products and wavelengths, as illustrated in Figure 7-2.
o Search sky by object name(s) or position and filter related literature
by keyword(s).
During the first year, the data publishing/linking effort will focus on identifying the
right strategy and technologies that can be used to promote and sustain its goals.
Engaging the AAS-sponsored journals (Astrophysical Journal and Astronomical
Journal) as partners in this effort is essential. However, we will also collaborate
with efforts underway between arXiv and the Data Conservancy project, which
include the deployment of a ―data submission‖ module as part of arXiv
submissions. From the technology point of view, there is a lot of activity in the
Digital Library community on data curation and preservation, and we will take
advantage of existing frameworks developed for this purpose. If a need for a VAO
data repository is identified for high-level or secondary data products associated
with publications, it is essential that it expose metadata for its data holdings using
standard protocols such as Open Archives Initiative’s Protocol for Metadata
Harvesting (OAI-PMH) and OAI’s Object Reuse and Exchange (OAI-ORE). The
unique contribution that the VAO and its partner organizations bring to this work,
complementing the Digital Library community, is domain knowledge and expertise
with data and metadata specific to astronomical images and spectra. A
scientifically useful data repository goes far beyond storing the bits, and metadata
that apply to particle physics or geophysics experiments are typically very different
from astronomical data sets.
The first-year delivery goals of the topic-based literature search are limited by
what can be integrated into a semantic knowledge base during this period of time
with limited resources. However, the system will be built on a framework that can
integrate heterogeneous sets of metadata, allowing searches that span over
bibliographic, observational and instrumental metadata. Over time, we plan to
leverage on additional resources, including metadata records from the VO
registry, object metadata from NED, SIMBAD, and Vizier, and instrument
metadata from the NASA mission archives. The integration of lightweight
ontologies for objects and observations will allow search results to be further
refined by what kinds of objects or data they refer to (galaxies vs. clusters of
VAO Project Execution Plan 129 Data Preservation and Curation
galaxies vs. stars, optical vs. infrared vs. X-ray vs. radio, pointed observation vs.
survey). Further progress in the creation of a data publishing framework
discussed above will enable an automated integration of these metadata in the
knowledge base.
In summary, the long-term outcome of this work will be:
1. Comprehensive capture of unique and valuable data associated with journal
publications utilizing assistance from authors at the time of publication, to
include machine-readable object lists, tables, images and spectra where
possible
2. Rapid and efficient integration of data and metadata into object databases and
repositories for digital astronomical data objects
3. New search capabilities that utilize extensive metadata involving observation
attributes, astrophysical object attributes and a rich set of topical keywords to
facilitate powerful science queries
Year one is largely a research and development phase for this task area, and the
primary deliverables will be a design and deployment plan, complemented with
prototype user interfaces for data capture and faceted searches of the literature.
The data linking and faceted search project will, in addition, be able to provide
limited user functionality thanks to in-kind resource contributions from SAO,
Harvard College Observatory, and Microsoft Research.
Deliverables
Data publishing tools: prototype in year one; streamlined, operational system
by year three
Vocabulary and Ontology development
o Instrument knowledge base
o ADS bibliographic model
o NED/SIMBAD object ontology
Building/populating data-literature triplestore4
Programmatic service interface (SPARQL) to triplestore
4
Triplestore contain subject-predicate-object relations; see also:
http://en.wikipedia.org/wiki/Triplestore
VAO Project Execution Plan 130 Data Preservation and Curation
Limited faceted browsing of literature through objects and data
Advanced triplestore supporting fast query and inferencing by year three
Advanced discovery tool integrating astronomical publications, objects, and
data (APOD) by year three
VAO Portal integration
Develop a data storage plan
All end-user applications will be Web-based (but SAMP-enabled via the current
SAMP-to-Web bridge). Therefore basic integration in the VAO Portal can be trivial
if all that is desired is a "VAO skin" over the service provided. As such, integration
of the interfaces described here can be accomplished periodically with the
collaboration of the VAO Portal team. More sophisticated integration can be
accomplished by writing Web applications that use VOSI- and SPARQL-compliant
queries.
VAO Project Execution Plan 131 Data Preservation and Curation
Section 8. Technology Assessment
Technology Assessment (TA) advises the VAO on technology choices, including
established and emerging technologies, that best support VAO goals based on
their potential benefit in the areas of:
Cost, including:
o Capital expenses
o Labor costs
o License and support fees
o Maintenance costs
New functionality
Extensibility
Performance and scalability
Security
Emerging technologies are more likely than established technologies to have risks
associated with their adoption, especially in the area of compatibility with existing
technologies. As a result, TA will pay special attention to backwards compatibility,
and work with the Risk Management Officer to develop risk mitigation strategies,
where needed.
We anticipate that TA will be carried out by staff supporting each task area in the
research and development phase of the development of deliverable services and
products. Consequently, TA will coordinate such assessments across all task
areas, and will contribute resources and expertise as needed, for example:
Gathering information and experience for how product improves end-user
capabilities of a VAO science product.
The choice of user support tools in User Support.
The options for monitoring tools used by Operations.
The choices of mass storage for repositories for Data Curation and
Preservation.
Finally, TA provides advice to the broader community on technology strategies:
for example, to those who are looking to scale up their usage or use the VAO in
innovative ways. It shall provide or connect them with the necessary help and
expertise, either in-house or third party, to meet their operational requirements.
VAO Project Execution Plan 132 Technology Assessment
Sources of Requests
A technology assessment can be both solicited and unsolicited. TA shall accept
requests for assessments in the form of:
Task Leads requesting advice on technology choices to support development
of archive and analysis services
Direction from the Director's Office, and especially the Technology Advisor, to
evaluate particular technologies
Requests for advice from members of the VO community
All TA activities undertaken within the VAO shall have the concurrence of the
appropriate Task Lead, the Technology Adviser, the Program Manager and the
Director.
Process for Managing TA Requests
TA activities in support of a VAO deliverable product shall take priority over all
other activities. The TA Lead shall advise the Program Manager on the level of
effort (LoE) required to support specific assessments. Cases where resources
need to be reallocated from other tasks to adequately support an assessment
shall have the knowledge and approval of the PM. Cases where resource
allocations may affect a deliverable shall also require the approval of the Director.
If there are insufficient resources to meet a specific request, the TA Lead shall
work with the PM to assess the impact of the request and choose to:
Reschedule the requested assessment to a later date when a sufficient (TA)
resource is available
Identify new resources outside the VAO with which to collaborate in order to
achieve the requested assessment
Roles and Responsibilities
The TA Lead shall:
Provide oversight and monitoring of all TA activities with the VAO to ensure
they remain on schedule and within resource allocations.
Report the status of TA activities to the Program Council, Program Manager,
Director, and other persons and groups as necessary.
VAO Project Execution Plan 133 Technology Assessment
Liaise with Task Leads, the Program Council, the Program Manager, the
Director and other person and groups as necessary regarding the requesting,
scheduling, provisioning, performance and completion of TA activities.
Identify project risks associated with specific technologies and coordinate
mitigation strategies with the Risk Management Officer.
Review and sign off a technology assessment report when completed.
Ensure all relevant TA documentation is archived.
Ensure all relevant TA software and tests are entered into a version control
system.
Where appropriate, disseminate technology assessment results to the broader
community through conference proceedings and papers in peer-reviewed
journals.
VAO staff undertaking a technology assessment shall:
Ensure a suitable description of the TA request is made available to the TA
Lead. The rationale for an assessment in support of a VAO deliverable will be
described in associated Product Development documentation. For other
assessments, a one-page summary describing the scope and nature of the
assessment, including the area of functionality it is addressing and an
indication of resource level required, shall be sufficient.
Carry out the technology assessment in a timely and efficient fashion.
Work with the TA Lead to produce a report on the technology assessment.
Document any software or tests produced as part of the technology
assessment in line with current VAO practice.
Place any software or tests developed as part of the technology assessment in
a version control system.
Report any problems and unexpected scheduling or resourcing issues in
connection with the assessment to the TA Lead.
Deliverables and Exit Criteria
The deliverable from a technology assessment shall normally be a report that
responds to the initiating request and focuses on the cost benefits of the assessed
technology. This document should detail:
The reason for an assessment, including use cases
VAO Project Execution Plan 134 Technology Assessment
Specific requirements that must be met, both in terms of functionality and
performance, along with details of any metrics to be used to quantize such
requirements, e.g. CPU load, network latency, etc.
The technologies being assessed in broad terms
How well the assessed technologies meet the specific requirement
A set of recommendations based on the assessment result
A list of issues the assessment may have generated
The precise nature of TA activity will vary in scope and level of detail, depending
on the assessment required, e.g., major new functionality or upgrading an existing
library. However, to determine whether a specific technology is valuable to the
VAO, the report should additionally address:
The reliability of the technology, including bug statistics (reporting system,
number open/pending, typical turnaround time, etc.)
The maturity of the technology:
o How widely it is used
o How well it is maintained
o Release history
o Number of active developers
o How well it is documented
The installation experience for a technology
The dependencies of a technology on other technologies
The impact of a technology on our code base, development and operational
procedures including an analysis of how disruptive it might be to adopt the
technology
The report should also identify the sources of all information presented, including:
Prototyping technologies
Vendor Web sites and specifications
Professional blogs and forums
Peer-reviewed literature
Conference proceedings
Technical meetings, especially IVOA meetings
VAO Project Execution Plan 135 Technology Assessment
Discussions with colleagues
All reports shall be reviewed and signed off by the TA Lead, Program Manager,
and the Technology Advisor. Each report shall be archived and curated and be
made publicly accessible on the VAO Web site.
Code developed as part of a technology assessment, including prototypes based
on particular technologies to test against specific requirements, shall not normally
be subject to VAO Product Development procedures but shall be kept in a version
control system and have sufficient documentation associated with it.
Example Areas of Interest
Specific areas of technology to address will be determined by the VAO product
deliverables and what Task Leads request. However, there are a number which
already suggest themselves:
Connected and Seamless Astronomy
There is a strong desire for greater connectivity between astronomical
applications and archives on the one hand, and between literature and data sets
on the other hand. The former can be achieved through the use of connecting
technologies such as SAMP or industry solutions such as Yahoo! Pipes and
YQL—the latter via semantic technologies such as RDFa or HTML5 micro
formats.
Backbone to Support Astronomy of Transient Sources
An existing problem in time domain astronomy support is determining which
technology to employ to disseminate VOEvent. It needs to be scalable, reliable,
and easy to use. Potential technologies include the existing TCPV protocol, XMPP
(Jabber), PUSH (PubSubHubbub) and Atom.
Large-Scale Cross-Matching
A proposed technology to support an unlimited cross-match capability is VOPipe.
This requires a proper evaluation before proceeding into a full product
development cycle; for example, is the proposed technology sufficient or does it
have hidden latencies that might render it ineffective?
VAO Project Execution Plan 136 Technology Assessment
The Cloud
To what extent should the VAO look at cloud computing as a solution? Does it
make sense to deploy VOPipe or VOSpace in the cloud? What about service
composability and orchestration in the cloud?
VAO Project Execution Plan 137 Technology Assessment
Section 9. Education and Public Outreach
The Education and Public Outreach (EPO) program for the VAO will be led by
Space Telescope Science Institute (STScI), in collaboration with the High Energy
Astrophysics Science Archive Research Center (HEASARC) EPO program and
Johns Hopkins University (JHU). It is the intent that our VAO EPO efforts will
result in a comprehensive package of resources that will enable educators to
address the theme of information technology while using real-world astronomical
data products in their classrooms. These resources will include field-tested
astronomical data sets, accompanying classroom support materials, background
materials about the development and infrastructure of the VAO, and professional
development opportunities.
A vast collection of data sets from many of the world's astronomical ground-based
and space-based observatories are housed within, and served through, various
pre-existing archives. The variety and complexity of these data sets create a high
barrier to entry for the public and scientifically competent EPO professionals,
making it difficult for them to make use of the data in a meaningful way. We will
structure our EPO efforts to provide uniform access to VAO information in order to
enable educational and research opportunities across multiple wavelengths and
time series data sets. We will also develop and support a cadre of EPO VAO
users, who in turn will make use of the facility in disparate ways. The components
of our VAO EPO program will include education initiatives, a media plan, a VAO
EPO Web site, and an EPO Data Archive. We will collaborate with HEASARC to
create a work plan to provide a comprehensive EPO program and intensify our
impact for the VAO.
Specifically, our efforts will have a clear focus on EPO professionals in formal
education. Based upon a needs assessment of the education community, we will
establish an archive of EPO-qualified data sets and develop accompanying
materials that highlight the development and operation of the VAO. To do this
most effectively, we will work within STScI's proven education infrastructure to
communicate the educational benefits of data available through the VAO, and
how the data and accompanying education activities can be used to address
critical skills and concepts in information technology. We will follow a top-down
approach by working with school districts, agencies, and other organizations with
which we already have an established partnership. Also, we plan to connect with
the public through existing organizations, including Google Sky, Microsoft's
WorldWide Telescope, ComPADRE and Zooniverse, to expand our reach. Team
members have experience with these organizations and have worked with them to
incorporate and disseminate astronomical data. Specifically, we plan to enhance
our established collaboration with Zooniverse—the Internet’s largest and most
VAO Project Execution Plan 138 Education and Public Outreach
successful citizen science project—and explore opportunities for new VAO-
enabled science, such as transient event classifications.
To extend the impact of VAO efforts further, we can capitalize upon the existing
infrastructure of the Hubble Education, the HEASARC EPO program, and Johns
Hopkins University’s SkyServer. A key component of the Hubble Education EPO
program is the Amazing Space Web site, which is used by K-12 education
institutions in all 50 states, including 42 of the 100 largest school districts in the
United States. Since its creation in 1996, the Amazing Space Web site has been
well received by educators seeking online, space science education materials.
The Amazing Space Web site has become a key vehicle for delivering up-to-date
space science content and education resources to the formal and informal
education communities. Specifically, we can apply lessons learned from the
development and revision of Amazing Space data-based activities, including the
Hubble Deep Field online exploration, to the process of bringing VAO data to the
education community in a relevant and usable format.
HEASARC will provide expertise with regard to high-energy astronomy satellite
data sets from X-ray and gamma-ray missions such as Suzaku, the Rossi X-ray
Timing Explorer, XMM, Swift, and Fermi. They will also provide expertise from
their work on Imagine the Universe!, which for nearly 15 years has provided
content and classroom resources on high-energy astronomy to high school
students and their teachers. Most pertinent to the VAO EPO effort, HEASARC
will provide expertise with regard to their experience in developing Student Hera,
a program that provides access to data and analysis tools based on the
HEASARC’s FTOOLS package.
In addition, we will use the existing infrastructure of SkyServer. SkyServer is an
online database that presents data from the Sloan Digital Sky Survey, a project to
make a map of a large part of the universe. The site also includes K-12 exercises
and projects.
Lastly, we will directly connect to the public through a VAO EPO-focused Web site
and conduct a media campaign to inform the public of the benefits of VAO-
enabled research. All of these efforts, in concert, will benefit from our partnership
with ComPADRE’s Astronomy Center—an NSF-funded collaborative project
among the American Association of Physics Teachers (AAPT), the American
Astronomical Society (AAS), the American Institute of Physics/Society of Physics
Students (AIP/SPS), and the American Physical Society (APS). This project helps
teachers and learners find and use high-quality astronomy resources through
collections and services tailored to their specific needs. This online repository
provides an effective mechanism for disseminating VAO materials to target
audiences.
VAO Project Execution Plan 139 Education and Public Outreach
The following outlines STScI’s efforts for years one through five in developing and
implementing the VAO EPO program. Throughout years one through five, STScI
will work with HEASARC in the areas of Education and Evaluation, Data Archive,
and selected areas of Program Development and Implementation.
Broader Impacts of the VAO EPO Plan
The VAO collaboration will foster diversity and broader participation at a number
of levels. First, within our development and Operations team, we will work to
engage a diverse staff. In our professional training and outreach programs, such
as summer schools and end-user tutorials, we will make sure the opportunities
are known broadly and advertised at colleges and universities with
underrepresented populations in the astronomy community. Our education and
public outreach programs will leverage on widely known astronomy-based
applications, such as Google Sky and WorldWide Telescope (WWT, Microsoft).
These have user bases in the millions and engage people from all walks of life.
Various ―citizen science ― or ―crowd-sourcing‖ programs will also be considered,
such as time series classification and evaluation of position-based cross-matching
results.
Year One
Program Development and Implementation
Identify and hire Technical Coordinator/Advocate(s) and provide him or her with
the necessary background.
Integrate VAO EPO plan into the existing infrastructure of participating
organizations, including HEASARC, Microsoft, Google, JHU, ComPADRE, and
Zooniverse.
Develop a coordination plan with HEASARC and JHU.
Work with the HEASARC and JHU to establish a work plan that focuses on
areas of the VAO EPO program (i.e. Program Development, Education and
Evaluation, Media, Web Site Development, and EPO Data Archive).
Education and Evaluation
Conduct a literature review to determine how data sets are currently used by
the education community.
VAO Project Execution Plan 140 Education and Public Outreach
Conduct a literature review to identify key technology topics that are addressed
by high school and higher education curricula
Assess and identify the needs of the internal/external EPO community:
o User community
o EPO partners
Work with HEASARC and JHU to search for and identify astronomical data
sets and tools designed for the education community.
Based on the literature review and needs assessment, develop a plan to guide
the future direction of VAO EPO.
Utilize the HST and VAO partner infrastructure to inform the education
community about the benefits of using real astronomical data via a virtual
observatory.
Star Witness curriculum support tool released to the K-12 community on the
Amazing Space Web site that tells the VAO technology story.
Media
Initiate the development of a media plan/package for promoting VAO science
results and EPO programs (package contains graphics, captions,
backgrounders, etc. to be used with press releases).
Web Site Development
Define the content and the information architecture for the VAO EPO Web site.
EPO Data Archive
Work with the HEASARC and JHU to identify common criteria for EPO-
qualified data and develop a work package for the EPO Data Archive.
Year Two
Program Development and Implementation
Work HEASARC to develop and conduct an EPO workshop for Program
Developers.
VAO Project Execution Plan 141 Education and Public Outreach
Identify, and contact as appropriate, professional organizations and meetings
for presenting pertinent findings, i.e. AAS, NSTA, ASP.
Education and Evaluation
Connect VAO technology and infrastructure to high school and higher
education curricula.
Connect VAO technology and infrastructure to national education standards for
the K-12 community.
Connect VAO data sets to high school and higher education curricula.
Connect VAO data sets to national education standards for the K-12
community.
Work with HEASARC to develop and pilot educational background materials
about the development and infrastructure of the VAO.
Develop and conduct a pilot professional development workshop for local
educators.
Media
Complete the media package for VAO.
Web Site Development
Launch the VAO EPO Web site at the beginning of year two to communicate
projects and activities to the public.
EPO Data Archive
Interface with other organizations, including Microsoft, Google, ComPADRE,
and Zooniverse, and embed VAO within their infrastructure as appropriate.
Year Three
Program Development and Implementation
Identify, and contact as appropriate, professional organizations and meetings
for presenting pertinent findings (e.g., AAS, NSTA, ASP)
VAO Project Execution Plan 142 Education and Public Outreach
Education and Evaluation
Work with HEASARC to develop and pilot a data set and supporting
documentation for the education community.
Work with HEASARC to develop a plan for disseminating data sets and
supporting documentation to the education community.
Work with HEASARC to conduct professional development workshops at
national teacher conferences.
Conduct an interim program evaluation.
Media
Conduct a science writers’ workshop, dependent upon VAO science findings
and results.
Web Site Development
Completion of the VAO EPO Web site with the inclusion of a data set and
education resources.
EPO Data Archive
Populate the EPO Data Archive with the pilot-tested data set.
Year Four
Program Development and Implementation
Apply lessons learned from the interim evaluation.
Based on community needs, conduct a second EPO Developer workshop.
Identify, and contact as appropriate, professional organizations and meetings
for presenting pertinent findings (e.g., AAS, NSTA, ASP).
Education and Evaluation
Determine usage and evaluate effectiveness of the data set.
VAO Project Execution Plan 143 Education and Public Outreach
Continue to conduct workshops at national teacher conferences with
HEASARC.
Media
Continue to implement the VAO media plan and promote efforts with press
releases as appropriate.
Web Site Development
Maintain the VAO EPO Web site and update it as appropriate
EPO Data Archive
Determine usage and evaluate effectiveness of the data set
Year Five
Program Development and Implementation
Publish research results to benefit other scientific infrastructures.
Identify, and contact as appropriate, professional organizations and meetings
for presenting pertinent findings (e.g., AAS, NSTA, ASP).
Education and Evaluation
Depending on the evaluation results of the pilot data set, make revisions and
develop a new data set.
Continue to conduct workshops at national teacher conferences with
HEASARC.
Complete a final program evaluation, document lessons learned, and write a
final program report.
Media
Continue to implement the VAO media plan and promote efforts with press
releases as appropriate.
VAO Project Execution Plan 144 Education and Public Outreach
Website Development
Establish a long-term plan for VAO EPO Web site maintenance and
sustainability.
EPO Data Archive
Work with HEASARC to populate the archive with the new data set.
VAO Project Execution Plan 145 Education and Public Outreach
Acronyms, Abbreviations and Glossary
Full Name
Term Definition
(proper nouns are capitalized)
2MASS Two Micron All Sky Survey
American Association of Physics
AAPT
Teachers
AAS American Astronomical Society
An NCSA-hosted collection of
astronomical, research-quality images that
ADIL Astronomy Digital Image Library
are made available to the astronomical
community and general public.
NASA Astrophysics Data System
ADS
Virtual Library
Astronomy Data Query
ADQL
Language
A compact region at the center of a galaxy
that has a much higher luminosity over at
AGN active galactic nucleus (or nuclei)
least some portion, and possibly all, of the
electromagnetic spectrum.
American Institute of
AIP/SPS Physics/Society of Physics
Students
A software package by NRAO used for
interactive and batch calibration and
editing of radio interferometric data, and
Astronomical Image Processing
AIPS for the calibration, construction, display
System
and analysis of astronomical images
made from those data using Fourier
synthesis methods.
Interactive sky atlas software by CDS that
Aladin helps users visualize and interact with
digitized astronomical images.
An array of 66 high-precision antennae
ALMA Atacama Large Millimeter Array located in Chile used for sub-millimeter
observing.
An interface implemented by a software
application programming
API program that enables it to interact with
interface
other software.
VAO Project Execution Plan 146 Acronyms, Abbreviations and
Glossary
Full Name
Term Definition
(proper nouns are capitalized)
A popular Web site that showcases an
astronomical photo each day. The site is a
APOD Astronomy Picture of the Day service of NASA’s Astrophysics Science
Division (ASD) and Michigan
Technological University
APS American Physical Society
An automated electronic archive and
Pronounced as ―archive,‖ with
distribution server for research articles.
arXiv the X as a hard K sound as in
maintained and operated by the Cornell
―LaTex‖
University Library.
NSF Division of Astronomical
AST
Sciences
AUI Associated Universities, Inc.
Associated Universities for
AURA
Research in Astronomy, Inc
Center for Advanced Computing
CACR
Research
Cosmic Assembly/Near-infrared
CANDELS Deep Extragalactic Legacy
Survey
A software packaged developed by NRAO
Common Astronomy Software with the goal of supporting the data post-
CASA
Applications processing needs of the next generation
of radio astronomical telescopes.
CCB Configuration Control Board
Centre de donneés
CDS
astronomiques de Strasbourg
From ―s’sciavo,‖ meaning ―I am Software developed to analyze data
CIAO your servant‖ in Italian (Venetian returned by the Chandra X-ray
dialect) Observatory.
A system that periodically and
CIS continuous integration system automatically builds a software tool and
(possibly) runs its unit tests.
An ontology for describing the nature of
CiTO Citation Typing Ontology reference citations in scientific research
articles and other scholarly works.
VAO Project Execution Plan 147 Acronyms, Abbreviations and
Glossary
Full Name
Term Definition
(proper nouns are capitalized)
configuration management, or
CM
Configuration Manager
A network of free online resource
collections supporting faculty, students
and teachers in physics and astronomy
ComPADRE education.
A European space mission that searches
for extrasolar planets with short orbital
COnvection ROtation and periods, and performs asteroseismology
COROT planetary Transits by measure solar-like oscillations in stars.
A consortium of North American
universities, colleges, and independent
research libraries that acquire and
preserve traditional and digital resources
CRL Center for Research Libraries for research and teaching.
A VAO Web-based tool for searching for
DataScope all known information about a given target
or region of the sky.
DALServer Data Access Layer Server
A software package for stellar photometry
used for various tasks, including finding
Dominion Astrophysical
DAOPhot objects, aperture photometry, obtaining
Observatory Photometry
the point spread function and profile-fitting
photometry.
A system for writing software reference
documentation within the programming of
Doxygen
various languages, including Java, C++
and IDL.
DRMO Deputy Risk Mitigation Officer
An astronomical imaging and data
DS9 visualization application developed at
Harvard/SAO.
A variety of activities conducted by
research institutes, universities, and
EPO Education and Public Outreach institutions such as science museum that
are aimed at promoting public awareness
and understanding of science.
VAO Project Execution Plan 148 Acronyms, Abbreviations and
Glossary
Full Name
Term Definition
(proper nouns are capitalized)
A system that provides general tools for
European Southern image processing and data reduction, with
ESO-MIDAS Observatory—Munich Image emphasis on astronomical applications,
Data Analysis System including imaging and special reduction
packages for ESO instrumentation.
An array of 27 antennas with 25-meter
EVLA Expanded Very Large Array
diameters that are located in New Mexico.
A compilation of facts and information
FAQ Frequently Asked Questions about a given topic provided in a question-
and-answer format.
A digital file format used to store, transmit,
FITS Flexible Image Transport System
and manipulate astronomical images.
A designation describing the amount of
FTE full-time equivalent hours of human resources allocated to a
specific task or project.
An orbiting, ultraviolet space telescope
launched in April 2003 to determine the
GALEX Galaxy Evolution Explorer distance between Earth and sever
hundred thousand galaxies, as well as the
rate of star formation in each galaxy.
An online sky and outer space viewer
showing the sky view made up of a
Google Sky
collaboration of Hubble Telescope space
images.
Also known as the Hubble Space
Telescope, Guide Catalog (HSTGC). It is
GSC2 Guide Star Catalogue II
a star catalog compiled to support HST
with targeting off-axis stars.
GSFC Goddard Space Flight Center
High Energy Astrophysics
HEASARC Science Archive Research
Center
HST Hubble Space Telescope
IAU International Astronomical Union
VAO Project Execution Plan 149 Acronyms, Abbreviations and
Glossary
Full Name
Term Definition
(proper nouns are capitalized)
A proprietary software system and
computer language commonly used by
IDL Interactive Data Language scientists and engineers in the analysis of
one-, two-, and three-dimensional data
sets.
A multi-mission center of expertise for
long-wavelength astrophysics that carries
Infrared Processing and Analysis
IPAC out data-intensive processing tasks for
Center
NASA's infrared and sub-millimeter
astronomy programs.
A general purpose software system for the
Image Reduction and Analysis
IRAF reduction and analysis of astronomical
Facility
data.
An online archive of data and science
IRSA Infrared Science Archive products for NASA’s infrared and sub-
millimeter missions.
An international consortium formed in
June 2002 to facilitate the international
coordination and collaboration necessary
for the development and deployment of
International Virtual Observatory
IVOA the tools, systems and organizational
Alliance
structures necessary to enable the
international utilization of astronomical
archives as an integrated and
interoperating virtual observatory."
A group composed of the chairs and vice
International Virtual Observatory
chairs of IVOA working group and interest
IVOA TCG Alliance Technical Coordination
groups that reviews and proposed IVOA
Group
recommendations.
A compiled programming language that
Java allows application developers to write
code that can run on any platform.
A proprietary issue-tracking product the
Truncation of "Gojira," the
JIRA VAO will use as its Help Desk issue/bug
Japanese name for Godzilla
tracking system.
JPL Jet Propulsion Laboratory
kindergarten through twelfth
K-12
grade
VAO Project Execution Plan 150 Acronyms, Abbreviations and
Glossary
Full Name
Term Definition
(proper nouns are capitalized)
A group within the IVOA that will develop
Knowledge Discovery in and test scalable data mining algorithms
KDD
Databases and the accompanying new standards for
VO interfaces and protocols.
A flexible form of business enterprise that
blends elements of partnership and
corporate structures. It is a legal form of a
LLC limited liability company
company that provides limited liability to
its owners within the vast majority of
United States jurisdictions.
Work of a general or supportive nature
(such as coordination, follow up, or
LoE level of effort
liaison) that does not result in a definitive
end product or outcome.
A facility that will produce a wide-field
Large Synoptic Survey
LSST astronomical survey of the Universe using
Telescope
an 8.4-meter ground-based telescope.
Multimission Archive at Space
MAST
Telescope
National Archives and Records
NARA
Administration
National Aeronautics Space
NASA
Administration
National Center for
NCSA
Supercomputing Applications
NASA/IPAC Extragalactic An online, knowledge-based database
NED
Database and archive for extra-galactic astronomy.
National Optical Astronomical
NOAO
Observatory
National Radio Astronomy
NRAO
Observatory
NSF National Science Foundation
National Science Teachers
NSTA
Association
A general purpose stellar and exoplanet
NASA/IPAC/NExScI Star and
NStED archive to support NASA's planet finding
Exoplanet Database
and characterization activities.
VAO Project Execution Plan 151 Acronyms, Abbreviations and
Glossary
Full Name
Term Definition
(proper nouns are capitalized)
NVO National Virtual Observatory
PC Program Council
PEP Project Execution Plan
PI Principal Investigator
PM Program Manager
A programming language for expressing
PQL Program Query Language
patterns of events on objects.
A specification that defines standards for
Open Archives Initiative—Object the description and exchange of
OAI-ORE
Reuse and Exchange aggregations of Web resources, including
text, images, data and video.
A set of six services invoked within HTTP
Open Archives Initiative—
OAI-PMH that provide a low-barrier mechanism for
Protocol for Metadata Harvesting
repository interoperability.
An IVOA data model that describes the
ObsCore Observation Core necessary metadata for multi-wavelength
data discovery queries.
ObsDM Observation Data Model
Observation Table Access
ObsTAP
Protocol
A non-profit network of computer library
service and research organizations
OCLC Online Computer Library Center dedicated to furthering access to
information and reducing information
costs.
An NVO tool that allows users to cross-
match astronomical catalogs and select
Open SkyQuery
subsets of catalogs with a general and
powerful query language.
A design for a wide-field imaging facility
Panoramic Survey Telescope &
Pan-STARRS being developed at the University of
Rapid Response System
Hawaii’s Institute for Astronomy.
PDD product definition document
PDP project development plan
VAO Project Execution Plan 152 Acronyms, Abbreviations and
Glossary
Full Name
Term Definition
(proper nouns are capitalized)
An open source, interpreted, object-
Python
oriented programming language.
QA&T
Quality Assurance and Testing
(or just QA)
A programming language and software
R environment for statistical computing and
graphics.
A standard model for data interchange on
Resource Description the Web that allows structured and semi-
RDF
Framework structured data to be mixed, exposed, and
shared across different applications.
A style of software architecture for
REST REpresentational State Transfer distributed hypermedia systems, such as
the World Wide Web.
A U.S.-based library consortium that
developed standards and tools for
RLG Research Libraries Group
computerized library systems. It merged
with OCLC in 2006.
RMO Risk Mitigation Officer
A messaging protocol that enables
Simple Application Messaging
SAMP astronomy software tools to interoperate
Protocol
and communicate.
Smithsonian Astrophysical
SAO
Observatory
A platform-independent layer that facilities
SEAP Simple Event Access Protocol
the interchange of VOEvent packets.
SC Science Council
SEDs Spectral Energy Distributions
A software program that builds a catalog
SExtractor Source Extractor
of objects from an astronomical image.
A formal document that describes, in
SDP software development process detail, an organization’s software
development process.
SDSS Sloan Digital Sky Survey
VAO Project Execution Plan 153 Acronyms, Abbreviations and
Glossary
Full Name
Term Definition
(proper nouns are capitalized)
A modeling and fitting software tool that is
Sherpa part of the CIAO software packaged
developed by Chandra X-ray Observatory
The second generation of an IVOA
standard that defines a protocol for
SIAV2 Simple Image Access Protocol 2 retrieving image data from a variety of
astronomical image repositories through a
uniform interface.
A clearinghouse of VOEvent notices that
individuals may subscribe to and/or
SkyAlert
monitor the from various event streams
and feeds.
A Web site for end-users to get data from
SkyServer
the Sloan Digital Sky Survey.
Technology that allows users to gain
SSO single sign-on access to multiple systems using a single,
central authentication method or interface.
An astronomical database that provides
Set of Identifications,
basic data, cross-identifications and a
SIMBAD Measurements, and Bibliography
bibliography for astronomical objects
for Astronomical Data
outside the solar system.
SMC Small Magellanic Cloud
A programming language that is
SPARQL Protocol and RDF considered a key semantic Web
SPARQL
Query Language technology by enabling programmers to
write more robust queries.
A tool developed at STScI for 1-D spectral
Specview visualization and analysis of astronomical
spectrograms.
A recommended data model for
describing spatial and temporal
STC Library Space-Time Coordinate Library astronomical observations, and
encapsulating them in single metadata
object.
Space Telescope Science
STScI
Institute
VAO Project Execution Plan 154 Acronyms, Abbreviations and
Glossary
Full Name
Term Definition
(proper nouns are capitalized)
A revision control system used to maintain
current and historical versions of files
Subversion
such as source code, Web pages, and
documentation.
A Web-based collaborative program that
Semantic Web Applications in
SWAN organizes and annotates knowledge about
Neuromedicine
neurodegenerative disorders.
One of the seven work areas of the VAO
project that advises the VAO on
TA Technology Assessment technology choices that best support VAO
goals.
A VO standard that defines a service
protocol for accessing general table data,
TAP Table Access Protocol
including astronomical catalog and
general database tables.
A project at Harvard University dedicated
to creating the world’s largest data center
TSC Time Series Center for time series, and developing algorithms
to understand various aspects of those
time series.
Tool for Operations on An interactive and graphical viewer and
TOPCAT
Catalogues And Tables editor for tabular data.
A type of database that is very efficient at
storing character information. Queries
triplestore
consist of short, three-part statements
(subject-predicate-object).
A form of testing that focuses on the
usability and scientific validation of
services and applications, with detailed
UAT user acceptance testing
attention to how the software responds to
possible/probable error conditions,
especially in the case of misuse.
An all-sky catalog of digitized
United States Naval Observatory
USNO-B photographic plates from observations
B1.0 catalog
covering the years 1949-2002.
An IVOA specification that defines how to
UWS Universal Worker Service manage asynchronous execution of jobs
on a service.
VAO Virtual Astronomical Observatory
VAO Project Execution Plan 155 Acronyms, Abbreviations and
Glossary
Full Name
Term Definition
(proper nouns are capitalized)
Virtual Astronomical
VAO, LLC Observatory, Limited Liability
Corporation
Virtual Observatory Command A set of tools developed by the NVO that
VO-CLI
Line Interface use a command-line interface.
VO Virtual Observatory
A software package that provides a high-
level, programmable interface between
desktop applications and the distributed
VOClient Virtual Observatory Client VO framework, providing access to
remote VO data and services, reference
implementations for VO data-providers
and end-user applications.
A group convened by NSF Division of
Astronomical Sciences (AST) to provide
VOG VAO Oversight Group oversight of the VAO and act as a point-
of-contact for NSF and NASA and the
Project Director.
An interface for asynchronous data
VOPipe Virtual Observatory Pipe
messaging using VOSpace.
VOResource Virtual Observatory Resource
Virtual Observatory Support
VOSI
Interfaces
VOSpace Virtual Observatory Space The IVOA interface to distributed storage.
An IVOA specification for presenting
VOTables Virtual Observatory Table tabular data in a flexible and interoperable
format.
A Web utility used to view large data
VOView Virtual Observatory View
tables within a Web browser.
A definition and description of a project’s
discrete work elements and tasks that
WBS Work Breakdown Structure
helps organize and define the total work
scope of the project.
Software developed by Microsoft
Research that enables desktop computers
WWT WorldWide Telescope
to view imagery from ground- and space-
based telescopes.
VAO Project Execution Plan 156 Acronyms, Abbreviations and
Glossary
Full Name
Term Definition
(proper nouns are capitalized)
An Internet-based citizen science project
Zooniverse
run by the Citizen Science Alliance.
VAO Project Execution Plan 157 Acronyms, Abbreviations and
Glossary
List of Tables and Figures
Table 1-1: Summary of FTEs For the Four Science Deliverables ..................................22
Table 1-2: Work Breakdown Structure Staffing Levels ...................................................36
Figure 1-1: VAO Schedule of Deliverables and Milestones ...........................................39
Figure 2-1: VAO Management Structure .......................................................................40
Table 2-1: VAO Work Areas and Designated Leads ......................................................41
Figure 2-2: VAO Risk Management Process .................................................................52
Figure 2-3: Sample Communication Assessing Risk Item .............................................54
Table 3-1: Candidate Heritage NVO Applications ..........................................................58
Table 3-2: Issue Streams ..............................................................................................59
Figure 3-3: Roles and Responsibilities in the VAO Project Life Cycle............................67
Figure 3-4: Flow chart showing decision points in the project life cycle .........................70
(Figure 3-4, continued)....................................................................................................71
Table 4-1: Test Case Specification .................................................................................89
Table 4-2: Definition of Test Processes ..........................................................................92
Table 6-1: Year One Priorities .....................................................................................118
Figure 7-1: Overview of the Components and Capabilities of a Data Capture Workflow
for Data in the Astrophysics Literature ..........................................................................123
Table 7-1: Data capture from the Astrophysics Literature: Challenges and Solutions ..124
Figure 7-2: Concept Diagram of a Three-Pane (Data, Publication, and Astronomical
Object) Browsing Interface ............................................................................................126
VAO Project Execution Plan 158 List of Tables and Figures
Appendix A: Traceability Matrix
The following table maps the requirements described in the Cooperative
Agreement with their locations in the Project Execution Plan.
Requirement PEP Sections In Link to Sections or
(as it appears in the CA) Which Requirement Sub-sections
Appears
Governing and Management
Section 2: Management Management
Structure
Organizational Structure Section 2: Management Governance
Baseline Project Definition Section 1: Project Baseline Project Baseline
Work Breakdown Structure Section 1: Project Baseline Work Breakdown Structure
Risk Assessment and
Section 2: Management Risk Management
Management
Contingency Management Section 2: Management Contingency Management
Configuration Management and
Section 3: Operations Configuration Management
Change Control
Quality Assurance and Quality
Section 3: Operations Quality Assurance
Control
Safety, Environment, and Health
Section 2: Management Safety, Environment and Health
Plan
Financial and Business Operations Business and Financial
Section 2: Management
Controls Management
Deploying and Monitoring
Plan For Operations and Services
Section 3: Operations
Maintenance
VAO Support Services
VAO Project Outcomes
Project Metrics Section 1: Project Baseline
Metrics
Section 9: Education and
Education and Public Outreach Education and Public Outreach
Public Outreach
Quarterly and Annual Reports Section 2: Governance Reporting
VAO Project Execution Plan 159 Appendix A: Traceability Matrix
