NRAO End to End (e2e) Operations
Initial Customer Assessment for Quality System Development
NRAO’s primary business is to build and operate large-scale instrumentation for radio
astronomy, providing the expertise to help researchers plan their experiments and generate
useful, scientifically interpretable data products. Software drives the “data manufacturing”
process at all stages, and so is critical to quality of operations. The steps of the manufacturing
1. Data is acquired from multiple component instruments: antennas, receivers, signal
processing backends; some of this is science data, some is monitor data
2. Data is combined in software to form raw data products (one or more files or databases)
3. Calibration is performed in software to remove instrumental effects
4. Data processing software is applied (manually or automatically) to generate science data
products, which are usually spectra or images. Additional calibration may be done to
remove atmospheric effects, which may require the assimilation of weather monitoring
data collected at step 1.
Here are four examples of spectra (L2 data; definitions on the next page).
Here’s an example of an image from the VLA telescope that’s generated after lots and lots of
manual refinement using software. Images are also L2 data.
So what do we mean by L2 data? There are many types of data, described in the following chart.
NRAO deals with L0 through L2 data, and is typically not concerned with L3 or L4 data which is
outside the domain of observational astronomy.
NRAO Instruments and Products
NRAO currently operates three telescopes: VLA (New Mexico), VLBA (New Mexico + world) and
GBT (West Virginia). You can find out more about them at http://www.nrao.edu.
Our product (right now) is L0 data. That means NRAO manages steps 1 and 2 of the data
manufacturing process described on page 1, and leaves it up to the scientist to perform step 3
(which would result in an L1 data product) and step 4 (which would result in an L2 data
Within the next five years, VLA will be replaced by EVLA, VLBA and GBT will continue to be
operated, and ALMA (a large international collaboration that NRAO has 50% interest in) will
start operations in Chile. We would like to offer L1 and L2 products for at least ALMA and
EVLA, and are developing software that we call “automated processing pipelines” to do this.
Bottom line, we’re upgrading our capabilities to offer more products.
Because software is such a critical capability, NRAO End to End Operations was launched in
April 2006 to provide oversight of all the software and customer service activities at the
Observatory. It is a quality organization whose goals are to eliminate waste and rework, improve
customer service, and ultimately increase innovation. Variation reduction is not an explicit goal.
The e2e Operations Division was established with the following official mission:
• To broaden access to NRAO facilities, enabling all astronomers to focus on the science
rather than the instrumental details
• To increase science impact and throughput via improved quality of data and efficient
delivery of science products.
• To help NRAO better optimize its investments in software, services and resources (e.g.
through continuous improvement and reuse of ideas, models, systems and information
So the link with quality is pretty clear: we need to eliminate waste and rework; identify,
establish & audit key processes; and become the continuous improvement arm of the
Observatory. Technically speaking, we (e2e Operations Division) have primary responsibility for
three technology areas: science archive and metadata repository, proposal administration for all
NRAO telescopes, and automated data processing. In addition, the Division plays a key role in
telescope scheduling for each instrument through scheduling metadata management, which
facilitates a centralized view while preserving the authority of the resident experts at each
telescope for creating and administering observations. This last point is kind of confusing, but
basically it means that we will also build “business intelligence” dashboards for senior &
executive management to keep track of what observing has been done, how successful it was,
how many publications were developed from our instruments – ie. how well are we satisfying
The e2e Operations Division is a umbrella organization with managerial authority over
standards, software and services for all NRAO telescopes – about one sixth of NRAO employees
total. It directly manages three workgroups (the Data Postprocessing Group, the Strategic
Technology Group and the Data Products Group), and co-manages the Core Science Group
which provides e2e Operations with a unified perspective on Observatory-wide developments.
In addition to the Data Postprocessing group, these workgroups are:
Core Science Group (CSG): Observatory-wide science representation for cross-telescope interests
• Provide a common science vision and change control for shared services across NRAO
(e.g. proposal administration, archive, scheduling & metrics)
• Synthesize requirements and specifications from operations groups to evaluate and
continuously improve prototypes managed by Strategic Technology Group
• Other scientists with an interest in e2e are welcome to participate
Data Products Group (DPG): The nexus for generating archivable science data products while “learning
from experience” with VLA/VLBA/GBT
• Produce best available automated science products using available pipeline tools, and
ensure that the recipes developed are transitioned to appropriate software packages
• Identify functionality gaps, transfer knowledge, assess performance, regression tests
• Responsible for data curation (ensuring high quality metadata and science products)
Strategic Technology Group (STG): Technology and telescope generalists focused on efficiency and
• Incubate functions that will become part of NVO facility, and spin-off projects that can
be externally funded (e.g. data quality expert systems)
• Evolve models for future software innovation into NRAO operations (cooperative
effort between e2e Operations and Rick Fisher, Observatory Technical Leader)
Organizational Chart for the e2e Operations Division
In addition to maintaining primary responsibility in the four core Observatory-wide efforts, the
e2e Operations Division coordinates the efforts of software development groups across the
Observatory. The Division maintains central coordination authority over each of the software
development organizations in the operations groups to facilitate cross-Observatory
communication while preserving the local autonomy that is critical for effective day-to-day
I spent some time trying to identify our key processes. It would be really helpful to have a list of
key processes in various areas to choose from, because coming up with your processes by scratch
is extremely difficult!
Here are some results (definitely not comprehensive):
Strategic Planning Process
Priority Setting Process
Innovative Idea Percolation Process
Quarterly Plan Alignment Process
Process for Generating Funding and “New Business”
Data Resource Standard Determination Process
Software Tool Standard Determination Process
Document Promotion Process (getting Observatory-wide memos published &
For each telescope (some processes can share common elements):
Raw Data (L0) Manufacturing Process
Calibrated Data (L0->L1) Manufacturing Process
Derived Data (L1->L2) Manufacturing Process
New Algorithm Development Process
Process for Publishing New Data to the Virtual Observatory
Process for Ingesting L2 data produced independently by scientists
Customer Service Processes
Proposal Submission Process
Proposal Handling Process