Ease of Use Project & Development Plan
GBT e2e Software Review
August 30, 2004
Nicole Radziwill (nradziwi@nrao.edu)
Goals
• To streamline the observing process:
– Enabling capabilities such as automated dynamic scheduling and
remote observing
– Make the GBT easier to use for both visiting observers and staff
astronomers
– Reduce setup times significantly
– Reduce the reliance of visiting observers on scientific staff
• Includes the ability to:
– define observations in advance of observing,
– the ability to execute those observations,
– improved monitor and status information during execution, and
– an improved real-time display
The Past
• Many disparate software applications, much duplication of code:
– GBT Observe (GO), a GUI application written in glish
– Control Library for Engineers and Operators (CLEO), a Tcl/tk GUI
– Ygor, a distributed object-oriented telescope control system in C++
– IARDS, an aips++ “quick look” real time data display
– DISH, the aips++ single dish data reduction environment
– Glen’s Spectrometer Tool
– Frank’s pulsar GUI and pulsar monitors
• In addition to being difficult to test functionality from end to end through all
these systems, it was often difficult to determine which application should
host a particular function. There was no clearly defined structure by which
one would know where to develop a particular new requirements.
• Operational problems can also result! (Ex. An individual’s application
using up too many connections to devices, causing cascading errors that
require extensive troubleshooting and operational support)
The Future
• Code is written once, used by multiple applications as necessary (example
of this is calibration)
• Code is written to accommodate a discrete number of well-defined levels of
abstraction:
– Applications
– Application Components
– High-Level APIs
– Low-Level APIs
– Data Systems
• Distinct categories of usage:
– Standard users interact with Applications and Application Components
– Expert users interact with High-Level APIs
– Programmers interact with Low-Level APIs
High-Level Architecture
GUIs
Application
Application
0..n
Component
uses
uses
Expert User HLAPIs Programmer
uses
LLAPIs
Control
System
Observed Real-Time
Data Data
(in an EDF) (streaming)
Data Processing (*)
Obs Prep Tool
ObsProposal ObsProject
Proposal Submission Tool consists of
Science
ObsUnitSet Products
* Science products may be
produced as part of the
pipeline or in an offline
mode
Scheduling Block
Quick Look
Display (GFM)
APIs
Quick Look
Execution Block
Results
Goal for the
Logical Structure Calibration
of a GBT Scan Results
Observing Project
(software subsystem Tel Cal Tool
boundaries noted by
dashed boxes) Subscan
Data Capture
Project Description
Quick Look
Data Display
GBT OT SB Executor
Status Screen uses
BUILD Configuration API
EDIT Observing API
Console Windows
RUN Balancing API
MONITOR
Easy Access to
Documentation, Help
• Scheduling Blocks
• Annotated Index of
Integrated Desktop Observation
• Execution Log File
* Note that APIs are very telescope-specific, yet are used
to abstract the observation into terms not specific to the
telescope Observation
Management Database
(*e2e = Project Database)
Integrated Desktop
Development Phases
1. Configuration – 10/1/03 to 3/31/04 – 6 mo elapsed
2. Observing – 11/15/03 to 9/17/04 – 6 mo elapsed
3. Scheduling Blocks – 5/17/04 to 9/17/04 – 3 mo elapsed
4. Full Functionality – 2005
5. Program Level Integration / Remote Observing – 2006
(Items in #4 and #5 have been referred to as “Advanced Ease of Use”)
1. Configuration API
• Provides ability to simply configure hundreds of parameters on the telescope by
specifying a small subset (<15) of astronomical metakeywords
• Also provides the ability to directly set control system parameters, for expert users
• Also provides diagnostics, including the ability to check devices, the state of the IF
path, frequencies and velocities of the LO1 and LO2, channels, switches, drivers &
modules in use
• Can be done through the Python command line interactively for testing purposes,
or by preparing a script and executing the entire script (recommended for expert
users)
• See Data.ConfigurationCases, Data.ConfigurationToolUsage,
Knowledge.ConfigKeywords on the Green Bank Wiki (http://wiki.gb.nrao.edu)
Can be accessed in the following ways:
– Directly, from the command line, for engineering and commissioning
– Via a Python script for expert users
– By other application components, such as the GBT OT
2. Observing API
• The Observing API, nicknamed “Turtle”, is modeled after the LOGO graphics
builder utility of the same name
• The purpose of Turtle is to provide a series of building blocks that observers
may utilize to create their own observation procedures; these building blocks
can then be used to create complex movements of the beam across the sky.
• Turtle also provides a pre-defined suite of commonly used scans created by on-
site astronomers for observers, which in turn may also be used as building
blocks to create even more complex beam movements.
Amy Shelton will provide a complete description of the structure of the GBT
Scheduling Block and how the new Observing Tool is to be used
3. Scheduling Blocks
• APIs are used in the execution of Scheduling Blocks
• Has identical entities/attributes as the ALMA Scheduling Block, with the possible
exception of setup/configuration
• Although these will be eventually archived in XML with the observer’s calibrated data
products, right now we simply aim to store them in plain text, and have chosen a
format similar to the user preferences file
• Uses the Configuration API and Observing API to execute complete observations on
the telescope; Scheduling Block Executor (implemented within Turtle) controls the
flow of information:
SB EXECUTOR
Configuration API Observing API Balancing API
4. Full Functionality (2005)
• Release of SB-based observation management system delayed from
8/31/04 to 9/17/04 because of this review
• It will cover all GBT Standard Observing Modes except Radar, or
observations with fast-moving near Earth objects (comets/satellites)
– Pulsar observing and some enhancements for spigot ease of use
are scheduled as maintenance activities for Fall 2004
• Required capabilities to be developed include:
– Support for generating ephemeris tables and observing fast-
moving sources like comets and satellites
– Be able to use and define source catalogs
– A Balancing API to eliminate user dependence on CLEO, which
requires expert level knowledge
5. Program Level Integration (2006)
• Work has focused on building SB’s and executing them by populating a
queue and letting those SBs execute first-in-first-out (FIFO), but we will
still need solutions for:
– Queue Management, which integrates constraints on the SBs (such as
opacity) and puts the management of observation execution more in
the hands of the operators
– Building the logical relationships between SBs to create ObsUnitSets
that fulfill scientific intent
– Kicking off pipeline processing (this should be done at the Program
level since the products of multiple SBs may be required to yield data
products – ex. maps)
– Low-Bandwidth Remote Observing Solution
– High-Bandwidth Remote Observing Solution
We may already have the tools to implement both remote observing solutions,
however, our constraints are specifications and science staff time
Development Plan
2H 03 1H 04 2H 04 1H 05 2H 05 1H 06 2H 06 2007
1: CONFIGURATION
API
2: OBSERVING API
TEST
QUEUE
3: SBs BALANCING API
MGMT
SOURCE CONSTRAINTS
CATALOGS
EPHEMERIS INTEG W/PIPELINE
COMPLETED BY END 2004
PHASE 4 TO BE SCHEDULED (2005)
PHASE 5 TO BE SCHEDULED (2006)
Relationship to ALMA and EVLA
• All development has been done with strong reliance on:
– Design documentation from ALMA PDR (March 2003)
– ALMA SB Definitions (January 2004)
– NRAO e2e Archive Model (May 2004)
– NRAO e2e Observing Model (May 2004)
– NRAO e2e Project Model (May 2004)
• Basic functionality of ALMA OT has been replicated:
– Less effort in prototype stage than customizing ALMA OT to use GBT
APIs
– We didn’t want things like “perspectives” and were concerned that we
are not ready to support ACS in -Lite or -Lighter forms
• We could adopt the ALMA OT and Queue Management system in the future
if this is desired; in any case, staff astronomers and observers will be able to
observe according to a “common look and feel” with ALMA upon completion
of the testing period (end 2004)
• GBT could be useful to prototype and test ALMA concepts with users; ALMA
interest is critical
Risks
• No Project Scientist for Spectral Line Data Display; no written specifications
or singular point of responsibility yet for what should be contained in
spectral line data display
– Although we do have the capability now to display waterfall diagrams,
which has been mentioned as a desirable enhancement
– Once specified, development could be done in a reasonably short
amount of time if approved by the GB Project Planning Committee
– Data display work has been driven by PTCS needs so far
• Adoption of SB-based system by staff astronomers. Writing down
observation plans a priori in the form of SBs may require a shift in behavior
for some staff.
• Adoption of OT (which does not look like former GO application!)
• User interfaces historically controversial – lots of gray areas between
needs and wants, and individual preferences
• Community buy-in to ALMA/e2e concepts and directions