Get documents about "7 5 SAF User Conf Test and Distribution Process Lopez
Shared by: HC120831174450
-
Stats
- views:
- 5
- posted:
- 8/31/2012
- language:
- English
- pages:
- 25
Document Sample
OneSAF Test and
Distribution Process
Stephen Lopez-Couto
Session 7.5
Discussion Points
• Introduction
• Major and Minor Releases
• The Content of a Release
• Planning for a Major Release
• PM (Internal) Testing
• Assessment Testing
• Miscellaneous Release Preparation
Tasks
• Conclusion
Introduction
• Prior to any official distribution of
OneSAF a lot of preparation,
coordination and testing occurs
– Major “X.0” releases
• Process begins approx. ten months prior to the
release
– Minor “X.Y” releases
• Process begins approx six months prior to the
release
– This presentation will discuss all of the tasks
that lead up to packaged versions of
OneSAF being shipped to users
Major and Minor Releases
• Major Releases
– Generally occur once per year
• Based on a full fiscal year’s worth of work
– Are subject to the most thorough T&E process
• Test is managed by TPO OneSAF
– At a user site
• Release decision falls on TPO OneSAF
• Minor Releases
– Generally occur twice per year
• PM Determines when OneSAF is going to provide a useful
new capability or set of bug fixes
– Release test managed by PM
• Has TPO and user participation
• Release decision falls on PM OneSAF
A Year’s Releases
• Version 1.x of OneSAF set and followed this
release formula:
– 1.0: The major release for the year
• Very thorough testing and the most new capabilities
– In this case it was the first major OneSAF release, so it is a slightly
different case than in future major releases
– 1.1: Completed shortly (3-4 months) after the Major
release
• Includes mostly non critical bug fixes found during Major
release testing
– 1.5: Completed approximately 2/3 through the year
• Contains a sampling of the features that will be fully
available in the next Major release
Determination of Version Content
• Two primary sources
– PM OneSAF Developed
• Bi-annual requirements boards meet and
determine the items that should be developed
during the year
– Those items are developed based on availability of
primary program (Army) funding
• Customer Funded
– A customer pays the PM to perform development work
– Co-Developers
• Build/Modify OneSAF and handover the code to
the PM for integration
Determination of Version Content
• The year’s development and integration tasks are
distributed into the system build schedule
1.0 1.1 1.5
October November December January February March April May June July August September October November December
Build 1 Build 2 Build 3 Build 4 Build 5 Build 6
Release Process Flow Chart
System Integration and Test
Analyze PTRs, CRs, Produce List of Use List to Select
Identify
CoDeveloper RIB items, Tests from test
Handovers specific tests
capabilities categories
[List of Tests to execute]
Begin ECCB Approval Complete Integration,
PM/TPO Witness
of Final Capabilities System Testing,
Integration Test, creates PTRs
Regression Testing
And PTRs
Start PRB Reviews
ECCB [All PTRs completed for this release]
PTRs, sets
severity reviews
[Sev 1 PTR, to be [Other PTRs (not Sev 1)]
fixed for this release]
Sev 2 PTR Urgent, provide patch prior to next release,
Fix PTR
Sev 2 PTR for next release are noted, remaining
PTRs are prioritized at future RIB meetings.
Begin Release Documentation Create Media
Documentation Review
(VDD, Installation Inst.)
for testing
Test Media and
installation
Create Media for
distribution
End
Major Release Planning
• TPO and PM OneSAF select an approximate date for
the major release
• First major task is to plan for the pre-release software
assessment
– Logistical planning
• Location
• Attendees
• Hardware, Software, Network
• C2 systems
– Assessment content
• Threads
– New and regression
• Domain Events
– Three face to face meetings conducted during the year in
preparation
Major Release Planning
• Assessment site selection
– TPO queries various user sites to ascertain
interest in hosting
• Site must satisfy a number of requirements
– Equipment
– Accessibility
– Availability
– Space
– TPO and PM personnel travel to perform site
surveys at interested locations
– TPO selects best site for that year’s
assessment
Major Release Planning
• Initial / Mid / Final Planning Conferences
(IPC / MPC / FPC)
– IPC (10 months out)
• Finalize the location for the assessment
• Discuss the general approach that will be taken
for the assessment
– Examine the previous event and look for changes
– Determine success criteria
• Discuss in general the content of the assessment
– Development is still in the early stages
Major Release Planning
• Initial / Mid / Final Planning Conferences
– MPC (6 months out)
• Discuss the technical details of the assessment
– Machine configurations and placement
– Network
– C2
• Finalize the schedule and content
• Begin the development of the test threads and larger scenarios
• Agree on each domain’s involvement
– Assign specific thread and scenarios
– FPC (3 months out)
• Held onsite at the assessment facility
• Finalize all technical details
• Review thread and scenario development
• Cover any remaining details
Major Release Planning
• One of the most troublesome areas in
selecting a site is the availability of
tactical C2 systems
– Some have little to no equipment
– Others have incompatible versions
• PEO STRI Digital Integration Lab has
assisted in the past
– This is not sustainable and will not be the
approach for the 2.0 assessment
PM Testing
• Occurs informally to prepare for both
Major and Minor releases
– This is more stringent than standard
development testing
– Status is tracked and reported to the PM
regularly
• Formally prior to a minor release
– Has TPO and user involvement
– PM Uses the results of the testing as an
input to his decision as to whether the
software should be released
PM Testing
• Approach
– A&I determines a set of threads and vignettes that
will best test the baseline applying known resource
constraints
• It is already impossible to test all threads for each release
• Thread tests will continue to grow as new capabilities are
added
• Threads and vignettes primarily test model and tool
capabilities
– Weekly Wars (large scale distributed tests)
• Run over two days
– OneSAF is supposed to remain executing overnight
• This is the primary test of system stability and
performance
Assessment Testing
• Similar in structure to PM tests, but…
– TPO/Users determine the threads that will
be run
• PM generally does not review these
– Users create the vignettes and large scale
distributed exercise
Miscellaneous Release Preparation
Tasks
• Regardless of the state of the software or
outcome of the assessments there are other
tasks that must be completed before OneSAF
can be distributed
– Security Accreditation
– Disc Layouts
– Installer Creation
– Documentation creation / cleanup
– Packaging preparation
– Media Creation
• All of these tasks are tracked in regularly
occurring distribution meetings
Security Accreditation
• Before any Army information technology system is
fielded it is supposed to complete a security
accreditation and receive an Authority to Operate
(ATO)
– Certifies that the system is generally secure and all major
vulnerabilities are known and protected
– Lasts for three years or until the system undergoes a major
change
• OneSAF completed the DoD Information Technology
Security Certification and Accreditation Process
(DITSCAP) and received an ATO for version 1.0
– If within the three year timeframe the PM must present all
system changes to the Designated Approving Authority (DAA)
to determine if a new accreditation is required under the “major
change” clause
• At this time it is most likely not required for OneSAF 2.0
Disc Layouts
• OneSAF contains lots of executable code, data
and documentation
– 1.5 release encompasses 17CDs or 6 DVDs
• Determining the best layout of the content onto
optical media helps with
– Lowering media requirements
– Aiding user understanding of content/installation
• Terrain Databases are by far the largest
consumers of space
– 8/17 CDs, 3/6 DVDs
– Future terrains will far surpass current requirements
• We may need to come up with unique distribution
approaches to handle
Installer Creation
• OneSAF 1.0 and 1.1 had very difficult installation
processes
– The PM received numerous support requests just about
installation
– The old approach was very generalized and required limited
work by the development team for any single release
• 1.5 has a new “unified” installer that greatly simplifies
the process for the end user
– More like commercial software installations
– Much more complex for the development team
• Each release requires rework of the installer to handle
changes in:
– OneSAF code and data
– Java and other 3rd party software
Documentation Creation and Cleanup
• Prior to each release a detailed Version
Description Document (VDD) is created
– Includes input from all task orders and
Government
• Installation Manuals updated
• Maintenance and User’s Manuals are
presently extracted from the onesaf.net
website
– Requires work to ensure that all data is
placed onto discs properly and links function
as designed
Packaging and Media Creation
• OneSAF is released in commercial grade
packaging to ensure that media remains
organized and accessible to end users
– PM provides the PEO STRI graphics office data and
design ideas for disc labels and case graphics
– Software Distribution and Test (SWDT) Team
• Purchases and prepares media cases and any printed
documentation
• Coordinates with media duplication vendor to have discs
created
• Executes final media testing
– Test a random sample of discs after handover from
duplication vendor
Correcting Distribution Problems
• OneSAF 1.1 users experienced
numerous problems with the DVD
version
• Multiple actions to correct this have been
implemented
– New duplication vendor selected
– PM No longer provides “Gold discs”
• Everything is kept on hard drives
– More thorough final disc verification will
occur
Conclusion
• Getting a version of OneSAF out the door
is not a quick or simple process
• The content and quality of the software is
of most importance
– but without these events occurring the user’s
(You) would never get to use it
Questions
Stephen Lopez-Couto
Stephen.lopezcouto@us.army.mil
