Document Sample
2011_04_11_DVED_ITEC Powered By Docstoc
					     Synthetic Environment (SE) Core Database Development Process

                                                Robert Cox
                                         PEO STRI, PM CONSIM

                                           Guillermo Flores
                                        PEO STRI, PM CONSIM

                                             Mark Johnson
                                       PEO STRI, PM CONSIM

                                              Dolores Lowe
                                        PEO STRI, PM CONSIM

                                              Connie Perry
                                        PEO STRI, PM CONSIM


ABSTRACT: The SE Core is part of the United States Army's overarching strategy of developing
simulation systems that help make our Warfighters the best trained in the world. The two primary
initiatives under the SE Core program are the Architecture and Integration (A&I) and the Database Virtual
Environment Development (DVED). The A&I is not the focus of this presentation and will not be discussed.
The SE Core's primary mission is to rapidly generate correlated simulation system terrain databases. The
master SE Core database is populated from a union of multiple authoritative data sources by using a suite
of commercial and government-off-the-shelf database tools. The SE Core DVED architecture and tools will
enable the generation of SE Core runtime terrain databases in days or weeks vice months or years.

The focus of this presentation is to describe the SE Core DVED production process. Key areas that will be
discussed will include the overall process and then focusing on the details to include: the initial database
request process; how source data for the requested database is gathered and prepared for use; the
refinement and enhancement of the source data; the standardization process; and the verification and
quality measures employed before delivery. Additionally, the authors will present the current production
along with the various formats and map projections that are supported. The presentation will end with
exemplars of future efforts that SE Core DVED will be pursing in the future.

1. Introduction                                     last three decades the price-performance benefits
                                                    of computer technology and the demand for
                                                    reconfigurable, less expensive, and more realistic
The acquisition of simulation training databases    representations have driven simulation systems
has been a very complex, lengthy, and costly        towards real-time digital computing and fully
endeavor. SE Core’s effort is developing            synthesized environments.
processes and tools to create non-proprietary,
open format, image generator (IG) independent
                                                    Until the early 1980's, simulators were mostly
Environmental Representation (ER) databases1.
                                                    stand-alone systems designed for a specific task
                                                    training purpose. Until the introduction of the
The SE Core Master Database (MDB) is the
                                                    SIMNET program, no one had ever used a
central repository for the creation of correlated
                                                    multitude of simulators in a combined forces
databases used to train, mission plan, or mission
                                                    training environment, interacting over a network
rehearsal in the Live, Virtual, or Constructive
                                                    in real-time. SIMNET technology allowed
(LVC) domains. Within the DVED the database
                                                    crewmembers in one simulator, to interact with
architecture is coupled with a suite of COTS
                                                    many other manned or unmanned simulators
tools that enable database development.
                                                    located at the same or other training sites.
The SE Core effort will also develop common
virtual vehicle models, common virtual sensor       Environmental Representation databases are
simulation software, and the virtual simulation     comprised of many parts. Those parts include
component of the dynamic environment. The           but are not limited to the terrain surface, terrain
dynamic environment will include approximate        features, 3-dimensional models, textures, images,
visual effects from simulation (e.g., munitions,    and other data such as system specific data to
mobility, engineering, rubbled buildings, etc.)2.   satisfy computational requirements and labels for
                                                    producing electronic and paper maps.
The SE Core process is flexible, data-driven, and
extensible. The Standard/Rapid Database             There are several constraints on the creation of
Generation Capability (STDGC) is designed to        environmental databases. Perhaps the most
standardize, align, and clean source data in the    prominent of these is dictated by the real-time
Master Terrain Database Generation Toolkit          computation requirements. Since processing
(MTDGT), which also is used to generate             power is often limited, once a computing
synthetic environment data using the Runtime        platform is chosen for a given cost-performance
Database Generation Toolset (RDGT). Data is         range, the software must be designed for best
exported in various formats from a standard         real-time performance. This means both the data
application programmer interface (API)3.            and the data structures stored in a platform-
Furthermore, the SE Core process addresses          specific version (runtime) of an environmental
other challenges such as miscorrelation between     database play an important role in optimizing the
simulations and lower database generation costs.    system performance.         Given     the     fixed
                                                    computational budget of a system, the database
In this paper we will first provide some            designer must take into account the application-
background on the terrain database generation       specific requirements, the size and extents of the
process. Next, we will present the SE Core          database, the desired density and fidelity, as well
methodology to include how a database is            as the type and amount of the available raw data
requested, the procedure for gathering source       elements that must be incorporated into the
data, the standards that are used by SE Core, and   database.
the techniques used to refine and enhance the
data. We will end with a discussion of the          There are other application-specific requirements
verification and quality process used by SE Core.   that must be taken into account such as those
                                                    related to whether the database will be used for
                                                    air, ground, sea, near ground, or any combination
2. Background4                                      of these simulations. The detail needed by a
                                                    simulator that allows an individual to walk on
Military training simulation systems have been      the terrain is much greater than that expected for
around for decades. Most of these systems have      a helicopter simulator typically flying several
used some derivation of the real environment to     hundred feet in the air. Database density, size
represent the three-dimensional reality. In the

and extents, viewing range, field of view, and                                                                                              condition for achieving it. Since the variables
other important simulation requirements dictate                                                                                             affecting interoperability are many and complex,
the amount and type of data that can be included                                                                                            effective mechanisms for making the database
in a database without overwhelming the                                                                                                      interchange     process      successful    become
performance requirements.                                                                                                                   significantly more difficult and challenging.

Often the intended simulation platform imposes                                                                                              A critical factor in constructing and sharing
specific constraints. The specific special purpose                                                                                          environmental data is incorporation and use of
hardware architectures, designed for the sole                                                                                               good tools. Most tools are special purpose, given
purpose of real-time image generation, impose                                                                                               the various criteria and techniques employed by
vastly different constraints on the database                                                                                                different suppliers for constructing and tailoring
contents. Two database designers building a                                                                                                 databases. As the domain of networked
database of the same region for two different IGs                                                                                           simulation expands and other information
often arrive at entirely different end results. The                                                                                         technology sectors emerge, the need for more
polygon or object processing capacity of an IG                                                                                              common and yet more sophisticated tools will
limits the database density to levels that can be                                                                                           increase.
processed in real-time. Similarly, image
rendering techniques drive whether a database                                                                                               Figure 1 is the SE Core processes with the
can contain textures and how many, or if the                                                                                                objective to be able to collect, generate, and
image generator can render all the objects that                                                                                             manage the Master Database (MDB) with the
are potentially viewable in a scene within a fixed                                                                                          highest fidelity of source data available. There
frame time. There are other architecture-specific                                                                                           are four major parts of the process. For this
features such as caching scheme, processing of                                                                                              paper the focus will be on how a database is
transparent objects, or image enhancement                                                                                                   requested, the gathering of source data and its
techniques like anti-aliasing drive how a                                                                                                   refinement. We will include a discussion of the
database may be partitioned.                                                                                                                standards and quality measures that are used.

These types of computation
constraints are not unique to
visual       systems.      Any
information         technology
                                                                                                       Standard / Rapid Terrain Database Generation Capability
application that must achieve                                                                                                                                          Supervisor
specific real-time performance        Master Terrain Database Generation                                                                                                                              Run Time Database
measures has to reflect such                   Toolset (MTDGT)                                                                                                                                     Generation Toolset (RDGT)
                                                                                                                                                                                                                      Filtered Viewer

                                                                                                                                                                        MDB Repository Quality
objectives in its design. This                                                                                                                                                                                                                                                                     CGF

                                                                                                                                                          3D Models



                                                                                                                                                                                                                                                                            SEDRIS Plug-In

in turn impacts the data                                                                                                                                                 Write
                                                                                                                                                                                                                                                                           Paper Map Plug-In       PVD

                                                                                                                                                                                                 Configuration                                                             Elec. Map Plug-In
                                                                                                                                                                                                                                                                 SEE                             Sensors
structures, data types, and the                                                                                      Refinement
                                                                                                                                                                                                 Management                                                CER
                                                                             Source Data Quality

                                        Source Data Inporter

                                                                                                                                                                                                                                                                               Plug-Ins           Visual
information that is needed and
                                                               Source Data


                                                                                                                                                                                                    Run Time                                                                 DBDD Plug-In
                                                                                                         Source                                                                                                                                                                                   Maps
expected to be available from                                                                                                                                                 Master

                                                                                                                                                                              (MDB)                                                                                                              Mission
a database.                                                                                                                                                                    Storage                                                                                                          Run Time
                                                 Raw                                               Source Interchange                                                                                                                                                                          Applications
                                                                                                                                             CAD                             Architecture
                                                Sources                                                 Formats                             Sources

Interoperability of multi-                                              Source Data Asset Management

fidelity systems on the same
network is highly desirable; in
fact it is often demanded. The
challenge is in determining the "right" type and
amount of environmental data that each                                                                                                                                Figure 1: SE Core Process
simulator should use to ensure interoperability.
Most successful heterogeneous networked
exercises have been conducted under restricted                                                                                              3. Database Request Process
                                                                                                                                            The SE Core Database Request process is shown
A common misconception is to equate success in                                                                                              in Figure 2.
the interchange of data with success in
interoperability. Interchange of data does not
guarantee interoperability, but is one necessary

Figure 2: SE Core Database Request Process             data is collected (discussed in the next section)
                                                       and any revisions to the request are processed
                                                       depending on the needs and availability of data.
                                                       If all is acceptable to the customer, the request is
                                                       passed to the database development team. The
                                                       current and future data formats available from
                                                       SE Core will be discussed in a later section.

                                                       The customer is provided with a database request
                                                       form. That form allows the customer to specify
                                                       the content they require. This includes but is not
                                                       limited to the features they require (e.g. bridges,
                                                       roads, agricultural areas), areas of significance to
                                                       include their location (e.g. specific buildings,
                                                       airfields), and the format of the database
                                                       transmittal. The customer can also use this form
                                                       to request any high resolution inserts that are of
                                                       importance to them.

                                                       4. Source Data

                                                       When a database request form has been
                                                       submitted and it has been approved, the process
                                                       of gathering source data begins (Figure 3).

Initially a program fills out a database request. If
the data already exists in the MDB and a means
to generate the required formats is available (e.g.
plug-ins) the request is forwarded directly to the                                                            START

production team and the desired data in the                                                              DATABASE

requested format is prepared and sent back to the
requesting program.                                                                                        DATA

                                                                                         DETERMINE      DETERMINE    COORDINATE
If the desired data is not already in the MDB, the                          TECHNICAL
                                                                                                       STORAGE AND SOURCE DATA
                                                                                                           SPACE      ENTERING/
                                                                                                        ALLOCATION EXITING FACILITY

request is validated by the US Army TRADOC                           NO

Capabilities Manager (TCM) Virtual to include                                                            REVIEW

the size of the database region, the level of
fidelity required, and any other requirements of                                                         USEABLE

interest to the customer which have been                                                                 SOURCE?

validated by TCM Virtual.                                                                                     YES

The database can be divided into separate zones                                VECTOR SOURCE
                                                                                                                              3D MODELING

if the customer requires. Those zones range
from providing only the terrain skin for a reqion
in the database to high fidelity inserts of urban                                                              END

environments. Currently, there are five (5)
different zones available for the customer to
request and they include levels of fidelity of the
transportation network, population density to            Figure 3: SE Core Source Data Collection
include buildings and building interiors, any                            Process
hydrography, significant military operational
features, and various vegetation that is required.     There are five (5) questions that must be
                                                       answered as the database request is fulfilled.
Once the request has been validated, it is passed      Those questions are:
to the program manager. At this point, source

    1.   What are the technical requirements          information as the user that inserted the data into
         (database and system)?                       the database, notes on why the data was
    2.   What are the resources required for          populated, and the origin of the data. SE Core
         acquisition?                                 will be expanding the level and type of metadata
    3.   What       are     the      storage/space    in the coming year.
         requirements for the data?
    4.   What coordination of acceptance of the       The SE Core data model is based on the SEDRIS
         source data must be done and how do          ISO/IEC 18023-1 Data Representation Model
         we deliver the resultant data?               (DRM)8 and all data concepts are defined by the
    5.   Are there any conversions that are           SEDRIS ISO/IEC 18025 Environmental Data
         expected?                                    Coding Specification (EDCS).9

The SE Core has also developed an extensive           Along with the above standards, the SE Core
document that outlines the source data                program is in compliance with the US NGA
investigated for use5. The reference document         standards that are developed by ISO Technical
lists over 160 different types of data that are       Committee (TC) 211.
currently investigated for use in building an SE
Core database. The principle data types that SE       Regional information is stored as areal features
Core uses include:                                    with the attribution that defines the region. For
                                                      example, a feature for a given area has the
    1.    Imagery:      CIB, Buckeye, and             attribution that defines possible textures that can
         Quickbird                                    be applied to models within the spatial extent of
    2.   Vector:     VMAP, Urban Tactical             that region. In addition, regions can be stacked
         Planner, NAVTEQ, DAFIF, and Shape            with a priority based on user needs.
    3.   Elevation: DTED, LIDAR                       There are several tools that are used to certify the
    4.   Models:      Site Photos, Building           data is ready for use. These tools check for
         diagrams, CAD                                invalid and null geometry and attribution. Plus,
                                                      the tools check for compliance to the SE Core
Once the five questions have been answered and        standards for geometry, spatial referencing,
the source data identified and collected, SE Core     definition, labelling, etc.     There will be a
is now ready to start processing of the data to       complete section on the SE Core quality process
include refining and enhancement. However,            and where in the SE Core process checks are
before this discussion is presented, a brief          performed to make sure the user request has been
discussion of source data standardization will be     met.
                                                      6. Data Refinement and Enhancement
5. Standards
                                                      A database, while required for a specified
All data is spatially partitioned based on MIL-       purpose or training event in totality, oftentimes
PRF-89041A6. The imagery is stored based on           contains certain areas in it that are more essential
the US NGA CIB format. For consistency, the           than others for that particular use. Therefore, SE
vectors are also partitioned based on the CIB         Core has developed a stratification of “zone” that
schema with one important difference; they are        help both the user and SE Core to focus
not chopped to the tiles, but references the          resources in those areas that are required by the
groups of features to the CIB tiles they intersect.   customer to have greater attribution, while
All data is stored in latitude/longitude using the    developing the full database.
WGS-84 reference datum. If source data uses a
different coordinate system or reference datum,       There are five separate zones that are based on
the SEDRIS developed ISO/IEC 18026 – Spatial          map projections (e.g. 1:250, 1:100). The zones
Reference Model7 is used for coordinate               contain the level of fidelity that the map
conversion or datum transformation.                   projection would have, but in some cases there is
                                                      more information added based on the
Data element also have metadata attached.             requirements that must be met. The additional
Currently, the metadata includes such                 information could be based on:

                                                                                  have the correct height, and the SE Core
    A.   SAF only padding                                                         automated tool checks are passed.
    B.   Ancillary areas                                        3.                Elevation: Checks for elevation data
    C.   Navigationally significant areas                                         include checking for spikes in the data,
    D.   Training areas                                                           making sure the proper projection is
    E.   High resolution Inset                                                    used along with the requested resolution
                                                                                  is present. Also, a check is made to
Also, within each of these zones/map projections                                  make certain the data is correlated to the
there are different significant characteristics that                              ground truth.
are organized based on the user’s need. Below is                4.                Models: The models are reviewed for
an example of five of those characteristics.                                      polygon attribution, footprint, LoD,
                                                                                  lights, etc. Plus, there is a texture
    1.   Transportation                                                           review that includes the format, sizing,
    2.   Population                                                               and application.
    3.   Hydrography
    4.   Military Operations                           Once the data has been refined and enhanced, it
    5.   Vegetation                                    is ready for the quality process checks.

Thus, if an area of the database is to be              7. Verification and Quality
developed at the Zone D level, the following
would be way the area is prepared for each of the
                                                       Figure 4 shows the SE Core quality steps
five characteristic areas:
                                                       (marked by letters “QC”). While there are steps
                                                       throughout the SE Core process, the majority of
    1.    Transportation:      All transportation
                                                       steps are found in the Master Terrain Database
         features extracted as linears. Parking
                                                       Generation Process. This is where source data is
         lots extracted as areals.
                                                       taken in and a sequence of steps are executed to
    2.   Population: All buildings represented
                                                       guarantee the data is ready for uploading into the
         with either 3D buildings or areal
                                                       SE Core MDB.
         features that are extruded by the RDGT.
    3.   Hydrography: Hydro features validated
                                                       Each of the four principle source data types has a
         against the imagery for location and
                                                       quality check. For instance, the vector data is
                                                       checked for duplicate features, self-intersecting
    4.   Military Operations:        All features
                                                       features, feature overlap, and valid attribution.
         extracted from imagery. Features not
                                                       For imagery, the data is checked for the
         recognizable in overhead images will be
                                                       projection type and orthorectification. There are
         placed by hand when appropriate source
                                                       elevation data checks for spikes and gaps and the
         is provided.
                                                       models are checked for conformance to the SE
    5.   Vegetation: Individual tree locations
                                                       Core model specification.
         will be provided in the source data.
         Forests will be extracted as areal
         features. Trees will be scattered in the
         RDGT.                                                                   Standard / Rapid Terrain Database Generation Process
                                                         Master Terrain Database Generation                                                          Run Time Database
                                                                  (MTDGT) Process                                                                 Generation (RDGT) Process
As presented in the previous section, there are           Sources Standardize        QC        Vector Digitizing                                                                                      DBRR
                                                                                                                         MDB Repository Quality

four fundamental data types that SE Core uses.                   QC

                                                                3D Models
                                                                             Vector Cleaning                                  Assurance                    Pre-Validated
                                                                                                                                                               RDGT                                     QC

The following are the general processing that SE                    QC           Textures            QC            DB

Core does for each of these data types.                                  Elevation                   QC
                                                                                                                                                       QC           QC     QC                           QC

                                                                                Imagery              QC                      Database              Project Set Up
                                                                                                                                                                                                      Free Fly
                                                                                                                              (MDB)                             CCDS

                                                                                                                                                                                                    Run Time
                                                                                                                                                                                                    Check Out

    1.    Imagery: A check is performed for                Raw           Source Interchange         CAD                      Architecture          Technical                    Quarterly
                                                          Sources             Formats              Sources                                        Interchange In
                                                                                                                                                                         SME    Working     DB
                                                                                                                                                    Meetings Process             Group Request
                                                                                                                                                             Reviews                      Process

         overall aesthetic quality to include                         Source Data Collection                                                               Database Request from TCM-V

         colour and cloud cover. In addition the
         resolution, projection, and correlation to    Figure 4:                                      Quality Checks Built into the
         ground truth data are checked.                Process
    2.   Vector: Checks include ensuring that
         vectors are aligned with the imagery,         At the runtime database generation step a series
         attributions are consistent, buildings        of tests are performed to validate the database

being generated for a user meets their               Here is a brief list of tools that have been
requirements. Checks at this step include such       created.
items as missing textures, terrain through
bridges, disconnected bridges, bad vertices,             1.   AutoSnapper:        Used to enhance
missing blocks, and specific tests for the various            ArcMap snapping functionality.
formats that SE Core can produce (e.g. DTED,             2.   MADV: Extends ArcMap and provides
Shape Files, OTF, CTDB, and STF).                             techniques to move, add, and delete
One such tool that SE Core has developed to              3.   Google Earth Sync tool: Syncs the area
assist in this process is called the Vector                   of the world loaded within ArcMap to
Analyzer (VAZER). This tool was developed as                  Google Earth.
an extension to the ESRI ArcGIS and contains a           4.   CM2 FID SMC tool: View and edit the
suite of tools that assist in finding issues with             feature ID (FID) and Surface Material
vector data the production process. This tool can             Code (SMC) setting of a moving model
find issues such as duplicate features, or                    in Multigen Creator.
improper attribution, or improper vector                 5.   The Attributor:        Applies materials
alignment.                                                    attribution to a selected polygon based
                                                              on the data model specified.
8. Current Databases and Formats                         6.   EDM Editor: Used for editing the data
                                                         7.   Validation Plug-in: Used with Terra
The SE Core program has produced several
                                                              Vista to validate data within the terrain
databases and supports over 15 different formats.
                                                              database specified by user inputs.
This phase of the effort has focused on the PEO
                                                         8.   Illuminati: Calculates and stores default
STRI virtual programs Close Combat Tactical
                                                              intensity values or and average
Trainer    (CCTT)      and    Aviation     CATT
                                                              luminance of a group of polygons.
(AVCATT). As such, the primary databases that
                                                         9.   Tune Town: Allows a user to tune
have been developed by SE Core have been
                                                              terrain database data.
those to support these programs. The databases
include Fort Hood, Fort Riley, Hawaii, Fort
Stewart, and several overseas databases.             9. Future Efforts

As stated above, the following list is some of the   The SE Core program has been focused on
formats that are supported by SE Core:               supporting the virtual domain. Now the program
                                                     will begin to add other domains. The first of
    1.   CADRG                                       which is the constructive domain. In particular,
    2.   CCTT PVD                                    SE Core will start the effort to provide OneSAF
    3.   CTDB                                        with its required environmental representation
    4.   DTED                                        databases.
    5.   Open Flight
    6.   OTF                                         Also, with the addition of supporting
    7.   SEDRIS Transmittal Format                   constructive simulations, the type and amount of
    8.   VBS2                                        metadata required will increase. The SE Core
                                                     program will re-evaluate all of its current
SE Core DVED also supports these and other           metadata types and add other data types to not
image generators:                                    only support constructive, but live simulation
                                                     and training programs.        As new required
    1.   EPX 50                                      metadata types are identified, they will be
    2.   Radon                                       compared with ISO 19115-2, the currently used
    3.   SAGE                                        metadata standard by the US NGA. If a new
    4.   S2 Focus                                    metadata type is identified and it is not currently
                                                     in this standard, the SE Core program will
Along with the various databases, format, and        approach ISO TC 211 and request that it be
image generators that SE Core supports, the          added to the standard.
program has created over 60 tools that are used
to support these and other processes in DVED.

In addition to these efforts, the program has        (3)     Johnson M.D., J. Freeman, C.M. Perry;
several other required tasks that will be worked     SE Core DVED – An Introduction to the
as resources are made available when existing        Standard/Rapid Database Generation Capability
efforts are completed.                               (STDGC), July 2007 IMAGE Conference.

10. Summary                                          (4)      The Background Section was taken in
                                                     part from
This paper provided the current status of the SE
                                                     (5)      SE Core SVR Management Program
Core program. We provided background on why
                                                     Attachment 19: Baseline Source Data Providers
the SE Core program is essential to the US Army
                                                     List,    W900KK11R0002,      October    2010,
simulation community and provided the top level
SE Core process. From there, we outlined the
database request process; how to get a database
form SE Core and then described the source data
and the process used to acquire source data. The
                                                     (6)     US National Geospatial-Intelligence
principle data types used by SE Core are
                                                     Agency, MIL-PRF-89041A. Controlled Image
imagery, vector, elevation, and models (both
                                                     base. Bethesda, Maryland, USA: NIMA, 2000.
static and moving).         Next, the SE Core
standardization process was discussed. The
                                                     (7)     ISO/IEC 18026:2005, Information
process includes the use of several international
                                                     technology – Spatial Reference Model (SRM).
standards. A key area of discussion was how SE
Core refines and enhances the source data
                                                     (8)     ISO/IEC 18023-1:2006, Information
collected. In particular, a discussion of how SE
                                                     technology – Data Representation Model
Core has defined five characteristic areas (or
zone) to enhance the level and fidelity of the
databases produced. By building a database
                                                     (9)      ISO/IEC 18025:2005, Information
defined by these zones, enables the production
                                                     technology – Environmental Data Coding
process to focus on user required areas for high
                                                     Specification (EDCS).
definition/details and focus resources in those
areas. The verification and quality process was
outlined, too. There are over a dozen steps in the
SE Core process that have detailed quality
checks to make sure a database is developed and
delivered to a user correctly. The SE Core
database can be delivered in multiple formats
and can support multiple image generators.
Lastly, the future efforts were outlined. It is SE
Core’s intent to begin supporting constructive
simulation in the near future along with
reviewing the current metadata for possible

11. Reference
(1)     SE Core Home page, Database-Virtual
Environment Development (DVED). Available
from World Wide Web:

(2)      SE Core Program Begins helping Army
Warfighters train as they fight, Military
Simulation & Training News, Issue Number 19,
Fall 2008/Winter 2009, CAE. Available from
World                Wide                Web:


Shared By: