These data were fitted to models to predict rates of skill acquisition by Y51jP30



                 Improved Performance Research Integration Tool

                           Training Algorithm Enhancements

Alion Microanalysis & Design

Individual Operator Study
IMPRINT, the Improved Performance Research Integration Tool, is used to model the effects of
manpower and workload issues on human performance. This project augmented the ability of
IMPRINT to predict training effects. Alion built an IMPRINT Pro Advanced Training Effects
plug-in that allows an operator to model training strategies and how those strategies impact both
skill acquisition and performance after decay. Experiments were conducted to collect empirical
performance data on two realistic military tasks: flying a Predator Unmanned Aerial System in a
Synthetic Task Environment and controlling a simulated autonomous ground vehicle. The data
were analyzed at the taxon level, an integrated human performance taxonomy already contained
in IMPRINT. Taxons allowed us to generalize training effects to different types of tasks. Alion
built a computational model of the Predator Unmanned Aerial System domain to demonstrate
training effects in action and context. In addition, we studied improvement to the maintenance
modeling capabilities in IMPRINT Pro.

Science Applications International Corporation

Individual Operator Study
For several decades, computer-based modeling and simulation tools (e.g., IMPRINT) have been
used to support the systems acquisition process. Historically, the emphasis applied to
simulation-based acquisition activities has focused on system capabilities that do not include the
process of training operators that are an integral part of the system. The Air Force Research
Laboratory’s Warfighter Readiness Research Division is sponsoring a number of studies aimed
at quantifying the relationship between training strategies and operator performance. In this
study, sixty university students participated in a training program to learn five tasks
accomplished by unmanned aircraft system sensor operators. The training program followed one
of three strategies that varied interactivity between the student and instructor. Data were
collected during two simulator sessions, the first during initial training and the second after a
retention interval. The data were analyzed to determine effects of training strategy, retention
interval, and nature of the task learned on performance measured by time to complete the task
and the accuracy to which it was performed. These data were fitted to models to predict rates of
skill acquisition, retention, and reacquisition, and were implemented in an IMPRINT model.
Model development and validation processes are presented, and recommendations are discussed.

Science Applications International Corporation

Team Training Study
            An experiment was conducted to quantify the effects of teamwork training on
             mission performance and team processes. A baseline training condition (no
             formal teamwork training) and three teamwork training strategies were examined.
             These included Communication skills training, Supportive skills training, and
             Corrective skills training. Differences between teams’ pre-test (pre-team training)
             and post-test (post-team training) performance and process scores served as a
             measure of the effect of teamwork training. Trends in the data indicated that
             teamwork training enhanced both team performance and teamwork processes,
             with task time being affected more than accuracy. A high degree of variability in
             the data limited the statistical significance of the teamwork training strategy
             effects. Implications of the findings are discussed and an approach to
             implementing teamwork training effects in the IMPRINT modeling environment
             is offered.

JXT Applications

Maintenance Training Study
The goal of this study was to make the training forecasting capability of IMPRINT more robust.
The effort aimed to increase IMPRINT’s capability to forecast performance gains for
maintenance tasks depending on the instructional strategy used. The goal was to be achieved by
conducting empirical studies to estimate the impact of instructional strategies on learning and
forgetting in the maintenance domain. Working with automotive program students at Florida
Community College Jacksonville and general university students at Florida State University, the
research team trained the subjects using computer based training, and observed and recorded the
subjects’ performance on five maintenance tasks. Based on the results of the descriptive
statistics, Accuracy lacked variance. Specifically, subjects exhibited performance at nearly
100% across all conditions. While this study of Air Force maintenance training is reflective of
current practices, the Air Force has indicated that it is changing the way it does business. If the
Air Force moves training into a collaborative world where best practices are shared across time
and space, then this new environment will have implications for Air Force maintenance training.
This is a different picture than the one reflected by the data examined in this report.

Florida State University

Literature Review & Meta Analysis
This technical report describes the major activities and results from a contract between the
United States Air Force Research Lab and the Florida State University Learning Systems
Institute. The general project goal was to employ a model-based approach for aligning
instructional strategies with technical task performance. The modeling system used in this effort
was the Improved Performance Research Integration Tool (IMPRINT). IMPRINT has been used
successfully by the United States Military to predict human performance in complex and
dynamic operational environments. At the outset of this project, however, IMPRINT did not
include a training component to determine the effects of instructional approaches on task
performance within various learning taxonomic domains. In order to achieve the project goal, the
project team carried out an extensive literature review on the effects of training on technical task
performance and developed a training effects algorithm based on the meta-analysis of relevant
studies. The training effects algorithm acts as a plug-in to the IMPRINT system and has been
shown to effectively model training effects in a technical mission.

To top