Situation Awareness In General Aviation Pilots by tyndale

VIEWS: 173 PAGES: 204

									                                               SATECH-00-01




Situation Awareness In
General Aviation Pilots
                                       Mica R. Endsley, Ph.D.
                                      Daniel J. Garland, Ph.D.
                                         SA Technologies, Inc.

                                         Richard W. C. Shook
                                                 John Coello
                                             Marco Bandiero
                           Embry-Riddle Aeronautical University



                                                January, 2000




  Prepared for NASA-Ames Research Center
        Under Contract #NAS2-99073
           Annual Report - Year 1
         Dr. Judith Orasanu, COTR




                             SA technologies
                                     4731 East Forest Peak
                                       Marietta, GA 30066
SA technologies                                                                            Page 1


     On July 16, 1999, a Piper Saratoga II crashed into the water off of Martha’s
     Vineyard, killing the pilot, John F. Kennedy, Jr. and his two passengers. The crash
     occurred at 9:41 p.m. under visual meterological conditions, with reportedly
     reduced in-flight visibility. The plane began a normal descent 34 miles from the
     airport. At 2,300 feet of altitude it turned to the right and climbed 300 feet where it
     remained for 1 minute. It then descended at 700 feet per minute and made a left
     turn back. Thirty seconds later, it turned to the right again and entered a rapid rate
     of descent exceeding 4,700 feet per minute, crashing into the ocean. Preliminary
     findings indicate the plane had no major engine or propellor failures, no structural
     failures and no evidence of fire (NTSB, 1999). Mr. Kennedy possessed a private
     pilot’s license and approximately 300 hours of flight experience. Although the
     circumstances vary, in many ways this story is typical of the approximately 1,900
     accidents involving GA aircraft each year, 19% of which are fatal.



1.0 Introduction

1.1 Situation Awareness

In the aviation domain, maintaining a high level of situation awareness is one of the most critical
and challenging features of a pilot’s job. Situation awareness (SA) can be thought of as an
internalized mental model of the current state of the flight environment. This integrated picture
forms the central organizing feature from which all decision making and action takes place. A
vast portion of the pilot’s job is involved in developing SA and keeping it up to date in a rapidly
changing environment. Many definitions of SA have been developed, some very closely tied to
the aircraft domain and some more general (see Dominguez (1994) or Fracker (1988) for a
review). A general, applicable definition describes SA as “the perception of the elements in the
environment within a volume of time and space, the comprehension of their meaning and the
projection of their status in the near future” (Endsley, 1988). Thus, it includes more than
perceiving or attending to information (Level 1 SA), but also the integration of multiple pieces of
information and a determination of their relevance to the person’s goals (Level 2 SA), and the
ability to forecast future situation dynamics (Level 3 SA), thus providing for timely and effective
decision making.

Having a high level of SA can be seen as perhaps the most critical factor for achieving successful
performance in aviation. Problems with SA were found to be the leading causal factor in a
review of military aviation mishaps (Hartel, Smith, & Prince, 1991). In a study of accidents
among major airlines, 88% of those involving human error could be attributed to problems with
SA technologies                                                                              Page 2


situation awareness as opposed to problems with decision making or flight skills (Endsley,
1995a). Although similar studies have not been performed for general aviation (GA) accidents,
SA is reported to be considerable challenge in this population as well, particularly as GA pilots
are frequently less experienced and less current than operators for major airlines (Hunter, 1995).
NASA has identified situation awareness as one of seven major task areas targets for human
error reduction in its Aviation Safety Program which is seeking to meet the government’s goal of
reducing fatal aircraft crashes by 80% over the next ten years (Huettner & Lewis, 1997). Given
the key role of situation awareness problems in pilot error, programs to deal with this issue
should be particularly effective at achieving the accident reduction goal.

1.2 Research Goals


Due to its importance and the significant challenge it poses, finding new ways of improving SA
has become one of the major design drivers for the development of new aircraft systems.
Interest has also increased within the operational community, which is interested in finding ways
to improve SA through training programs. Due to the important role that SA plays in the pilot
decision making process and its substantial role in aviation accidents and incidents, the goal of
the present research is to develop and validate methods for training to improve SA in aircraft
pilots.

This research is focused on SA in general aviation. This focus was selected for a number of
reasons: (1) General aviation accidents and incidents account for 94% of accidents and 92% of
fatalities (based on July, 1999 accident data), an accident/flight hour rate over 27 times that of
scheduled part 121 carriers (NTSB, 1999). Reduction of accidents in this arena will contribute
substantially towards the overall goal of reducing aircraft accidents. (2) Improvement of SA in
GA pilots will also contribute towards improving the overall safety of the airspace system as
these aircraft share the same congested airport areas, runways and enroute environment with
commercial aircraft. (3) Improvement of SA in GA pilots may also have a trickle-up effect. As
major airlines and commuters are more frequently seeking to hire pilots from the GA population,
partially due to limits in the numbers of pilots available from the military ranks, improved skills
in GA pilots may translate into better pilot capabilities in other areas of aviation. (4) Many of the
research and training techniques developed for the GA pilot may also be transferable to other
areas of aviation. Many of the issues that can be found to underlie problems in SA in the GA
pilot appear to also be a problem for pilots at-large (as will be discussed in more detail). (5) By
focusing this developmental research on the GA pilot, a more readily available population can be
studied at much lower cost than can commercial pilots. This focus also allows us to study
individual SA separately from the problems of team SA.
SA technologies                                                                              Page 3


Pursuant to the goal of developing and validating methods for training to improve SA, research
that describes issues pertinent to human performance in general aviation will be briefly reviewed.
In addition, challenges for SA and recent work on training SA will be discussed in more detail,
laying a foundation for the development of this research program. Finally, Year 1 research
results will be presented and discussed, followed by the approach to be taken for Years 2 and 3
of the research project.

1.3 General Aviation


General aviation represents a wide variety of activities including private, corporate, government,
and individual business travel, as well as training, and aerial applications such as law
enforcement, forest fire fighting, freight transportation, air ambulance, and other work use. A
total of 1,907 U.S. registered general aviation aircraft were involved in accidents in 1998. Of
these 1,907 accidents, 361 accidents resulted in fatal injuries, with a total of 621 fatalities. U.S.
general aviation accidents accounted for 94% of all U.S. civil aviation accidents and 94% of U.S.
civil aviation fatalities in 1998. U. S. general aviation accounted for 7.12 accidents per 100,00
flight hours as compared to 0.26 accidents per 100,000 flight hours for scheduled U.S. air
carriers operating under 14 CFR 121. In addition, general aviation accounted for 1.35 fatalities
per 100,000 flight hours as compared to 0.006 fatalities per 100,00 flight hours for U.S. part 121
air carriers.

Through July, 1999 1102 general aviation accidents have occurred. Of these 1102 accidents, 189
accidents resulted in fatal injuries, with 336 total fatalities. These general aviation accidents
have accounted for 94% of all U.S. civil aviation accidents and 92% of all fatalities in civil
aviation through July 1999.

 Accident statistics indicate GA accidents have increased slightly from 1,858 in 1997 to 1,907 in
1998. As previously stated, 621 people were killed in GA accidents in 1998. The 1998 fatalities
were the lowest in 15 years in which there were an estimated 26.8 million hours by GA pilots
(NTSB, 1998). According to the 1998 Nall Report (AOPA, 1998) the overall GA accident rate
per 100,000 flying hours has declined over the past 25 years. The declining trend has, however,
slowed in the past 10 to 12 years with a relatively stable trend over the past six years. The GA
fatal accident rate also has declined over the past 25 years, but has remained fairly stable over the
past 16 to 17 years.

While the GA accident statistics are generally good, GA pilots continue to crash and be killed
due to a number of pilot related factors. A 1989 NTSB review of 361 GA accidents concluded
that 97% of the probable causes were attributable to pilot error. In 1985, the pilot was found to
be a "broad cause/factor" in 84% of all GA accidents, and 90.6% of all fatal accidents (Trollip &
SA technologies                                                                              Page 4


Jensen, 1991). In a 1982 review, Jensen attributed 85% of GA accidents to pilot error, with
faulty decision making cited as the primary cause.

Consistent with the past, pilot related factors continue to be involved in most of today's GA
accidents (AOPA, 1997; NTSB, 1998). Initial accident reports for most years indicate that
between 60 and 70 percent of accidents are attributed to pilots as the main causal factor. Final
reports traditionally indicate this percentage increases to 70 to 80 percent when the investigations
are complete. Pilot related accidents are generally attributed to the following factors, in order of
frequency (Trollip & Jensen, 1991).
        •       loss of directional control
        •       poor judgment
        •       airspeed not maintained
        •       poor preflight planning and decision making
        •       clearance not maintained
        •       inadvertent stalls
        •       poor crosswind handling
        •       poor inflight planning and decision making
Most of these issues indicate problems with pilot SA.

In its 1987 "Annual Review of Aircraft Accident Data"(NTSB, 1989), the NTSB profiled GA
pilots most likely to have accidents. Based on the profile, the greatest number of accidents
involved "pilots between 35 to 39 years of age with between 100 to 499 hours total time who
were engaged in personal flying, took place in daylight visual meteorological conditions (VMC),
were precipitated by a loss of power in the landing and takeoff phases, were complicated by a
loss of directional control for which pilots decision making and/or weather conditions were to
blame, ended with a collision with terrain or obstacles, and resulted in no fire or injuries" (Wells,
1992).

Based on this profile, Trollip and Jensen (1991) have concluded there is a period between
100–500 hours in which a pilot's confidence level exceeds their ability level. They suggest two
periods that are particularly dangerous, (1) approximately 100 hours, after the pilot has
accumulated about 50 hours beyond the private pilot certificate, and (2) between 50 to 100 hours
after earning an instrument rating. Both periods are marked by an increase in confidence,
without a substantial experience gain. These data indicate that an appropriate SA training
intervention strategy would be at this stage, after basic flight skills have been acquired but an in-
depth level of expertise on which to build SA has not yet been accumulated.

1.4 Situation Awareness Challenges and Skills
SA technologies                                                                        Page 5


One way of identifying methods for improving SA is to examine in what ways SA errors occur.
A second method is to identify the ways in which pilots successfully develop and maintain SA as
compared to pilots who do a poorer job at these tasks.

A number of studies have been performed pertinent to these issues. An analysis of SA Errors in
aviation was conducted (Jones & Endsley, 1996) using reports from NASA’s Aviation Safety
Reporting System (ASRS) using an SA error taxonomy based on a model of SA (Endsley,
1995b),Table 1-1.
SA technologies                                                                                 Page 6

                                     Table 1-1: SA Error Taxonomy


                  Situation Awareness Errors                                    Overall
                  Level 1 SA Errors                                             76.3 %
                  •   Data not available –                                          11.6 %
                      System & design failures, failure of communication,
                      failure of crew to perform needed tasks
                  •   Data hard to detect —                                         11.6 %
                      Poor runway markings, poor lighting, noise in the
                      cockpit
                  •   Failure to monitor/observe data —                             37.2 %
                      Omission from scan, attentional narrowing, task related
                      distractions, other distractions, workload
                  •   Misperception of data —                                           8.7 %
                      Prior expectations
                  •   Memory loss —                                                 11.1 %
                      Disruptions in routine, high workload, distractions
                  Level 2 SA Errors                                             20.3 %
                  •   Incomplete mental model —                                     3.5 %
                      Automated systems, unfamiliar airspace
                  •   Incorrect mental model —                                      6.4 %
                      Mismatching information to expectations of model or
                      model of usual system
                  •   Over-reliance on default values —                             4.7 %
                      General expectations of system behavior
                  Level 3 SA Errors                                             3.4 %
                  •   Incomplete/Poor mental model                                  .4 %
                  •   Over-projection of current trends                             1.1 %
                  •   Other                                                         1.9 %



Gibson, Orasanu, Villeda and Nygren (1997) also performed a study of SA errors based on
ASRS reports. They found problems with workload/distraction (86%),
communications/coordination (74%), improper procedures (54%), time pressure (45%),
equipment problems (43%), weather (32%), unfamiliarity (31%), fatigue (18%), night conditions
(12%), emotion (7%) and other factors (37%). Consequences of loss of SA resulted in altitude
deviations (26%), violations of FAR (25%), heading deviations (23%), traffic conflicts (21%),
and non-adherence to published procedures (19%). Dangerous situations were found to result
from 61% of the cases. Clearly loss of SA should be taken seriously in aviation.

While both of these studies included loss of SA by controllers and commercial pilots in addition
to GA, they shed some light on the nature of the problems that occur in situations where SA is
SA technologies                                                                            Page 7


lost. Several other researchers have investigated the differences in SA between pilots who
perform well and pilots who do not.

Prince and Salas (1998) studied the situation assessment behaviors of GA pilots (mean
experience level = 720 hours), airline pilots (mean experience level = 6,036 hours), and
commercial airline check airmen (mean experience level = 12,370 hours). They found several
key differences with experience level.

   •   Increasing levels of preflight preparation - GA pilots talked about personal preparation
       before the flight, while line pilots also emphasized knowing the equipment and its limits
       and briefing for the flight. Check airman focused on planning and preparation specific to
       the flight and gathered as much information as possible about the conditions and flight
       elements (e.g. weather, ATC, airport status) in order to prepare in advance.

   •   More focus on understanding and projection – GA pilots described themselves as passive
       recipients of information with an emphasis on information in the immediate environment
       (Level 1 SA). Line pilots dealt more at the level of comprehension (Level 2 SA) and
       emphasized their active role in seeking out information. Check airmen were more likely
       to deal with Level 3 SA, seeking to be proactive. They dealt with large numbers of
       details and the complex relationships between factors in this process.

In conducting critical incident reviews with pilots they identified four major actions that are
important for team SA in commercial pilots: (1) identifying problems or potential problems, (2)
demonstrating knowledge of the actions of others, (3) keeping up with flight details, and (4)
verbalizing actions and intentions. Prince, Salas and Stout (1995) found that those aircrews who
performed better on an objective measure of SA demonstrated more actions in these areas. They
seemed to solve problems faster and recognized problem situations developing. While the focus
of this research was on team SA rather than individual SA, it does point at certain characteristics
that are important for developing SA in aviation settings.

Orasanu and Fischer (1997) studied the characteristics of commercial aircrews in making various
types of decisions through an analysis of ASRS data and observations from simulator studies.
They found that in making go/no-go decisions about an approach, the better performing crews
attended to more to cues signaling deteriorating weather and sought out weather updates
allowing them to plan for a missed approach in advance. When studying a choice type task
which involved picking an alternate airport, however, the better performing crews took longer.
They were much more attuned to the constraints imposed by a hydraulic failure and reviewed
other alternates in light of the constraints. They gathered more information allowing them to
make a better decision, whereas poorer performing crews went right to evaluating options.
SA technologies                                                                              Page 8


Analysis of a hydraulic failure, which represented a scheduling type task, showed the better
performing crews taking active steps to manage what would become a high workload task. They
planned in advance for actions that would occur in the high workload periods and thus were
more effective in these situations.

From this research, Orasanu and Fischer have focused on a two step decision model: situation
assessment and action selection. Time availability, risk level and problem definition are
indicated as critical components of the situation assessment phase. Situation ambiguity and the
availability of responses were hypothesized to be critical factors dictating the difficulty of the
decision. When cognitive demands are greater, the higher performing crews managed their
effort by performing actions that would buy them extra time (e.g. holding) and by shifting
responsibilities among the crew. Good situation assessment, contingency planning and task
management were highlighted as critical behaviors associated with success. Less effective pilots
appeared to apply the same strategies in all cases rather than matching their strategy to the
situation.

 In examining accident reports, Orasanu, Dismukes and Fischer (1993) also report that pilots who
had accidents tended to interpret cues inappropriately and often under-estimated the risk
associated with a problem and over-estimated their ability to handle dangerous situations.
Wiggens, Connan and Morris (1995) found that GA pilots who preformed poorly in deciding to
continue into inclement weather were poorly gauged in terms of matching their skill level to the
situation. “In the absence of extensive task-related experience, pilots are more inclined to rely
on their self-perceived risk-taking behavior than their self-perceived ability to resolve various
decisions.” The more experienced pilots demonstrated behaviors that were much more related to
perceptions of their own ability. It would appear that inexperienced pilots may be deficient in
their ability to properly assess risk and capabilities (Level 2 SA) from the situational cues at
hand.

In studies of individual differences in SA abilities, Endsley and Bolstad (1994) found that
military pilots with better SA were better at attention sharing, pattern matching, spatial abilities
and perceptual speed. O’Hare (1997) also found evidence that elite pilots (defined as
consistently superior in gliding competitions) performed better on a divided attention task
purported to measure SA. Gugerty and Tirre (1997) found evidence that people with better SA
performed better on measures of working memory, visual processing, temporal processing and
time-sharing ability. While some of these characteristics may not be trainable, at least attention
sharing has shown some indication that it can be improved through training (Damos & Wickens,
1980). Reducing loads on working memory may also help.
SA technologies                                                                          Page 9


In summary, from these various studies several key factors that indicate where SA can be
improved can be identified:

      •   Task management – Interruptions, task-related distractions, other distractions and
          overall workload pose a high threat to SA. Good task management strategies appear
          critical for dealing with these problems. Schutte and Trujillo (1996) found that the
          best performing crews in non-normal situations were those whose task management
          strategies were based on the perceived severity of the tasks and situations. Those
          who used an event/interrupt driven strategy (dealing with each interruption as it came
          up) and those who used a procedural based strategy performed more poorly. The
          ability to accurately assess the importance and severity of events and tasks is an
          important component of Level 2 SA. This understanding also allows pilots to
          actively manage their task and information flow so as not to end up in situations in
          which they are overloaded and miss critical information.

      •   Development of comprehension (Level 2 SA) - In addition to problems with properly
          assessing the importance or severity of tasks and events, pilots will also perform
          poorly if they are unable to properly gauge the temporal aspects of the situation, the
          risk levels involved and both personal and system capabilities for dealing with
          situations. Simmel and Shelton (1987), in analyzing accident reports, note that
          accurately determining the consequences of non-routine events appeared to be the
          problem for these pilots. Each of these factors (timing, risk, capabilities,
          consequences and severity) are major components of Level 2 SA (Endsley, et al.,
          1998). The research suggests that more inexperienced pilots are less able to make
          these important assessments, remaining more focused at Level 1.

      •   Projection (Level 3 SA) and planning – Amalberti and Deblon (1992) found that a
          significant portion of experienced pilots’ time was spent in anticipating possible
          future occurrences. This gives them the knowledge (and time) necessary to decide on
          the most favorable course of action to meet their objectives. Experienced pilots also
          appear to spend significant time in pre-flight planning and data gathering and engage
          in active contingency planning in flight. Each of these actions serves to reduce
          workload in critical events. Using projection skills (Level 3 SA) these pilots are able
          to actively seek important information in advance of a known immediate need for it
          and plan for various contingencies. Not all planning is equally effective, however.
          Taylor, Endsley and Henderson (1996) found that teams who viewed only one plan
          were particularly susceptible to Level 2 SA errors, failing to recognize cues that
          things were not going according to plan. Actively planning for various contingencies
          and not just the expected in flight is critical.
SA technologies                                                                           Page 10



       •   Information seeking and self-checking activities - Pilots with high levels of SA
           actively seek out critical information. They are quicker to notice trends and react to
           events because of this. Furthermore, it has been noted that these pilots are good at
           checking the validity of their own situation assessments, either with more information
           or others (Taylor, et al., 1996). This was found to be effective in dealing with false
           expectations and incorrect mental models. Other researchers have also suggested a
           “Devil’s Advocate” strategy where people are encouraged to challenge their
           interpretations of situations (Klein, 1995; Orasanu, 1995).

1.5 Training for SA


Relatively few programs have attempted to specifically train SA to-date. Most work has been
directed towards design and automation issues and the more fundamental research issues
regarding what to train discussed above, although there are a few exceptions. Many major
airlines, for example, have recently introduced short training courses on SA that mainly serve to
acquaint pilots with what SA is and some of the factors that can impact it. While these type of
SA awareness courses may be helpful and is certainly a good first step for airlines to undertake,
no validation work has been done as to their effectiveness. Rather than acquainting pilots with
SA as a concept, the research proposed here seeks to improve the skills and knowledge bases that
are critical for achieving good SA in flight.

Other research has sought to work with the SA of teams. Endsley and Robertson (in press)
developed a two-day training course for team SA in the aviation maintenance environment.
Based on a detailed analysis of the SA requirements of maintenance technicians and the
interactions between the various teams involved in aircraft maintenance, the course features
modules in CRM review, shared mental models, verbalizations of decisions, shift meetings and
teamwork, feedback, and factors that impact SA. Implemented at a major airline, early results
from the program show that 89% of the recipients viewed it as either useful or very useful and
83% reported a moderate or large change in behavior on the job. While increases in various
relevant safety behaviors were demonstrated, the course is still too new to determine whether
these changes will result in improvements in organizational performance goals.

In a separate effort, Prince (1998) outlines a list of training suggestions for teams of aircrew,
many of which are focused on the communications and coordination of the crew and feature
Crew Resource Management principles. While both of these programs are important for
expanding our knowledge of team SA and show promise for improving SA in their intended
settings, their focus may not be appropriate for the GA pilot. Taking the view of Salas, Prince,
Baker and Shrestha (1995) that team situation awareness is based on both individual situation
SA technologies                                                                           Page 11


awareness and team processes, these two efforts have focused on team process issues primarily.
It is the goal of the present research to focus on individual SA. Not only is this appropriate for
the GA pilot, but also, even for pilots working in teams, it is hard for the team to develop good
SA unless at least one of its members has good SA.

Some basic approaches to training individual SA were outlined by Endsley (1989) to include
higher order cognitive skills training, intensive pre-flight briefings, the use of structured
feedback, and SA-oriented training programs.

   •   Higher order cognitive skills training – Training programs devoted specifically to
       teaching higher order cognitive skills related to SA would include programs for teaching
       attention sharing, task management, contingency planning, information seeking/filtering,
       self-checking and other meta-skills identified through research to be important for SA
       would fall into this category.

   •   Intensive pre-flight briefings – As the acquisition and interpretation of information is
       highly influenced by expectations, pre-flight briefing is critical. Prince and Salas (1998)
       found that the issues focused on in the pre-flight preparation varied considerably for
       pilots of different experience levels. The idea behind intensive pre-flight briefings is to
       use multi-media tools to help pilots develop a clear picture of their flight: where the
       hazard areas are, where the weather is, what a new airport’s approach pattern looks like.
       By being able to ‘fly-through” the flight in advance, pilots can develop a better mental
       picture of the environment. This can be extremely helpful when flying into a new airport;
       for instance, being able to picture where to look for needed cues and runway
       configurations. But more importantly, such a tool could be incorporated within a
       contingency planning assistant, prompting pilots to look for potential hazards, such as
       deteriorating weather conditions, heavy air traffic or in-flight mechanical failures, and
       developing appropriate contingency plans.

   •   SA-oriented training programs – Current training programs could be greatly enhanced by
       incorporating training that specifically focuses on the development of pilot SA. Current
       training for GA focuses primarily on the basic skills of flying the aircraft. At some point
       after the psychomotor skills and basics of flight have been mastered, a training regime
       focused on SA and decision making in flight would be of the most benefit. This type of
       training would focus on developing the schema and mental models that allow
       experienced pilots to have a much better understanding of the importance, consequences,
       timing, risk levels and capabilities associated with different events and options. This type
       of training would specifically focus on creating Level 2 and 3 SA from the basic
       information available in flight.
SA technologies                                                                            Page 12



       It also would help to train pilots in the critical cues that signify prototypical classes of
       situations. More experienced pilots know where to look for cues and understand the
       significance of these cues when they occur. Kass, Herschler and Companion (1990)
       showed success with training subjects to recognize critical cues in a simulated battlefield
       environment. Other research, however, has shown that the cues attended to in early
       training can largely affect what is attended to later on (Doane, Alderton, Sohn, &
       Pellegrino, 1996). It is therefore important that a broad range of representative sets of
       situational cues be included when training subjects in this way.

   •   Structured Feedback – Feedback is critical to the learning process. In order to improve
       SA, pilots need to receive feedback on the quality of their SA. For example,
       inexperienced pilots may fail to appreciate the severity of deteriorating weather because
       unless they have an accident, or at least a bad scare, they have come through similar
       weather in the past just by luck. Unfortunately, this also reinforces poor assessments.
       Due to the probabilistic link between SA and outcome, it is difficult for pilots to develop
       a good gauge of their own SA in normal flight. Feedback on SA can be used to help train
       SA, however, through the Situation Awareness Global Assessment Technique (SAGAT),
       a measure of SA developed for design evaluation. SAGAT uses freezes in the flight
       simulation to query pilots about important aspects of their SA. The accuracy of the
       pilot’s perceptions are then compared to the real situation to provide an objective and
       direct measure of pilot SA. This technique could be adapted for a training application by
       providing feedback to the pilot on how accurate he or she was in the responses given.
       For instance: “You thought you were here when actually you are there. You have traffic
       at one O’clock, but were unaware of it. You are actually very close to stall speed.” This
       type of technique could be integrated with the higher-order cognitive skills training and
       SA-oriented training programs to help pilots fine-tune their information acquisition
       strategies and schema.

1.6 Year 1 Research Objectives


This overall objective of the research project is to develop and validate effective methods and
tools for training situation awareness skills in GA pilots. In support of this objective, the first
year of the research was devoted to investigating SA specifically in the GA pilot. Most of the
research on situation awareness in aviation to date has been conducted on more experienced
military or commercial pilots. Relatively little research has been done to identify where pilots at
lower levels of experience, skill and currency have the most critical problems with SA. In order
to create effective training programs for the GA pilot, it is first necessary to insure that the
appropriate problems and skill areas are being addressed by any proposed training technique.
SA technologies                                                                            Page 13



An investigation of situation awareness in GA pilots was conducted through three related
activities.

       (1) An analysis of accidents and incidents involving general aviation pilots (at different
       levels of experience) based on reports collected at a general aviation flight school;

       (2) A survey of flight instructors to determine observed problems with SA in GA pilots,
       including key flight tasks, environmental features and other factors associated with SA
       problems; and

       (3) Observation of GA pilots at different experience levels in critical flight scenarios to
       ascertain SA problems and behaviors associated with good and poor SA.

Results from these three activities will be presented in this report and an analysis made of the
problems and issues for SA experienced by GA pilots from early training through the first
several hundred hours of experience. Recommendations for SA training programs will be
generated from these results for exploration in Phase I of this research effort.
SA technologies                                                                          Page 14



2.0 Analysis of General Aviation Incident Reports

Incident reports from ongoing training of general aviation pilots at Embry-Riddle Aeronautical
University were analyzed to determine the areas where low SA is apparent. The incident report
database at ERAU contains documented reports completed by instructor pilots and flight students
concerning problems and incidents encountered during GA pilot training exercises. This
database encompasses both early flight training experience (prior to initial certification), and
later flight training experience as student pilots work up to instrument and multi-engine ratings.
Incident reports available from pilots at a range of experience levels were examined.


2.1 Method


Over 500 general aviation incident reports from flight training exercises at Embry-Riddle
Aeronautical University's Daytona Beach, Florida campus and Prescott, Arizona campus from
the years 1995–1999 were reviewed. The reports were reviewed by two subject matter experts
(SMEs), both experienced flight instructors, to identify those which involved identifiable
problems with situation awareness. The review resulted in 222 incident reports which had SA
related problems. The other reports were eliminated because they either did not provide enough
information to identify the causal factors or they involved problems unrelated to SA, such as
mechanical failures and bird strikes.

The 222 SA-related incident reports were analyzed to determine the SA error causal factor(s)
according to a SA error taxonomy based on Endsley's (1995) model of SA. Several of the
reports contained more than one SA error. Consequently, if the errors occurred during different
times in the flight, each error was listed individually.

In addition, the 222 SA incident reports were categorized by the training class the pilot was
enrolled in. These groups correlate roughly to experience levels. This resulted in the following
four pilot categories.

Students working on a Private Pilot’s License: The private pilot understands the basics of
aerodynamics, aircraft performance, density altitude, visual flight rules (VFR), cross country
navigational techniques, procurement of weather reports and forecasts, application of Federal
Aviation Regulations (FARs) and information contained in the Airman's Information Manual
(AIM). The private pilot is able to take passengers on flights and operate an aircraft in the
SA technologies                                                                           Page 15


National Airspace System (NAS). To earn the private pilot certificate typically requires 40 to60
flight hours.

Students working on an Instrument Pilot Qualification: The instrument pilot has an
understanding of cross-country operations at higher altitudes, instrument flight rules (IFR) of
operation, operation into high-density airports as well as IFR flight into the National Airspace
System. Pilot experience will include resource management for single pilot operations, high
performance aircraft operation and the development of interpersonal work load management and
communication skills. This group has all completed the requirements for a private pilot’s
license. Instrument pilot student experience levels are typically 175 hours and more.
Students working on a Commercial Pilot License: The commercial pilot understands the
basics of high altitude flying, mountain flying, single and multi-engine aircraft operations,
regulations that cover commercial operations and what a professional pilot must do to comply
with scheduled or non-scheduled air carrier service. This group have all completed the
requirements for a private pilot’s license and have obtained an instrument rating. Commercial
pilot student experience levels are usually 200 hours and up.

Students working on a Multi-Engine Rating: Multi-engine pilot preparation allows the single
engine pilot (Private, Instrument, and Commercial) to gain aeronautical skills and knowledge
necessary to fly a multi-engine aircraft. Multi-engine pilots have knowledge in single engine
operations and high altitude flight operations, instrument flight operations and emergencies.
This group have all completed the requirements for a private pilots license and have obtained an
instrument rating. Almost all have also obtained the commercial pilot’s license. Multi-engine
student experience levels can be as low as 100 hours with private pilots obtaining their multi-
engine qualifications, however they are typically 200 hours and up.

The 222 incidents were categorized by the SMEs into the SA taxonomy error categories by GA
pilot level. In cases where the two SMEs did not agree on the error classification, they discussed
the incident until an agreement was reached.

2.2 Results and Discussion

The 222 reports included 266 situation awareness errors by the pilot. Of the 222 reports, 179 had
1 identifiable error, 42 had 2 identifiable errors, and 1 report had 3 identifiable errors. The
causal factors associated with the 266 SA errors were identified and are presented as a percent of
the 266 total SA errors. The overall results across flight student category will be presented first,
followed by the results for each flight student category.
SA technologies                                                                            Page 16

2.2.1 SA Errors Across All Groups


As shown in Table 2-1, Level 1 SA errors accounted for 57.9% of all SA errors, 15.0% were
categorized as Level 2 SA errors and 27.1% were categorized as Level 3 SA errors. Level 1 SA
errors involved the failure of the pilot to detect the necessary information or to misperceive the
information. Each of the Level 1 errors was attributed to one of five causal factor categories,
also shown in Table 2-1. The vast majority of Level 1 errors (42.1% of all SA errors) involved a
failure to monitor or observe the relevant information that was clearly present in the situation.
For example, 23.7% of the SA errors were related to vigilance problems on the part of the pilot
not attending to some relevant aspect of the situation. This included failure to actively monitor
traffic in the traffic pattern or airspace, failure to realize restricted airspace had been entered,
failure in the instrument scan, and failure to detect instrument malfunctions. In addition, 18.4%
of the SA errors were associated with pilot failures to observe and monitor information as a
result of distractions from other relevant tasks or due to an overall high level of workload. These
situations involved missing ATC or other pilot communications and primarily concerned traffic
advisories and messages on other pilot intentions in uncontrolled airspace.

Level 1 SA errors associated with information which was difficult to perceive, detect or
discriminate accounted for 6.0% of the total SA errors. Such errors involved visual difficulties
(2.3% of the total SA errors) such as the pilot not being able to see other traffic in-flight and
during taxiing because they were in the aircraft's blind spot and failure to detect other traffic
coming out of direct sunlight. Communication problems involving flight instructors, ATC, and
other pilots accounted for 3.7% of the total SA errors. Such problems included (a) difficulties
with radio communications with ATC and other pilots, such as sending and receiving poorly
worded and unclear radio transmissions and (b) difficulties with the communication process
between the student pilot and the flight instructor, such as information not being properly
communicated and consequently misunderstood.

Additional communication problems that resulted from misperception of transmitted information
accounted for 5.6% of all SA errors. The misperceived or misunderstood information was
primarily due to ATC or other pilot communications being misunderstood due to task
distractions (4.9% of all SA errors).

               Table 2-1: Situation Awareness Error Causal Factors As A Percent
                          Of The 266 Total SA Errors For All GA Pilot Groups.


                        Situation Awareness Errors            Overall
                        Level 1 SA Errors                     57.9%
                        •   Data not available                     0.8%
SA technologies                                                                           Page 17

                       •   Data hard to detect                    6.0%
                       •   Failure to monitor/observe data        42.1%
                       •   Misperception of data                  5.6%
                       •   Memory loss                            3.4%
                       Level 2 SA Errors                       15.0%
                       •   Incomplete mental model                7.5%
                       •   Incorrect mental model                 1.9%
                       •   Over-reliance on default values        5.6%
                       Level 3 SA Errors                       27.1%
                       •   Incomplete/Poor mental model           11.7%
                       •   Over-projection of current trends      15.4%


Level 2 SA errors (15% of all SA errors) involved the inability to integrate multiple pieces of
information and determine their relevance in light of the operational goals. These errors
involved three SA error causes as presented in Table 2-1. Situations in which the pilot lacked a
mental model or had an incomplete mental model with which to process the task specific
information accounted for 7.5% of all SA errors. Many of these situations involved the pilot
becoming lost and disoriented during a solo cross country flight because they were unfamiliar
with the airspace and the geographical area. Others involved the pilot not being familiar with the
arrival and departure procedures for specific uncontrolled areas and the improper understanding
of how to perform a short field take-off and landing.

The use of an incorrect mental model accounted for 1.9% of all SA errors. These errors occurred
primarily because of expectations of the pilot based on information that was perceived by the
pilot or received from another source. For example, one pilot made a series of false position
reports in uncontrolled airspace after misidentifying landmarks, which resulted in a near mid-air
collision.

Incidents involving pilots over-relying on default values in their mental model accounted for
5.6% of all SA errors. These cases involved habitual behavior on the part of the pilot when
landing and taxiing instead of acting according to instructions. Several pilots habitually followed
their usual landing procedures without realizing they were landing on the wrong runway.

Level 3 SA errors (27.1% of all SA errors) involved incidents in which the pilots failed to predict
or forecast future situation dynamics. Level 3 SA includes the pilot's ability to perform such
things as (a) understanding the implications of current information, (b) recognizing current
trends, (c) doing contingency planning, and (d) analyzing future weather conditions. Poor
projection due to the lack of a proper mental model accounted for 11.7% of all SA errors. For
example, several cases involved hard landings due to the pilot not considering information such
SA technologies                                                                            Page 18


as the projected crosswinds, the entry point into the approach, and the necessary glide path and
sink rate based on weather conditions. They did not appear to be able to form projections that
took these factors into account. More than half of the Level 3 SA errors (15.4% of all SA errors)
involved misjudging changing trends in the airspeed and heading of other aircraft in adjoining
airspace or in the pattern.

In order to examine the impact of experience on factors associated with SA errors, the 266 SA
errors were categorized according to the experience level of the involved pilot. Figure 2-1
provides a breakdown of the 266 total SA errors and their causal factors for each student level
category. The private pilot category accounted for 53.4% (142 SA errors) of the total 266 SA
errors in the group. Instrument pilots accounted for 18.8% (50 SA errors) of the 266 total SA
errors. The commercial pilots experienced 18.4% (49 SA errors) of all the SA errors, while the
multi-engine pilots accounted for 9.4% (25 SA errors) of all the SA errors experienced.


                                    Multi-Engine
                                         9%




                           Commercial
                              18%

                                                              Private
                                                               54%



                              Instrument
                                  19%




               Figure 2-1: Situation Awareness Errors by Student Pilot Categories.


In order to comment on this distribution, it is necessary to determine whether these numbers
represent true differences in the types of problems experienced by each group or merely
differences in the base rates. (i.e. A higher rate of SA errors by those working on their private
pilot’s license may reflect the larger number of these students in the flight schools rather than an
increased likelihood of problems in this group.)
SA technologies                                                                            Page 19


The number of flight hours for each category of students at the flight school was collected for
1998. These numbers indicate that students working on their private pilot’s licenses comprise
approximately 45.3% of the total flight hours, students working towards their instrument ratings
comprise 15.1% of the flight hours, students working on their commercial licenses comprise
14.4% of the flight hours, and students working on multi-engine ratings made up the remaining
25.2%. Based on this distribution, the expected number of SA errors for each category of student
pilot were calculated (assuming that SA errors are distributed proportionately). A Chi-square
analysis showed that the observed SA errors are not distributed proportionately across the four
groups, χ2(df=3) = 25.5, p < .001, as shown in Figure 2-2.


        150                                                                              Expected
                                                                                         Observed

        100


          50


           0
                  Private        Instrument      Commercial     Multi-Engine


                        Figure 2-2: Distribution of SA Errors Across Groups


SA errors were more highly represented in the lower experience levels. Fewer SA errors than
expected were found to occur in the multi-engine student group. The tendency for SA errors to be
attributable to the different SA error categories within each group will be examined.

2.2.2 SA Errors in GA Pilots Working on a Private Pilot’s License


The least experienced group of pilots were those working towards their private pilot’s license
(usually less than 60 hours of flight experience). As shown in Table 2-2, Level 1 SA errors
accounted for 57.0% of all private pilot student SA errors, 16.2% were Level 2 SA errors and
26.8% were categorized as Level 3 SA errors. Each of the Level 1 errors was attributed to one of
four causal factor categories, also shown in Table 2-2. (No cases of missing data were noted.)
The majority of Level 1 errors (43.7% of all private pilot student SA errors) involved the pilot
failing to monitor or attend to the relevant information that was clearly present in the situation.
For example, 25.4% of the private pilot student SA errors were related to vigilance problems on
the part of the pilot in not monitoring or attending to other traffic in the airspace. This included
failing to see and avoid traffic in the traffic pattern resulting in several near mid-air collisions.
Several cases involved pilots performing a 45 degree entry into the traffic pattern not realizing
SA technologies                                                                            Page 20


another aircraft was either directly above or below converging at the same entry point. Another
case involved the pilot executing a steep turn to avoid a certain head-on collision with an
approaching aircraft. In addition, 18.3% of the private pilot student SA errors were associated
with failures to observe and monitor information as a result of distractions or due to a high level
of workload. These situations primarily involved missed ATC radio calls or missed radio calls
from other pilots in uncontrolled airspace. One case actually involved the pilot taxiing into a
light pole while being distracted during taxi-in.

                Table 2-2. Situation Awareness Error Causal Factors as a Percent
                           of the142 SA Errors For Private Pilot Students .


                         Situation Awareness Errors               Private
                                                              Pilot Students
                     Level 1 SA Errors                            57.0 %
                     •   Data not available                          0.0 %
                     •   Data hard to detect                         4.2 %
                     •   Failure to monitor/observe data           43.7 %
                     •   Misperception of data                       7.0 %
                     •   Memory loss                                 2.1%
                     Level 2 SA Errors                            16.2 %
                     •   Incomplete mental model                     7.8 %
                     •   Incorrect mental model                      3.5 %
                     •   Over-reliance on default values             4.9 %
                     Level 3 SA Errors                            26.8 %
                     •   Incomplete/Poor mental model                8.5 %
                     •   Over-projection of current trends         18.3 %


Private pilot student Level 1 SA errors associated with information that was difficult to perceive,
detect or discriminate accounted for 4.2% of the total private pilot student SA errors. These
errors involved communication problems with flight instructors, ATC, and other pilots. The
communication problems included (a) difficulties with radio communications with ATC and
other pilots, and (b) difficulties with the communication process between the private pilot student
and their flight instructor. Such cases involved information not being properly communicated
and consequently misunderstood.

Further communication problems resulted from private pilot students misperceiving or
misunderstanding clearly transmitted information (7.0% of all private pilot student SA errors).
The misperceived or misunderstood information was primarily due to ATC or other pilot
communications being misunderstood while distracted by other tasks and/or high workload.
SA technologies                                                                            Page 21


Memory loss resulted in 2.1% of all private pilot student SA errors. These cases involved the
pilot forgetting pertinent tasks, often associated with distractions or high workload. During
preflight, one pilot left the fuel cap on the ground, while another left the master switch ON after
taxi-in. In another, a solo student failed to make the required position reports while returning
into controlled airspace, resulting in a near mid-air collision.

Private pilot student Level 2 SA errors (16.2% of all private pilot student SA errors) involved the
inability to integrate multiple pieces of information and understand their relevance to the
situation. These errors involved three SA error causal factors as presented in Table 2-2.
Situations in which the pilot lacked a mental model or had an incomplete mental model with
which to process information accounted for 7.8% of all private pilot student SA errors. These
cases involved incidents in which pilots did not understand the necessary flight procedures or
were unfamiliar with the airspace procedures. For example, many of these situations involved
pilots becoming lost and disoriented during a solo cross country flight because of their
unfamiliarity with the airspace and geographical area (e.g., terrain, landmarks). They did not
understand the information they were viewing. Others involved pilots who were unfamiliar with
the arrival and departure procedures for specific uncontrolled airports and with ATC procedures.
For example, two cases involved the pilot failing to make appropriate position reports resulting
in near mid-air collisions. A pilot who was not familiar with the departure airport and area
performing a non-standard take-off procedure caused another near mid-air collision. Incidents
also included improper understanding of how to perform short field take-off and landings and the
use of improper cross wind techniques. For example, one pilot attempted a short-field take-off
without a proper understanding of the procedure, which resulted in a near stall during climb-out.

The use of an incorrect mental model (improperly matching perceived cues to an incorrect model
of picture of the situation) accounted for 3.5% of all private pilot student SA errors. For
example, one pilot made a series of false position reports after misidentifying landmarks, which
resulted in a near mid-air collision. One incident involved the pilot misidentifying landmarks
and not being aware of his position and consequently entered class B airspace without a
clearance. Another accepted a landing clearance that did not allow for safe operation. A final
example involved a pilot taking off with ice on the wings after the pilot was given misleading
information about the de-icing procedure.

The final category for the SA Level 2 private pilot student SA errors involved pilots over-relying
on default values in their mental model. These cases (4.9% of all private pilot student SA errors)
involved pilots habitually following their usual landing procedures without realizing they were
landing on the wrong runway.
SA technologies                                                                             Page 22


SA Level 3 private pilot student SA errors (26.8% of all private pilot student SA errors) involved
incidents in which the pilots failed to predict or forecast future situation dynamics. The use of an
incomplete or poor mental model accounted for 8.5% of all private pilot student SA errors. For
example, five incidents (3.5% of all private pilot student SA errors) involved hard landings with
the nose gear first due to the pilot not considering information such as the projected crosswinds,
and the necessary glide path and sink rate based on weather conditions. Their mental models
were not sufficiently developed to be able to predict the effects these factors would have on
future dynamics. Two incidents involved the pilot failing to properly analyze trends in future
weather conditions, resulting in one pilot flying into IFR conditions, while the other failed to
account for changing wind direction and velocity during landing, which resulted in a hard nose
gear first landing. The majority of Level 3 SA errors (18.3% of all private pilot student SA
errors) involved the pilot misjudging changes in airspeed and heading of other aircraft in
adjoining airspace or in the pattern. Most of these incidents involved the pilot misjudging
changes in their own and another aircraft's relative airspeed and heading in performing a 45
degree entry into the traffic pattern, resulting in near mid-air collisions due to converging aircraft
at the same place.
SA technologies                                                                             Page 23

2.2.3 SA Errors in Pilots Working on an Instrument Rating


The next most experienced category of pilots were those who possessed a private pilot’s license
and were working towards an instrument rating (typically between 60 and 175 hours of flight
experience). Table 2-3 shows Level 1 SA errors accounted for 66.0% of all instrument pilot
student SA errors, 4.0% were Level 2 SA errors and 32.7% were categorized as Level 3 SA
errors. The Level 1 SA errors were attributed to one of five causal factor categories, also shown
in Table 2-3. The majority of Level 1 errors (44.0% of all instrument pilot student SA errors)
involved the pilot failing to monitor or attend to the relevant information that was clearly present
in the situation. For example, 22.0% of the instrument pilot student SA errors were associated
with failures to observe and monitor information, often a result of distractions or due to a high
level of workload. These situations primarily involved missed ATC radio calls due to task
saturation. One incident involved the pilot continuing an ILS approach after ATC
communications were missed creating a near mid-air collision as another aircraft passed 100 feet
below the aircraft on approach. In addition, 22.0% of the instrument pilot student SA errors
were related to vigilance problems. This included a failure to see and avoid traffic in the traffic
pattern resulting in several near mid-air collisions. Another pilot entered restricted airspace after
failing to monitor and properly read the chart.

                Table 2-3: Situation Awareness Error Causal Factors as a Percent
                           of 50 SA Errors For Instrument Pilot Students.


                         Situation Awareness Errors            Instrument
                                                              Pilot Students
                     Level 1 SA Errors                            66.0 %
                     •   Data not available                          2.0 %
                     •   Data hard to detect                        10.0 %
                     •   Failure to monitor/observe data            44.0 %
                     •   Misperception of data                       4.0 %
                     •   Memory loss                                 6.0 %
                     Level 2 SA Errors                             4.0 %
                     •   Incomplete mental model                     2.0 %
                     •   Incorrect mental model                      0.0 %
                     •   Over-reliance on default values             2.0 %
                     Level 3 SA Errors                            30.0 %
                     •   Incomplete/Poor mental model               14.0 %
                     •   Over-projection of current trends          16.0 %


Instrument pilot student Level 1 SA errors associated with information that was difficult to detect
accounted for 10.0% of the total instrument pilot student SA errors. These errors involved
SA technologies                                                                            Page 24


communication problems with flight instructors and ATC. The communication problems
included (a) difficulties with the communication process between the instrument pilot student
and their flight instructor, and (b) difficulties with radio communications with ATC. Such cases
involved information not being properly communicated (transmitted) and consequently
misunderstood. Memory loss resulted in 6.0% of all instrument pilot student SA errors. These
cases involved the pilot forgetting pertinent tasks due to distractions. During preflight, one pilot
left the oil dipstick unscrewed, while another left the magnetos on after the completion of the
flight.

Only 4.0% of the incidents involving instrument pilot students were attributed to Level 2 SA
errors, which involved the inability to integrate multiple pieces of information and understand
their relevance to the situation. One incident involved failure in the instrument scan pattern and
cross-checking, resulting in the pilot becoming disoriented and descending below the minimum
descent altitude (MDA). Another incident involved the pilot automatically following his/her
usual landing procedures without realizing the plane was landing on the wrong runway.

Level 3 instrument pilot student SA errors (30.0% of all instrument pilot student SA errors)
involved incidents in which the pilots failed to predict or forecast future situation dynamics. The
use of an incomplete or poor mental model accounted for 14.0% of all instrument pilot student
SA errors. For example, several incidents involved hard landings with nose gear first or
propeller strikes. These cases involved the pilot not considering information such as the
projected crosswinds, and the necessary glide path and sink rate based on weather conditions, the
implications of which were necessary for projecting the dynamics of the aircraft. One incident
involved the pilot failing to recognize and properly correct for windshear resulting in a propeller
strike. Other incidents involved the pilot not properly analyzing the dynamics of future weather
conditions, resulting in one pilot flying into a hail storm, while another flew into a thunderstorm.
Another involved the pilot failing to realize the implications of severe weather ahead, resulting in
ice forming on the wings. The majority of Level 3 instrument pilot student SA errors (16.0% of
all instrument pilot student SA errors) involved the pilot misjudging changes in the airspeed and
heading of other aircraft in adjoining airspace or in the traffic pattern resulting in near mid-air
collision situations.

2.2.4 SA Errors in Pilots Working on a Commercial License


The next most experienced group of pilots were those who possessed an instrument rating and
were working towards a commercial pilot’s license (typically more than 200 hours of flight
experience). Level 1 SA errors accounted for 57.1% of all commercial pilot student SA errors,
10.2% were Level 2 SA errors and 32.7% were categorized as Level 3 SA errors, as shown in
Table 2-4. The Level 1 SA errors were attributed to one of five causal factor categories, with the
SA technologies                                                                              Page 25


vast majority of Level 1 errors (40.8% of all commercial pilot student SA errors) involving the
pilot failing to monitor or attend to the relevant information that was clearly present in the
situation. For example, 16.3% of the commercial pilot student SA errors were associated with
failures to observe and monitor information, often due to distractions or a high level of workload.
One incident involved a near mid-air collision because the pilot failed to look for traffic in the
traffic pattern while talking with the instructor. In addition, 20.4% of the all commercial pilot
student SA errors were related to vigilance problems, which included failure to see and avoid
traffic in the traffic pattern resulting in several near mid-air collisions. Another pilot failed to
notice the parking brake was on while performing touch and go landings.

Commercial pilot student Level 1 SA errors associated with information that was difficult to
detect accounted for 8.2% of the total commercial pilot student SA errors. These errors involved
communication problems with flight instructors and ATC. One incident involved the pilot
failing to see other traffic because it was in the aircraft's blind spot, which resulted in a near mid-
air collision. Another had to perform a steep turn to avoid an aircraft that was obscured by bright
sunlight.

                Table 2-4: . Situation Awareness Error Causal Factors as a Percent
                         of 49 SA Errors For Commercial Pilot Students.


                          Situation Awareness Errors             Commercial
                                                                Pilot Students
                     Level 1 SA Errors                              57.1 %
                     •   Data not available                            2.0 %
                     •   Data hard to detect                           8.2 %
                     •   Failure to monitor/observe data              40.8 %
                     •   Misperception of data                         2.0%
                     •   Memory loss                                   4.1%
                     Level 2 SA Errors                              10.2%
                     •   Incomplete mental model                       8.2%
                     •   Incorrect mental model                        0.0%
                     •   Over-reliance on default values               2.0%
                     Level 3 SA Errors                              32.7%
                     •   Incomplete/Poor mental model                  18.4%
                     •   Over-projection of current trends             14.3%



Level 2 SA errors, accounted for 10.2% of all commercial pilot student SA errors. These were
primarily due to incomplete mental models. One incident involved the pilot over-rotating on
take-off (due to not understanding the aircraft) resulting in a near spin. Another pilot became
SA technologies                                                                            Page 26


disoriented because of unfamiliarity with the area and failed to make contact with other aircraft,
creating traffic conflicts. A final example involved a pilot being instructed by ATC to follow a
specific aircraft in the pattern. The pilot followed the wrong aircraft (mismatching cues to the
situation model), resulting in the pilot cutting off the traffic he/she was assigned to follow.

SA Level 3 commercial pilot student errors accounted for 32.7% of all commercial pilot student
SA errors. The use of an incomplete or poor mental model accounted for 18.4% of all
commercial pilot student SA errors. A majority of these incidents involved hard landings with
nose gear first, propeller strikes, or tail strikes. These cases involved the pilot not considering
information such as the projected crosswinds, and the necessary glide path and sink rate based on
weather conditions, which were necessary for predicting the aircraft dynamics. The remaining
Level 3 commercial pilot student SA errors (14.3% of all commercial pilot student SA errors)
involved the pilot misjudging changes in the airspeed and heading of other aircraft in adjoining
airspace or in the traffic pattern resulting in near mid-air collisions. For example, one case
involved a traffic conflict because the pilot misjudged the approach rate of another aircraft,
resulting in the pilot descending too late for an approach.

2.2.5 SA Errors in Pilots Working on a Multi-Engine Rating


Finally, the most experienced of this group of pilots were those working on their multi-engine
rating. (Although it is not necessary to obtain a commercial pilot’s license before the multi-
engine rating, it is the normal path of progression at this flight school.) As shown in Table 2-5,
Level 1 SA errors accounted for 48.0% of all multi-engine pilot student SA errors, 40.0% were
Level 2 SA errors and 12.0% were categorized as Level 3 SA errors.

                Table 2-5: Situation Awareness Error Causal Factors as a Percent
                         of 25 SA Errors for Multi-Engine Pilot Students.


                          Situation Awareness Errors           Multi-Engine
                                                              Pilot Students
                      Level 1 SA Errors                           48.0 %
                      •   Data not available                         0.0 %
                      •   Data hard to detect                        4.0 %
                      •   Failure to monitor/observe data           32.0 %
                      •   Misperception of data                      8.0 %
                      •   Memory loss                                4.0 %
                      Level 2 SA Errors                           40.0 %
                      •   Incomplete mental model                   16.0 %
                      •   Incorrect mental model                     0.0 %
                      •   Over-reliance on default values           24.0 %
SA technologies                                                                               Page 27

                       Level 3 SA Errors                            12.0 %
                       •   Incomplete/Poor mental model                12.0 %
                       •   Over-projection of current trends            0.0 %


 Each of the Level 1 SA errors were attributed to one of four causal factor categories. (No
incidents were attributed to needed data not being available.) The majority of Level 1 SA errors
(32.0% of all multi-engine pilot student SA errors) involved the pilot failing to monitor or attend
to the relevant information that was clearly present in the situation. For example, 16.0% of the
multi-engine pilot student SA errors were related to the pilot not monitoring or attending to other
traffic in the airspace. This included failing to see and avoid traffic in the traffic pattern resulting
in near mid-air collision situations. One case involved the pilot failing to maintain proper
clearance between the wing tip and a building. Another pilot failed to notice an altimeter
malfunction, which resulted in a loss of electrical power. Two incidents involved pilots not
monitoring fuel level, resulting in critically low fuel levels at landing. In addition, 16.0% of the
multi-engine pilot student SA errors were associated with failures to observe and monitor
information as a result of distractions or due to a high level of workload. These situations
primarily involved missed ATC radio calls or missed radio calls from other pilots in uncontrolled
airspace.

Multi-engine pilot student Level 1 SA errors associated with information that was difficult to
detect accounted for 4.0% of the total multi-engine pilot student SA errors. One incident
involved a pilot who was unable to detect another aircraft in the pattern because it was in the
aircraft's blind spot, resulting in a near mid-air collision situation. Another incident involved a
pilot nearly colliding with another aircraft during taxi-out because the other aircraft had no taxi
lights.

Communication problems resulted from multi-engine pilot students misperceiving or
misunderstanding transmitted information (8.0% of all multi-engine pilot student SA errors).
The misperceived or misunderstood information was primarily due to ATC or other pilot
communications being misunderstood due to task distractions and/or high workload.

Multi-engine pilot student Level 2 SA errors accounted for 40.0% of all multi-engine SA errors.
Situations in which the pilot lacked a mental model or had an incomplete mental model with
which to process information accounted for 16.0% of all multi-engine pilot student SA errors.
For example, many of these situations involved the pilot becoming lost and disoriented during a
cross country flight because of their unfamiliarity with the airspace and geographical area (e.g.,
terrain, and landmarks). Another incident involved the pilot failing to understand and follow
ATC instructions, resulting in a near mid-air collision situation while maneuvering in the pattern.
SA technologies                                                                           Page 28


The remaining Level 2 SA errors (24% of all multi-engine pilot student SA errors) involved
over-reliance on defaults, primarily due to pilots automatically following their usual routine
during take-off or landing without adherence to ATC instructions. Two cases involved pilots
performing their usual landing procedures without realizing they were landing on the wrong
runway. Another incident involved a pilot continuing onto an active runway despite instructions
from ATC to hold short of the runway.

Level 3 multi-engine pilot student SA errors (12.0% of all multi-engine pilot student SA errors)
involved incidents in which the pilots failed to predict or forecast future situation dynamics. The
use of an incomplete or poor mental model accounted for all of these SA errors. For example,
one incident involved a hard landing with a propeller strike due to the pilot not considering the
projected crosswinds, and their impact on the glide path and sink rate during descent.

2.2.6 Comparison of SA Errors Across Pilot Groups


In looking at the trends in SA errors across pilot groups, Figure 2-3, Level 1 SA errors account
for the greatest number of problems across all groups, followed by Level 3 SA errors and Level 2
SA errors for most groups. Multi-engine pilot students were slightly different with more
problems in Level 2 SA than in Level 3 SA.

When the distribution of Level 1 errors across the three groups are examined, as compared to
that which would be expected based on the distribution of flight hours across these fours groups,
a significant difference is found, χ2=26.56 (df=3), p<.001. Significantly more Level 1 SA errors
were found in the lower experience level groups and fewer in the multi-engine student group.
Level 2 SA errors were proportionately distributed across the four groups, however, χ2 = 3.14
(df=2). None of the four groups were significantly more or less likely to experience Level 2 SA
errors. Level 3 SA errors were not evenly distributed, χ2(3)=17.92, p<.001. There were
significantly fewer Level 3 SA errors in the multi-engine student group and significantly more in
the other three groups. Therefore it would appear that the lower overall SA error rate observed
for the most experienced group is attributable to a decrease in Level 1 and Level 3 SA errors, but
not Level 2 SA errors.
SA technologies                                                                             Page 29


                                                                             Private
                                                                             Instrument
                                                                             Commercial
                                                                             Multi-Engine



       40

       30                                                               Multi-Engine

       20                                                       Commercial

                                                           Instrument
       10
                                                     Private
         0
             Level 1     Level 2     Level 3
               SA          SA          SA

                     Figure 2-3: SA Error Causal Factors Across Pilot Groups


A more detailed analysis of the reasons behind these trends can be found in Figure 2-4. An
examination of Figure 2-4 shows that for all groups, failure to monitor or observe data that was
available was the leading factor leading to SA errors. While this trended downward with more
experience, it remained the leading problem. Across all four groups, the most commonly cited
contributing problems were vigilance/monitoring and distractions/ high workload. A
comparatively smaller percentage of SA Level 1 errors were associated with problems in
perceiving relevant information (ranging from 4-10%) or in misperceiving inforamtion (ranging
from 2-7%). These problems tended to center around difficulties in clearly hearing the radio
communications with ATC and other pilots, or the flight instructor, and was sometimes
accompanied by other distractions or high workload. No clear trends across experience levels
was apparent, asside from a decrease in errors for the multi-engine group. SA errors due to
memory problems also accounted for a small but consistent percentage of the Level 1 SA errors
(ranging from 2-6%).

The overall percentage of Level 2 SA errors appeared to go down with experience, but became
relatively high for the multi-engine pilot students. This may reflect a gradual learning trend,
where with experience the pilots develop better mental models with which to process data. The
multi-engine pilot students had relatively more problems in this area, however. This may reflect
the lesser proportion of other causal factors (e.g. a lowing of Level 1 SA errors), or it may reflect
new problems encountered as they attempted to learn to fly multi-engine aircraft, relying on
mental models learned with single engine flight. Thus, the benefit of experience observed in
other categories did not appear to carry over to Level 2 SA for this group.
SA technologies                                                                            Page 30



Problems with Level 3 SA errors, similarly appeared to reflect problems with insufficient mental
models for understading the effect of factors encountered in flight (crosswinds, glide paths, sink
rates, weather) on flight dynamics. Projecting changing weather and wind conditions was
particularly noted in these groups. Surprisingly the private, commercial and intrument pilot
students all encountered significant difficulties in projecting the relative heading and airspeed of
other aircraft in the pattern. The multi-engine student pilots were notably higher in the
percentage of SA errors attributable to over-reliance on defaults in the mental model, although
their rate of SA Level 3 errors was down overall. Many instances of pilots in this group
reverting to their usual patterns and proceedures, even when directed differently by the relevant
procedures or ATC, were noted. It may be that this group had gained suffient experience to begin
developing some automaticity, but had not yet developed the mental models needed to
understand when to make changes or exceptions to those learned behaviors, or to manage such
changes on the fly.


  50
  45                                                    Private         Instrument
  40
                                                        Commercial      Multi-Engine
  35
  30
  25
  20
  15
  10
   5
   0




                    SA Level 1                                         SA Level 3
                                                 SA Level 2

                Figure 2-4: Detailed SA Error Causal Factors Across Pilot Groups



2.3 Conclusions


This analysis represents an examination of SA problems experienced by GA pilots at low levels
of experience. No information on SA specifically in this group of pilots had been previously
available. The information gathered here is in rough agreement with earlier analysis of SA
SA technologies                                                                           Page 31


problems across the pilot community (Jones and Endsley, 1996). A comparison of Table 1-1 to
Table 2-1 shows that in both studies, failure to monitor or observe was the leading SA error
causal factor. Relatively more problems in Level 3 SA were noted in the present study, however,
as compared to Level 2 SA errors. Projection from existing information may be more of a
problem at low levels of experience.

Overall, the following problems were noted as particularly difficult, leading to SA problems in
this group of relatively inexperienced student pilots.

•   Distractions and high workload – Many of the SA errors could be linked to problems with
    managing task distractions and task saturation. This may reflect the high workload associated
    with tasks that are not learned to high levels of automaticity, problems with multi-tasking, or
    insufficiently developed task management strategies.

•   Vigilance and monitoring deficiencies – While associated with overload in about half of the
    cases, in many incidents vigilance and monitoring deficiencies were noted without these
    accompanying problems. This may reflect insufficiently learned scan patterns, attentional
    narrowing, or an inability to priortize information.

•   Insufficiently developed mental models — Many errors in both understanding perceived
    information and projecting future dynamics could be linked to insufficiently developed
    mental models. As these memory constructs take time and experience to develop, this
    finding could be expected. In particular, these pilots had significant difficulties with new
    operations in new geographical areas, including recognizing landmarks and matching them to
    maps, and understanding new procedurs for flight, landings and departures in unfamiliar
    airspace. They also had significant difficulties with understanding the implications of many
    environmental factors on aircraft dyanmics/behaviors. Pilots at these relatively low levels of
    experience exbited problems with judging relative motion and rates of change in other traffic.
    Additional training in these areas, to supplement that gained through normal flight
    experience, would appear to be warrented to develop more robust memory structures needed
    for dealing with these factors under the dynamic, time-constrained conditions of flight.

•   Over-reliance on mental models — Reverting to habitual patterns (learned mental models)
    when new behaviors were called for was also a problem for this group. They need to
    understand the limits of learned models and how to properly extend these models to new
    situations. This may involve the need for a wider set of models (e.g. learning many
    variations on landing profiles as appropriate for different airports).
SA technologies   Page 32
SA technologies                                                                            Page 33



3.0 Survey of Flight Instructors

A survey was developed and distributed to certified flight instructors at Embry-Riddle
Aeronautical University. The survey sought their perceptions of SA and SA problems in the
general aviation environment, in terms of where they have observed their students or other pilots
having problems. The survey elicited data on the factors and consequences associated with a
loss of SA during flight training operations for the four flight student experience levels
(presented in Section 2.0). This information was sought to supplement that available from the
incident and accident analysis (Section 2.0) with information on SA problems across a wider
group of GA pilots, and not just those experiencing significant incidents. The survey sought to
gain more information from flight instructors about observed SA challenges for this group of
pilots.


3.1 Method


The survey used in this research was developed and organized to elicit information on
dimensions related to SA. To develop the survey cognitive, systems and environmental factors
associated with Level 1, Level 2, and Level 3 SA errors were generated based on previous
studies of SA error (Jones and Endsley, 1996; Gibson, Orasanu, Villeda and Nygren, 1997) and
reviewed by three subject matter experts for coherency and completeness. In addition, phases of
flight and a list of typical consequences that may result from SA errors were generated with the
assistance of three subject matter experts.

Participants were asked to rate each of the factors on the survey on a three-point scale: (1) not a
major problem for SA, (2) moderate amount of problems for SA, and (3) frequent problems for
SA. They were asked to make these ratings for each item on the survey for each of the four
student levels (1) students with flight hours up to the private pilot’s license, (2) students with
flight hours up to the instrument rating, (3) students with flight hours up to the commercial
license, and (4) students with flight hours up to the multi-engine rating. Participants were only
asked to provide ratings for the categories of students that they had experience in instructing (e.g.
they did not provide ratings for multi-engine students if they did not instruct students at that
level).

A draft survey was subsequently reviewed and evaluated by eight flight instructors. The survey
was reworked and finalized based on the feedback from the flight instructors. A copy of the
survey is included in Appendix A.
SA technologies                                                                               Page 34



An informed consent form for this research was approved by Embry-Riddle Aeronautical
University's and NASA's Human Rights Committees. The survey was distributed only to those
flight instructors who voluntarily consented to participate. All flight instructors signed an
informed consent form.

The survey was completed by 58 certified flight instructors at Embry-Riddle Aeronautical
University. The purpose of the research was described to them, and any questions they had
about the research were answered. The surveys were distributed and completed during weekly
flight instructor team meetings. The 58 flight instructors reported a mean of 1383 hours of
general aviation flight experience (mean of 617 hours in the past 12 months), and a mean of 1495
hours of other aviation flight experience (e. g. military, commercial). The instructors reported a
mean of 858 hours of experience in instructing general aviation pilots. This included a mean of
287 hours of instructing students working towards a private pilot’s license, 202 hours of
instructing students working towards an instrument rating, 146 hours of instructing students
working towards a commercial license, and 84 hours of instructing students working towards a
multi-engine rating. (Participants only provided ratings for those student groups they were
involved in instructing.)

3.2 Results and Discussion


The responses of the 58 flight instructors are provided in an annotated survey in Appendix B.
The annotated survey provides the percentages of responses for each survey item across the four
flight student experience levels. The analysis of the survey data sought to determine the impact
of flight experience level on the rating responses by the flight instructors for each survey item.
In order to take advantage of the ordinal nature of both the flight experience variable and the
three-point rating scale, the Somer's D test of association was used to analyze the survey data.
The Somer’s D is a non-parametric statistical analysis that allows one to determine if the
response to one variable (the survey item in question) depends on the other variable (experience
level), and in which direction. Details on this analysis are presented in Appendix C. Responses
to the survey and statistically significant differences (at the .05 level of significance) across pilot
student experience levels will be presented.
3.2.1 SA Problems Across Phases of Flight


As shown in Figure 3-1, student pilots working towards their private pilot’s license are reported
to have at least moderate to frequent problems with SA across all phases of flight, with ratings
for these two categories exceeding 50% in all cases. The landing and approach phases were
reported to be problematic most often, followed by the take-off, taxi-out and climb phases.
SA technologies                                                                         Page 35

                        Figure 3-1: Ratings of SA Problems for Private Pilot Students
                     100%

                       90%

                       80%

                       70%

                       60%

                       50%

                       40%

                       30%

                       20%

                       10%

                        0%
         Frequent Problems for
         SA
         Moderate amount of
         problems for SA
         Not a major problem
         area for SA                               Phases of Flight


Student pilots working towards their instrument ratings were reported to have fewer SA
problems overall, as shown in Figure 3-2. Approach and landing still were rated frequently
(>70%) as phases of flight with moderate to frequent problems for SA, however, and moderate
problems with SA were indicated across all phases of flight. Take-off, taxi-out, climb and
descent were rated as posing moderate to frequent problems for this group by more than 50% of
the flight instructors.

                    Figure 3-2: Ratings of SA Problems for Instrument Pilot Students
                     100%

                       90%

                       80%

                       70%

                       60%

                       50%

                       40%

                       30%

                       20%

                       10%

                        0%
         Frequent Problems for
         SA
         Moderate amount of
         problems for SA
         Not a major problem
         area for SA                               Phases of Flight




Student pilots working towards their commercial license were rated as having fewer problems
with SA across the phases of flight than the previous two groups. Moderate to frequent problems
SA technologies                                                                         Page 36


with SA were indicated for this group by fewer than 50% of the flight instructors, although this
is still quite high.

                   Figure 3-3: Ratings of SA Problems for Commercial Pilot Students
                     100%

                       90%

                       80%

                       70%

                       60%

                       50%

                       40%

                       30%

                       20%

                       10%

                        0%
         Frequent Problems for
         SA
         Moderate amount of
         problems for SA
         Not a major problem
         area for SA                             Phases of Flight


Multi-engine pilot students were more frequently rated as having moderate to frequent problems
with SA across most phases of flight. The approach and landing phases continued to be problem
phases, followed by climb, take-off and taxi-out. As was found in the incident analysis (Section
2.0), the problems involved in learning to handle two engines may pose problems for this group,
independent of their hours of flight experience.

                   Figure 3-4: Ratings of SA Problems for Multi-Engine Pilot Students


                     100%

                       90%

                       80%

                       70%

                       60%

                       50%

                       40%

                       30%

                       20%

                       10%

                        0%
         Frequent Problems for
         SA
         Moderate amount of
         problems for SA
         Not a major problem
         area for SA                             Phases of Flight
SA technologies                                                                            Page 37


The Somer’s D analysis showed that pilots were rated as having problems with SA significantly
(p<.01) less frequently with increasing levels of experience across all phases of flight except for
engine start and engine shut-down (Appendix C).

3.2.2 Problems with Level 1 SA: Perception of Situation Information


The frequency of flight instructor ratings of SA problems related to Level 1 SA (perception of
information) for each of the four groups are shown in Figures 3-5 through 3-8. The factors rated
included elements of SA (e.g. altitude, heading, airspeed,…) and well as behaviors believed to be
important for maintaining SA (e.g. maintaining instrument scan, completing checklists…). As
shown in Figure 3-5, at low levels of experience, the pilots were rated as having moderate to
frequent problems with SA (> 80%) across almost all the elements listed. They appeared to have
the fewest problems with cross-checking instruments and maintaining an instrument scan,
however both of these items had more than 65% of the flight instructors indicate moderate to
frequent SA problems. There was a high preponderance of ratings of frequent SA problems
across all the items. Maintaining SA under high workload and distractions appeared to be the
biggest problems.

Pilots in the group that had received their private pilot’s license but not yet their instrument
rating received slightly fewer moderate to frequent ratings, Figure 3-6, but still appeared to
demonstrate many problems with SA across all the factors listed. They did better in maintaining
an awareness of separation (problems noted by > 50%) and in communicating with the instructor
(problems noted by > 75%). They also did better in maintaining an awareness of altitude,
heading, airspeed, other traffic and weather conditions. Never-the-less, moderate to frequent
problems were noted on these factors by the majority of the flight instructors. Dealing with high
workload remained a problem for this group.

By the time the pilots had gained more flight experience and were working towards their
commercial licenses, SA problems appear to have diminished across the board, Figure 3-7. More
than 50% of the flight instructors still felt that these pilots had moderate to frequent SA problems
associated with high workload, distractions, dealing with malfunctions, completing checklists,
carrying out standard operating procedures, maintaining awareness of other traffic and
maintaining awareness of airspeed.

The multi-engine pilot group was rated as having more problems with SA than the commercial
pilot group across most factors, Figure 3-8, in agreement with the previous findings for this
group. Problems with high workload, distractions, dealing with malfunctions, completing
checklists, and maintaining awareness of other traffic topped the list of problem areas. Failure to
carry out standard operating procedures, cross-check instrument settings, misreading information
SA technologies                                                                             Page 38


and attentional narrowing were also noted as problems by more than 60% of the flight
instructors.

The Somer’s D analysis revealed that the pilots were rated as having problems with SA
significantly (p<.01) less frequently with increasing levels of experience across all of the factors
listed. (Appendix C).

In addition to these factors, detailed information about the pilots’ SA regarding weather factors
was elicited, as difficulty in projecting and dealing with weather has often been cited as a
problem for inexperienced GA pilots. The flight instructors’ ratings regarding SA problems on
weather factors are shown in Figures 3-9 through 3-12. Amongst the private pilot students,
Figure 3-9, SA regarding surface winds was particularly problematic (> 80%), followed by
problems with SA on winds aloft (> 70%). Problems with SA regarding turbulence both in cruise
and an approach were noted by many instructor pilots (> 60%), as was difficulties with SA
regarding ceilings (> 60%) in VRF, IFR and MVFR flight. In general moderate to frequent
problems with SA across all weather factors was found. Instrument pilots were rated only
slightly better, Figure 3-10. More than 50% of the flight instructors noted moderate to frequent
problems with awareness of surface winds, winds aloft, turbulence (in cruise and approach),
thunderstorms (at all stages) and icing conditions en route. They did not appear, however, to
have as much problem with ceilings as the private pilot students.

The commercial pilot students were also only slightly improved, Figure 3-11. More than 40% of
the flight instructors still noted problems with awareness of surface winds and winds aloft,
ceilings, and turbulence (in cruise and approach). More than 20% reported moderate to severe
problems across all weather conditions. The multi-engine students faired similarly, Figure 3-12.
Moderate and severe problems with awareness of thunderstorms and ceilings were noted slightly
more frequently than with the commercial pilot group.

The Somer’s D analysis revealed that the pilots were rated as having problems with SA
significantly (p<.01) less frequently with increasing levels of experience across all of the weather
factors, except for awareness of icing (enroute and ground) and stratus clouds (Appendix C).
SA technologies                                                                                                                                 Page 39


                                                           Figure 3-5: SA Level 1 Problems for Private Pilot Students




         Failure to maintain an awareness of separation

             Failure to maintain an awareness of altitude

            Failure to maintain an awareness of heading

           Failure to maintain an awareness of airspeed

         Failure to maintain an awareness of other traffic

         Failure to determine current weather conditions

      Failure to carry-out standard operating procedures

         Failure to detect information due to distractions

              Failure to cross-check instrument settings

                     Undetected equipment malfunction

                                          Memory failure

Failure to detect information due to attentional narrowing

               Misreading or misinterpreting information

       Failure to detect information due to high workload

                            Failure to complete checklist

                   Failure in the instrument scan pattern

                  Failure to detect instrument reading(s)

             Failure in ATC communication/coordination

            Failure in student/instructor communications

                                                             0%    10%     20%     30%      40%        50%           60%   70%   80%   90%   100%

            Not a major problem area for SA                                                       Rating Frequency

            Moderate amount of problems for SA

            Frequent problems for SA
SA technologies                                                                                                                            Page 40


                                                        Figure 3-6: SA Level 1 Problems for Instrument Pilot Students




            Failure to maintain an awareness of separation

                Failure to maintain an awareness of altitude

               Failure to maintain an awareness of heading

              Failure to maintain an awareness of airspeed

            Failure to maintain an awareness of other traffic

            Failure to determine current weather conditions

         Failure to carry-out standard operating procedures

            Failure to detect information due to distractions

                 Failure to cross-check instrument settings

                        Undetected equipment malfunction

                                             Memory failure

   Failure to detect information due to attentional narrowing

                  Misreading or misinterpreting information

          Failure to detect information due to high workload

                               Failure to complete checklist

                      Failure in the instrument scan pattern

                     Failure to detect instrument reading(s)

                Failure in ATC communication/coordination

               Failure in student/instructor communications

                                                                0%   10%   20%    30%     40%        50%           60%   70%   80%   90%   100%

                                                                                                Rating Frequency
             Not a major problem area for SA

             Moderate amount of problems for SA

             Frequent problems for SA
SA technologies                                                                                                                               Page 41


                                                         Figure 3-7: SA Level 1 Problems for Commercial Pilot Students




             Failure to maintain an awareness of separation

                 Failure to maintain an awareness of altitude

                Failure to maintain an awareness of heading

               Failure to maintain an awareness of airspeed

             Failure to maintain an awareness of other traffic

             Failure to determine current weather conditions

          Failure to carry-out standard operating procedures

             Failure to detect information due to distractions

                  Failure to cross-check instrument settings

                         Undetected equipment malfunction

                                              Memory failure

    Failure to detect information due to attentional narrowing

                   Misreading or misinterpreting information

           Failure to detect information due to high workload

                                Failure to complete checklist

                       Failure in the instrument scan pattern

                      Failure to detect instrument reading(s)

                 Failure in ATC communication/coordination

                Failure in student/instructor communications

                                                                 0%   10%   20%    30%     40%        50%           60%   70%   80%   90%   100%

                                                                                                 Rating Frequency

                        Not a major problem area for SA

                        Moderate amount of problems for SA

                        Frequent problems for SA
SA technologies                                                                                                                               Page 42


                                                         Figure 3-8: SA Level 1 Problems for Multi-engine Pilot Students




             Failure to maintain an awareness of separation

                 Failure to maintain an awareness of altitude
                Failure to maintain an awareness of heading

               Failure to maintain an awareness of airspeed
             Failure to maintain an awareness of other traffic

             Failure to determine current weather conditions

          Failure to carry-out standard operating procedures

             Failure to detect information due to distractions

                  Failure to cross-check instrument settings

                         Undetected equipment malfunction

                                              Memory failure

    Failure to detect information due to attentional narrowing

                   Misreading or misinterpreting information
           Failure to detect information due to high workload

                                Failure to complete checklist
                       Failure in the instrument scan pattern

                      Failure to detect instrument reading(s)
                 Failure in ATC communication/coordination

                Failure in student/instructor communications

                                                                 0%   10%   20%     30%     40%        50%           60%   70%   80%   90%   100%

                                                                                                  Rating Frequency
               Not a major problem area for SA

               Moderate amount of problems for SA

               Frequent problems for SA
SA technologies                                                                                                              Page 43



                                               Figure 3-9: SA Level 1(Weather) Problems for Private Pilot Students


               Ceilings (MVFR)

                 Ceilings (VFR)

                  Ceilings (IFR)

    Thunderstorms (Dissipating)

       Thunderstorms (Mature)

      Thunderstorms (Building)

               Cumulus Clouds

                 Stratus Clouds

     Lightning (cloud-to-ground)

      Lightning (cloud-to-cloud)

                   Precipitation

            Turbulence (cruise)

         Turbulence (approach)

       Icing conditions en-route

                 Icing (Ground)

                 Surface Winds

                    Winds Aloft

                                   0%        10%       20%       30%      40%        50%           60%   70%     80%   90%   100%

                                                                                Rating Frequency
                 Not a major problem area for SA

                 Moderate amount of problems for SA

                 Frequent problems for SA
SA technologies                                                                                                              Page 44


                                          Figure 3-10: SA Level 1 (Weather)Problems for Instrument Pilot Students




               Ceilings (MVFR)

                 Ceilings (VFR)

                  Ceilings (IFR)

    Thunderstorms (Dissipating)

       Thunderstorms (Mature)

      Thunderstorms (Building)

               Cumulus Clouds

                 Stratus Clouds

     Lightning (cloud-to-ground)

      Lightning (cloud-to-cloud)

                   Precipitation

            Turbulence (cruise)

         Turbulence (approach)

       Icing conditions en-route

                 Icing (Ground)

                 Surface Winds

                    Winds Aloft

                                   0%         10%   20%      30%       40%        50%           60%   70%   80%     90%   100%


         Not a major problem area for SA                                     Rating Frequency
         Moderate amount of problems for SA
         Frequent problems for SA
SA technologies                                                                                                                 Page 45


                                              Figure 3-11: SA Level 1(Weather) Problems for Commercial Pilot Students


                 Ceilings (MVFR)

                   Ceilings (VFR)

                    Ceilings (IFR)

      Thunderstorms (Dissipating)

         Thunderstorms (Mature)

        Thunderstorms (Building)

                 Cumulus Clouds

                   Stratus Clouds

       Lightning (cloud-to-ground)

        Lightning (cloud-to-cloud)

                     Precipitation

              Turbulence (cruise)

           Turbulence (approach)

         Icing conditions en-route

                   Icing (Ground)

                   Surface Winds

                      Winds Aloft

                                     0%           10%   20%      30%       40%        50%           60%   70%    80%    90%   100%

                                                                                 Rating Frequency
                Not a major problem area for SA

                Moderate amount of problems for SA

                Frequent problems for SA
SA technologies                                                                                                               Page 46


                                            Figure 3-12: SA Level 1(Weather) Problems for Multi-engine Pilot Students



              Ceilings (MVFR)

                Ceilings (VFR)

                 Ceilings (IFR)

   Thunderstorms (Dissipating)

      Thunderstorms (Mature)

     Thunderstorms (Building)

              Cumulus Clouds

                Stratus Clouds

    Lightning (cloud-to-ground)

     Lightning (cloud-to-cloud)

                  Precipitation

           Turbulence (cruise)

        Turbulence (approach)

      Icing conditions en-route

                Icing (Ground)

                Surface Winds

                   Winds Aloft

                                  0%         10%      20%       30%      40%         50%           60%   70%     80%    90%   100%


               Not a major problem area for SA                                  Rating Frequency

               Moderate amount of problems for SA

               Frequent problems for SA
SA technologies                                                                               Page 47

3.2.3 Problems with Level 2 SA: Comprehension of Situation Information


The frequency of flight instructor ratings of SA problems related to Level 2 SA (comprehension
of information) for each of the four groups are shown in Figures 3-13 through 3-16. The factors
rated included lack of mental models about key information (e.g. unfamiliarity with aircraft,
geographical areas, etc.) and using the wrong mental model (e.g. misinterpreting information
through confusion with a different aircraft, airport or procedure). The survey also included
general comprehension problems such as misdiagnosing equipment problems, and an inability to
form a coherent picture from the given information. It elicited information on which
comprehension requirements posed problems (e.g. impact of system failures, timing of events
and tasks, etc.).

As shown in Figure 3-13, private pilot students were rated as having moderate to frequent
problems with comprehension across all factors (problems indicated by > 70%). Some of their
biggest problems (problems indicated by > 90%) appeared to be due to an inability to understand
task priorities, and unfamiliarity with the airspace, ATC procedures, and destination airports.
With their increased level of experience, the instrument pilot students were rated as having
moderate to frequent problems with comprehension far less often across all items, Figure 3-14.
Moderate to frequent problems for SA were still noted by more than 50% of the flight instructors
on all comprehension items, however. They were most frequently rated as having moderate to
frequent SA problems related to understanding the timing and criticality of events and tasks,
understanding task priorities, understanding problems and emergencies, misdiagnosing
equipment problems, and unfamiliarity with destination airports (problems indicated by >70%).

Commercial pilot students appear to suffer less from these problems, Figure 3-15. Only
unfamiliarity with destination airports and aircraft, inability to understand one’s own capabilities
and limitations, misdiagnosis of equipment problems and an overall inability to form a coherent
picture were rated as having moderate to frequent problems for SA by more than 50% of the
flight instructors. They improved the most in terms of familiarity with the departure airport
(which would likely remain fairly constant for most student pilots). Moderate to frequent
problems with SA were indicated by between 30 and 50% of the flight instructors on the
remaining items.

The multi-engine pilot students were generally rated more poorly than the commercial pilots,
Figure 3-16. More than 60% of the flight instructors found there were moderate to frequent SA
problems related to understanding problems and emergencies, understanding the impact of
failures, misdiagnosing equipment problems, and unfamiliarity with aircraft. These factors may
be highly related to the new demands for systems knowledge associated with multi-engine
aircraft. In addition, an inability to understand timing and criticality of events and tasks, inability
SA technologies                                                                                 Page 48


to understand task priorities and general inability to form a coherent picture were noted as
problems by more than 60% of the flight instructors.

The Somer’s D analysis revealed that the pilots were rated as having problems with SA
significantly (p <.01) less frequently with increasing levels of experience across all of the
comprehension factors (Appendix C).
SA technologies                                                                                                                               Page 49


                                                             Figure 3-13: SA Level 2 Problems for Private Pilot Students




      Inability to understand aircraft capabilities/limits

         Inability to understand own capabilities/limits

      Inability to understand criticality of events/tasks

         Inability to understand timing of events/tasks

                  Inability to understand task priorities

     Inability to understand impact of failure on safety

          Inability to understand problem/emergency

                    Unfamiliarity with ATC procedures

                         Unfamiliarity with the airspace

  Unfamiliarity with uncontrolled airspace procedures

                  Unfamiliarity with destination airport

                    Unfamiliarity with departure airport

                      Unfamiliarity with flight instructor

                  Unfamiliarity with geographical area

                              Unfamiliarity with aircraft

                         Inability to form cohert picture

Confusion with different airport, aircraft or procedures

                 Misdiagnosis of equipment problems

                                                             0%    10%      20%     30%      40%        50%           60%   70%   80%   90%     100%

             Not a major problem area for SA                                                       Rating Frequency
             Moderate amount of problems for SA

             Frequent problems for SA
SA technologies                                                                                                                             Page 50



                                                          Figure 3-14: SA Level 2 Problems for Instrument Pilot Students




        Inability to understand aircraft capabilities/limits

           Inability to understand own capabilities/limits

        Inability to understand criticality of events/tasks

           Inability to understand timing of events/tasks

                    Inability to understand task priorities

       Inability to understand impact of failure on safety

            Inability to understand problem/emergency

                      Unfamiliarity with ATC procedures

                           Unfamiliarity with the airspace

    Unfamiliarity with uncontrolled airspace procedures

                    Unfamiliarity with destination airport

                      Unfamiliarity with departure airport

                        Unfamiliarity with flight instructor

                    Unfamiliarity with geographical area

                                Unfamiliarity with aircraft

                           Inability to form cohert picture

  Confusion with different airport, aircraft or procedures

                   Misdiagnosis of equipment problems

                                                               0%   10%   20%     30%      40%        50%           60%   70%   80%   90%   100%

                                                                                                 Rating Frequency
                   Not a major problem area for SA

                   Moderate amount of problems for SA

                   Frequent problems for SA
SA technologies                                                                                                                                 Page 51



                                                             Figure 3-15: SA Level 2 Problems for Commercial Pilot Students




           Inability to understand aircraft capabilities/limits

              Inability to understand own capabilities/limits
           Inability to understand criticality of events/tasks

              Inability to understand timing of events/tasks
                       Inability to understand task priorities

          Inability to understand impact of failure on safety
               Inability to understand problem/emergency

                         Unfamiliarity with ATC procedures

                              Unfamiliarity with the airspace

       Unfamiliarity with uncontrolled airspace procedures

                       Unfamiliarity with destination airport

                         Unfamiliarity with departure airport

                           Unfamiliarity with flight instructor

                       Unfamiliarity with geographical area

                                   Unfamiliarity with aircraft
                              Inability to form cohert picture

     Confusion with different airport, aircraft or procedures
                      Misdiagnosis of equipment problems

                                                                  0%   10%   20%      30%     40%        50%           60%   70%   80%   90%   100%

                                                                                                    Rating Frequency
                       Not a major problem area for SA

                       Moderate amount of problems for SA

                       Frequent problems for SA
SA technologies                                                                                                                              Page 52


                                                          Figure 3-16: SA Level 2 Problems for Multi-engine Pilot Students




        Inability to understand aircraft capabilities/limits

           Inability to understand own capabilities/limits

        Inability to understand criticality of events/tasks

           Inability to understand timing of events/tasks

                    Inability to understand task priorities

       Inability to understand impact of failure on safety

            Inability to understand problem/emergency

                      Unfamiliarity with ATC procedures

                           Unfamiliarity with the airspace

    Unfamiliarity with uncontrolled airspace procedures

                    Unfamiliarity with destination airport

                      Unfamiliarity with departure airport

                        Unfamiliarity with flight instructor

                    Unfamiliarity with geographical area

                                Unfamiliarity with aircraft

                           Inability to form cohert picture

  Confusion with different airport, aircraft or procedures

                   Misdiagnosis of equipment problems

                                                               0%   10%    20%      30%     40%        50%           60%   70%   80%   90%   100%

                                                                                                  Rating Frequency
                Not a major problem area for SA

                Moderate amount of problems for SA

                Frequent problems for SA
SA technologies                                                                          Page 53

3.2.4 Problems with Level 3SA:Projection of Situation Information


The frequency of flight instructor ratings of SA problems related to Level 3 SA (projection of
information) for each of the four groups are shown in Figures 3-17 through 3-20. The factors
rated included lack of mental models about key information (e.g. lack of understanding of
implications of information), and problems with recognizing trends. In addition analysis of
future weather conditions was rated, as well as the degree to which these pilots developed
contingency plans (which are believed to be critical for SA).

Pilots with little experience, those working towards their private pilot’s license, figure 3-17, were
found to be deficient in all four areas related to projection of information, with moderate to
frequent problems for SA indicated by more than 90% of the flight instructors. By the time these
pilots were working on their instrument ratings, the problems had decreased, Figure 3-18,
however more than 75% of the flight instructors still noted moderate to frequent problems across
all four factors. Recognizing current trends appeared to be the most problematic.

The pilots working towards their commercial pilot’s license were rated as having a much better
ability to project into the future, the highest level of SA, Figure 3-19. Moderate to frequent
problems with SA in each of the four areas were indicated by fewer than 50% of the flight
instructors in all cases. None-the-less, frequent problems in analyzing future weather conditions
were still indicated by 20% of the flight instructors for this group. Multi-engine pilot students
again experienced a slight dip, compared to commercial pilot students, Figure 3-20. They were
rated as not having a contingency plan leading to moderate to severe problems for SA by 58% of
the flight instructors, and were slightly more likely to have problems in recognizing current
trends.

The Somer’s D analysis revealed that the pilots were rated as having problems with SA
significantly (p <.001) less frequently with increasing levels of experience across all of the
projection factors (Appendix C).
SA technologies                                                                                        Page 55




                            Figure 3-17: SA Level 3 Problems for Private Pilot Students




  Failure to analyze future weather
              conditions

  Lack of a contingency plan for the
              given task


      Recognition of current trends


          Lack of understanding of
         implications of information

                                   0%        10%   20%   30%   40%    50%      60%   70%   80%   90%   100%
           Not a major problem area for SA
                                                                 Rating Frequency
           Moderate amount of problems for SA
           Frequent problems for SA




                           Figure 3-18: SA Level 3 Problems for Instrument Pilot Students



   Failure to analyze future weather
               conditions

  Lack of a contingency plan for the
              given task


       Recognition of current trends


           Lack of understanding of
          implications of information

                                       0%    10%   20%   30%   40%    50%      60%   70%   80%   90%   100%
           Not a major problem area for SA                       Rating Frequency
           Moderate amount of problems for SA
           Frequent problems for SA
SA technologies                                                                                             Page 56


                      Figure 3-19: SA Level 3 Problems for Commercial Pilot Students




Failure to analyze future weather
            conditions


   Lack of a contingency plan for
           the given task


   Recognition of current trends



       Lack of understanding of
      implications of information

                               0%       10%       20%   30%   40%    50%     60%   70%   80%   90%   100%
         Not a major problem area for SA
                                                                Rating Frequency
         Moderate amount of problems for SA
         Frequent problems for SA




                     Figure 3-20: SA Level 3 Problems for Multi-engine Pilot Students




 Failure to analyze future weather
             conditions


    Lack of a contingency plan for
            the given task



     Recognition of current trends


         Lack of understanding of
        implications of information

                                    0%      10%   20%   30%   40%    50%     60%   70%   80%   90%   100%

          Not a major problem area for SA                       Rating Frequency
          Moderate amount of problems for SA
          Frequent problems for SA
SA technologies                                                                            Page 57

3.2.5 Problems with Other Factors Related to SA


In addition to providing ratings on the factors related to Level 1, 2 and 3 SA problems,
information on other factors and behaviors that have been theorized to be related to SA problems
was collected. These factors included pre-flight planning, in-flight planning, task prioritization
and task management, aircraft handling/psychomotor skills, divided attention in and out of
cockpit, understanding ATC communications, maintaining separation, performing safe flight
procedures, seeking out flight information, personal fatigue and time constraints. Flight
instructor ratings of problems related to SA associated with each of these factors for each of the
four groups are shown in Figures 3-21 through 3-24.

Moderate to frequent problems for SA stemming from problems with understanding ATC
communications, considering future weather conditions during pre-flight, and problems with in-
flight planning and decision making were noted by more than 90% of the flight instructors for
the private pilot students, Figure 3-21. In addition, significant difficulties in task prioritization
and management, divided attention, and crosswind takeoff and landing techniques were deemed
problematic. All factors were rated as moderate to frequent in terms of SA problems by more
than 70% of the flight instructors, with the exception of maintaining aircraft separation.

The pilots working towards their instrument rating were rated as having fewer SA problems
related to these factors, Figure 3-22. The biggest problem areas for this group included
crosswind takeoff and landing techniques, task prioritization and task management, and
considering future weather conditions during pre-flight planning. Approximately 70% of the
flight instructors also indicated time constraints and personal fatigue, poor in-flight planning and
decision making, inability to understand ATC radio communications, and not seeking out flight
information were rated as moderate to frequent problems for SA with this group.

The more experienced group working on their commercial pilot’s rating received more favorable
ratings, Figure 3-23. Moderate to frequent problems for SA stemming from these factors were
noted by 30 to 50 percent of the flight instructors for this group. Poor crosswind take-off and
landing techniques, divided attention problems and time constraints topped the list of problem
areas. In addition to less frequently mentioned problems in all the areas, they appeared to have
proportionately less trouble with understanding ATC radio communications. SA problems
reported for the multi-engine pilot students displayed very similar patterns, Figure 3-24.

The Somer’s D analysis revealed that the pilots were rated as having problems with SA
significantly (p <.001) less frequently with increasing levels of experience across all of the
factors included in this section, with the exception of personal fatigue and time constraints
(Appendix C).
SA technologies                                                                                                                         Page 58


                                               Figure 3-21: Factors Associated with SA Problems for Private Pilot Students




                                        Time constraints

                                        Personal fatigue

    Inability to understand ATC radio communications

                Inability to maintain aircraft separation

             Inability to perform safe flight procedures

         Inability to seek out specific flight information

       Poor crosswind landing and takeoff techniques

            Inability to divide attention in/out of cockpit

                         Poor cockpit task prioritization

                        Poor cockpit task management

             Poor aircraft handling/psychomotor skills

           Poor in-flight planning and decision making

    Poor pre-flight planning: future weather conditions

   Poor pre-flight planning: current weather conditions

                                     Poor flight planning

  Poor pre-flight planning: NAS equipment/capabilities

                                                              0%   10%   20%   30%    40%        50%           60%   70%   80%   90%   100%

              Not a major problem area for SA
                                                                                            Rating Frequency
              Moderate amount of problems for SA

              Frequent problems for SA
SA technologies                                                                                                                         Page 59



                                             Figure 3-22: Factors Associated with SA Problems for Instrument Pilot Students




                                         Time constraints

                                         Personal fatigue

     Inability to understand ATC radio communications

                 Inability to maintain aircraft separation

              Inability to perform safe flight procedures

          Inability to seek out specific flight information

        Poor crosswind landing and takeoff techniques

             Inability to divide attention in/out of cockpit

                          Poor cockpit task prioritization

                         Poor cockpit task management

              Poor aircraft handling/psychomotor skills

            Poor in-flight planning and decision making

     Poor pre-flight planning: future weather conditions

    Poor pre-flight planning: current weather conditions

                                      Poor flight planning

   Poor pre-flight planning: NAS equipment/capabilities

                                                               0%   10%   20%   30%   40%        50%           60%   70%   80%   90%   100%

                                                                                            Rating Frequency
             Not a major problem area for SA

             Moderate amount of problems for SA

             Frequent problems for SA
SA technologies                                                                                                                       Page 60




                                            Figure 3-23: Factors Associated with SA Problems for Commercial Pilot Students




                                        Time constraints

                                        Personal fatigue

    Inability to understand ATC radio communications

                Inability to maintain aircraft separation

             Inability to perform safe flight procedures

         Inability to seek out specific flight information

       Poor crosswind landing and takeoff techniques

            Inability to divide attention in/out of cockpit

                         Poor cockpit task prioritization

                        Poor cockpit task management

             Poor aircraft handling/psychomotor skills

           Poor in-flight planning and decision making

    Poor pre-flight planning: future weather conditions

   Poor pre-flight planning: current weather conditions

                                     Poor flight planning

  Poor pre-flight planning: NAS equipment/capabilities

                                                              0%   10%   20%   30%   40%        50%           60%   70%   80%   90%   100%

                                                                                           Rating Frequency
                Not a major problem area for SA

                Moderate amount of problems for SA

                Frequent problems for SA
SA technologies                                                                                                                       Page 61


                                           Figure 3-24: Factors Associated with SA Problems for Multi-engine Pilot Students




                                        Time constraints

                                        Personal fatigue

    Inability to understand ATC radio communications

                Inability to maintain aircraft separation

             Inability to perform safe flight procedures

         Inability to seek out specific flight information

       Poor crosswind landing and takeoff techniques

            Inability to divide attention in/out of cockpit

                         Poor cockpit task prioritization

                        Poor cockpit task management

             Poor aircraft handling/psychomotor skills

           Poor in-flight planning and decision making

    Poor pre-flight planning: future weather conditions

   Poor pre-flight planning: current weather conditions

                                     Poor flight planning

  Poor pre-flight planning: NAS equipment/capabilities

                                                              0%   10%   20%   30%   40%        50%           60%   70%   80%   90%   100%

                                                                                           Rating Frequency
          Not a major problem area for SA

          Moderate amount of problems for SA

          Frequent problems for SA
SA technologies                                                                           Page 62

3.2.6 Typical Consequences of SA Problems


Finally, information was gathered on the typical consequences that are observed to result from
SA problems, when there are consequences. It is possible for many lapses in SA to result in no
negative consequences due to an intervention by ATC or the flight instructor, the pilot
recovering from the lapse in a timely manner or simply due to luck. The likelihood of various
transgressions or incidents occurring due to a loss of SA for each of the four groups was rated by
the flight instructors, as shown in Figures 3-25 through 3-28.

Overall, the flight instructors felt there was a risk of negative consequences associated with loss
of SA for these pilot groups. At least 30% of the flight instructors believed private pilot students
were more than 50% likely to experience all the negative consequences listed with a loss of SA,
Figure 3-25. The negative consequences listed as more than 50% likely to result from SA loss
by the majority of flight instructors (more than 50%) included becoming lost or disoriented,
airspace incursion, landing on the wrong runway or airport, continuing an unstabilized approach,
violation of FARs, heading or altitude deviations and near mid-air collision situations.

With experience, the flight instructors were less likely to feel there was a high probability of
negative consequence associated with a loss of SA, as shown for the instrument pilot students,
Figure 3-26. This may reflect a belief that the more experienced pilots would be better able to
detect and recover from a loss of SA. Becoming lost or disoriented, altitude or heading
deviations, violating crossing restrictions, controlled flight into terrain, non-adherence to
procedures and near mid-air collision situations were most frequently cited (> 30%) as
consequences with a greater than 50% likelihood of occurring due to a loss of SA for this group
of pilots.

Similarly the commercial pilots were not believed to be at quite as high a probability of negative
consequence, Figure 3-27. Becoming lost or disoriented, altitude or heading deviations,
violating FARs, near mid-air collision situations, critical fuel quantity situations, airspace
incursions, taxiway and runway transgressions, and landing without a clearance were most
frequent cited (> 20%) as consequences with a greater than 50% likelihood of occurring due to a
loss of SA for this group of pilots. The multi-engine pilot students were most frequently (>
30%) believed to be at a greater than 50% chance of becoming lost or disoriented, inadvertently
shutting down an engine, losing directional control, a fuel quantity critical situation, near mid-air
collision situation and violation of FARS, Figure 3-28.

The Somer’s D analysis revealed that the pilots were significantly (p <.01) less likely to be rated
as at high risk of negative consequence with increasing levels of experience for all items, with
SA technologies                                                                           Page 63


the exception of engine shut down, critical fuel quantity, fueling error, and loss of directional
control (Appendix C).
SA technologies                                                                                                                Page 64


                                              Figure 3-25: Consequences of SA Problems for Private Pilot Students




      Loss of directional control
                Inadvertent stall
                Speed deviation
  Controlled flight toward terrain
         Taxiway transgression
         Runway transgression
    Landing without a clearance
             Crossing restriction
   Non-adherence to procedure
                  Traffic conflict
              Heading deviation
                Violation of FAR
              Airspace incursion
        Propeller/tail wing strike
         Significant fueling error
   Inadvertent engine shut down
Landing on wrong airport/runway
Continued unstabilized approach
   Inadequate terrain separation
              Lost or disoriented
            Fuel quantity critical
           Near mid-air collision
               Altitude deviation

                                     0%        10%         20%   30%      40%        50%           60%   70%    80%   90%   100%
       Not likely to occur with loss of SA
                                                                                Rating Frequency
       Less than 50% likelihood to occur with loss of SA
       More than 50% likelihood to occur with loss of SA
SA technologies                                                                                                                 Page 65


                                               Figure 3-26: Consequences of SA Problems for Instrument Pilot Students




         Loss of directional control
                   Inadvertent stall
                   Speed deviation
     Controlled flight toward terrain
            Taxiway transgression
            Runway transgression
       Landing without a clearance
                Crossing restriction
      Non-adherence to procedure
                     Traffic conflict
                 Heading deviation
                   Violation of FAR
                 Airspace incursion
           Propeller/tail wing strike
            Significant fueling error
      Inadvertent engine shut down
   Landing on wrong airport/runway
   Continued unstabilized approach
      Inadequate terrain separation
                 Lost or disoriented
               Fuel quantity critical
              Near mid-air collision
                  Altitude deviation

                                        0%        10%        20%   30%      40%        50%           60%   70%   80%    90%   100%

         Not likely to occur with loss of SA
                                                                                  Rating Frequency
         Less than 50% likelihood to occur with loss of SA
         More than 50% likelihood to occur with loss of SA
SA technologies                                                                                                                     Page 66



                                                  Figure 3-27: Consequences of SA Problems for Commercial Pilot Students


           Loss of directional control
                     Inadvertent stall
                     Speed deviation
       Controlled flight toward terrain
              Taxiway transgression
              Runway transgression
         Landing without a clearance
                  Crossing restriction
        Non-adherence to procedure
                       Traffic conflict
                   Heading deviation
                     Violation of FAR
                   Airspace incursion
             Propeller/tail wing strike
              Significant fueling error
        Inadvertent engine shut down
     Landing on wrong airport/runway
     Continued unstabilized approach
        Inadequate terrain separation
                   Lost or disoriented
                 Fuel quantity critical
                Near mid-air collision
                    Altitude deviation

                                          0%          10%           20%   30%   40%        50%           60%   70%   80%   90%   100%

                                                                                      Rating Frequency

                Not likely to occur with loss of SA
                Less than 50% likelihood to occur with loss of SA
                More than 50% likelihood to occur with loss of SA
SA technologies                                                                                                                 Page 67


                                            Figure 3-28: Consequences of SA Problems for Multi-engine Pilot Students




        Loss of directional control
                  Inadvertent stall
                 Speed deviation
   Controlled flight toward terrain
           Taxiway transgression
           Runway transgression
     Landing without a clearance
              Crossing restriction
    Non-adherence to procedure
                    Traffic conflict
               Heading deviation
                 Violation of FAR
               Airspace incursion
         Propeller/tail wing strike
          Significant fueling error
    Inadvertent engine shut down
 Landing on wrong airport/runway
Continued unstabilized approach
    Inadequate terrain separation
               Lost or disoriented
             Fuel quantity critical
            Near mid-air collision
                Altitude deviation

                                       0%       10%       20%    30%       40%        50%           60%   70%   80%    90%   100%

      Not likely to occur with loss of SA
                                                                                 Rating Frequency
      Less than 50% likelihood to occur with loss of SA

      More than 50% likelihood to occur with loss of SA
SA technologies                                                                           Page 68



3.3 Conclusions


The results are overwhelming in showing that, according to flight instructors, the inexperienced
private pilot students have significantly more problems than the more experienced pilots in (a)
their ability to perceive, detect, and attend to key elements of the situation, (b) their ability to
integrate multiple pieces of information and determine their relevance in light of the operational
goals, and (c) their ability to project or forecast future situation dynamics. Even the more
experienced pilots working on commercial and multi-engine ratings were rated as having a fair
degree of problems with SA across the items in the survey, however. Flight experience appears
to be reducing, but not eliminating the SA problems in the survey. This is particularly apparent
when comparing the multi-engine student ratings to the commercial pilot ratings. Lower ratings
of SA were present on many of the items, indicating that flight experience particular to the class
of aircraft, and not just flight hours in general, is important in building up the skills and
knowledge bases upon which SA depends.

The SA problems were pervasive across all phases of flight, although approach and landing were
the most frequently noted as problematic, particularly for the less experienced pilots. Pilots who
were still working on the private pilot’s license received extremely low ratings with regard to
their SA on most areas of the survey. This is not too surprising. Even though they may be able to
solo and gain flight hours, they have not yet developed the needed skills. Pilots in the remaining
three groups (typically with 100 to 200 hours) also were also rated as having significant
difficulties with SA, even though they were all in possession of a private pilot’s license. This
corresponds to the experience level that is particularly prone to GA accidents (NTSB,1989;
Trollip and Jensen, 1991) and best targeted for SA training. Key problem areas to be addressed
for these groups include:

           (1) Task Management:
               • Managing high workload
               • Dealing with distractions
               • Task criticality and prioritization
               • Task management
               • Divided attention
               • Avoiding attentional narrowing
           (2) Basic procedures
               • Completing checklists
               • Carrying out procedures
               • Performing instrument cross-checks
               • Cross wind take-off and landing
SA technologies                                                                          Page 69


               • Understanding ATC radio communications
           (3) Vigilance
               • Maintaining awareness of traffic
               • Maintaining awareness of airspeed
           (4) Awareness and effects of weather
               • Surface winds
               • Winds aloft
               • Turbulence(cruise and approach)
               • Thunderstorms
               • Icing (en route)
               • Considering changes in weather (future predictions) in the pre-flight
           (5) Dealing with malfunctions
               • Diagnosis
               • Understanding problems and emergencies
               • Impact of failures
           (6) Building mental models
               • Understanding of own capabilities and limitations
               • Impact of time constraints & fatigue
               • Building familiarity with destination airports, airspace, and ATC procedures
               • Recognizing current trends
               • Implications of information
           (7) Critical Skills
               • Contingency planning
               • In-flight planning and decision making
               • Seeking out relevant information

One potential limitation of this study is that it relies on subjective impressions and considers
primarily a group of pilots who are under training. As such, these findings may exaggerate the
degree to which SA problems exist in the wider population of GA pilots. These findings
generally agree with those of Trollip and Jensen, (1991) who found pilot related accidents
attributable to: loss of directional control, poor judgment, airspeed not maintained, poor preflight
planning and decision making, clearance not maintained, inadvertent stalls, poor crosswind
handling, and poor in-flight planning and decision making. This provides some level of support
for their applicability to the wider GA population, and is probably representative of pilots at
these levels of experience who are at higher risk for GA accidents and incidents. These findings
also are in agreement with the review of factors that are important for SA (Section 1.0) as
generated from studies involving more experienced commercial and military pilots, and support
the proposed areas that were identified as potentially fruitful for training SA.
SA technologies                                                                           Page 70



4.0 Observation of Critical Scenarios

As a third means of examining SA problems in GA pilots, we conducted a series of simulation
trials. The objective of these trials was to acquire more direct detailed evidence regarding the
observable behaviors and skills that are associated with high and low SA in GA pilots. To
conduct this study we recruited two groups of participants: (1) low experience level GA pilots,
and (2) highly experienced GA pilots who are certified flight instructors. By examining
differences between these two groups, we sought to establish key differences in strategies and
behaviors that might be related to differences in SA abilities between these two groups.

4.1 Method
4.1.1 Participants


Pilots with levels of experience between 41 hours and 9000 hours were included in the study.

•   Novice Pilots: Ten certified private pilots participated in the flight simulation. Based on their
    certification, private pilots were expected to understand the basics of aerodynamics, aircraft
    performance, density altitude, visual flight rules (VFR), cross country navigational
    techniques, procurement of weather reports and forecasts, application of Federal Aviation
    Regulations (FARs) and information contained in the Airman's Information Manual (AIM).
    The 10 private pilots had a mean general aviation flight experience level of 109 hours (range
    41 to 300) with a mean of 37 flight hours within the last 90 days (range 0 to 60). None of this
    group of pilots had other than general aviation experience. One pilot was also instrument
    rated.

•    Experienced Pilots: Ten certified flight instructors served as the experienced pilots in the
    study. Based on their certification, the 10 certified flight instructors were expected to have an
    understanding of cross-country operations at higher altitudes, instrument flight rules (IFR) of
    operation, operation into high-density airports as well as IFR flight into the National
    Airspace System. These pilots have experience in resource management for single pilot
    operations, high performance aircraft operation and the development of interpersonal work
    load management and communication skills. The 10 flight instructors had a mean general
    aviation flight experience level of 1790 hours (range 300 to 9000), with a mean of 138 flight
    hours within the last 90 days (range 25 to 300). Only one of these pilots reported flight hours
    other than general aviation experience.
SA technologies                                                                          Page 71

4.1.2 Task


In the simulation task the pilots were asked to complete a flight scenario which was constructed
containing features known to be challenging for SA in GA pilots and which required basic
attitude instrument (BAI) flight skills. (Specifics of the flight scenario are provided in Appendix
D). The flight scenario was designed by experienced flight instructors to observe the pilots in a
variety of situations that relate to the skill of flying an aircraft in the instrument environment.
The scenarios were designed to include SA challenging factors such as deteriorating weather,
ATC communications problems, other traffic, and equipment problems (failed alternator, and
turn coordinator, oil pressure drop/engine temperature increase, fuel loss, engine ice, pitot tube
ice, failed attitude indicator, failed compass locator). The scenario included an instrument take-
off, and responses to ATC commands (turn to new heading, climb and descent), and instrument
approach.

4.1.3 Procedures


Each pilot participated in the study voluntarily. An informed consent form for this research was
approved by Embry-Riddle Aeronautical University's and NASA's Human Rights Committees.
All research participants signed the form acknowledging their informed consent.

Pilots participated in the simulation individually. The pilots were provided with general ATIS
and clearance information, as well as radio frequencies, weather information and airport
operations information in their pre-brief. The pilots were also instructed to provide verbal
protocols to allow more insight into the cognitive processes of the pilots in performing their
tasks. The scenario took approximately 1.5 hours to complete for each participant.

Subject matter experts (SMEs who are experienced flight instructors served as ATC during the
study, providing needed clearances and manipulating the events in the scenario. The SMEs
completed an observational rating form during the trial. This form (Appendix E), used in flight
training, listed key activities and behaviors that were expected during each of the tasks and
phases of flight included in the scenario. The behaviors were marked as not observed, never,
sometimes or always.

Following the completion of each simulated scenario, the SME completed a subjective rating of
the participant’s performance on a 1 (low) to 7 (high) scale. The rating sheet (Appendix F)
included ratings on a number of factors:
    • Overall SA
    • Level 1 SA
SA technologies                                                                         Page 72


   •   Level 2 SA
   •   Level 3 SA
   •   Flight tasks/skills
       • Pre-flight planning
       • Utilizing NAS equipment
       • In-flight planning and decision making
       • Aircraft handling/psychomotor skills
       • Cockpit task management
       • Cockpit task prioritization
       • Maintaining divided attention
       • Cross-wind landing and takeoff
       • Seeking out specific flight information
       • Maintaining aircraft separation
       • ATC communication/coordination
       • Detection of instrument readings
       • Checklist procedures
       • Memory
       • Detection of equipment malfunctions
       • Cross-checking instrument settings.

In addition they noted any negative consequences that occurred during the trial as a result of poor
SA. This included incidents of:
    • Altitude deviation
    • Near mid-air collision
    • Fuel quantity critical (in-flight)
    • Lost or disoriented
    • Inadequate terrain separation
    • Propeller/tail wing strike
    • Airspace incursion
    • Violation of FAR
    • Heading Deviation
    • Traffic Conflict
    • Non-adherence to published procedure
    • Controlled flight toward terrain
    • Speed deviation
    • Inadvertent stall
    • Continued unstabilized approach
    • Landing on wrong runway
    • Landing without clearance
SA technologies                                                                          Page 73


   •   Runway/Taxiway transgression
   •   Engine shutdown inadvertent
   •   Other

4.1.4 Apparatus


The scenarios were implemented in a Frasca 141 instrument training flight simulator which
allowed the collection of flight data and audio/video recording. The simulator includes all basic
cockpit equipment, including flight, engine and system controls and indicators. It is an FAA
qualified device (Level 2 or 3) that uses aerodynamic models collected from actual flight aircraft.
While the private pilots participating in the study were not instrument qualified, their private
pilot training provided them training on the instrument requirements needed for the flight
simulation.

4.2 Results


The scenario proved quite difficult for the participants. Although the flight tasks were straight
forward, the number of ATC commands was fairly taxing, controlling the simulator proved to be
a demanding task for many, and there were a number of equipment malfunctions.

4.2.1 Flight Task Performance


Of the novice pilots, six were unable to complete the simulation trial, three after crashing the
simulator and three giving up. Detailed comments on the problems observed for each pilot is
presented in Appendix G. All of the novice pilots were noted to have problems with altitude,
and speed deviations and all violated at least one FAR. Nine of the ten were also observed to
have problems with heading deviations, and seven ended up with critical fuel quantity problems.
Four of the novices continued with an unstabilized approach and four experienced an inadvertent
stall. Three pilots became lost or disoriented and two inadvertently shut down the wrong engine.
There were a total of 64 negative consequences reported for the ten novice pilots. It is likely the
number of negative outcomes would have been higher had they all completed the simulated
flight.

The experienced group did somewhat better, however their performance was far from perfect.
All ten were noted as violating a FAR, eight were noted to have altitude and heading deviations
(although not as chronically as the novice pilots), and six were noted to have speed deviations. It
is interesting that the SMEs tended to attribute problems in maintaining conformance to desired
flight parameters to “aircraft transition issues” for this group, but attributed it to problems with
SA technologies                                                                                      Page 74


basic aircraft instrument control for the novice group. Four of the experienced group got into a
critical fuel quantity situation, two continued with unstabilized approaches and two encountered
an inadvertent stall. Three pilots were found to be lost or disoriented and one inadvertently shut
down the wrong engine. There were a total of 47 negative consequences reported for this group.

4.2.2 SA Results


Table 4-1 shows the mean SA ratings for both private pilot and flight instructors. In general, the
experienced flight instructor pilots were scored as having higher SA than the novice pilots
across all three levels as expected. In examining the data closer, however, it becomes apparent
that these differences were not uniform among the two groups.

                       Table 4-1: Mean SA Ratings for Novice and Experienced Pilots


Participant                       Overall                    Level 1               Level 2          Level 3
Category                         SA Rating                  SA Rating             SA Rating        SA Rating
Novice Pilots                           2.9                    2.6                   1.9              2.2
Experienced Pilots                      5.7                    5.2                   5.1              5.0


To further examine differences in pilot SA, the pilots were categorized into four groups: (1)
Novice pilots with low SA (scores of 1, 2, or 3 on the overall SA scale), (2) Novice pilots with
moderate SA (scores of 4 or 5 ), (3) Experienced pilots with moderate SA (scores of 4 or 5), and
(4) Experienced pilots with high SA (scores of 6 or 7). The mean SA scores of these four groups
are shown in Figure 4-1. Across the different levels of SA, no real differences between SA
scores for the better scoring novice pilots and lower scoring experienced pilots can be found,
although the high scoring experienced pilots clearly outscored the other groups at all levels.

                                                  Novice - Low SA
                                                  Novice - Moderate SA
                                                  Experienced - Moderate SA
                                                  Experienced - High SA
                7.00
                6.00
                5.00
                4.00
                3.00
                2.00
                1.00
                0.00
                         Overall SA            SA Level 1            SA Level 2       SA Level 3


                                      Figure 4-1: Mean SA Scores by Pilot Group


As shown in Figure 4-2, differences in the number of flight hours possessed by the pilots in each
of the four groups were present. Surprisingly, however, not all of the most experienced members
SA technologies                                                                                      Page 75


of the two groups (novice and experienced) ended up the higher SA groupings. In the novice
category two of the more experienced GA pilots (180 and 300 hours) were rated as among the
seven having low SA. Of the three novice pilots with moderate SA, two were fairly low time
pilots (55 and 80 flight hours). In the experienced category, while all of the very high time pilots
(more than 1000 hours) were rated as having high SA, so were two pilots with only 450 hours.
Of the four flight instructors that were rated as having only moderate SA, one had as many as
730 flight hours of experience. Flight hours was therefore not significantly predictive of group,
F(3,16) = 2.48, p>.05.


               3000

               2500

               2000
                                                                         Novice - Low SA
                                                                         Novice - Moderate SA
               1500
                                                                         Experienced - Moderate SA
                                                                         Experienced - High SA
               1000

                500

                  0
                           Flight Hours      Flight Hours Last 90 Days


                       Figure 4-2: Mean Flight Experience of Pilot Groups


Number of flight hours in the past 90 days was significantly related to SA group, however,
F(3,16) = 3.434, p=.04. A Tukey test revealed that only the Novice-low SA and Experienced-
high SA groups were different however. There were no differences in number of flight hours in
the past 90 days between better and worse performing pilots in each of the two categories.

To examine whether the SA ratings were related to performance in the simulation, the number of
negative consequences per pilot was examined, as shown in Figure 4-3. There were significantly
fewer negative events for the Experienced-high SA group, as compared to the other three groups,
F(3,16) = 15.00, p<.001. It is likely that the two novice groups would have had more negative
events had all of the pilots completed the simulation. It is interesting to note that the
Experienced-moderate SA pilots did not generate fewer negative events than the two novice pilot
groups.
SA technologies                                                                                   Page 76


                       8.00
                       7.00
                       6.00
                       5.00                                           Novice - Low SA
                                                                      Novice - Moderate SA
                       4.00
                                                                      Experienced - Moderate SA
                       3.00                                           Experienced - High SA
                       2.00
                       1.00
                       0.00
                                    Negative Consequences Per Pilot


                       Figure 4-3: Performance in Simulation by Pilot Group


4.2.3 Flight Tasks/Skills


The ability of pilots in each of the four groups to perform key flight tasks and skills are presented
in Figure 4-4. There was no opportunity to observe maintaining attention inside and outside of
the cockpit, crosswind landing and take-off, or maintaining aircraft separation during the trial,
therefore these three variables were omitted from analysis. A MANOVA revealed significant
differences between the four groups across the remaining 13 tasks/skills, F(12,39) = 5.573,
p=.003. ANOVAs on each of the thirteen tasks/skills showed that all were rated differently
between the four groups at the .05 level of significance, except for pre-flight planning, as shown
in Table 4-2.

                     Table 4-2: ANOVAs of Flight Tasks/Skills by Pilot Group


                              Task/Skill                       df      F                p
                Pre-flight                                    3,16    1.773           .193
                Utilizing NAS                                 3,16     4.54           .017
                In-flight planning and decision               3,16    14.22           .001
                making
                Aircraft handling/ psychomotor                3,16    21.06           .001
                skills
                Cockpit task management                       3,16    33.41           .001
                Cockpit task prioritization                   3,16    20.86           .001
                Seeking out specific flight                   3,16    18.26           .001
                information
                ATC communication/                            3,16    38.39           .001
                coordination
                Detection of instrument                       3,16     7.36           .003
                readings
                Checklist procedures                          3,16     5.50           .009
                Memory                                        3,16     7.60           .002
SA technologies                                                                         Page 77

                Detection of equipment            3,16         9.03      .001
                malfunctions
                Cross-checking instrument         3,16        21.69      .001
                settings



As shown in Figure 4-4, utilization of NAS was rated higher for both groups of experienced
pilots, F(3,16) = 4.54, p = .017. Across all the other skills and tasks, the Experienced-high SA
group were rated significantly higher than the other groups. The Novice-moderate SA group and
Experienced-moderate SA group both were rated very similarly across most skills/tasks. The
Novice-low SA group was rated lowest.

In examining the scores for individual pilots in each group, it is interesting to note that certain
skills/tasks were rated uniformly higher for the higher SA group for both novice and experienced
pilots. Those pilots which were scored as having better SA (in both the novice and experienced
categories) all received much higher ratings for aircraft handling/psychomotor skills, cockpit task
SA technologies                                                                                                                                     Page 78


                                                                   Novice - Low SA
                                                                   Novice - Moderate SA
                                                                   Experienced - Moderate SA
                                                                   Experienced - High SA
         7.00
         6.00
         5.00
         4.00
         3.00
         2.00
         1.00
         0.00
                   Utilizing NAS      In-flight Planning and          Aircraft        Cockpit Task       Cockpit Task         Seeking out Specific
                                         Decision Making       Handling/Psychomotor   Management         Prioritization        Flight Information
                                                                       Skills




         7.00

         6.00

         5.00

         4.00

         3.00

         2.00

         1.00

         0.00
                ATC Communication/   Detection of Instrument   Checklist Procedures     Memory       Detection of Equipment     Cross-checking
                   Coordination             Readings                                                     Malfunctions         Instrument Settings




                                         Figure 4-4: Flight Task and Skill Ratings by Pilot Group
SA technologies                                                                          Page 79


management, cockpit task prioritization and ATC communication/coordination than did the
pilots who were rated as having lower SA.

To further examine this finding, the variables were submitted to a step-wise regression with
overall SA score as the dependant variable. The final model, accounting for 91.7% of the
variance in SA score across all pilots included aircraft handling/psychomotor skill and ATC
communication and coordination, F(2,17) = 94.39, P<.001. This is a very interesting finding that
is in agreement with previous studies of individual differences that have found a relationship
between psychomotor skills and SA, presumably because of issues associated with limited
attention (Endsley and Bolstad, 1994, O’Hare, 1997). The SMEs noted significant problems
with basic flight skills in the novice pilots and indicated they were quickly overloaded by the
tasks. Keeping up with ATC communications was also noted as challenging for many of the
pilots. They had to ask for many repeats and it used up their attentional resources. These issues
were less of a problem for the experienced pilots.

4.3 Discussion and Conclusions

The simulated trial contained many ATC commands for heading and altitude changes along with
numerous system failures. It was thus very high in workload, overloading the novice pilots and
seriously fatiguing the experienced pilots. This extreme situation was designed to tax the pilots
to bring out coping skills or failures, and it was successful at doing so.

It is not too surprising that the novice pilots performed worse and were attributed with lower SA
than the more experienced flight instructor pilots, overall. Interesting differences in skills were
found between high and low performers in each group, however.

Amongst the experienced pilots with high SA, good aircraft handling skills and good task
prioritization were noted frequently. Their performance was not perfect, but this group seemed
more able to detect and recover from their own errors. Many were noted as flying first and only
responding to ATC clearances or equipment malfunctions when they had the plane under control.
The experienced pilots who were rated as having only moderate SA were more likely to have
been noted as having difficulty in controlling the simulated aircraft and poorer prioritization and
planning skills.

The inexperienced group of pilots had so many problems that clear differences between those
with poor SA and those rated as having moderate SA are harder to determine. The later group
was consistently rated higher in task management and prioritization kills, aircraft handling
ability, ATC communication and coordination, and seeking out specific flight information.
SA technologies                                                                        Page 80


These differences are important to note, as several of the novice pilots in the moderate SA group
had very few flight hours, while several who had more hours were rated with very poor SA.

The possibility of a halo effect in the SME ratings cannot be discounted. That is there may have
been a tendency to rate all skills as higher for pilots who seemed to be performing better. In a
study relying on subjective ratings, this is always a general caveat. When combined with the
results of the previous two studies (Section 2 and Section3), however, this study serves to
provide more detailed insights into the problems of low time GA pilots in maintaining SA under
challenging flight conditions. In particular this study illuminates the considerable workload
problems that these pilots face. Basic flight control and ATC communications quickly
overloaded them and left little attention available for maintaining SA. Attentional narrowing and
fixation added to this problem. Amongst GA pilots with more experience, task prioritization and
task management skills were also important markers associated with the pilots rated as having
better SA. This group had the additional resources (or knowledge bases) to think ahead of the
aircraft and plan for contingencies. Fatigue and aircraft transition problems appeared to pose a
challenge even for this group, however.
SA technologies                                                                           Page 81



5.0    Recommendations


The three studies conducted under the present research will be used to guide the development of
SA training programs. The techniques to be explored include higher-order cognitive skills
training, intensive pre-mission briefings, SA-oriented training and the use of structured feedback.
In addition, specific skills training for low-time pilots is recommended. Examples of these
concepts will be expanded into deliverable training modules and tested in Year 3 of this program.
PC-based modules and flight simulator modules will be created, as appropriate.

5.1 Training support for low-time pilots

Low-time pilots were found to suffer from a host of problems that left them with poor SA. Most
notably, these pilots were overwhelmed with the basic tasks of flying the aircraft (psychomotor
skills) and communicating with ATC. More experienced pilots who have developed these tasks
to a level of automaticity have a far greater reserve of attention for maintaining SA and thinking
ahead of the aircraft. Part-task trainers for developing these skills to automaticity are highly
recommended for low-time GA pilots. These part-task trainers should also allow the students to
move forward with practice in dual tasking (performing both types of tasks together and problem
solving while performing the tasks) once each individual task has been learned to automaticity.
Additionally, these pilots could benefit from part-task trainers that allow practice in judging the
relative speed and heading of traffic and other aircraft in the pattern. This type of trainer should
be practical in nature, allowing for many variations in position and rates of change in both the
trainee’s aircraft and other aircraft. Practice in avoiding other aircraft or merging into the pattern
would make this most useful.

5.2 Higher order cognitive skills training


Training programs devoted specifically to teaching higher order cognitive skills related to SA
may include programs for teaching attention sharing, task management and prioritization,
contingency planning, information seeking/filtering, checklist completion, self-checking and
other meta-skills identified to be important for SA . These skills would probably be most useful
for GA pilots who have reached proficiency in the basic flight skills (Section 5.1) and will have
the attentional resources to employ towards the meta-cognitive skills. While these modules may
feature instruction in the benefits and skills involved in performing these basic strategies, in
order to be effective, it is most likely they will need to be employed in conjunction with intensive
pre-flight briefings and SA-oriented training that will help them develop the needed schema and
mental models. In order for pilots to be able to effectively prioritize and manage workload, they
SA technologies                                                                         Page 82


also need to develop the requisite knowledge bases to understand what the timing and priority of
tasks should be. The skills may be generic, but the ability to employ them effectively is domain
and knowledge dependent. Therefore, training that is oriented to flight tasks and conditions is
recommended.

5.3 Intensive pre-flight briefings


As the acquisition and interpretation of information is highly influenced by expectations, pre-
flight briefing is critical. Prince and Salas (1998) found that the issues focused on in the pre-
flight preparation varied considerably for pilots of different experience levels. The idea behind
intensive pre-flight briefings is to use multi-media tools to help pilots develop a clear picture of
their flight: where the hazard areas are, where the weather is, what a new airport’s approach
pattern looks like. By being able to ‘fly-through” the flight in advance, pilots can develop a
better mental picture of the environment. This can be extremely helpful when flying into a new
airport; for instance, being able to picture where to look for needed cues and runway
configurations. But more importantly, such a tool could be incorporated within a contingency
planning assistant, prompting pilots to look for potential hazards, such as deteriorating weather
conditions, heavy air traffic or in-flight mechanical failures, and developing appropriate
contingency plans.

This research indicates that pre-flight briefing tools may be particularly useful for GA pilots at
all levels of experience. Considerable problems were noted in flying into new airports,
recognizing the geography in a new area (and matching it to a map) and understanding when and
how to employ the appropriate procedures in these environments. This training approach will
help to build the schema and mental models that greatly reduce the workload in new flight
environments and should create a much higher level of SA. This training will emphasize key
decision points and should train a variety of circumstances, not just the usual. For example, a
missed approach should be shown, low minimums and alternate approach/runway/wind
conditions can be demonstrated. To be effective, intensive pre-flight training module should
incorporate different contingencies and guide students through evaluating and responding to
hazard cues.

5.4 SA-oriented training programs

At some point after the psychomotor skills and basics of flight have been mastered, a training
regime focused on SA and decision making in flight would be of the most benefit. This type of
training would focus on developing the schema and mental models that allow experienced pilots
to have a much better understanding of the importance, consequences, timing, risk levels and
capabilities associated with different events and options. This type of training would specifically
SA technologies                                                                       Page 83


focus on creating Level 2 and 3 SA from the basic information available in flight. It also would
help to train pilots in the critical cues that signify prototypical classes of situations.

SA-oriented training modules for GA should focus on:
   (1) Recognizing weather patterns and trends, including surface winds, winds aloft,
       turbulence, ceilings and icing. The implications of the weather for aircraft handling and
       key decision points should be emphasized.
   (2) Problem diagnosis, including a consideration of critical cues, options available, timing
       and constraints.
   (3) Projection of events, including weather conditions, winds and equipment malfunctions.

To be effective this type of training should help build the mental models and schema necessary
to put cues together to form the needed comprehension and projection. As such it should
probably be experientially based (i.e. involve flight scenarios and examples) and should allow
students to build an understanding of cues and events in a multi-modal manner. Experiential
training is likely to be more effective than textbook learning for this type of robust knowledge
that must be readily accessed and manipulated under flight conditions.


5.5 Structured Feedback


Feedback is critical to the learning process. In order to improve SA, pilots need to receive
feedback on the quality of their SA. For example, inexperienced pilots may fail to appreciate the
severity of deteriorating weather because unless they have an accident, or at least a bad scare,
they have come through similar weather in the past just by luck. Unfortunately, this also
reinforces poor assessments. Due to the probabilistic link between SA and outcome, it is
difficult for pilots to develop a good gauge of their own SA in normal flight. Feedback on SA
can be used to help train SA, however, through the Situation Awareness Global Assessment
Technique (SAGAT), a measure of SA developed for design evaluation. SAGAT uses freezes in
the flight simulation to query pilots about important aspects of their SA. The accuracy of the
pilot’s perceptions are then compared to the real situation to provide an objective and direct
measure of pilot SA. This technique can be adapted for a training application by providing
feedback to the pilot on how accurate he or she was in the responses given. For instance: “You
thought you were here when actually you are there. You have traffic at one O’clock, but were
unaware of it. You are actually very close to stall speed.” To be most effective structured
feedback will be employed following higher-order cognitive skills training and SA-oriented
training programs to help pilots fine-tune their information acquisition strategies and schema.
SA technologies                                                                         Page 84


5.6 Future Research


Many factors were found to impact on the GA pilot in this research. As the least trained and
generally least experienced group of pilots in aviation, they suffer from significant disadvantages
in maintaining SA under challenging flight conditions. In order to significantly reduce the
number of incidents and accidents involving GA pilots, focused structured training designed to
improve SA should be of significant benefit and should create a much safer aviation system. It is
recommended that the training techniques outlined here be developed into deliverable training
modules and tested to determine their benefits in increasing situation awareness in GA pilots.
SA technologies                                                                        Page 85



6.0 References
Amalberti, R., & Deblon, F. (1992). Cognitive modeling of fighter aircraft process control: a step
    towards an intelligent on-board assistance system. International Journal of Man-machine
    Systems, 36, 639-671.
AOPA (1998). Nall Report: General Aviation Accident Trends and Factors for 1997. Fredrick,
    MD: Aircraft Owners and Pilots Association Air Safety Foundation.
Damos, D., & Wickens, C. D. (1980). The acquisition and transfer of time-sharing skills. Acta
    Psychologica, 6, 569-577.
Doane, S. M., Alderton, D. L., Sohn, Y. W., & Pellegrino, J. W. (1996). Acquisition and transfer
    of skilled performance: Are visual discrimination skills stimulus specific? , 22(5), 1218-
    1248.
Dominguez, C. (1994). Can SA be defined? In M. Vidulich, C. Dominguez, E. Vogel, & G.
    McMillan (Eds.), Situation awareness: Papers and annotated bibliography (AL/CF-TR-
    1994-0085) (pp. 5-15). Wright-Patterson AFB, OH: Armstrong Laboratory.
Endsley, M. R. (1988). Design and evaluation for situation awareness enhancement. In
    Proceedings of the Human Factors Society 32nd Annual Meeting (pp. 97-101). Santa
    Monica, CA: Human Factors Society.
Endsley, M. R. (1989). Pilot situation awareness: The challenge for the training community. In
    Proceedings of the Interservice/Industry Training Systems Conference (I/ITSC) (pp. 111-
    117). Ft Worth, TX: American Defense Preparedness Association.
Endsley, M. R. (1995a). A taxonomy of situation awareness errors. In R. Fuller, N. Johnston, &
    N. McDonald (Eds.), Human Factors in Aviation Operations (pp. 287-292). Aldershot,
    England: Avebury Aviation, Ashgate Publishing Ltd.
Endsley, M. R. (1995b). Toward a theory of situation awareness. Human Factors, 37(1), 32-64.
Endsley, M. R., & Bolstad, C. A. (1994). Individual differences in pilot situation awareness.
    International Journal of Aviation Psychology, 4(3), 241-264.
Endsley, M. R., Farley, T. C., Jones, W. M., Midkiff, A. H., & Hansman, R. J. (1998). Situation
    awareness information requirements for commercial airline pilots (ICAT-98-1). Cambridge,
    MA: Massachusetts Institute of Technology International Center for Air Transportation.
Endsley, M. R., & Robertson, M. M. (in press). Situation awareness in aircraft maintenance
    teams. International Journal of Industrial Ergonomics.
Fracker, M. L. (1988). A theory of situation assessment: Implications for measuring situation
    awareness. In Proceedings of the Human Factors Society 32nd Annual Meeting (pp. 102-
    106). Santa Monica, Ca: Human Factors Society.
Gibson, J., Orasanu, J., Villeda, E., & Nygren, T. E. (1997). Loss of situation awareness: Causes
    and consequences. In R. S. Jensen & R. L. A. (Ed.), Proceedings of the Eighth International
SA technologies                                                                        Page 86


    Symposium on Aviation Psychology (pp. 1417-1421). Columbus, OH: The Ohio State
    University.
Gugerty, L., & Tirre, W. (1997). Situation awareness: a validation study and investigation of
    individual differences. In Proceedings of the Human Factors and Ergonomics Society 40th
    Annual Meeting (pp. 564-568). Santa Monica, CA:
Hartel, C. E., Smith, K., & Prince, C. (1991, April). Defining aircrew coordination: Searching
    mishaps for meaning. Paper presented at the Sixth International Symposium on Aviation
    Psychology, Columbus, OH.
Huettner, C. H., & Lewis, M. S. (1997). Aviation safety program: report to industry.
Hunter, D. R. (1995). The airman research questionnaire: characteristics of the American pilot
    population. In R. S. Jensen & L. A. Rakovan (Ed.), Proceedings of the Eighth International
    Symposium on Aviation Psychology (pp. 795-800). Columbus, OH: Ohio State University.
Jones, D. G., & Endsley, M. R. (1996). Sources of situation awareness errors in aviation.
    Aviation, Space and Environmental Medicine, 67(6), 507-512.
Kass, S. J., Herschler, D. A., & Companion, M. A. (1990). Are they shooting at me?: An
    approach to training situation awareness. In Proceedings of the Human Factors Society 34th
    Annual Meeting (pp. 1352-1356). Santa Monica, CA: Human Factors Society.
Klein, G. A. (1995). A user's guide to naturalistic decision making (DASWO-1-94-M-9906).
    Fairborn, OH: Klein Associates.
NTSB (1989). General Aviation Accidents Involving Visual Flight Rules Flight Into Instrument
    Meteorological Conditions (NTSB/SR–89/01). Washington, DC: Author.
NTSB (1998). 1997 U.S. Airline Fatalities Down Substantially From Previous Year; General
    Aviation Deaths Rise. NTSB Press Release 2/24/98 (SB 98-12). Washington, DC: Author.
NTSB (1999). Preliminary Findings: NYC99MA178 Accident at Vineyard Haven, MA.
    Washington, DC: Author.
O'Hare, D. (1997). Cognitive ability determinants of elite pilot performance. Human Factors,
    39(4), 540-552.
Orasanu, J. (1995). Situation awareness: It's role in flight crew decision making. In R. S. Jensen
    & L. A. Rakovan (Ed.), Proceedings of the Eighth International Symposium on Aviation
    Psychology (pp. 734-739). Columbus, OH:
Orasanu, J., Dismukes, R. K., & Fischer, U. (1993). Decision errors in the cockpit. In
    Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting (pp. 363-
    367). Santa Monica, CA: Human Factors and Ergonomics Society.
Orasanu, J., & Fischer, U. (1997). Finding decisions in natural environments: the view from the
    cockpit. In C. E. Zsambok & G. Klein (Eds.), Naturalistic Decision Making (pp. 343-357).
    Mahwah, NJ: Lawrence Erlbaum.
Prince, C. (1998). Guidelines for situation awareness training. Orlando, FL: Naval Air Warfare
    Center Training Systems Division.
SA technologies                                                                      Page 87


Prince, C., & Salas, E. (1998). Situation assessment for routine flight and decision making.
     International Journal of Cognitive Ergonomics, 1(4), 315-324.
Prince, C., Salas, E., & Stout, R. (1995). Situation awareness: team measures, training methods.
     In D. J. Garland & M. R. Endsley (Eds.), Experimental Analysis and Measurement of
     Situation Awareness. Daytona Beach, FL: Embry-Riddle University Press.
Salas, E., Prince, C., Baker, D. P., & Shrestha, L. (1995). Situation awareness in team
     performance: Implications for measurement and training. Human Factors, 37(1), 123-136.
Schutte, P. C., & Trujillo, A. C. (1996). Flight crew task management in non-normal situations.
     In Proceedings of the Human Factors and Ergonomics Society 40th Annual Meeting (pp.
     244-248). Santa Monica, CA:
Simmel, E. C., & Shelton, R. (1987). A assessment of nonroutine situations by pilots: A two-part
     process. Aviation, Space and Environmental Medicine, 58(November), 1119-21.
Taylor, R. M., Endsley, M. R., & Henderson, S. (1996). Situational awareness workshop report.
     In B. J. Hayward & A. R. Lowe (Eds.), Applied aviation psychology: Achievement, change
     and challenge (pp. 447-454). Aldershot, UK: Ashgate Publishing Ltd.
Trollip, S. R., & Jensen, R. S. (1991). Human Factors for General Aviation. Englewood, CO:
     Jeppesen Sanderson.
Wells, A. T. (1992). Flight Safety: A Primer for General Aviation Pilots. Blue Ridge Summit,
     PA: TAB Books.
Wiggens, M., Connan, N., & Morris, C. (1995). Self-perceptions of weather-related decision-
     making ability amongst pilots. In R. S. Jensen & L. A. Rakovan (Ed.), Proceedings of the
     Eighth International Symposium on Aviation Psychology (pp. 845-850). Columbus, OH:
     Ohio State University.
SA technologies                                                                       Page 88



                                        Appendix A

                                Flight Instructor Survey




                          Flight Instructor Survey
         on Situation Awareness in General Aviation
With the support of NASA, SA Technologies, Inc. and Embry-Riddle Aeronautical University
are obtaining information to develop and validate methods for training to improve situation
awareness in general aviation pilots. Your participation is deeply appreciated. Participation in
this survey is completely voluntary. It is not necessary to give your name at any point. You may
decline to answer any of the questions in this survey. All surveys will be de-identified and all
information obtained from any individual survey will be kept confidential by SA Technologies,
Inc., Embry-Riddle Aeronautical University, and NASA.




                                Defining Situation Awareness

Maintaining a high level of situation awareness is one of the most critical and challenging
features of a pilot’s job. Situation awareness (SA) can be thought of as an internalized mental
model of the current state of the flight environment. This integrated picture forms the central
organizing feature from which all decision making and action takes place. A vast portion of the
pilot’s job is involved in developing SA and keeping it up to date in a rapidly changing
environment. Situation awareness can be described as a person’s state of knowledge or mental
model of the situation around them. Many definitions of SA have been developed. A general,
applicable definition describes SA as “the perception of the elements in the environment within a
volume of time and space, the comprehension of their meaning and the projection of their status
in the near future”. Thus, it includes more than perceiving or attending to information, but also
the integration of multiple pieces of information and a determination of their relevance to the
person’s goals, and the ability to forecast future situation dynamics, thus providing for timely
and effective decision making.
SA technologies                                                                             Page 89


                        Flight and Flight Instruction Experience

Your total hours of general aviation flight experience:
Your total hours of general aviation flight experience within the last 12 months
Your total hours of other flight experience (e.g., military, commercial):

Your total hours of general aviation flight instruction experience:
Your total hours of general aviation flight instruction experience within the last 12 months

Your total hours of other flight instruction experience (e.g., military, commercial):

Please indicate the number of flight instruction hours you have had in the following flight
student experience levels:

       Hours up to Private                    hours
       Hours up to Instrument                 hours
       Hours up to Commercial                 hours
       Hours up to Multi-engine               hours

I. Loss of situation awareness may occur during different aspects of any flight. With regards to
   this please rate each of the following categories as you perceive them as a problem for the
   pilot in maintaining situational awareness. For each item, please circle one rating in each
   of the four flight student experience levels (i.e., hours up to Private, hours up to
   Instrument, hours up to Commercial, hours up to Multi-engine).

   Rating Scale
   1 Not a major problem area for SA
   2 Moderate amount of problems for SA
   3 Frequent problems for SA
                                                                    Flight Student Experience Levels
                                                        Private       Instrument   Commercial         Multi-engine

               Engine Start                             1   2   3      1   2   3    1   2   3          1   2   3
               Preflight                                1   2   3      1   2   3    1   2   3          1   2   3
               Taxi–out                                 1   2   3      1   2   3    1   2   3          1   2   3
               Takeoff                                  1   2   3      1   2   3    1   2   3          1   2   3

               Climb                                    1 2 3          1 2 3        1 2 3              1 2 3
               Cruise                                   1 2 3          1 2 3        1 2 3              1 2 3
SA technologies                                  Page 90


           Descent            1 2 3   1 2 3   1 2 3        1 2 3
           Approach           1 2 3   1 2 3   1 2 3        1 2 3

           Landing            1 2 3   1 2 3   1 2 3        1 2 3
           Taxi–in            1 2 3   1 2 3   1 2 3        1 2 3
           Engine Shut Down   1 2 3   1 2 3   1 2 3        1 2 3
SA technologies                                                                           Page 91


II. The following factors may be related to losing situation awareness. Please rate the following
    with regard to the degree to which you comprehend each is a problem for student pilots in
    the four flight experience levels (i.e., hours up to Private, hours up to Instrument, hours up to
    Commercial, hours up to Multi-engine). For each item, please circle one rating in each of
    the four flight student experience level.

   Rating Scale
   1 Not a major problem area for SA
   2 Moderate amount of problems for SA
   3 Frequent problems for SA
                                                                   Flight Student Experience Levels
                                                            Private    Instrument   Commercial    Multi-engine
A. Failure to correctly detect the critical
   information in the situation due to:

   Failure in student/instructor communications            1   2   3    1   2   3     1   2   3   1     2   3
   Failure in ATC communication/coordination               1   2   3    1   2   3     1   2   3   1     2   3
   Failure to detect instrument reading(s)                 1   2   3    1   2   3     1   2   3   1     2   3
   Failure in the instrument scan pattern                  1   2   3    1   2   3     1   2   3   1     2   3

   Failure to complete checklist                           1   2   3    1   2   3     1   2   3   1     2   3
   Failure to detect information due to high workload      1   2   3    1   2   3     1   2   3   1     2   3
   Misreading or misinterpreting information               1   2   3    1   2   3     1   2   3   1     2   3
   Failure to detect information due to attentional        1   2   3    1   2   3     1   2   3   1     2   3
   narrowing (e.g., fixation on an instrument)

   Memory failure (e.g., forgetting radio frequency)       1   2   3    1   2   3     1   2   3   1     2   3
   Undetected equipment malfunction                        1   2   3    1   2   3     1   2   3   1     2   3
   Failure to cross-check instrument settings              1   2   3    1   2   3     1   2   3   1     2   3
   Failure to detect information due to distractions       1   2   3    1   2   3     1   2   3   1     2   3
    (e.g., distracted by communications
    from other aircraft)

   Failure to carry-out standard operating procedures      1   2   3    1   2   3     1   2   3   1     2   3
   Failure to determine current weather conditions         1   2   3    1   2   3     1   2   3   1     2   3
   Failure to maintain an awareness of other traffic       1   2   3    1   2   3     1   2   3   1     2   3
   Failure to maintain an awareness of airspeed            1   2   3    1   2   3     1   2   3   1     2   3

   Failure to maintain an awareness of heading             1 2 3        1 2 3         1 2 3       1 2 3
   Failure to maintain an awareness of altitude            1 2 3        1 2 3         1 2 3       1 2 3
SA technologies                                                                   Page 92


  Failure to maintain an awareness of separation      1 2 3       1 2 3       1 2 3       1 2 3
  Failure to maintain an awareness of the following
  weather factors:
          Winds Aloft                                 1   2   3   1   2   3   1   2   3   1   2   3
          Surface Winds                               1   2   3   1   2   3   1   2   3   1   2   3
          Icing (Ground)                              1   2   3   1   2   3   1   2   3   1   2   3
          Icing conditions en-route                   1   2   3   1   2   3   1   2   3   1   2   3
SA technologies                                                                           Page 93

   Rating Scale
   1 Not a major problem area for SA
   2 Moderate amount of problems for SA
   3 Frequent problems for SA
                                                                        Flight Student Experience Levels
                                                          Private         Instrument    Commercial     Multi-
engine
A. Failure to correctly detect the critical
   information in the situation due to:

   Failure to maintain an awareness of the following
   weather factors:
           Turbulence (approach to landing)               1   2     3       1   2   3      1   2   3      1     2   3
           Turbulence (cruise)                            1   2     3       1   2   3      1   2   3      1     2   3
           Precipitation (penetration of rain)            1   2     3       1   2   3      1   2   3      1     2   3
           Lightning (cloud-to-cloud)                     1   2     3       1   2   3      1   2   3      1     2   3

           Lightning (cloud-to-ground)                    1   2     3       1   2   3      1   2   3      1     2   3
           Stratus Clouds                                 1   2     3       1   2   3      1   2   3      1     2   3
           Cumulus Clouds                                 1   2     3       1   2   3      1   2   3      1     2   3
           Thunderstorms (Building Stage)                 1   2     3       1   2   3      1   2   3      1     2   3

           Thunderstorms (Mature Stage)                   1   2     3       1   2   3      1   2   3      1     2   3
           Thunderstorms (Dissipation Stage)              1   2     3       1   2   3      1   2   3      1     2   3
           Ceilings (IFR)                                 1   2     3       1   2   3      1   2   3      1     2   3
           Ceilings (VFR)                                 1   2     3       1   2   3      1   2   3      1     2   3
           Ceilings (MVFR)                                1   2     3       1   2   3      1   2   3      1     2   3

B. Failure to comprehend the situation due to:

   Misdiagnosis of equipment problems                     1 2 3             1 2 3          1 2 3          1 2 3
   Misinterpretation of information due to confusion
      with a different aircraft, airport, or
      operational procedure                               1 2 3             1 2 3          1 2 3          1 2 3
   Inability to relate bits of information with each
      other in order to form a coherent picture.          1 2 3             1 2 3          1 2 3          1 2 3

   Unfamiliarity with aircraft                            1 2 3             1 2 3          1 2 3          1 2 3
   Unfamiliarity with geographical area (e.g., terrain)   1 2 3             1 2 3          1 2 3          1 2 3
   Unfamiliarity with flight instructor                   1 2 3             1 2 3          1 2 3          1 2 3
SA technologies                                                                 Page 94


  Unfamiliarity with departure airport                  1 2 3       1 2 3       1 2 3       1 2 3

  Unfamiliarity with destination airport                1   2   3   1   2   3   1   2   3   1   2   3
  Unfamiliarity with uncontrolled airspace procedures   1   2   3   1   2   3   1   2   3   1   2   3
  Unfamiliarity with the airspace                       1   2   3   1   2   3   1   2   3   1   2   3
  Unfamiliarity with air traffic control procedures     1   2   3   1   2   3   1   2   3   1   2   3
SA technologies                                                                              Page 95

   Rating Scale
   1 Not a major problem area for SA
   2 Moderate amount of problems for SA
   3 Frequent problems for SA
                                                                      Flight Student Experience Levels
                                                            Private     Instrument         Commercial    Multi-engine
   Inability to development of an understanding of:
      Problem/emergency situation                       1     2   3          1    2    3     1   2   3     1   2   3
      Impact of system failure on flight safety         1     2   3          1    2    3     1   2   3     1   2   3
      Task priorities                                   1     2   3          1    2    3     1   2   3     1   2   3
      Timing of events/tasks                            1     2   3          1    2    3     1   2   3     1   2   3
      Criticality of events/tasks                       1     2   3          1    2    3     1   2   3     1   2   3
      Own capabilities/limitations                      1     2   3          1    2    3     1   2   3     1   2   3
      Aircraft capabilities/limitations                 1     2   3          1    2    3     1   2   3     1   2   3

C. Failure to predict future situations due to:

   Lack of understanding the implications of the
      current information                               1    2    3      1    2    3        1    2   3     1   2   3
   Recognition of current trends                        1    2    3      1    2    3        1    2   3     1   2   3
   Lack of a contingency plan for the given task        1    2    3      1    2    3        1    2   3     1   2   3
   Failure to analyze future weather conditions         1    2    3      1    2    3        1    2   3     1   2   3

D. Failure to effectively perform the necessary
   flight tasks due to:

   Poor Pre-Flight planning:
    Inability to utilize NAS equipment and
      their capabilities                                1    2    3      1    2    3        1    2   3     1   2   3
    Poor flight planning                                1    2    3      1    2    3        1    2   3     1   2   3
    Inability to analyze current weather conditions     1    2    3      1    2    3        1    2   3     1   2   3
    Inability to analyze future weather conditions      1    2    3      1    2    3        1    2   3     1   2   3
   Poor in-flight planning and decision making          1    2    3      1    2    3        1    2   3     1   2   3
   Poor aircraft handling/Psychomotor skills            1    2    3      1    2    3        1    2   3     1   2   3
   Poor cockpit task management                         1    2    3      1    2    3        1    2   3     1   2   3
   Poor cockpit task prioritization                     1    2    3      1    2    3        1    2   3     1   2   3

   Inability to maintain division of attention inside
    and outside the cockpit                             1 2 3            1 2 3              1 2 3          1 2 3
SA technologies                                                               Page 96


  Poor crosswind landing and takeoff techniques       1   2   3   1   2   3   1   2   3   1   2   3
  Inability to seek out specific flight information   1   2   3   1   2   3   1   2   3   1   2   3
  Inability to perform safe flight procedures         1   2   3   1   2   3   1   2   3   1   2   3
  Inability to maintain aircraft separation           1   2   3   1   2   3   1   2   3   1   2   3
  Inability to understand ATC radio communications    1   2   3   1   2   3   1   2   3   1   2   3
  Personal fatigue                                    1   2   3   1   2   3   1   2   3   1   2   3
  Time constraints                                    1   2   3   1   2   3   1   2   3   1   2   3
SA technologies                                                                          Page 97


III. Loss of situation awareness in aviation may lead to several consequences. Please rate the
     following consequences with regard to their likelihood of occurring due to a loss of
     situation awareness in the four flight experience levels (i.e., hours up to Private, hours up to
     Instrument, hours up to Commercial, hours up to Multi-engine). For each item, please
     circle one rating in each of the four flight student experience level.

     Rating Scale
     1 Not likely to occur with loss of SA
     2 Less than 50% likelihood to occur with loss of SA
     3 More than 50% likelihood to occur with loss of SA
                                                                    Flight Student Experience Levels
                                                            Private     Instrument     Commercial       Multi-engine


     Altitude deviation                                     1   2   3    1   2   3       1   2   3        1   2   3
     Near mid-air collision                                 1   2   3    1   2   3       1   2   3        1   2   3
     Fuel quantity critical (in-flight)                     1   2   3    1   2   3       1   2   3        1   2   3
     Lost or disoriented                                    1   2   3    1   2   3       1   2   3        1   2   3
     Inadequate terrain separation                          1   2   3    1   2   3       1   2   3        1   2   3
     Continued unstabilized approach                        1   2   3    1   2   3       1   2   3        1   2   3
     Landing on wrong airport/runway                        1   2   3    1   2   3       1   2   3        1   2   3
     Engine shut down inadvertent                           1   2   3    1   2   3       1   2   3        1   2   3

     Significant fueling error                              1   2   3    1   2   3       1   2   3        1   2   3
     Propeller/tail wing strike                             1   2   3    1   2   3       1   2   3        1   2   3
     Airspace incursion                                     1   2   3    1   2   3       1   2   3        1   2   3
     Violation of FAR                                       1   2   3    1   2   3       1   2   3        1   2   3
     Heading deviation                                      1   2   3    1   2   3       1   2   3        1   2   3
     Traffic Conflict                                       1   2   3    1   2   3       1   2   3        1   2   3
     Non-adherence to published procedure                   1   2   3    1   2   3       1   2   3        1   2   3
     Crossing restriction                                   1   2   3    1   2   3       1   2   3        1   2   3

     Landing without clearance                              1   2   3    1   2   3       1   2   3        1   2   3
     Runway transgression                                   1   2   3    1   2   3       1   2   3        1   2   3
     Taxiway transgression                                  1   2   3    1   2   3       1   2   3        1   2   3
     Controlled flight toward terrain                       1   2   3    1   2   3       1   2   3        1   2   3
     Speed deviation                                        1   2   3    1   2   3       1   2   3        1   2   3
     Inadvertent stall                                      1   2   3    1   2   3       1   2   3        1   2   3
     Loss of directional control                            1   2   3    1   2   3       1   2   3        1   2   3
SA technologies                                                 Page 98


    Other:                                 1 2 3    1 2 3       1 2 3     1 2 3
    Other:                                 1 2 3    1 2 3       1 2 3     1 2 3



             Thank you for your participation in this survey.
SA technologies                                                                        Page 99



                                         Appendix B

                                    Annotated Survey




                                  Flight Instructor Survey

                     on Situation Awareness in General Aviation

                           Flight and Flight Instruction Experience

Your total hours of general aviation flight experience:        Mean = 1383
Your total hours of general aviation flight experience within the last 12 months Mean = 617
Your total hours of other flight experience (e.g., military, commercial):   Mean = 1495

Your total hours of general aviation flight instruction experience: Mean = 858
Your total hours of general aviation flight instruction experience within the last 12 months Mean
= 405
Your total hours of other flight instruction experience (e.g., military, commercial):       Mean
= 46

Please indicate the number of flight instruction hours you have had in the following flight
student experience levels:

       Hours up to Private                   Mean = 287     hours
       Hours up to Instrument                Mean = 202     hours
       Hours up to Commercial                Mean = 146     hours
       Hours up to Multi-engine              Mean = 84      hours
SA technologies                                                                       Page 100


I. Loss of situation awareness may occur during different aspects of any flight. With regards to
   this please rate each of the following categories as you perceive them as a problem for the
   pilot in maintaining situational awareness. For each item, please circle one rating in each
   of the four flight student experience levels (i.e., hours up to Private, hours up to
   Instrument, hours up to Commercial, hours up to Multi-engine).

   Rating Scale
   1 Not a major problem area for SA
   2 Moderate amount of problems for SA
   3 Frequent problems for SA




                         Private            Instrument           Commercial          Multi-Engine
       Phase of
                       1    2    3          1    2   3           1  2    3           1     2   3
        Flight
    Engine Start      48.3   34.5   17.2   72.4   27.6   00.0   87.3   12.7   00.0   51.3   43.6    5.0
    Preflight         50.0   39.7   10.3   77.6   20.7    1.7   85.5   14.6   00.0   71.8   25.6    2.5
    Taxi-out          19.0   53.5   27.6   48.3   44.8    6.9   74.6   25.5   00.0   48.7   48.7    2.5
    Takeoff           13.8   51.8   34.5   44.8   43.1   12.1   67.3   30.9    1.8   48.8   41.0   10.0
    Climb             20.7   60.3   19.0   46.6   46.6    6.9   72.7   23.6    3.6   38.5   46.2   15.0
    Cruise            39.7   44.8   15.5   56.9   34.5    8.6   80.0   20.0   00.0   64.1   33.3    2.5
    Descent           32.8   48.3   19.0   48.3   37.9   13.8   72.7   23.6    3.6   51.3   38.5   10.0
    Approach          10.3   43.1   46.6   15.5   48.3   36.2   58.2   30.9   10.9   25.6   59.0   15.0
    Landing            6.9   36.2   56.9   32.8   53.5   13.8   54.6   40.0    5.6   25.6   59.0   15.0
    Taxi-in           36.2   50.0   18.8   65.5   34.5   00.0   81.8   18.2   00.0   56.4   41.0    2.5
    Engine Shut       72.4   17.2   10.3   87.9   12.1   00.0   90.9    9.1   00.0   76.9   23.1   00.0
    Down
SA technologies                                                                                           Page 101


II. The following factors may be related to losing situation awareness. Please rate the following
    with regard to the degree to which you comprehend each is a problem for student pilots in
    the four flight experience levels (i.e., hours up to Private, hours up to Instrument, hours up to
    Commercial, hours up to Multi-engine). For each item, please circle one rating in each of
    the four flight student experience level.

    Rating Scale
    1 Not a major problem area for SA
    2 Moderate amount of problems for SA
    3 Frequent problems for SA

A. Failure to correctly                 Private           Instrument           Commercial          Multi-Engine
   detect the critical            1        2       3      1      2      3      1      2      3      1       2        3
   information in the
   situation due to:
Failure in student/              15.5    60.3     24.1   27.6   60.3   12.1   52.7   43.6   3.6    41.0    51.3   7.5
instructor communications
Failure in ATC                   17.2    31.0     51.7   15.5   53.5   31.1   65.5   29.1   5.5    56.4    38.5   5.0
communication/coordination
Failure to detect                20.7    43.1     36.2   20.7   46.6   32.8   63.6   32.7   3.6    43.6    48.7   7.5
instrument reading(s)
Failure in the                   34.5    37.9     26.7   10.3   48.3   41.4   72.7   27.3   0.0    51.3    46.2   2.5
instrument scan pattern
Failure to complete              10.3    48.3     41.4   20.7   65.5   13.8   43.6   47.3   9.1    28.2    51.3   20.0
checklist
Failure to detect information    1.7     31.0     67.2   5.2    32.8   62.1   43.6   45.5   10.9   15.4    46.2   37.5
due to high workload
Misreading or                    13.8    53.5     32.8   13.7   55.2   31.0   58.2   38.2   3.6    38.5    51.3   10.0
misinterpreting information
Failure to detect                12.1    41.4     46.6   10.3   41.4   48.3   61.8   36.4   1.8    41.0    46.2   12.5
Info (attentional narrowing)
Memory failure                   17.2    34.5     48.3   24.1   55.2   20.7   60.0   34.6   5.5    51.3    43.6    5.0
Undetected equipment             12.1    50.0     37.9   17.2   51.8   31.0   40.0   50.9   9.1    28.2    51.3   20.0
malfunction
Failure to cross-check           31.0    50.0     19.0   12.1   51.8   36.2   61.8   34.6   3.6    38.5    51.3   10.0
instrument settings
Failure to detect                6.9     48.3     44.8   15.5   53.5   31.0   36.4   49.1   14.6   25.6    46.2   27.5
information due (distractions)
Failure to carry-out             20.7    50.0     29.3   39.7   51.7   8.6    49.1   45.5   5.5    41.0    48.7   10.0
standard operating procedures
Failure to determine             19.0    46.6     34.5   43.1   39.7   17.2   54.6   36.4   9.1    51.3    41.0   7.5
current weather conditions
Failure to maintain an           8.6     41.4     50.0   41.4   36.2   22.4   43.6   45.5   10.9   33.3    46.2   20.0
SA technologies                                                                                   Page 102

awareness of other traffic
Failure to maintain an       13.8   56.9   29.3   39.7   48.3   12.1   50.9   45.5   3.6   43.6    46.2   10.0
awareness of airspeed
Failure to maintain an       12.1   67.2   20.7   29.3   55.2   15.5   61.8   38.2   0.0   53.9    46.2   0.0
awareness of heading
Failure to maintain an       8.6    69.0   22.4   29.3   51.7   19.0   67.3   32.7   0.0   51.3    48.7   0.0
awareness of altitude
Failure to maintain an       13.8   51.7   34.5   56.9   27.6   15.5   58.2   38.2   3.6   53.9    38.5   7.5
awareness of separation
SA technologies                                                                                        Page 103


    Rating Scale
    1 Not a major problem area for SA
    2 Moderate amount of problems for SA
    3 Frequent problems for SA

•   Failure to correctly detect the critical information in the situation due to:

Failure to maintain an               Private            Instrument          Commercial          Multi-Engine
awareness of the               1        2       3      1      2      3      1      2      3      1       2        3
following weather
factors:
Winds Aloft                   25.9    32.8     41.4   34.5   39.7   25.9   52.7   36.4   10.9   51.3    35.9   12.5
Surface Winds                 17.2    39.7     43.1   37.9   46.6   15.5   52.7   36.4   10.9   51.3    33.3   15.0
Icing (Ground)                53.5    17.2     29.3   55.2   24.1   20.7   63.6   18.2   18.2   59.0    23.1   17.5
Icing conditions en-route     53.5    19.0     27.6   48.3   25.9   25.9   63.6   21.8   14.6   59.0    25.6   15.0
Turbulence (approach)         26.3    49.1     24.6   35.1   45.6   19.3   59.3   35.2    5.6   50.0    42.1    7.7
Turbulence (cruise)           36.8    47.4     15.8   43.9   47.4    8.8   61.1   35.2    3.7   57.9    36.8    5.1
Precipitation                 43.9    40.4     15.8   52.6   38.6    8.8   79.6   18.5    1.9   71.1    26.3    2.6
Lightning (cloud-to-cloud)    45.6    40.4     14.0   57.9   28.1   14.0   70.4   22.2    7.4   68.4    21.1   10.3
Lightning (cloud-to-ground)   52.6    35.1     12.3   56.1   35.1    8.8   70.4   22.2    7.4   68.4    23.7    7.7
Stratus Clouds                54.4    35.1     10.5   66.7   28.1    5.3   74.1   22.2    3.7   65.8    26.3    7.7
Cumulus Clouds                49.1    36.8     14.0   63.2   28.1    8.8   75.9   18.5    5.6   71.1    23.7    5.1
Thunderstorms (Building)      40.4    35.1     24.6   45.6   40.4   14.0   63.0   27.8    9.3   52.6    34.2   12.8
Thunderstorms (Mature)        42.1    31.6     26.3   45.6   36.8   17.5   63.0   29.6    7.4   57.9    31.6   10.3
Thunderstorms (Dissipating)   43.9    31.6     24.6   50.9   33.3   15.8   70.4   22.2    7.4   63.2    26.3   10.3
Ceilings (IFR)                38.6    35.1     26.3   59.7   29.8   10.5   61.1   35.2    3.7   60.5    34.2    5.1
Ceilings (VFR)                38.6    42.1     19.3   68.4   22.8    8.8   66.7   29.6    9.3   71.1    21.1    7.7
Ceilings (MVFR)               28.1    45.6     26.3   61.4   28.1   10.5   57.4   33.3    9.3   60.5    26.3   12.8
SA technologies                                                                                            Page 104


    Rating Scale
    1 Not a major problem area for SA
    2 Moderate amount of problems for SA
    3 Frequent problems for SA



                                         Private           Instrument           Commercial          Multi-Engine
B. Failure to                      1       2        3      1      2      3      1      2      3      1       2        3
comprehend the
situation due to:
Misdiagnosis of equipment         24.1    43.1     32.8   27.6   51.7   20.7   41.8   52.7   5.5    33.3    48.7   17.5
problems
Misinterpretation of              17.2    32.8     50.0   34.5   51.7   13.8   54.5   34.6   10.9   48.7    38.5   12.5
information (confusion…)
Inability to relate bits of       10.3    43.1     46.6   17.2   58.6   24.1   45.5   45.5   9.1    33.3    46.2   20.0
Info. with each other…
Unfamiliarity with aircraft       13.8    37.9     48.3   48.3   36.2   15.5   47.3   38.2   14.6   30.8    38.5   30.0

Unfamiliarity wit                 15.5    34.5     50.0   48.3   39.7   12.1   60.0   32.7   7.3    51.3    41.0   7.5
geographical area
Unfamiliarity with                25.9    43.1     31.0   51.7   36.2   12.1   63.6   27.3   9.1    56.4    33.3   10.0
flight instructor
Unfamiliarity with                25.9    36.2     37.9   51.7   29.3   19.0   72.7   18.2   9.1    69.2    20.5   10.0
departure airport
Unfamiliarity with                8.6     31.0     60.3   27.6   50.0   22.4   45.5   34.6   20.0   46.2    30.8   22.5
destination airport
Unfamiliarity with                20.7    34.5     44.8   41.4   39.7   19.0   61.8   27.3   10.9   61.5    23.1   15.0
uncontrolled airspace
Unfamiliarity with                6.9     51.7     41.4   43.1   39.7   17.2   63.6   23.6   12.7   61.5    23.1   15.0
the airspace
Unfamiliarity with ATC            8.6     36.2     55.2   39.9   37.9   24.1   63.6   23.6   12.7   59.0    25.6   15.0
procedures
Failure to comprehend the
situation due to an inability
to develop an understanding
of:
   Problem/emergency              14.0    49.1     36.8   26.3   57.9   15.8   51.9   42.6    5.6   29.0    63.2    7.7
   Impact failure on safety       14.0    45.6     40.4   31.6   54.4   14.0   59.3   35.2    5.6   31.6    57.9   10.3
   Task priorities                 3.5    42.1     54.4   22.8   43.9   33.3   59.3   27.8   13.0   36.8    36.8   25.6
   Timing of events/tasks         15.8    45.6     38.6   21.1   57.9   21.1   53.7   35.2   11.1   39.5    44.7   15.4
   Criticality of events/tasks    12.3    42.1     45.6   29.8   45.6   24.6   51.9   37.0   11.1   39.5    47.4   12.8
   Own capabilities/limits        15.8    45.6     38.6   36.8   40.4   22.8   46.3   35.2   18.5   42.1    39.5   18.0
   Aircraft capabilities/limits   26.3    45.6     28.1   43.9   38.6   17.5   55.6   25.9   18.5   42.1    47.4   10.3
SA technologies   Page 105
SA technologies                                                                                            Page 106



                                       Private             Instrument           Commercial          Multi-Engine
C. Failure to predict            1        2       3       1      2       3      1      2      3      1       2      3
future situations due to:
Lack of understanding           10.5     43.9    45.6    24.6   57.9    17.5   63.0   27.8   9.3    60.5    23.7   15.4
implications of information
Recognition of                  10.5     38.6    50.9    15.8   66.7    17.5   59.3   31.5   9.3    52.6    36.8   10.3
current trends
Lack of a contingency           10.5     42.1    47.4    22.8   45.6    31.6   53.7   31.5   14.8   42.1    39.5   18.0
plan for the given task
Failure to analyze future       10.5     38.6    50.9    21.1   47.4    31.6   51.9   27.8   20.4   52.6    29.0   18.0
weather conditions


                                        Private               Instrument        Commercial           Multi-Engine
D. Failure to perform             1        2       3       1      2      3      1      2      3      1       2        3
   flight tasks due to:
Poor pre-flight planning:
 Inability to utilize NAS        17.5    47.4     35.1   38.6    43.9   17.5   53.7   35.2   11.1   50.0    39.5   10.3
 Equipment/capabilities
 Poor flight planning            17.5    52.6     29.8   38.6    45.6   15.8   53.7   40.7   5.6    52.6    42.1   5.1

   Inability to analyze          14.0    47.4     38.6   43.9    42.1   14.0   55.6   29.6   14.8   47.4    39.5   12.8
   current weather conditions
   Inability to analyze          5.3     52.6     42.1   28.1    52.6   19.3   48.2   37.0   14.8   44.7    44.7   10.3
   future weather conditions
Poor in-flight planning          10.5    50.9     38.6   31.6    43.9   24.6   50.0   38.9   11.1   47.4    31.6   20.5
and decision making
Poor aircraft handling/          21.1    40.4     38.6   49.1    31.6   19.3   64.8   24.1   11.1   47.4    34.2   18.0
psychomotor skills
Poor cockpit task                10.5    47.4     42.1   28.1    38.6   33.3   55.6   33.3   11.1   44.7    31.6   23.1
management
Poor cockpit task                10.5    49.1     40.4   22.8    47.4   29.8   50.0   37.0   13.0   36.8    39.5   23.1
prioritization
Inability to maintain            12.3    29.8     57.9   50.0    32.1   17.9   44.4   38.9   16.7   36.8    39.5   23.1
attention in/out of cockpit
Poor crosswind landing           10.5    22.8     66.7   22.8    47.4   29.8   46.3   33.3   20.4   42.1    34.2   23.1
and takeoff techniques
Inability to seek out            19.3    49.1     31.6   29.8    56.1   14.0   53.7   42.6   3.7    52.6    34.2   12.8
specific flight information
Inability to perform safe        29.8    47.4     22.8   50.9    40.4    8.8   66.7   31.5   1.9    57.9    31.6   10.3
flight procedures
Inability to maintain            36.8    42.1     21.1   61.4    31.6    7.0   72.2   25.9   1.9    63.2    31.6   5.1
aircraft separation
Inability to understand          10.5    40.4     49.1   31.6    45.6   22.8   68.5   24.1   7.4    65.8    23.7   10.3
SA technologies                                                                                  Page 107

ATC radio communications
Personal fatigue           29.8   45.6   24.6   31.6   47.4   21.1   51.9   29.6   18.5   39.5    36.8   23.1

Time constraints           29.8   43.9   26.3   33.3   42.1   24.6   46.3   27.8   25.9   39.5    31.6   28.2
SA technologies                                                                                          Page 108


III. Loss of situation awareness in aviation may lead to several consequences. Please rate the
     following consequences with regard to their likelihood of occurring due to a loss of
     situation awareness in the four flight experience levels (i.e., hours up to Private, hours up to
     Instrument, hours up to Commercial, hours up to Multi-engine). For each item, please
     circle one rating in each of the four flight student experience level.
     Rating Scale: 1 = Not likely to occur with loss of SA,
                      2 = Less than 50% likelihood to occur with loss of SA,
                      3 = More than 50% likelihood to occur with loss of SA

                                       Private            Instrument          Commercial          Multi-Engine
Consequences due                 1        2       3      1      2      3      1      2      3      1       2        3
to a loss of SA
Altitude deviation              8.8      36.8    54.4   12.3   42.1   45.6   31.5   48.1   20.4   29.0    44.7   25.6
Near mid-air collision          12.3     33.3    54.4   33.3   35.1   31.6   33.3   38.9   27.8   31.6    31.6   35.9
Fuel quantity critical          24.6     29.8    45.6   35.1   36.8   28.1   44.4   31.5   24.1   31.6    34.2   33.3
Lost or disoriented             0.0      28.1    71.9   14.0   35.1   50.8   38.9   31.5   29.6   23.7    34.2   41.0
Inadequate terrain separation   28.1     40.4    31.6   38.6   36.8   24.6   61.1   22.2   16.7   50.0    23.7   25.6
Continued unstabilized app.     8.8      33.3    57.9   19.3   50.9   29.8   42.6   38.9   18.5   39.5    42.1   18.0
Landing on wrong airport        5.3      35.1    59.7   36.8   38.6   24.6   44.4   37.0   18.5   42.1    34.2   23.1
Engine shut down                45.6     22.8    31.6   52.6   24.6   22.8   64.8   18.5   16.7   26.3    34.2   38.5
inadvertent
Significant fueling error       38.6     29.8    31.6   52.6   22.8   24.6   61.1   20.4   18.5   42.1    34.2   23.1
Propeller/tail/wing strike      19.3     49.1    31.6   56.1   31.6   12.3   55.6   35.2   9.3    52.6    31.6   15.4
Airspace incursion              3.5      31.6    64.9   45.6   29.8   24.6   51.9   25.9   22.2   44.7    26.3   28.2
Violation of FAR                8.8      33.3    57.9   31.6   40.4   28.1   44.4   29.6   25.9   36.8    31.6   30.8
Heading deviation               5.3      43.9    50.8   14.0   50.8   35.1   35.2   40.7   24.1   36.8    36.8   25.6
Traffic conflict                12.2     52.6    35.1   35.1   42.1   22.8   46.2   37.0   16.7   39.5    36.8   23.1
Non-adherence to procedure      21.1     52.6    26.3   21.1   47.4   31.6   57.4   27.8   14.8   52.6    29.0   18.0
Crossing restriction            33.3     35.1    31.6   22.8   43.9   33.3   59.3   25.9   14.8   50.0    31.6   18.0
Landing without a clearance     10.5     43.9    45.6   43.9   31.6   24.6   55.6   24.1   20.4   39.5    34.2   25.6
Runway transgression            12.3     42.1    45.6   38.6   36.8   24.6   51.9   27.8   20.4   44.7    31.6   23.1
Taxiway transgression           10.5     43.9    45.6   40.4   36.8   22.8   48.2   29.6   22.2   42.1    31.6   20.5
Controlled flight toward        31.6     33.3    35.1   35.1   33.3   31.6   59.3   24.1   16.7   47.4    31.6   20.5
terrain
Speed deviation                 19.3     43.9    36.8   19.3   57.9   14.0   42.6   46.3   11.1   31.6    42.1   25.6
Inadvertent stall               19.3     42.1    38.6   45.6   40.4   14.0   59.3   29.6   11.1   50.0    34.2   15.4
Loss of directional control     28.1     40.4    31.6   49.1   35.1   15.8   57.4   29.6   13.0   34.2    31.6   33.3
SA technologies                                                                         Page 109



                                          Appendix C

   Statistical Analysis of Differences in SA Ratings for Pilots of Four
                      Experience Levels: Somer’s D

The Somer’s D is a non-parametric statistical analysis that allows one to determine if the
response to one variable (the survey item in question) depends on the other variable (experience
level), and in which direction. For example, if "low" experience levels (e.g. Private pilot) tend to
imply "low" responses for that item (e.g., Rating of 1 = Not a major problem area for SA), the
result is a positive association. However, if the "low" experience level tends to imply a "high"
response (e.g., Rating of 3 = Frequent problems for SA), the result is a negative association.

Tables C-1 through C-6 provide the Somer's D value and the level of significance for each of
the survey items. Taking the first survey item, Engine Start, as an example, the Somer's D value
is negative (-.101), consequently the direction is reversed. That is, "low" experience levels (i.e.,
Private Pilot) tend to imply "high" responses (Rating of 3) for that survey item — or that the
private pilot experience level category is rated to have "frequent problems for SA" more often.
The level of significance for this research is set at .05.




                   Table C-1: Somer's D value and the level of significance for
                        phase of flight across all pilot experience levels.


                                                     Somer's D Analysis
                        Phase of Flight               Value        Significance
                    Engine Start                      – .101            NS
                    Preflight                         – .153           .002
                    Taxi-out                          – .281           .001
                    Takeoff                           – .292           .001
                    Climb                             – .179           .001
                    Cruise                            – .202           .001
                    Descent                           – .173           .001
                    Approach                          – .274           .001
                    Landing                           – .298           .001
                    Taxi-in                           – .181           .001
                    Engine Shut Down                  – .054            NS
SA technologies   Page 110
SA technologies                                                                            Page 111

           Table C-2:    Somer's D value and the level of significance for survey items
         related to failure to detect critical information across all pilot experience levels.
Failure to correctly detect the critical information in the                 Somer's D Analysis
situation due to:                                                            Value        Significance
Failure in student/instructor communications                                 – .232              .001
Failure in ATC communication/coordination                                    – .399              .001
Failure to detect instrument reading(s)                                      – .283              .001
Failure in the instrument scan pattern                                       – .269              .001
Failure to complete checklist                                                – .218              .001
Failure to detect information due to high workload                           – .298              .001
Misreading or misinterpreting information                                    – .285              .001
Failure to detect information due to attentional narrowing                   – .352              .001
Memory failure                                                               – .353              .001
Undetected equipment malfunction                                             – .202              .001
Failure to cross-check instrument settings                                   – .169              .001
Failure to detect information due to distractions                            – .203              .001
Failure to carry-out standard operating procedures                           – .185              .001
Failure to determine current weather conditions                              – .247              .001
Failure to maintain an awareness of other traffic                            – .233              .001
Failure to maintain an awareness of airspeed                                 – .236              .001
Failure to maintain an awareness of heading                                  – .328              .001
Failure to maintain an awareness of altitude                                 – .349              .001
Failure to maintain an awareness of separation                               – .273              .001
Failure to maintain an awareness of the following weather factors:
        Winds Aloft                                                          – .230              .001
        Surface Winds                                                        – .261              .001
        Icing (Ground)                                                       – .064              NS
        Icing conditions en-route                                            – .080              NS
        Turbulence (approach to landing)                                     – .208              .001
        Turbulence (cruise)                                                  – .161              .003
        Precipitation (penetration of rain)                                  – .209              .001
        Lightning (cloud-to-cloud)                                           – .137              .008
        Lightning (cloud-to-ground)                                          – .106              .041
        Stratus Clouds                                                       – .084              NS
        Cumulus Clouds                                                       – .146              .004
        Thunderstorms (Building Stage)                                        – 123              .027
        Thunderstorms (Mature Stage)                                         – .146              .008
SA technologies                                                                           Page 112


       Thunderstorms (Dissipating Stage)                                      – .157           .003
       Ceilings (IFR)                                                         – .162           .003
       Ceilings (VFR)                                                         – .179           .001
       Ceilings (MVFR)                                                        – .179           .001


       Table C-3: Somer's D value and the level of significance for survey items related to
        failure to comprehend the relevant information across all pilot experience levels.


                                                                     Somer's D Analysis
Failure to comprehend the situation due to:                         Value              Significance
Misdiagnosis of equipment problems                                   – .143                .011
Misinterpretation of information due to confusion…                   – .289                .001
Inability to relate bits of information with each other…             – .264                .001
Unfamiliarity with aircraft                                          – .156                .006
Unfamiliarity with geographical area (e.g., terrain)                 – .307                .001
Unfamiliarity with flight instructor                                 – .219                .001
Unfamiliarity with departure airport                                 – .288                .001
Unfamiliarity with destination airport                               – .302                .001
Unfamiliarity with uncontrolled airspace procedures                  – .289                .001
Unfamiliarity with the airspace                                      – .328                .001
Unfamiliarity with air traffic control procedures                    – .364                .001
Inability to development an understanding of:
        Problem/emergency situation                                  – .237                .001
        Impact of system failure on flight safety                    – .253                .001
        Task priorities                                              – .311                .001
        Timing of events/tasks                                       – .246                .001
        Criticality of events/tasks                                  – .274                .001
        Own capabilities/limitations                                 – .193                .001
        Aircraft capabilities/limitations                            – .141                .007




         Table C-4: Somer's D value and the level of significance for survey items related
         to failure to predict future situation dynamics across all pilot experience levels.


                                                                         Somer's D Analysis
Failure to predict future situations due to:                              Value         Significance
SA technologies                                                              Page 113


Lack of understanding the implications of the current information   – .371    .001
Recognition of current trends                                       – .389    .001
Lack of a contingency plan for the given task                       – .285    .001
Failure to analyze future weather conditions                        – .311    .001
SA technologies                                                                       Page 114

       Table C-5. Somer's D value and the level of significance for survey items associated
                     with related factors across all pilot experience levels.


                                                                   Somer's D Analysis
Failure to effectively perform the necessary flight tasks
due to:                                                            Value           Significance
Poor pre-flight planning:
        Inability to utilize NAS equipment and capabilities        – .246              .001
        Poor flight planning                                       – .265              .001
        Inability to analyze current weather conditions            – .244              .001
        Inability to analyze future weather conditions             – .307              .001
Poor in-flight planning and decision making                        – .255              .001
Poor aircraft handling/psychomotor skills                          – .214              .001
Poor cockpit task management                                       – .264              .001
Poor cockpit task prioritization                                   – .231              .001
Inability to maintain division of attention in/out of cockpit      – .218              .001
Poor crosswind landing and takeoff techniques                      – .312              .001
Inability to seek out specific flight information                  – .260              .001
Inability to perform safe flight procedures                        – .207              .001
Inability to maintain aircraft separation                          – .192              .001
Inability to understand ATC radio communications                   – .406              .001
Personal fatigue                                                   – .084               NS
Time constraints                                                   – .050               NS
SA technologies                                                                   Page 115

        Table C-6:     Somer's D value and the level of significance for consequences
                     due to a loss of SA across all pilot experience levels.


                                                                Somer's D Analysis
          Consequences due to a loss of SA                      Value          Significance
Altitude deviation                                              – .239              .001
Near mid-air collision                                          – .147              .008
Fuel quantity critical (in-flight)                              – .098              NS
Lost or disoriented                                             – .258              .001
Inadequate terrain separation                                   – .152              .001
Continued unstabilized approach                                 – .305              .001
Landing on the wrong airport/runway                             – .287              .001
Engine shut down inadvertent                                    – .030              NS
Significant fueling error                                       – .069              NS
Propeller/tail/wing strike                                      – .204              .001
Airspace incursion                                              – .287              .001
Violation of FAR                                                – .222              .001
Heading deviation                                               – .247              .001
Traffic conflict                                                – .181              .001
Non-adherence to published procedure                            – .213              .001
Crossing restriction                                            – .171              .002
Landing without a clearance                                     – .209              .001
Runway transgression                                            – .225              .001
Taxiway transgression                                           – .206              .001
Controlled flight toward terrain                                – .153              .005
Speed deviation                                                 – .153              .008
Inadvertent stall                                               – .233              .001
Loss of directional control                                     – .070              NS
SA technologies                                             Page 116



                               Appendix D

                          Task 3 Flight Scenario




                  Situation Awareness in General Aviation

                              Flight Scenario
SA technologies                                                                      Page 117


Name ______________________
Date: ______________________
Certificates: _________________
Status of subject: N ________        S ____________


Pre flight briefing
Radio frequencies
        ATIS Frequency               123.45
        Ground Control               121.0
        Tower Frequency              123.5
        Departure Frequency          125.8
        Clearance Delivery           118.55
        Transponder code             0713
Weather Conditions
        Wind Direction        240
        Wind Speed            10 kts
        Visibility            2 sm
        Ceiling               1,000 ft AGL overcast
        Weather conditions    Rain showers, mist
        Temperature           40 C
        Dewpoint              40 C
        Barometric Pressure   30.10
Airport operations
        Runway in use        27R
        Local Time           1645Z
        ATIS Information     Foxtrot
        Field elevation      ______
        AC Call sign         N 443 BC
        Aircraft performance C-172R
        Climb AS             110 kts
        Cruise AS            125 @ 2450 RPM
        500 fpm climb        115kts @ 2400 RPM

Instructor action: Brief the subject on the following:
Q Instruct the subject to talk out loud during the entire flight exercise. Try and verbalize your
  thoughts about the flight. For example, talk about actions you are about to perform, tactics or
  strategies that you plan to use or are using, or just general comments about the flight such as
  your current heading and altitude or current weather conditions. We do not expect you to
SA technologies                                                                        Page 118


  comment about anything in particular, just tell us basically what you are thinking about
  regarding the flight. At certain times the simulator will be “frozen in order to de-brief the
  subject
Q At certain times the simulator may be “frozen” in order to debrief the subject pilot.

ATIS Information – XYZ, airport information Foxtrot, time 1654Z, weather: sky 1,000 ft
overcast visibility two miles, light rain showers, temperature 40 C, dewpoint 40 C, altimeter 30.10
wind 2400 at10. Landing and departing runway 27 left, ILS runway 27 left in use. Advise initial
contact you have ATIS information Foxtrot….”

Clearance - N443BC is cleared to the XYZ airport via radar victors. Climb and maintain 3,000
ft, departure frequency will be 125.8, Squawk 0713”
SA technologies                                                                       Page 119




Flight Sequence



                          Subject Action – Instrument Takeoff
No coaching from the instructor
      1. Prepare Cockpit – checklist, kneeboard & timer on and adjusted.

       2. Complete all checklists appropriate up to, but not including the start checklist.

       3. Obtain ATIS information

       4. Contact Clearance Delivery on 118.55 for departure clearance

       5. Complete instrument cockpit check, with the exception of those instruments, which

           will be checked during taxi.

       6. Set NAV radios, frequency and OBS for departure

       7. Complete start checklist

       8. Contact ground control (121.0), obtain taxi clearance

       9. Taxi and complete the instrument cockpit check.

       10. At the hold short line complete run-up, appropriate checklists.

       11. Contact tower (123.5), and advise ready for departure.

Subject observations: (See score sheet)

Instructor action – Giving ATC instructions
       1. Cleared for takeoff, maintain runway heading, climb to 1,000-ft level off, and turn

           right to a heading of 3150,
SA technologies                                                                                       Page 120


        2. Contact Departure on 125.8

        3. After the aircraft contacts departure instruct the pilot to turn right to a heading of 3600

             and climb to 3,000 ft MSL.

        4. End of the takeoff session. The pilot then will do the warm up and simulator checks.

                                Subject Action – Warm up exercise

Instructor action: Let the subject do the following for preparation of the experiment. Time limitation 5
min.

Subject action – Perform the following
Q   Calibrate the Turn coordinator
Q   Climbs and descents
Q   Straight and level flight
Q   Power and aircraft configurations practice


Subject observations: (see score sheet)

                                  Instructor Action – Turns to a heading

Observing basic instrument skills Task #1 turns to a heading


        1.   Establish the aircraft level at 3,000 ft MSL

        2.   Establish Heading of 3600

        3.   Establish cruise airspeed of 125 kts. straight and level flight.

        4.   Establish a right turn of 150 AOB for a 1200 heading change

                                                                           Course heading = 1200 Magnetic

        5.   Fly straight and level for 1 minute, and then commence a right turn at 150 AOB for an 1800 heading

             change.                                                                      Course heading = 3000

             Magnetic.


        6. Traffic Alert: traffic at your 3 o’clock position altitude 2,500 ft
SA technologies                                                                                    Page 121

     7.   Fly straight and level for 45 seconds on this heading and commence a right turn at 300 AOB for a 900

          heading change.                                                      Course heading = 0300 Magnetic

     8.   Fly straight and level for 1 minute and turn left at 450 AOB for a heading change of 1800.

                                                                      Course heading = 2100 Magnetic


     9. Change to departure frequency – 134.55

     10. Fly straight and level for 45 seconds, turn right 1500 at standard rate

                                                                      Course heading = 3600 Magnetic


     11. End of task #1
SA technologies                                                                      Page 122



                         Instructor Action – Climbs and descends

Observing basic instrument skills Task #2 Climbs and descends

        1. Set the simulator to turbulent air #6

        2. Establish the aircraft on a heading of 3600

        3. Establish the aircraft AS at 125 kts

        4. Commence a descent at 500 fpm at an indicated airspeed @ 125 kts (rate and airspeed

            descent)

        5. Change radio frequency of 123.45

        6. Level off at 2,000 ft MSL

        7. Fly straight and level for 1 minute

        8. Change transponder code to 0114

        9. Commence a climb of 500 fpm to an altitude of 3,000 ft MSL (rate climb)

        10. Level off at 3,000 ft MSL

        11. Commence a climb of 750 fpm to and altitude of 4,000 ft MSL (rate climb)

        12. Level off at 4,000 ft MSL

        13. Set turbulent level to #8

        14. Commence a 1,000-foot descent using the clock and a descent rate of 500 fpm to

            3,000 ft. Time of completion 2 minutes. (rate and timed descent)

        15. Fail Alternator
SA technologies           Page 123


     16. End of task #2
SA technologies                                                                                         Page 124




       Instructor Action – Combination: Climbs, and descends using clock

Observing basic instrument skills Task #3 Turns, Climbs and descends



Combination exercise

        1.   Establish the aircraft at 3,000 ft MSL

        2.   Establish the aircraft on a heading of 1800


        3. Fail turn coordinator

        4.   Turn right to a heading of 3600 using standard rate and the clock.

        5.   At the completion of the turn, descend the aircraft at 500-ft. per minute to an altitude of 2,500-ft MSL,

             turn left to a heading of 1800 complete the heading and altitude change at the end of 1 minute. Again

             using the clock and VSI.

        6. Oil Pressure drop to 10

        7.   Upon reaching 2,500-ft MSL level off and fly the aircraft for 1 minute.


        8. Increase engine temperature to 100%

        9.   At the completion of the minute, start to climb the aircraft using 750 fpm to an altitude of 3,000-ft

             MSL at the same time turning right to a heading of 0450.

        10. Level off and fly straight and level for 45 seconds.


        11. REDUCE FUEL IN RIGHT TANK TO 5 GAL

        12. After the 45 seconds of straight and level flight, turn left to a heading of 2250 and descend at 800 fpm

             to an altitude of 2,600-ft MSL.

        13. After this maneuver fly straight and level flight for 1 minute.
SA technologies                                                                                   Page 125

     14. Starting at 2,600 ft MSL note your time, climb to an altitude of 3,600 ft MSL using the clock, climb at

         a rate of 500 fpm ending your climb in 2 minutes and making a heading change of 3600.
SA technologies                                                                                  Page 126


     15. Ice up engine

     16. Maintain your heading of 2250 climb straight ahead at 200 fpm to an altitude of 4,000-ft MSL and level

         off.

     17. Ice up pitot tube

     18. Fly straight and level for 1 minute.

     19. Climb to an altitude of 5,000-ft MSL at 110 kts. IAS.

     20. Upon reaching 5,000-ft MSL level off for 45 seconds and then turn left to a heading of 1600 and

         commence to descend at 120 kts. IAS to an altitude of 2,500 ft MSL.

     21. Fail attitude indicator

     22. Change radio frequency to 119.35

     23. Level off after 1 minute and slow the aircraft to approach speed of 100 kts IAS.

     24. Turn right to a heading of 3600

     25. End of Task #3
SA technologies                                                                                     Page 127



                           Instructor Action – Instrument Approach

Observing basic instrument skills Task #4 Instrument approach

Instructor Actions: Approach set up


        1. Set the simulator to position IOCF

        2.   Instrument approach: LOC Rwy 36, Ocala FL

        3.   Aircraft will be established for the approach by the instructor

        4.   The subject will perform an approach briefing to the instructor (make notes on score sheet)

        5.   The following weather briefing as reported by AWOS-3

             a.   Wind 300 @ 10

             b.   Ceiling 450’ AGL

             c.   Visibility 1 mile, mist

             d.   Altimeter 30.00 in Hg.


        6. Fail the compass locator (OC)

        7. Change the surface winds to 045 @ 18kts.




Subject observations:
        1.   Is the approach below minimums?
        2.   What is the determination on the missed approach point?
SA technologies                                             Page 128



                                 Appendix E

                  Task 3 Flight Performance Score Sheet




                  Situation Awareness in General Aviation

Flight Performance Score sheet
SA technologies                              Page 129


Name: _____________________
Date: ______________________
Certificates: ________________
Status of subject: N______     E__________
SA technologies                                                                           Page 130




                                                                                         Sometimes

                                                                                          Observed
                                                                                Always


                                                                                           Never

                                                                                            Not
 Pre-planning observations
1. Plans the sequence of specific approaches and holding.
2. Has a back-up plan.
3. Checks all NOTAMS including FDC and graphic notices.
5. Anticipates the effect of the winds aloft prior to the flight.
6. Determines freezing levels and cloud heights on winter flights.
7. Considers the current weather, aircraft inspections, navigation equipment,
and inoperative items conclude we can accomplish the planned flight.
8. Determines whether or not there are specific departure procedures.
9.Insures that the aircraft can climb at least 200 feet per nautical mile.
10.Give full consideration to the amount of fuel needed in excess of the
legal reserve. (ex: Two attempted approaches at destination, two missed
approaches, holding, one approach at an alternate airport, and/or fuel to
fly out of mountain range in case both destination and alternate airports
go below minimums.)
11.Pre-plans what to do in the event of loss communications.
12. Writes down specific information pertaining to the flight
13. Handles all radio communications
SA technologies                                                                                       Page 131




                                                                                          Sometimes


                                                                                                      Observed
                                                                                 Always


                                                                                                       Never

                                                                                                        Not
  Observations General
1. Hears radio calls the first time.
2. Carefully listens to the initial clearance to determine if we are operating
under IFR or VFR.
3. Requests clarification if ATC’s instructions are unclear.
4. Understands why we were given a particular radar vector and where it
is taking us.( Concludes that it coincides with what we want to do.)
5. “Mentally” arrives at the scene 5 minutes prior to the “aircraft”.
(Visualizes position, airspeed, and configuration.)
6. Makes decisions in a timely manor
7. Knows where he and the aircraft are at all times
SA technologies                                                                          Page 132




                                                                               Sometim


                                                                                           Observe
                                                                                Always


                                                                                            Never

                                                                                             Not
                                                                                  es
 Departure Observations
1. Tunes and identifies all NAVAIDs as soon as safety permits after
departure.
2. Confirms that the gear and flaps actually move when making
configuration changes.
3. Accelerates to cruise airspeed when appropriate.
4. Utilizes the checklist.
5. Sets up the instrument flying “T’s” five miles prior to entering holding.

6. Adjusts all headings and timing for wind.
7. Chooses a heading to fly, tenaciously sticks to it, then monitors the
navigation instruments to see if an adjustment needs to be made.




                                                                                        Sometimes

                                                                                         Observed
                                                                               Always


                                                                                          Never

                                                                                           Not
  Warm up exercise

1. General Observations about the pilot habits and skills
2. Is the pilot comfortable with the aircraft?
3. Did the pilot use the warm up time efficiently?
SA technologies                                                                                    Page 133




                                                                                                              Not Observed
                                                                                       Sometimes
                                                                              Always
 Turns to a Heading




                                                                                                   Never
1. Sets the attitude and power for a particular performance.
2. Looks at the attitude indicator while initially changing power to an
approximate setting. Then glances at the power instrument to make
final adjustment.
3. Make simultaneous pitch, roll, and power inputs while looking at
the attitude indicator.
4. Trims during all airspeed changes (before prompted by the
altimeter).
5. Maintains positive pitch/bank control with the control wheel at all
times.
6.Uses power or configuration to change the total energy (altitude
and/or airspeed).
7.Views the airspeed, altimeter, and VSI as a whole before changing
the trim, power or pitch during straight and level flight.
8.Keeps wings level using the bank index on the top of the attitude
indicator rather than looking at the wings on the miniature airplane.
9. Leads the turn when intercepting courses.
10. Adjusts all aircraft headings towards the cardinal heading the wind
  is coming from. (i.e. Consistently applies wind correction toward the
  “North”, not to the left/right.)
11. Trims during all airspeed changes before being prompt by the altimeter.
12. Initiates a turn, then key the microphone to reply to ATC.
13. Did the subject respond to the traffic alert?
14. Did the subject respond to the change to departure frequency?
15. Did the subject know where he and the aircraft were at all times?
16. Did the subject maintain altitude?
SA technologies                                                                                   Page 134




                                                                                    Sometimes



                                                                                                        Observed
                                                                           Always



                                                                                                Never
  Climbs and Descends




                                                                                                          Not
1. Sets the attitude and power for a particular performance.
2. Looks at the attitude indicator while initially changing power to an
Approximate setting. Then glances at the power instrument to make
final adjustment.
3. Make simultaneous pitch, roll, and power inputs while looking at
the attitude indicator.
4. Leads leveloffs with power if wishing to maintain the climb or
descent airspeed. (Gently guides the pitch with the control wheel.)
5. Leads leveloffs with pitch if wishing to accelerate or decelerate.

6. After leveling off from a climb, maintains climb power while
accelerating to cruise airspeed.
7. Trims during all airspeed changes (before prompted by the
altimeter).
8. Sets the trim to “lock in” the airspeed, and sets the power to adjust
the rate of climb/descent for the trimmed airspeed.
9. Maintains positive pitch/bank control with the control wheel at all
times.
10.Uses pitch to adjust the balance of energy between altitude and
airspeed.
11.Uses power or configuration to change the total energy (altitude
and/or airspeed).
12.Views the airspeed, altimeter, and VSI as a whole before changing
the trim, power or pitch during straight and level flight.
13.Keeps wings level using the bank index on the top of the attitude
indicator rather than looking at the wings on the miniature airplane.
 14. Trims during all airspeed changes before being prompt by the
 altimeter.

15. Did the subject respond to the transponder code change?
16. Did the subject respond to the failed alternator?
17. Did the subject know where he and the aircraft were at all times?
18. Did the subject maintain altitude?
SA technologies                                                                                  Page 135


19. Did the subject maintain heading?
20. Did the subject respond to all radio calls?




                                                                                                          Not Observed
                                                                                     Sometimes
                                                                            Always
 Arrival Setup Observations




                                                                                                  Never
1. Gets ATIS 100-150 nm out when on a cross-country.
2. Prior to the approach, determines the active runway.
3. Consistently does a detailed or abbreviated approach briefing.
4. Considers the following during a detailed approach briefing:
   ( First 9 items for abbreviated briefings - Frequency/Course/Altitude)
Supporting navigation instruments tuned and identified
Course
Heading indicator set to magnetic compass
Final approach course set in
Altitude
Altimeter set
Procedure turn/glide slope intercept altitude (“Top of the approach”)
DH or MDA (“Bottom of the approach”)
Visibility minimum
Severe weather at the missed approach holding pattern that may prevent us
Notes on the approach chart
Procedure turn and how to fly it
Procedure turn timing adjustment for wind
Inbound and outbound headings adjusted for wind
Final approach fix ( how to identify )
SA technologies                                                                                  Page 136




                                                                                     Sometimes


                                                                                                  Observed
                                                                            Always



                                                                                                   Never

                                                                                                    Not
 Approach Observations
1. Prior to initiating an approach, informs ATC of our intentions to
follow so that ATC can determine the appropriate missed approach
instructions if any.
2. Request radar vectors to final or the full approach when initially
requesting the approach.
3. Requests a delay vector if more time is needed to brief and set up the
approach.
4. Plans when to decelerate to approach airspeed.
5. Sets cockpit, landing and runway lights as appropriate.
6. Adjusts all headings and timing for wind.
7. Chooses a heading to fly, tenaciously sticks to it, then monitors the
navigation instruments to see if an adjustment needs to be made.
SA technologies                                                                           Page 137



                                            Appendix F

             General Aviation Situation Awareness Rating Sheet

Research Participant Number:                    Observer(s):

1. Rate the research participant's SA during the entire flight. Consider their SA during
   Preflight, Taxi–out, Takeoff, Climb, Cruise, Descent, Approach, Landing, Taxi–in, and
   Engine Shut Down.
       Low                                                                                        High
         1               2              3              4               5              6                 7




2. Rate the research participant's Level 1 SA. Level 1 SA is the ability to perceive, detect, or
attend to key elements of the situation. Consider their ability to determine current weather
conditions, and their ability to maintain an awareness of other traffic, airspeed, heading, altitude,
and separation.
       Low                                                                                        High
         1               2              3              4               5              6                 7




3. Rate the research participant's Level 2 SA. Level 2 SA is the ability to integrate multiple
   pieces of information and determine their relevance in light of the operational goals.
   Consider their ability to diagnose equipment problems, integrate information, and their
   understanding of problem/emergency situation(s), impact of system failure on safety, task
   priorities, timing of events/tasks, criticality of events/tasks, own capabilities/limitations, and
   aircraft capabilities/limitations.
       Low                                                                                        High
         1               2              3              4               5              6                 7
SA technologies                                                                        Page 138


4. Rate the research participant's Level 3 SA. Level 3 SA is the ability to project or forecast
   future situation dynamics. Consider their ability to (a) understand the implications of current
   information, (b) recognize current trends, (c) do contingency planning, and (d) analyze future
   weather conditions.
       Low                                                                                    High
         1              2             3              4              5              6              7
SA technologies                                                                          Page 139


Rate the research participant's ability to perform each flight task/skill.
                                                                             Low                    High
                   Flight Tasks/Skills                            N/A        1   2   3    4     5   6   7
Pre-Flight planning
Utilizing NAS equipment and their capabilities
In-flight planning and decision making
Aircraft handling/Psychomotor skills
Cockpit task management
Cockpit task prioritization
Maintaining division of attention inside and outside cockpit
Crosswind landing and takeoff techniques
Seeking out specific flight information
Maintaining aircraft separation
ATC communication/coordination
Detection of instrument reading(s)
Checklist procedures
Memory (e.g., remembering radio frequency)
Detection of any equipment malfunctions
Cross-checking instrument settings

If applicable, check all the consequences which resulted from a loss of SA during the flight.
Consequences
Altitude deviation
Near mid-air collision
Fuel quantity critical (in-flight)
Lost or disoriented
Inadequate terrain separation
Propeller/tail wing strike
Airspace incursion
Violation of FAR
Heading deviation
Traffic Conflict
Non-adherence to published procedure
Controlled flight toward terrain
Speed deviation
Inadvertent stall
Continued unstabilized approach
SA technologies                   Page 140


Landing on wrong airport/runway
Landing without clearance
Runway/Taxiway transgression
Engine shut down inadvertent
Other
SA technologies                                                                                                                             Page 141



                                                                 Appendix G

                           Observations from Simulation of Critical Events


                                                                               Novice Pilots
                                                               Low SA                                                                      Moderate SA
                      #1             #2             #5                #6        #9               #10             #7              #8                #4                #3
Instrument
  Take-Off
   Prepare                                                                 Poor                            No
                                             Good              Good                          Needed Help                                    Good              Good
   Cockpit                                                                 Organization                    Organization
                                 Missed
                 Did not acquire
 Obtain ATIS                     Altimeter
                 ATIS
                                 Setting
                 Did Not Write                                                                                                              Missed radio
   Obtain                        Missed      Missed
                 Down                                                                                                                       and
  Departure                      Departure   clearances, re-
                 Clearances/                                                                                                                transponder
  Clearance                      Frequency   obtained
                 Frequencies                                                                                                                code
  Complete                                                                                                                                                    Set Wrong
              Did not                                                                        Did not       Did not         Did not
 Instrument                                                                                                                                                   Altimeter
              Perform                                                                        Perform       Perform         Perform
Cockpit Check                                                                                                                                                 Setting
                                                                                                           Wrong freq.,
   Set NAV                                                                                                                                  Good Check of
                                                                                                           wrong                                          Did not Check
 radios, freq.                                                                                                                              Entries
                                                                                                           altimeter                                      2nd Radio
   and OBS                                                                                                                                  Against Notes
                                                                                                           setting
Complete Start                                                             Did not Turn on
  Checklist                                                                Transponder
                                                                                                                           Did not write
                 Did Not Write
                                                                                                                           down
 Obtain Taxi     Down                                                                                                                                         Readback
                                                                                                                           Clearances,
 Clearance       Clearances/                                                                                                                                  Error
                                                                                                                           Needed
                 Frequencies
                                                                                                                           Repeats
  Taxi and
                                                                                             Did not do
  Complete                                                                                                 Did not check                    Aircraft not in
                                                                                             Instrument
 Instrument                                                                                                instruments                      Control
                                                                                             Cockpit Check
Cockpit Check

  Complete
               No Engine                                                   No Engine Run-
Engine Run-up
               Check                                                       up
and Checklists
Advise Tower
                                                                                                                                                              Missed Tower
 Ready for
                                                                                                                                                              Freq.
  Take-off
                                                                                                                                                              Did not
                                                                           Heading and
Climb to 1000                                                                                                                                                 Change
                                                                           Altitude      Poor Altitude
ft and Change                                                                                              Lost Heading                                       Heading,
                                                                           Deviation,    Control
    Heading                                                                                                                                                   Overshot
                                                                           Airspeed Loss
                                                                                                                                                              Altitude
                                                                                                           Called ATC to
   Contact                                   Used Wrong                                                    verify
  Departure                                  freq.                                                         Transponder
                                                                                                           Code
 Turn right to
                                             Altitude                                                      Poor altitude
360 and climb
                                             Overshot                                                      control
 to 3000 MSL
SA technologies                                                                                                                                    Page 142




                                                                                    Novice Pilots
                                                                 Low SA                                                                           Moderate SA
                      #1            #2               #5               #6             #9              #10                #7                #8             #4               #3
(1) Turn to
 Heading
              Did not
              establish at
                                                                Did not write                                     Poor altitude
Level at 3000 Flight Level,                                                     Overshot        Poor Altitude
                                                                down ATC                                          and Heading
    MSL       Altitude                                                          Altitude        Control
                                                                Clearances                                        Control
              Control Not
              within Limits
                              Altitude
                                                                Heading Off
                 Heading      Deviation,
Heading at 360                                                  and Altitude    Not stable                                                                           Lost 40 deg.
                 Overshoot    Overcorrectio
                                                                High
                              n
    Cruise
 Airspeed at
                                                                                                                                                                     Fixated on
  125 kts,                                      Airspeed High
                                                                                                                                                                     Instruments
Straight and
    Level
Right Turn at
                                                                                Could not                                           Gained         Gained
 15 AOB for
                                                                                maintain rate                                       Altitude       Altitude
  120 deg..
Straight and
 Level for 1
                                                                                                                                                                     Altitude
minute, Right                                                                                                                                      Gained
                                                                                Fixated                                                                              Control
 Turn at 15                                                                                                                                        Altitude
                                                                                                                                                                     Problems
AOB for 180
    deg..
                              Responded         Good                                            Responded         Responded                        Missed Radio
Traffic Alert at Did not                                                        Repeated to                                         Looked even
                              had Aircraft in   maintenance                                     had Aircraft in   had Aircraft in                  Calls,
  3 O'clock      respond                                                        ATC                                                 though IMC
                              Sight (IMC)       of Control                                      Sight (IMC)       Sight (IMC)                      Overloaded
 Straight and
                                                                                                                  Did not write
 Level for 45
                                                Missed ATC                                                        down
 sec., Right                                                                                                                    Misread            Overshot
                                                Heading                                                           Clearances,
  Turn at 30                                                                                                                    callsign           Heading
                                                change                                                            needed ATC to
 AOB for 180
                                                                                                                  repeat
    deg..
                              Gained
                                                                Put in Wrong
 Change to                    Altitude,
                                                                Frequency,      Lost Heading,                                                      Gained
 Departure                    Missed Radio                                                                        Lost Heading
                                                                Could not       Fixated                                                            Altitude
 Frequency                    Freq. Change,
                                                                correct
                              Overload
 Straight and
 Level for 45
                                                                                                                                                   Fixated, Failed
  sec, Right                                                                                                      Turned Wrong
                                                                                                                                                   to Correct
Turn 150 deg.                                                                                                     Direction
                                                                                                                                                   Altitude Gains
 at Standard
     Rate
SA technologies                                                                                                                             Page 143




                                                                              Novice Pilots
                                                           Low SA                                                                         Moderate SA
                   #1           #2             #5              #6              #9              #10              #7              #8               #4              #3
(2) Climbs
    and
 Descents
                          Gained
Heading at 360 Heading                    Altitude        Altitude                                         Airspeed
                          Altitude, Did                                                                                   Overshot          Wrote Down
& Airspeed at Control                     Control         Control                                          Control
                          Not maintain                                                                                    Heading           Instructions
  125 kts.     Problems                   Problems        Problems                                         Problems
                          Airspeed
  Descent at   Airspeed
                                                                                                                                            Excessive
500 fpm at 125 Control                                                                    Could not Hold
                                                                                                                                            Rate
      kts      Problems
                                                                         Lost Heading                      Missed
                                                                         and Airspeed,                     changing radio Lost airspeed,                    Missed
Change Radio
                                                                         Over-                             freq., Did not Did not control                   changing radio
 Frequency
                                                                         controlled in                     write down     Vertical Speed                    freq.
                                                                         correction                        instructions
                          Wrong
                                                          Repeated
Level at 2000             Altitude,
                                                          Altitude                                                        Lost altitude
    MSL                   Heading
                                                          Deviations
                          Overshoot
 Straight and
                                          Could not Use                                                                                                     Fixated on
  Level for 1
                                          clock                                                                                                             Instruments
    minute
                                                                                                                                            Rapid
  Change                                                  Input Wrong                     Gained
                                                                                                                                            Descent, Did
Transponder                                               Code                            Altitude
                                                                                                                                            not notice
                                                                                          Overcontrolled
 Climb at 500                                                            Incorrectly      Aircraft, Poor
 fpm to 3000                                                             Reduced          heading and
     MSL                                                                 Power            Altitude
                                                                                          Control
Level at 3000
                          Too High
    MSL
                                                                                                           Problems
 Climb at 750             Could not                                      Added power                                                                        Leveled off
                                                                                                           maintaining    Heading
 fpm to 4000              maintain rate                                  without                                                                            Below
                                                                                                           airspeed,      Drifted
     MSL                  of climb                                       knowing why                                                                        Assigned Alt.
                                                                                                           Fixation
Level at 4000
     MSL
Descend 1000
  feet using                                                             Overloaded,
 clock at 500                                                            not responding
 fpm to 3000                                                             with needed
   MSL in 2                                                              changes
   minutes
                                        Did not Trouble   Did not Trouble                                                 Problems
                          Confused, Did Shoot or ask      Shoot or ask                    Confused, Did                   maintaining       Confused, Did
  Alternator                                                              Fixated on
                          not Know How for Help,          for Help,                       not Know How                    heading,          not Know How
   Failure                                                                Problem
                          to Correct    Random            Wanted to                       to Correct                      altitude,         to Correct
                                        Response          Land                                                            airspeed
SA technologies                                                                                                                                                                          Page 144

                                                                                           Novice Pilots
                                                                      Low SA                                                                              Moderate SA
                          #1            #2               #5                #6               #9                 #10              #7                #8               #4                #3
       (3) Turns,
        Climbs &
       Descents
                                                                                      Loss of
       Level at 3000 Simulation                                                       Heading and
           MSL       Terminated                                                       Altitude
                                                                                      Control
        Heading 180
                          X
           deg.
                                                                                                          Noticed                         Noticed
                                                                     Detected and
        Failed Turn                                                                                       Problem,        Did not notice, Problem,
                          X       Did not notice   Did not notice    Responded        Did not Notice                                                         Did not notice
        Coordinator                                                                                       Responded       Fixation        Responded
                                                                     Correctly
                                                                                                          Correctly                       Correctly
        Right Turn to                                                                                                                                                          Heading
          360 deg.,                                Overshoot         Poor Altitude                                        Gained                             Did not use       Overshot, Did
                          X
       Standard rate                               Heading           Control                                              Altitude                           Clock             not use
          and Clock                                                                                                                                                            Standard Rate
         Descend at
         500 fpm to                                                                                                                         Multi-tasking
                                                                                                          Not
       2500 MSL and                                                                                                                         Problems,
                          X                                                                               responding to
         Left Turn to                                                                                                                       Missed timing
                                                                                                          commands
        180 deg. in 1                                                                                                                       of Turn
           minute
                                                                                      Did not Notice
                                                                                                                                            Detected,
        Oil Pressure                                                 Did not notice   for 15 min,    Simulation           Did not notice
                          X       Did not notice   Did not Notice                                                                           Developed                          Did not Notice
            Drop                                                     for 15 min       Declared       Terminated           for 15 min
                                                                                                                                            plan
                                                                                      Emergency
                                                                                                                                            Problems
                                                                                                                                            maintaining
       Level at 2500
                          X                                                                                     X                           heading,
           MSL
                                                                                                                                            altitude,
                                                                                                                                            airspeed
        Engine Temp                                                                                                                                          Overloaded,       Overloaded,
                          X                        Did not Notice Did not Notice Did not Notice                 X         Did not Notice
          Increase                                                                                                                                           Did not notice    Did not notice
       Climb 750 fpm
                                                                                                                                            Turned in
        to 3000 MSL
                          X                                                                                     X                           wrong
       and Turn Right
                                                                                                                                            direction
         to 45 deg..
        Straight and
                          X                                                                                     X
       Level 45 sec.
                                                                                                                                                             Took action
                                                                                                                                                             but did not
        Fuel loss in                                                 Did not Notice, Noticed and                          Noticed and                        think through
                          X                                                                                     X                           Did not Notice
        Right Tank                                                   Overloaded      Responded                            Responded                          causes/
                                                                                                                                                             implications of
                                                                                                                                                             problem
                                                                                                                          Problems
         Turn Left to             Loss of
                                                                                                                          maintaining
        225 deg. and              control,
                                                                     Turned Wrong                                         Altitude and
         descend at       X       unusual                                                                       X
                                                                     Direction                                            Heading While
         800 fpm to               attitude,
                                                                                                                          Problem
          2600 MSL                disoriented
                                                                                                                          Solving
         Straight and
                          X                                                                                     X
         Level 1 min.
        Climb to 3600
          MSL using
         Clock at 500                                                                                                                                                          Could not use
                          X                                                                                     X
        fpm, Heading                                                                                                                                                           clock
       to 360 deg., in
          2 minutes
                                                   Confused,                          Could tell
                                                                                                                                                             Did not
                                                   Task                               engine wasn't                                          Applied Carb
                                                                                                                                                             understand
                                                   Saturated,                         responding as                       Noticed and       Heat, Then
       Engine Ice up      X                                                                                     X                                            problem, Did
                                                   Losing Altitude                    well but did                        Responded         Turns off and
                                                                                                                                                             not request
                                                   and Heading                        not know why                                          ices again
                                                                                                                                                             ATC help
                                                   Control                            or correct for
                                                                                      Applied Carb
       Heading 225,
                                                   Finally Applied                    Heat to see if it
       Climb at 200                                                                                                                                          Simulation
                          X                        Carb Heat,                         would fix                 X
       fpm to 4000                                                                                                                                           Terminated
                                                   Then Turns off                     problem, then
           MSL
                                                                                      turned off
                                  Turns on Pitot   Did not                                                                Became
                                                                                                                                            Recognized
                                  Ice, Problem     understand or                                                          Confused,
       Pitot Tube Ice-                                                            Detected and                                              Problem,
                          X       Clears, Turns    know how to     Did not Notice                               X         Misdiagnosed,                             X
              up                                                                  Responded                                                 Responds
                                  Off and Ices     correct                                                                Acted
                                                                                                                                            correctly
                                  Engine           problem                                                                Randomly
                                                   Engine Iced
        Straight and                               up, lost power,                                                                          Gained
                          X                                                                                     X                                                   X
        Level 1 min.                               stalled and                                                                              Altitude
                                                   crashed
       Climb to 5000                               Simulation
                          X                                                                                     X                                                   X
       MSL at 110 kts                              Terminated
        Level for 45
       sec, Turn Left
                                                                     Failed to
        to 160 deg.
                          X                              X           comply to ATC                              X                                                   X
       and Descend
                                                                     Commands
       at 120 to 2500
            MSL
                                                                                                                          Did not
                                                                                      Lost control,                       understand
                                                                                                                                            Fixated, Went
                                                                                      fixated                             problem,
           Attitude                                                                                                                         to Unusual
                                                                     Lost Control     became                              Reacted
          Indicator       X                              X                                                      X                           Attitude,               X          Crashed
                                                                     and Crashed      confused,                           Randomly,
           Failure                                                                                                                          Correctly
                                                                                      Gave up and                         Almost stalled,
                                                                                                                                            Diagnosed
                                                                                      Crashed                             Could not
                                                                                                                          Continue
       Change Radio                                                  Simulation                                           Simulation                                           Simulation
                          X                              X                                                      X                                                   X
         Frequency                                                   Terminated                                           Terminated                                           Terminated
       Level at 1 min,
         Slow to 100      X                              X                                                      X                X                                  X                X
            kts.
        Turn Right to
                          X                              X                                                      X                X                                  X                X
          360 deg..
SA technologies                                                                                                                                           Page 145

                                                                                         Novice Pilots
                                                                    Low SA                                                                           Moderate SA
                      #1               #2              #5                #6               #9              #10               #7               #8                #4               #3
     (4)
Instrument
 Approach
                                                                                                                                       Overcontrolled
                                 Did not tune
 Approach                                                                                                                              aircraft, Failed
                       X         and ID nav             X                                                   X                X                                  X                X
  Briefing                                                                                                                             to maintain
                                 equipment
                                                                                                                                       Altitude
                                                                                                                                       Wrong freq.,
                                 Altimeter not                                      Wrong Freq.,
                                                                                                                                       Misread
                                 Set Correctly,                                     lost altitude,
Fly Approach           X                                X                                                   X                X         Approach                 X                X
                                 Loss of                                            Missed Final
                                                                                                                                       Chart, Lost
                                 Heading                                            Approach
                                                                                                                                       DME on Final
 Compass
                       X                                X                                                   X                X         Did not Detect           X                X
Locator Fails
Wind Change             X                               X                                                    X                X                                 X                 X
                Inadequate                                         Inadequate       Inadequate       Inadequate       Inadequate                                          Inadequate
 General
                Aircraft                                           Aircraft         Aircraft         Aircraft         Aircraft                                            Aircraft
Problems        Control Skills                                     Control Skills   Control Skills   Control Skills   Control Skills                                      Control Skills
                Inadequate       Inadequate                        Inadequate       Inadequate       Inadequate       Inadequate                          Inadequate      Inadequate
                Instrument       Instrument                        Instrument       Instrument       Instrument       Instrument                          Instrument      Instrument
                Skills/          Skills/                           Skills/          Skills/          Skills/          Skills/                             Skills/         Skills/
                Knowledge        Knowledge                         Knowledge        Knowledge        Knowledge        Knowledge                           Knowledge       Knowledge
                                 Scan Pattern     Scan Pattern                      Scan Pattern                      Scan Pattern                        Scan Pattern    Scan Pattern
                                 Deficiencies     Deficiencies                      Deficiencies                      Deficiencies                        Deficiencies    Deficiencies
                                 Overload         Overload         Overload         Overload         Overload         Overload         Overload           Overload        Overload
                                 Attentional      Attentional                       Attentional      Attentional      Attentional      Attentional                        Attentional
                                 Narrowing/       Narrowing/                        Narrowing/       Narrowing/       Narrowing/       Narrowing/                         Narrowing/
                                 Fixation         Fixation                          Fixation         Fixation         Fixation         Fixation                           Fixation
                                 Multi-Tasking    Multi-Tasking                     Multi-Tasking    Multi-Tasking    Multi-Tasking    Multi-Tasking                      Multi-Tasking
                                 Problems         Problems                          Problems         Problems         Problems         Problems                           Problems
                                 Followed                                                                             Followed
                                 Routine                                                                              Routine
                                 Behaviors                                                                            Behaviors
                                 ATC Comm         ATC Comm         ATC Comm                                                            ATC Comm                           ATC Comm
                                 Problems         Problems         Problems                                                            Problems                           Problems
                                 Failure to       Failure to       Failure to       Failure to       Failure to       Failure to                          Failure to
                                 Diagnose &       Diagnose &       Diagnose &       Diagnose &       Diagnose &       Diagnose &                          Diagnose &
                                 Problem Solve    Problem Solve    Problem Solve    Problem Solve    Problem Solve    Problem Solve                       Problem Solve
                                                  Inadequate       Inadequate       Inadequate       Inadequate       Inadequate                          Inadequate
                                                  Systems          Systems          Systems          Systems          Systems                             Systems
                                                  Knowledge        Knowledge        Knowledge        Knowledge        Knowledge                           Knowledge
                                                  Did not Seek     Did not Seek     Did not Seek                                                          Did not Seek
                                                  Information      Information      Information                                                           Information
                                                  from             from             from                                                                  from
                                                  Resources        Resources        Resources                                                             Resources
                                                                                                                      Did not Write
                                                                                                                      Down
                                                                                                                      Information
                                                                                    Skipped                           Skipped          Skipped
                                                                                    Tasks,                            Tasks,           Tasks,
                                                                                    Checklist                         Checklist        Checklist
                                                                                    Items                             Items            Items
                                                                                    Poor                              Poor
                                                                                    Organization &                    Organization &
                                                                                    Planning                          Planning
                                                                                                                      Over
                                                                                                                      Confident
                                                  Did not rush
                                                  into actions
                                                  after getting
                                                  ATC calls
                                                  Good Preflight
                                                  Planning
SA technologies                                                                         Page 146


                                 Low Experience Pilot Results


Private Pilot #1

Instrument Takeoff

Q   Did not complete a instrument cockpit check prior to takeoff
Q   Did not know how to obtain information from ATIS prior to the flight
Q   Did not complete an engine check.
Q   Failed to write down any information pertaining to airport frequencies or clearance

Task 1

Q   Did not establish aircraft at the flight level when told to do so
Q   Altitude control was not within standards +/- 400 ft
Q   Overshot heading by 30deg.
Q   Did not respond to the traffic alert on line 6

Comments

The subject seemed to be totally lost during this portion on the flight. There was very little
demonstrated SA during this segment, there were times that the subject just sat still and was
apparently awaiting coaching from the instructors before proceeding with the assigned tasks. The
subject has enough experience to be able to handle the simulator but was very nervous about the
experience.

Task 2

Q Subject could not fly the simulator and was confused during this phase.
Q Headings were not maintained and airspeed control was very difficult +/-20 knots.
Q Subject was struggling to maintain flying status and it was at this time the experiment was
  terminated.

Summary

All areas of SA were rated as ones and it was the opinion of the facilitators that this person
should not be flying at all unless supervised by another pilot.
SA technologies                                                                      Page 147



Private Pilot #2


Instrument Takeoff

Q   Subject completed a through pre-flight check and cockpit check
Q   Missed the altimeter setting from ATIS
Q   When asked to turn to a heading of 3150. Subject turned passed the heading by 300.
Q   Subject missed the departure frequency and had to go back to previous frequency to correct
    his miss understanding of the frequency. Good SA in this case because he had a plan of
    action to correct the situation.

Task 1

Q Subject altitude deviation was -200 ft when attempting to change radio frequencies
Q In line item 2 when asked to turn right and change heading, again the subject lost 200 ft in
  the process.
Q Tried to correct altitude deviation after radio call from ATC and overcorrected by 200 ft in
  attempting to return to assigned altitude.
Q On the traffic alert subject was in IMC conditions and said that he had the traffic at his 3
  o’clock position when in fact he could not see the traffic and gained 250 ft in the process.
  Very poor SA because the response was not thought out in it’s entirety and because of the
  confusion BAI was compromised.
Q Line #9 change to departure frequency the subject gained 250 ft and missed the radio
  frequency change and then he started to become disorientated and we started to see evidence
  of task overload.

Task 2

Q Line item #1 gained alt and did not establish the aircraft on the proper airspeed of 125 kts.
Q Leveled off on the wrong assigned altitude in line #6 and heading was overshot by 20
  degrees.
Q Line item #10 subject leveled on at 3,100 ft instead of going to the assigned altitude of 3,000
  ft.
Q Subject had a unusual time of completing the rate climb as assigned by ATC. ATC had to
  prompt the pilot into making the necessary correction.
Q Line #15 alternator failure subject was confused and did not have SA in determining how to
  correct the problem.
Q Altitude gained was 400 ft. and loss of direction was +/- 20 deg. Either way of center line
SA technologies                                                                       Page 148



Comments (task 1 and 2)

Subject seemed to be confused during this portion of the flight. His SA was very poor he did not
try to solve his current problems or attempt to solve them in the future, he just keep things the
way they were and continued on with the flight. BAI skills were lacking, but should have been
better given the fact the he has 180hrs. total time and more experience that most pilots at this
juncture of his career.

Task 3

Q Subject did not notice the failed TC and went on trying to complete a standard rate turn using
  the attitude indicator. This would be good SA except the pilot never noticed that the TC was
  inoperative.
Q Subject never caught the fact that there was a drop in oil pressure. Completely eliminated the
  engine instruments from his scan.
Q Increased engine temperature, subject did not notice the change and now his SA is very low
  and the aircraft has started to loose performance.
Q Line #12 subject looses control and gets himself into a unusual attitude and freezes at the
  controls, subject completely disorientated and not trying to solve the situation.
Q Line #17 iced up pitot tube and looses AS indicator, turns on the pitot ice and the AS returns
  to normal then turns off the pitot ice and the engine now has iced up and quit.

Comments

The subject did not utilize the best of BAI skills when performing this flight portion. He is a
certified instrument pilot and the procedures used in this flight scenario were in accordance with
those the subject has been trained with.


Task 4


Q Subject did no tune and identified the navigational equipment prior to the beginning of the
  approach.
Q Altimeter was not set to the proper pressure setting.
Q Loss of heading during approach



Comments
SA technologies                                                                          Page 149



Subject did not have good SA skills when it came to flying this approach. From the start, radios
were not set to the proper frequencies and he appeared to be confused while flying the aircraft.
His SA was very low because of his indecision about flying the approach.

Summary

The subject was rated a 2 for SA during the entire flight; he did not have good control over
altitude or heading for the entire flight. Attempts to correct the situation were not noticeable.
Rated a 2 in the Level 1 SA for his ability to stay ahead of the situation was not apparent.
Level 2 SA again a 2 because of his lack of ability to diagnose equipment problems and the
failure to understand the safety issues associated with the flight.
In level 3 SA is rated a 2 because his ability to analyze future trends and forecast future situation
dynamics was poor. No real attempt to predict future situations in order to evaluate his current
situations led him to this low rating.
SA technologies                                                                        Page 150


Private Pilot #3

Instrument Takeoff

Q   Set up the cockpit in an orderly fashion and followed his checklist.
Q   Set in the wrong altimeter setting
Q   Used only the number one radio and did not test the other one for proper working order.
Q   Missed tower frequency could be because of the associated with the DAB airspace.
Q   Read back wrong clearance from ATC
Q   Did not change heading when requested form ATC after takeoff
Q   Flew past altitude by 500 feet on takeoff




Task 1

Q Lost 400 on heading command and fixated on one instrument while on instruments.
Q Altitude control was a problem this could be do the fact that he was new to the simulator, but
  he attempted to correct in a timely fashion.
Q Altitude control and heading control was an important factor in why the subject was missing
  radio calls and overshooting and undershooting heading assignments.

Comments

This subject was a newly required private pilot and was trying very hard to control the aircraft
and follow instructions. This is why he was having so much trouble to control the simulator.
His SA was very good overall in trying to accomplish the mission.


Task 2


Q Line item 5 subject missed changing the radio frequency and did a good job of controlling
  the aircraft while attempting the task.
Q Subject very much in a fixation mode while flying aircraft
Q Adjusted power for the climb and then leveled off 500 ft below the assigned altitude.

Comments
SA technologies                                                                      Page 151


The subject is a newly certified private pilot. His training was very limited when it comes to
flying under instrument conditions and it was evident that he was nervous throughout task 2. His
SA level was high because he was trying to accomplish all instructions that were given to him
but he fixated on only one or two instruments and this led to his poor performance on BAI skills.


Task 3


Q Line item #4 started time after turn was established making his turn go past the assigned
  heading by 20 – 30 degrees
Q Subject was using 30 degrees of bank instead of the standard rate of approximately 15
  degrees of bank.
Q Subject was task saturated past line #8
Q Could not use the clock when making heading changes or altitude changes.
Q Did not pick up on the oil pressure drop nor engine temperature increase for the entire flight.
Q The aircraft had and engine failure and an iced up pitot tube, he did not apply carburetor heat
  to clear up the situation. The pilot has not flown an aircraft with carburetor heat and he
  therefore did not know to apply it when the situation called for it.
Q When line #21 was applied the subject crashed the simulator.

Comments

Lack of flight experience and instrument experience was the main factor for a low SA rating in
this flight. The subject was slow to react when it came to discovering problems and having a
plan of action ready when it came to making decisions on how to get out of the situation. He was
tasked saturated when it came to doing multi task assignments. This led to his crashing the
simulator and not knowing how to put in the right control imputes to correct the situation.


Task 4


Not applicable


Summary:

The subject was rated a 6 for his SA during the whole flight because he was making decisions
and trying to solve his problems however because of his lack of experience and skill level he was
not successful in solving all of those problems.
SA technologies                                                                       Page 152


Level 1 SA rated at a 4 because he perceived and detected the key elements of the flight but
again he did not have the skill level to solve the situation.
Level 2 SA was rated a 3. He was aware of the problems but due to the lack of experience he
could not cope with the emergency situations and make adjustments.
Level 3 SA was rated a 3. The subject tried to have a plan of action to solve his flight problems
but did not have a full understanding of the situation.
SA technologies                                                                        Page 153


Private Pilot #4

Instrument Takeoff

Q Good SA from the standpoint that the use of checklist was used and this kept the subject on
  task during pre-flight preparation.
Q Missed radio call and transponder coded given by ATC
Q During the taxi portion the aircraft was not in control. This is from the fact that the subject
  has not performed any taxi maneuvers before.
Q Good SA when the instructions from ATC stated what the code should be and the subject
  miss entered the code and then referred back to his notes for the correct code.
Q Very good SA when doing a pre-flight of the mission and the subject was thinking ahead of
  the airplane.


Task 1


Q Subject gained 200 feet when trying to accomplish line #4
Q Again, he gained 300 ft when doing line #5
Q After ATC commanded that the aircraft maintain the pilot flew to an altitude of 3,500 ft. a
  gain of 1,000 ft. The subject corrected back to 3,000 ft but was still 500 ft high. ATC had to
  repeat the instructions to the pilot for the correction.
Q The subject missed many radio calls due to work saturation.
Q Line #8 AOB exceeded 30 degrees, overshot heading by 50 degrees.
Q After contacting ATC the aircraft gained 1,000 ft to and altitude of 3,500 ft should have
  maintained 2,500 ft.
Q Subject did not attempt to correct for the altitude gain. ATC instructions were to descend
  back to the assigned altitude, but the subject was fixated on one instrument and failed to
  correct.

Comments

The SA at this point in the flight was poor. The subject tried to over control the simulator and
was having a difficult time because of his lack of experience. He was trying to accomplish all
tasks, but was hindered by fixated on one instrument and fell behind the aircraft.


Task 2
SA technologies                                                                       Page 154


Q Demonstrated good SA when writing down ATC instructions
Q Excessive rate of descent when asked to maintain 500-ft decent. Actual rate was in excessive
  of 1,000 fpm.
Q When asked to change transponder codes the aircraft was placed in a 500-fpm descent and
  the subject did not attempt to stop the change until notified by ATC.
Q Line #12 fail alternator subject had no idea as to what to do. Never asked for help from ATC
  and or notified them of the problem.

Comments


Task 3


Q Failed to detect that the turn coordinator was inoperative
Q Did not use the clock when conducting the turn. This may be due to the fact that this is a
  private pilot and these maneuvers have not been performed.
Q Failures to notice that the engine gages were giving indications that an engine problem is
  about to happen. Omission was the cause of bad SA
Q Noticed that the fuel in the right tank was being depleted and he decided to change tanks, but
  no notice to ATC about the problem. Did not trouble shoot the problems just accepted it and
  continued with the flight, elapsed time 7-min.
Q After line #14 the subject lost all track of what was happening to the aircraft. When the
  engine was iced up the subject did not know how to apply carburetor heat. The reason for
  this is due to the training he received in a fuel injected aircraft. After trying to continue to
  climb with an engine producing partial power and the climb rate of only 50-fpm the subject
  was confused as to why the performance was so low. He never asked for assistance from
  ATC. The aircraft rolled to the right and stalled spiraling into the ground. The pilot was
  confused and never applied the correct control imputes to correct the situation.




Task 4


Did not attempt


Summary
SA technologies                                                                       Page 155


SA overall for the entire flight was rated a 4 for the given hours the subject has accumulated. He
demonstrated good SA for those areas he was conformable with, but placing him into new
situations that stretched his training envelope was very evident that fixation and omission was
the primary downfall for his decision making processes.
Level 1 SA was rated a 3 because of his ability to detect the key elements of the situation that
was placed on his resources. He did try to comprehend how to do the best with what knowledge
he had, but fell short by not seeking more information from other resources.
Levels 2 & 3 rated a 1 and 2 respectively. These skills were almost nonexistent because he was
overwhelmed with task saturation at every level.
SA technologies                                                                      Page 156



Private Pilot #5


Instrument takeoff

Q   Cockpit checks were make and the subject seemed very organized
Q   Subject missed clearance delivery information and had to obtain information that he missed
Q   Line #11 subject contacted departure on the wrong frequency.
Q   Flew past the assigned altitude and had to be corrected by ATC


Task 1


Q   Established aircraft on the correct heading but airspeed was high
Q   Good AS on the callout of the traffic and did not loose control of the aircraft.
Q   Missed the heading change by ATC and turned past the assigned heading by 90 degrees
Q   Good SA from the stand point of not rushing the flight when receiving instructions from
    ATC

Comments

The subject did a fine job of flying the simulator and following ATC instructions. Radio calls
were a little sloppy and had to be corrected by ATC when mistakes were made.


Task 2


Q Altitude control is starting to be a problem +200 ft.
Q Failed to use the clock when asked to do so. This could be because of the lack of training in
  this area and that the subject is starting to have difficulty in task management due to the
  increase in workload.
Q Line #12 alternator failure- subject wanted to land the aircraft immediately and did not try to
  trouble shoot the problem. Did not notify ATC of the situation and did not consult the
  aircraft handbook for proper procedures. Poor SA because the subject did not try to solve the
  problem he just kept on flying and had no apparent plan of action to remedy the situation.
Q Subject went to change fuel tanks when the alternator failed. When asked why he did it he
  had no explanation.

Comments
SA technologies                                                                           Page 157



At this stage of the flight the subject is starting to show signs of saturation of his decision
making skills.


Task 3


Q Failed to notice that the turn coordinator had failed. Subject continued on with trying to do a
  standard rate turn without a TC no clock or other device to help in accomplishing the task.
  Flying by the seat of the pilot’s pants can best describe this maneuver.
Q Flew past the assigned heading by 90 degrees
Q Missed the drop in engine pressure
Q Missed the increase in engine temperature
Q Both engine pressure and temperature problems were not detected until the end of the flight
Q After line 15 the subject started to become confused as to what all the aircraft problems were.
  Task saturation was very evident. Altitude control and heading control was starting to drift
Q Line #17 iced up pitot tube causing the AS indicator to freeze and the pilot did not know how
  to turn on the pitot heat to eliminate the situation.
Q Line # 15 carburetor ice was having the effect of decreasing the engine RMP. The subject
  notices the condition and turned on the carburetor heat and then turned it off after clearing
  the engine. This left the engine free to ice up again. At this time the aircraft lost power and
  proceeded to stall and crash leaving the subject not knowing what happened to him or how to
  correct the situation.

Comments

The subject was experiencing work overload and had no idea as to what to do to correct the
situation or how to prioritize his work load to accomplish a successful out come. This subject
has only 66 total hours and is not an instrument trained. His SA was good for the training he
received but fell short when he was left on his own judgement to make decisions.

Summary

Subject rated a 3 on SA for the entire flight. His preflight planning was good in the fact that
cockpit organization was clean and organized. He knew what information was needed and
anticipated what would come next. This was very apparent from the start but as the workload
increased these traits started to fall.
Level 1 SA was rated a 4 very high for this low time pilot. The subject was certainly trained well
in his flight course and he was responding to different situations with control and skill.
SA technologies                                                                       Page 158


Level 2 SA was rated a 1. When the subject started to do multiple tasks he was loosing SA very
fast. He fixated on the instruments and BAI control was lost. Transfer of knowledge was
becoming very slow and he was not responding to the needs of the situation and integrating
thought patterns on how to solve these multiple tasks.
Level 3 SA rated at a 2 only for the fact that he could recognize trends and problems but the door
closed there. The subject is very dangerous at this point with his flying skills.
SA technologies                                                                       Page 159


Private Pilot #6


Instrument Takeoff
Q Performed pre flight cockpit checks good SA
Q Did not perform an instrument cockpit check when taxiing out to the active runway
Q This is a private pilot and has not had very much experience when performing this task




Task 1


Q Did not write down ATC clearance
Q Heading off by 60 degrees and altitude was high by 300 ft
Q When told to change radio frequency the subject set in the wrong frequency and he did not
  know how to correct the error.

Comments

This subject had a hard time adjusting to flying solely by reference to the instruments. His
altitude control and heading control were giving him difficulty and was starting to become
saturated with just trying to control the aircraft.


Task 2


Q   Heading control was the most noticeable problem for this subject
Q   Heading was wavering by 900 most of the time
Q   Line #4 altitude strayed by + 150’ and ATC instructed the subject to correct three times.
Q   Changing the transponder code was changed to the incorrect code.
Q   When the alternator was failed the pilot wanted to land immediately by declaring an
    emergency when asked what he was thinking, the pilot explained that there was no reason
    other than this is what he thought was expected of him to do. The point is that he did not
    demonstrate good SA because he did not evaluate all of his options and make a game plan to
    fit the situation.

Comments
The subject is a new private pilot and is somewhat overwhelmed with flying this profile. His
skill level should allow him to fly this task without much trouble. He could have done much
SA technologies                                                                          Page 160


better but his recognition of trends and direction of aircraft movement really hampered his SA.
He was behind the aircraft the entire time and lost.


Task 3


Q Subject established that the turn coordinator was failed and adjusted accordingly, good SA. I
  then asked how he recognized this fact he replied he saw the TC was not moving when
  changing headings
Q His BAI skills are again hampering his controlling the aircraft alt +500ft, heading change by
  30 degrees
Q Line 6 failure of oil pressure, subject did not notice this for 15 minutes
Q Line #8 engine temperature was not noticed for the entire flight
Q Failure to notice #11 is due to the pilot having trouble flying the aircraft and struggling with
  altitude and heading control
Q Subject turned the wrong direction in line #12 and ATC had to correct the mistake
Q Line #17 subject did not detect the pitot tube had iced up
Q A number of commands from ATC were not complied with the subject ultimately lost control
  of the aircraft and crashed.

Comments

This subject had just finished receiving training for his private pilot certificate and is very “new”
when asked to demonstrate such BAI skills. He probably did not have to demonstrate this type
of maneuvers when ask for his check ride and therefore felt uncomfortable when doing this for a
research project.

Task 4

N/A

Summary

The subject was rated a 3 for the entire flight due to the fact that he was trying hard to
accomplish all the tasks that were assigned to him. He attempted to stay ahead of the aircraft and
was seeking information but saturation was overriding his flying skills.
In level 1 SA, the subject rated a 3, again the reason for this was due to the fact that he was
trying to maintain a high level of SA but could not fly and perceive how different situation were
playing out.
SA technologies                                                                       Page 161


SA level 2 was non-existent (rated 1). Emergency situations were all around him and no
recognition of the severity as to what they were telling the pilot which ultimately ended in the
aircraft crashing.
Level 3 SA was rated a 2. The subject wanted to fly better but couldn’t because of his limited
flight skills. Resignation was the main reason he couldn’t progress further in his SA. Watching
him loose recognition of the situation and the aircraft crashing led to him blaming it on the
simulator and it’s sensitivity. The truth of the matter was he couldn’t read his instruments
properly.
SA technologies                                                                       Page 162


Private Pilot #7

Instrument takeoff
Q No real method of organizing information prior to takeoff
Q Failed to set the altimeter to the proper setting
Q Did not perform ICC
Q Set the radios to the wrong frequency
Q When taxiing he failed to check the instruments for proper working order.
Q Lost heading on takeoff 600
Q Missed the code for the transponder
Q Heading control was very poor altitude was +150 ft.
Q Had to call ATC for verification of transponder code

Comments

The pilot was not very organized in the beginning of the flight and he showed a cavalier attitude
as to his pilot skills. He was not very organized with the details in accomplishing this phase of
flight.

Task 1

Q Altitude control and heading control were outside limits of instrument flying
Q BAI skills were very weak
Q Line #6 called out the traffic and said he saw the aircraft even though it was IMC
Q Lost heading while changing radio frequency
Q Did not write down clearance therefore the subject lost information and had to go back to
  ATC for further instructions
Q Line #10 turned the wrong direction when instructed by ATC
Q Heading was off by 600

Comments

Subject was very confident that he could handle the flight and not have any problems by not
writing down clearances or instructions from ATC. The problem was he thought that this was a
simple flight and he found out otherwise.

Task 2
SA technologies                                                                       Page 163


Q Subject had trouble establishing the proper descent airspeed he was very poor in bracketing
  the power settings this is probably due to the fact that he was not trained to do so.
Q Missed the radio frequency change, again not writing down the instructions from ATC
Q BAI skills were very poor. Subject had trouble establishing airspeeds in climbs and descents
  because of fixation on certain instruments.
Q Line #15 alternator failure he did troubleshoot the situation nicely and showed good SA.

Comments

The subject had very poor cockpit organizational skills and was very lax in writing down
information and following ATC instructions. This could be due to the fact that he still is showing
a cavalier attitude throughout the flight.


Task 3


Q Flailed the turn coordinator: the pilot did not pick up that the TC had failed. When asked
  how he was making a standard rate turn he did not know. He was using the attitude indicator
  and was turning the aircraft to the requested heading. This is ok, but it shows that he is
  fixating on one instrument and not using the proper instruments.
Q Gained altitude when changing heading +200 ft
Q Line #8 noticed that the oil pressure had dropped after an elapsed time of 15 min.
Q Did not notice that the engine temperature was high and continued on with the flight.
Q Line #11 noticed that he had a fuel tank leakage and switched tanks good SA
Q Iced up engine and he correctly applied carburetor heat.
Q During the times of engine problems the subject reacted by loosing or gaining altitude and
  heading control was constantly changing. Very hard to do multiple tasks at once.
Q Line #17 subject was becoming confused as to what instruments or systems were failed.
Q Subject thought that the AS indicator was broken did not use pitot heat to solve the problem.
Q Line #21 when the AI was failed the subject reduced the power and almost stalled the
  aircraft. When asked why? He could not think of a reason, only he knew that something was
  wrong and reacted accordingly.
Q Line #21 the subject was totally confused as to what the situation really was and just sat there
  not doing anything to correct the flight of the aircraft.

Comments

The subject had a hard time adjusting to all of the problems that he was encountering during this
phase of the flight. He had shown signs of resignation and frustration during his flight. He
SA technologies                                                                      Page 164


seemed to give the impression that he was more qualified than his experience showed. When the
pressure was applied he appeared to blame the simulator rather than realizing his own
limitations.


Task 4


Did not attempt.


Summary

Overall SA was rated a 2 because the subject did not show much awareness as to how the flight
was going instead he was trying to fly the simulator and showed signs of frustration because his
performance was not what he thought it should be.
Level 1 SA was rated a 2 because he lacked good BAI skills and never got “ahead of the
airplane”
Level 2 SA rated as a 3 because he was trying to set priorities and having better success in
certain situations.
Level 3 SA was rated a 3 because the subject was behind the aircraft and very slow in planning
for contingency
Private Pilot #8



Instrument takeoff

Q No ICC prior to takeoff
Q Did not write down clearances by ATC
Q Taxi instructions were very hard to follow and had to be repeated by ATC several times



Task 1

Q Line #4 subject gained 200 ft on the first instruction by ATC
Q Situation on seeing the aircraft at your 3 o’clock position response was looking even thought
  the aircraft is in IMC conditions
Q Missed read the call sign of the aircraft several times

Comments
SA technologies                                                                       Page 165



The subject has a good SA in the fact that he has 160 hours of flight time and was in command
of the aircraft during the warm up phase. He knew what he wanted and where to get the
information to solve problems.



Task 2

Q Set the altimeter to the correct setting
Q Turned past his heading by 100
Q Subject was fixation on changing the radio frequencies and there fore lost airspeed and failed
  to control his vertical speed rate.
Q Started to rush when instructed by ATC and lost altitude
Q Line #11 heading drifted off by 300
Q Line #15 BAI deteriorated when concentrating on the alternator failure

Comments

Altitude and heading control was a problem and the subject tried to correct in a timely fashion.
It appears that he had a fair grasp to do one task and solve another one at the same time. Not bad
SA for all the problems encountered during this phase.

Task 3

Q Announced the failure of the TC and proceeded to use the AI for the completion of the turn.
  Good SA
Q Multi task of line #5 caused some problems for the pilot. He was trying to accomplished
  multi task and he lost timing of the turn and the VSI
Q Subject saw and announced the loss of oil pressure and proceeded to have a plan of action on
  solving the problem while flying the simulator. Altitude and heading varied slightly causing
  BAI to degrade
Q At line #9 the subject was starting to show signs of saturation and fatigue. He was turning
  the wrong direction when asked by ATC and altitude was getting out of standards.
Q Never caught the leak in the fuel tank for line #11
Q Line #15 engine iced up. Applied carburetor heat then turned it off. Given the weather
  conditions as reported in ATIS the subject should have realized this was a possibility and
  caught the problem earlier. When he turned off the carburetor heat the aircraft iced up again.
Q Line #17 caught the situation and applied pitot heat altitude gained 200 ft
SA technologies                                                                          Page 166


Q Line #21 the failure of the AI the subject was fixated on the instrument and placed the
  aircraft in an unusual attitude before realizing what the problem really was.



Comments
As task 3 started to get more complicated the subject was trying to catch up with the aircraft and
solve his problems. As he would catch the problems he would announce what they were and try
to make decisions about what to do to solve them. This was making for a game to play and try
to catch what the facilitators were doing to the simulator. He was behind the aircraft when it
came to BAI and really trying to solve his problems.

Task 4

Q   Over controlling BAI
Q   Altitude loss was 300 ft. on final approach
Q   Tuned in the wrong frequency
Q   Miss read the approach chart
Q   Lost DME on final because of tuning in the wrong station
Q   Did not identify JUMPI intersection
Q   Did catch nav 1 failure
Q   Elected to go missed approach when he could have done an loc approach
Q   Failure to maintain altitude during the approach

Comments

Subject is not an instrument rated pilot. He did a fairly good job of trying to control the aircraft
during his instrument approach. His SA was high because he wanted to do a good job and not
crash.

Summary

Overall SA rated a 5 because of his organizational cockpit skills and staying ahead of the
aircraft.
Level 1 SA rated a 4 mainly because of the way he maintained SA out of the cockpit.
Level 2 SA rated a 4 again he could maintain a good safety margin and set priorities
Level 3 SA rated a 4 because of he ability to analyze future conditions and act upon their trends
SA technologies                                                                     Page 167


Private Pilot #9


Instrument takeoff


Q Subject did not do an ICC and lacked cockpit organizational skills
Q The pre takeoff check was done without an engine run up
Q After takeoff the aircraft was allowed to deviated altitude by +200 ft and heading drifted to
  the right by 300
Q When the aircraft was on climb out the AS slowed down to 50 knots and it appeared that the
  subject was about to stall the aircraft
Q Transponder was not turned on and the pilot had to be reminded by ATC to do so

Task 1

Q Subject passed through assigned altitude by 300 ft
Q The pilot showed signs of not being able to establish a stable BAI flight while turning to a
  new headings
Q Could not establish a 150 AOB while turning the aircraft
Q Subject was fixation on one instrument at a time and therefore the flight was not going well
Q On line #6 the subject just repeated back ATC instructions about the traffic. It appeared that
  the pilot really did not know if he had the traffic or not
Q When changing frequency the pilot lost 30 – 45 degrees of heading and again it appeared that
  the subject was fixating on the radio change more than flying the aircraft



Comments

From the start it was evident that he has very bad instrument flying skills. He never got
established into any comfort zone while flying the aircraft and was behind more that ahead of
any situation in task 1.



Task 2

Q When requested to change frequency the subject could not maintain the desired vertical rate
  for the climb
Q When asked to change transponder code the subject lost heading and speed. When trying to
  recover the pilot over controlled the aircraft and this compounded the problem
SA technologies                                                                     Page 168


Q Line item #9 the subject reduced power for the climb when asked why he reduced the power
  the reply was that he did not have an answer, mean while the subject was having difficulty in
  maneuvering the aircraft to control altitude
Q On command line item #11 the subject still was adding power slowly and never really was
  sure of what he was trying to accomplish
Q Subject was saturated with an over work load and was not responding to the aircraft when
  obvious pitch and heading changes were necessary
Q Line #15 failed alternator, the subject lost all situation on what was going on in the cockpit
  and started to focus immediately on the failed alternator. He lost priorities of BAI and over
  concentrated on the alternator and

Comments

Same as in task 1. For the number of flight hour’s (300) this pilot was very uncomfortable with
his own skills.

Task 3

Q For this portion of the flight there were too many problems with altitude and heading control
  to list them all. The fact is that the subject was trying to make corrections and had to
  concentrate on flying the simulator rather than solving problems that were occurring at any
  given time. Altitude control was greater than 500 ft and heading was drifting off by more
  than 30 degrees during this portion of the flight.
Q He did notice that the TC had failed and stated that he could not accomplish what ATC was
  telling him to do. This was good SA in the fact that he included this instrument in his BAI
  scan.
Q When the engine oil pressure dropped he subject did not notice this indication and continued
  with the flight.
Q Line #8 engine temperature was climbing and the subject again did not notice it for the entire
  flight.
Q Subject noticed the failure of the oil pressure after 15 min. and announced that he was having
  a problems maintaining oil pressure and declared and emergency. He did not notice that the
  engine temperature was redlined but never indicated to the instructor. He should have caught
  this and associated it with the oil pressure problem.
Q Line #11 time elapse 5 min. subject did notice the fuel imbalance and switched the tanks.
  He was asked about the situation and his reply was the switching of the tanks should remedy
  the situation that was the extent of his judgement.
Q Line #15 the engine was producing less power and the aircraft could not climb at the desired
  rate the subject never added full power and tried to climb at a slower rate. When queried his
SA technologies                                                                       Page 169


  response was that he knew that the aircraft was not climbing at the desired AS, but he was
  just trying to maintain control of the aircraft.
Q After this point he added throttle and decided he needed to add carburetor heat to see if the
  engine would clear. Interesting behavior he then turned off carburetor heat and the problem
  started to occur again.
Q When the pitot tube started to ice up the AS indicator “froze” and the subject added pitot heat
  and this solved the problem
Q Line #21 was very trying for the subject. He started to chase the AI and the aircraft went out
  of control. The subject fixated on certain instruments and did not know what the indications
  of those instruments were telling him. He attempted to correct the situation only to think that
  he had another problem elsewhere. After 30 seconds the subject just stared at the floor and
  the aircraft crashed. When debriefed we asked the subject what he thought had happened he
  said that he had no idea as to the problem other than it appeared that all instruments had
  failed. We asked him some specific questions about the instruments and he though that they
  all were defective and that he could not solve the problem.

Comments

See above for comments

Task 4

Q   Tuned in the wrong frequency for nav 1
Q   Fixated on changing the radio frequency and the aircraft lost 200 ft altitude
Q   Flew past the final approach course
Q   Did not catch that the field was below minimums for the approach

Summary

Very bad flying skills were demonstrated on this flight. The subject seemed tired and lacked
proficiency in flying an instrument approach. Overall SA was graded at a 2 and this was mainly
due to the poor BAI flying of the simulator. He was very slow in seeking information outside of
the cockpit and having a plan to stay ahead of the aircraft.
Level 1 SA was rated at a 2 basically for the same reason as overall SA. His ability to determine
current situations was very poor and in some cases nonexistent.
Level 2 SA and level 3 SA were rated 2’s for the reason he crashed and never knew what went
wrong.
SA technologies                                                                      Page 170


Private Pilot #10

Instrument takeoff:

Q   Subject needed some coaching as to set up for this flight
Q   Subject seemed very uncomfortable the entire time
Q   Altitude control +300 ft at the first altitude command
Q   Seemed unstable about his BAI skills and appeared that he was not ahead of the aircraft

Comments

This subject need coaching and was very unsure of his skills for this flight.

For a 90-hour private pilot he seemed very unsure of his skills and abilities. His manor was very
cautious and he was showing signs of omission and reactionary to flying the simulator.

Task 1

Q No ICC during taxi or after engine start
Q Altitude gained +200 ft for the first ATC instruction
Q Said that he had traffic when in face he was IMC

Comments

Subject had to be given coaching instructions several times and it was obvious that he wanted to
end this task as soon as possible.

Task 2

Q Could not hold 500 fpm and 125 knots
Q Lost heading by 30 degrees
Q After being instructed to change transponder code the aircraft gained 300 ft.
Q Subject started to over control the aircraft and the altitude and heading were changing
  constantly
Q Line #15 when the alternator failed and the subject did not know what or how to trouble
  shoot the problem. He did not know the extent of the problem either.

Comments
SA technologies                                                                         Page 171


The same comments as in #2. Subject asked for coaching and was really behind the aircraft
causing attitude and heading to change constantly.



Task 3

Q Subject caught the TC failure and continued the turn with the AI
Q The subject was very imitated by the simulator and he was having trouble flying altogether.
Q With the different problems he was encountering with the engine and flight instruments he
  was not responding to any commands by us

Comments

Subject was really having a hard time adjusting to the environment of the simulator. He was not
conformable because his BAI skills were really bad and he was very embarrassed over the
situation. The experiment stopped after the subject could not respond to our commands. This is
very unusual because the subject has his private certificate and has 90 hours of flight time.

Summary

The subject was rated ones for all areas of SA. I think that this pilot was not trained as well as he
should have been. It is true that he is a certified private pilot and he has earned the rights and
privileges of the certificate, but I don’t think that he really wants to be a pilot--it scares him.
SA technologies                                                                                                                                      Page 172


                                                                                    Experienced Pilots
                                     Moderate SA                                                                              High SA
                       #1             #2               #6                #4               #5             #7                 #8              #9           #10             #3
Instrument
  Take-Off
                                                                                                                      Wrong
                                                                                    Did not catch                     transponder                                  Did not catch
   Prepare                      Caught runway Caught runway Missed flaps                                                                             Missed runway
                                                                                    runway                            setting,                                     runway
   Cockpit                      discrepancy   discrepancy   on checklist                                                                             discrepancy
                                                                                    discrepancy                       missed runway                                discrepancy
                                                                                                                      discrepancy
                                                                                                    Detect
                                                                                                                                       Turned on
 Obtain ATIS                                                                                        backwards
                                                                                                                                       Pitot heat
                                                                                                    ATIS
   Obtain                                                                                           Wrote down
  Departure                                                                                         wrong ground
  Clearance                                                                                         freq
                                                                    Did not                         Did not                            Did not
  Complete                                                                                                                                                         Did not check
              Did not                                               perform                         perform                            perform
 Instrument                     Did full check                                                                        Did full check                               turn
              complete                                              Instrument                      Instrument                         Instrument
Cockpit Check                                                                                                                                                      coordinator
                                                                    cross check                     cross check                        cross check
                 Set wrong
   Set NAV
                 tower freq.,
 radios, freq.
                 set wrong
   and OBS
                 altimeter
Complete Start
  Checklist
 Obtain Taxi
 Clearance
  Taxi and
  Complete
 Instrument
Cockpit Check

  Complete
Engine Run-up                                                       Checked flaps No Run-up
and Checklists
                                                                                    Missed turn
Advise Tower                                                                        direction, Set Stopped on roll-
                                                                                                                                                                   Missed Take-
 Ready for                                                                          wrong           out to copy
                                                                                                                                                                   off clearance
  Take-off                                                                          departure freq. clearance
                                                                                    Initially
                                Had trouble      Detected
                                with rudder,     wrong
Climb to 1000
                                stalled and      altimeter                                          Turned at 400 Exceed by 250
ft and Change
                                crashed,         setting on roll,                                   feet          feet
    Heading
                                missed           150 feet too
                                heading          high
                                                                                    Lost 300 feet
   Contact                                                          Gained100
                                                                                    after freq.
  Departure                                                         feet, switching
                                                                                    Change.
                                               Climbed at
 Turn right to
                                Did separate   2500 rpm
360 and climb
                                turn and climb (forgot it was
 to 3000 MSL
                                               Cessna)
SA technologies                                                                                                                             Page 173

                                                                                  Experienced Pilots
                                         Moderate SA                                                                        High SA
                        #1                #2              #6              #4           #5              #7                #8           #9         #10               #3
(1) Turn to
 Heading
Level at 3000                                        stayed at 100
    MSL                                              ft above
Heading at 360
    Cruise
 Airspeed at
   125 kts,
 Straight and
     Level
 Right Turn at                                                       Missed
  15 AOB for                        Wrong heading                    heading,
   120 deg..                                                         repeated
 Straight and
  Level for 1                       Lost 400 feet
                                                                                                 Lost 100 feet
minute, Right                       while fixating
                                                                                                 copying
  Turn at 15                        on heading
                                                                                                 clearance
 AOB for 180                        bug
     deg..
Traffic Alert at                                     Correct                      "Keeping eyes                   Correct                   Looked for       Looked for
                                                                                                "In sight"                       "In IMC"
   3 O'clock                                         response                     pealed"                         response                  Traffic in IMC   Traffic in IMC
 Straight and
                   Readback                                                       Readback
 Level for 45
                   error but                         Used 55                      error but
  sec., Right
                   turned to                         degree bank                  turned to
  Turn at 30
                   correct                           angle                        correct
 AOB for 180
                   heading                                                        heading
     deg..
  Change to                                                                                      Readback
                   Asked for                                         Strayed in                                   Prioritized
  Departure                                                                                      error, lost 10
                   repeat                                            bank angle                                   flying first
  Frequency                                                                                      feet solving
 Straight and
 Level for 45      Asked for
  sec, Right       repeat, Rolled
Turn 150 deg.      out at wrong
 at Standard       heading
     Rate
SA technologies                                                                                                                              Page 174

                                                                         Experienced Pilots
                                Moderate SA                                                                         High SA
                   #1            #2              #6           #4               #5                 #7              #8             #9             #10          #3
(2) Climbs
    and
 Descents
Heading at 360
                                            Stayed 100
& Airspeed at
                                            feet high
  125 kts.
  Descent at
500 fpm at 125
      kts
                                                                                            Leveled off
                                                                                            while copying
Change Radio               Leveled out to                                                                                   Asked for freq
                                                                                            freq, Lost 500
 Frequency                 perform                                                                                          repeat
                                                                                            feet setting
                                                                                            freq
Level at 2000
    MSL
Straight and
 Level for 1
   minute
  Change
Transponder
                                                         Increased to
 Climb at 500
                                                         1000 fpm when
 fpm to 3000               Working hard
                                                         doing other
     MSL
                                                         task
Level at 3000
     MSL
 Climb at 750
 fpm to 4000
     MSL
Level at 4000              Detected
     MSL                   turbulence
Descend 1000
  feet using
                                                                         Jumbled
 clock at 500 Completed 25                                                                                                                              Missed
                                                                         readback, but
 fpm to 3000 seconds early                                                                                                                              descent rate
                                                                         sorted through
   MSL in 2
   minutes
                                                                                                                                                        Detected,
                           Detected in 2                                 Detected                            Detected and                               leveled out
  Alternator                                             Changed rate
                           minutes, 8                                    quickly and                         responded                                  momentarily to
   Failure                                               of descent
                           seconds                                       prioritized well                    smoothly                                   evaluate and
                                                                                                                                                        correct
SA technologies                                                                                                                                                                    Page 175

                                                                                           Experienced Pilots
                                              Moderate SA                                                                             High SA
                             #1                #2               #6               #4             #5              #7                #8               #9              #10               #3
     (3) Turns,
      Climbs &
     Descents
     Level at 3000
         MSL
     Heading 180
         deg.
                                                                                                                                        Detected and
                                                                           Gained 120                     Detected 2 min Detected and
      Failed Turn                                         Detected and                     Detected and                                 corrected,                             Detected and
                                                                           feet, used last                30 sec, did not checked other                       Detected
      Coordinator                                         corrected                        corrected                                    gained 200                             corrected
                                                                           bank                           notify ATC      gyros
                                                                                                                                        feet
                                         Detected
     Right Turn to
                                         failed turn
       360 deg.,                                          Gained 150
                                         coordinator,
     Standard rate                                        feet
                                         did not calc
       and Clock
                                         correct angle
       Descend at
                                                                                                          Turned right
       500 fpm to
                                                                           Trouble                        instead of left,
     2500 MSL and
                                                                           maintaining                    lost 100 feet
       Left Turn to
                                                                           rate                           copying
      180 deg. in 1
                                                                                                          clearance
         minute
                                                          Detected in 4
                       Noticed after 3
      Oil Pressure                                        min, 30 sec.,    Detected in 2                                                                      Detected in 11 Detected in 40
                       min, 11                                                                                                                Missed
          Drop                                            Checked          min, 30 sec                                                                        min            seconds
                       seconds
                                                          temperature
     Level at 2500
          MSL
      Engine Temp
                                                                                                                                              Missed          Missed
        Increase
     Climb 750 fpm
                                         Leveled out
      to 3000 MSL
                                         while setting
     and Turn Right
                                         heading bug
       to 45 deg..
      Straight and
     Level 45 sec.
                                                                                                                                              Notice
                                                          Detected in 5 Gained 250                                                            imbalance but                    Detected but
      Fuel loss in
                                                          min, 30 sec,    feet trying to                                     Missed           did not       Missed             did not check
      Right Tank
                                                          Did not correct solve                                                               determine                        fuel selector
                                                                                                                                              cause
                                         Detected low
                                         oil, gained
                                         altitude while
      Turn Left to
                                         problem
     225 deg. and                                                                                         Missed
                                         solving,
      descend at                                                                                          heading in
                                         increased
      800 fpm to                                                                                          clearance
                                         descent rate,
       2600 MSL
                                         missed
                                         altitude and
                                         heading
      Straight and
      Level 1 min.
                                         Lost bank
      Climb to 3600
                                         angle and
        MSL using                                         Trouble
                                         vertical speed                                                                                       Started climb
       Clock at 500                                       holding 500                                     Omitted time , Excessive
                                         while setting                                                                                        before full
      fpm, Heading                                        fpm, did d not                                  used 1000 fpm rate
                                         heading bug,                                                                                         clearance
     to 360 deg., in                                      meet time
                                         turned to
        2 minutes
                                         wrong heading
                                                          Detected 1 min
                                                                                                                             Trouble
                                                          30 sec,
                                                                                                          Detected -         holding
                                                          applied heat                                                                                                         Detected after
                                                                                                          applied heat       altitude, didn't Detected after Detected
     Engine Ice up                                        then turned off Detected early                                                                                       2 min 25
                                                                                                          then turned off    realize engine 2 min 30 sec. immediately
                                                          again,                                                                                                               seconds
                                                                                                          again              ice was
                                                          reapplied 3
                                                                                                                             problem
                                                          times
     Heading 225,
                                         Leveled off                                                      Climbed 200                         Trouble
     Climb at 200                                                                                                            Unable to
                                         due to power                                                     feet while                          holding 200
     fpm to 4000                                                                                                             climb
                                         loss                                                             copying                             fpm
         MSL
                                                                                                                                                                                 Was not
                     Noticed after
                                                          Detected in 3                                   Detected,                                                              affected by
                     55 seconds,
     Pitot Tube Ice-                     Detected and     min, Applied     Detected in 2                  Leveled out        Detected and                                        as turned on
                     lost altitude
            up                           corrected for    heat then        min                            and declared       corrected                                           pitot heat
                     while
                                                          turned off                                      emergency                                                              after ATIS
                     responding
                                                                                                                                                                                 brief
      Straight and
      Level 1 min.
                                                                                                                                                              Detected Pitot
     Climb to 5000
                                                                                                                                                              ice problem
     MSL at 110 kts
                                                                                                                                                              here
      Level for 45
                                           Lost 100
     sec, Turn Left                                                                                                          Finally
                                           feet writing
      to 160 deg.                                                                          Missed                            detected
                                           down info,
     and Descend                                                                           heading                           engine ice and
                                           needed
     at 120 to 2500                                                                                                          corrected
                                           repeat twice
          MSL
                                                                         Detected in 20                                                       Lost descent
                       Spun down to                                                                       Detected, lost
        Attitude                                                         seconds,                                                             rate until      Detected and     Detected,
                       300 feet, not                      Detected in 30                                  heading,
       Indicator                                                         entered        Detected                                              detected,       responded        turned off carb
                       fully aware                        seconds                                         leveled out to
        Failure                                                          unusual                                                              declared        correctly        heat
                       why                                                                                trouble shoot
                                                                         attitude                                                             emergency
     Change Radio                                         Waited to do
                                                                                        Lost heading
       Frequency                                          change
     Level at 1 min,
       Slow to 100
          kts.
      Turn Right to
        360 deg..
SA technologies                                                                                                                                            Page 176

                                                                                      Experienced Pilots
                                       Moderate SA                                                                                High SA
                      #1                #2                #6               #4               #5               #7                #8               #9              #10               #3
     (4)
Instrument
 Approach
                                                                   Climbed 300                                        Varied by 100       Did not write
 Approach                                           Lost 400 feet, feet, setup                         Lost 20 deg on feet, missed        down AWOS,
                                                                                      Good briefing
  Briefing                                          used 2000 fpm wrong                                heading        AWOS, lost          repeated 3
                                                                   approach                                           heading             times
                                                                                                       Caught Nav 1
                                                    Pulled VOR
                                                                     Lost heading                      problem,                           Gained 200
             Turned 20            Altitude          rather than
                                                                     doing other                       caught                             feet during
             degrees tuning       fluctuated, Did   localizer,                        Detected                                                                              Did not tune
                                                                     tasks ,                           minimums                           briefing, 20
Fly Approach radial, Did not      not catch         altitude                          minimums                                                                              AWOS, did not
                                                                     Detected                          problem didn't                     deg off
             tune and             minimums          fluctuations,                     problem                                                                               use NAV 2
                                                                     minimums                          dial DME,                          heading, used
             identify             problem           no ID of
                                                                     problem                           assumed lost                       1000 fpm
                                                    localizer
                                                                                                       comm
                                                                                                                                                                            Gained
                Switched to                       Thought 100                                                            Caught OM        Detected and     Detected and
                                                                                                                                                                            altitude, asked
                localizer, Lost                   feet higher                      Detected OM                           failure, used    used marker      used DME,
                                  Lost 400 feet                                                                                                                             for radar ID or
                400 feet while                    than actual,       used DME,     failure,                              DME, used        beacon,          ID'd missed
                                  after detecting                                                 used marker                                                               FAF, missed
 Compass        solving                           got AWOS           missed        Detected DME                          Nav 2, caught    Detected         approach after
                                  problem, did                                                    beacon                                                                    approach time
Locator Fails   problem,                          late, missed       approach time problem, ID'd                         DME loss,        minimums         NAV 1 fail,
                                  not tune                                                        instead                                                                   late, used
                Gained 200                        minimums,          late          missed                                detected         problem, Went    Caught
                                  backup                                                                                                                                    wrong time, did
                feet trying to                    missed after                     approach point                        minimums         missed after     minimums
                                                                                                                                                                            not detect
                get position                      losing Nav 1                                                           problem          losing Nav 1     problem
                                                                                                                                                                            minimums
                                                                     Detected,
Wind Change Detected                                Missed           Descended        Missed           Missed            Detected         Missed           Detected         Detected
                                                                     below MDA
                Inadequate        Inadequate        Inadequate
 General                                                                              Good Aircraft    Good Aircraft     Good Aircraft    Good Aircraft    Good Aircraft    Good Aircraft
                Simulator         Simulator         Simulator
Problems        Control Skills    Control Skills    Control Skills
                                                                                      Control Skills   Control Skills    Control Skills   Control Skills   Control Skills   Control Skills
                Lack of           Lack of
                                                    Aircraft         Aircraft                          Aircraft          Aircraft         Aircraft         Aircraft         Lack of
                Recency in        Recency in
                                                    transition       transition                        transition        transition       transition       transition       proficiency in
                Maneuvers,        Maneuvers,
                                                    problems         problems                          problems          problems         problems         problems         approaches
                Instrument        Instrument
                Scan Pattern                                         Scan Pattern                                        Scan Pattern     Scan Pattern     Scan Pattern
                Deficiencies                                         Deficiencies                                        Deficiencies     Deficiencies     Deficiencies
                                                                                                                                                           Plenty of
                Overload/         Overload/                                                                                                                                 Not
                                                                     Overload                                                                              residual
                Saturation        Saturation                                                                                                                                overloaded
                                                                                                                                                           attention
                                  Attentional
                                                                                                       Attention shift
                                  Narrowing/
                                                                                                       problem
                                  Fixation
                                                                     Multi-Tasking
                Poor              Multi-Tasking     Good task        Problems,        Good task        Good task         Good task        Good task                         Good task
                Prioritization    Problems          prioritization   Poor             prioritization   prioritization    prioritization   prioritization                    prioritization
                                                                     prioritization
                                  ATC Comm
                                                 Ddi not write                                                           Ddi not write    Ddi not write
                                  Problems/
                                                 down                                                                    down             down
                                  Misinterpreted
                                                 clearances                                                              clearances       clearances
                                  Clearances
                                                                     Failure to
                                                                     Diagnose &
                                                                     Problem Solve
                Fatigue                                              Fatigue       Fatigue                               Fatigue          Fatigue                           Fatigue
                                                                                   Ahead of                              Ahead of                                           Good
                                                                                   aircraft                              aircraft                                           projection
                                                    High
                                                    frustration
                                                    level
                Poor Planning
SA technologies                                                                      Page 177


                      Experienced GA Pilot (Flight Instructor) Results



Flight Instructor #1

Instrument Takeoff:

   -     failed to run through a full instrument cockpit check prior to takeoff
   -     set wrong tower frequency
   -     did not set altimeter to current pressure

Task 1

Appeared cool and collected
  - during step 7, subject read back 15 degree angle of bank, but USED the correct angle of
     30 degrees
  - asked for step 8 to be read to him twice
  - asked for step 10 to be read to him twice
  - rolled out at 330 instead of 360 as stipulated by step 10

Comments

This pilot seemed to have a measure of difficulty paying attention to the clearances because he
was coping with the sensitivity of the simulator. In other words, more recent experience with a
sensitive simulator like this would probably have yielded better results.

Task 2
   - completed step 14 twenty five seconds early

Comments:

The subject most likely performed better during this task because he was warmed up. Failure to
complete the descent in exactly two minutes was probably due to both the simulator's sensitivity
and lack of recent experience with such basic BAI maneuvers. In other words, the pilot did not
get saturated throughout the task because he was warmed up and focused on the overall picture
and relative simplicity of the maneuvers and emergency events.

Task 3
SA technologies                                                                           Page 178



   -     Took 3 minutes 11 seconds to detect oil pressure drop
   -     Took 55 seconds to detect step 17 (pitot ice) and lost altitude while responding to the
         event
   -     The attitude indicator was failed during step 20, the subject spun down to 300 feet, the
         subject did not seem fully aware of what had actually caused him to spin. The recovery
         seemed more a matter of lucky handling of the controls rather than a deliberate attempt to
         recover.

Comments

The large amount of time it took for the pilot to detect the oil pressure drop was probably due
more to the location of the engine instruments than it was mental saturation. This type of error
best illustrates the type of oversight seen in "aircraft transition" accidents. The detection of pitot
ice was fairly timely considering how long it takes for it to be detectable in this simulator. The
spin resulting from the attitude indicator was definitely due to a loss of SA. However, the
subject's failure to recognize the attitude indicator's loss was probably due to lack of recent
practice and experience in recognizing such losses. In other words, the failure to recognize the
attitude indicator resulted from both mental saturation and lack of proficiency in BAI (which
includes instrument failure recognition.) The combination of icing, frequent clearances to
different altitudes and frequency/transponder code changes near the end of the flight definitely
precluded the subject from devoting full attention to his basic BAI.

Task 4
   -     turned 20 degrees trying to figure out his location by tuning in a radial
   -     did not tune and identify
   -     switched DME to the localizer rather than Ocala VOR
   -     lost 400 feet while determining DME was set wrong and outer marker was out of service
   -     gained 200 feet at 4.3 DME trying to ascertain position
   -     decided to continue on localizer to missed approach point
   -     caught minimums problem
   -     detected wind change

Comments:

To begin with, the subject's performance was probably degraded by fatigue in this stage of the
simulation. Altitude and heading deviations were due both to lack of planning (as in inadequate
approach briefing and setup) and poor prioritization. These problems can be due to a
combination of overall low experience and lack of recent practice doing approaches. Loss of SA
SA technologies                                                                          Page 179


in the form of altitude/heading deviations (poor BAI) was due to poor prioritization (i.e. setting
different frequencies while correcting an altitude rather than "flying the aircraft first" and then
setting radio frequencies.) This prioritization problem was exacerbated during emergency
situations (such as the compass locator failure,) where the pilot lost 400 feet trying to figure out
how to shoot the approach without the compass locator.

Summary

The subject rated a 5 for SA during entire flight, he kept cool and used good techniques.
The subject rated a 5 for Level 1 SA since he generally kept up with the aircraft at the most
difficult moments.
The subject rated a 4 for Level 2 SA since he did recognize the overall impact of a given adverse
situation, but could have been more proactive.
The subject rated a 4 for Level 3 SA since he basically kept up with the aircraft but was not
particularly proficient at adjusting to rapidly changing conditions.
SA technologies                                                                          Page 180


Flight Instructor #2

Instrument Take Off:

   -      caught runway discrepancy
   -      did full instrument cockpit check
   -      had trouble with rudder control during takeoff, stalled and crashed; fixated on the
          airspeed indicator
   -      agitated with the sensitivity of the simulator
   -      missed turn specified in step one (turn to 315 degree after level off)
   -      turned first and THEN climbed to 3000 feet instead of doing both simultaneously
   -      trouble determining what climb airspeed was best

Comments:

The simulator's sensitivity played a LARGE roll in the crash. A true evaluation of the pilot's loss
of SA is really not possible in this situation.

Task 1:

   -      commenced 110 degree heading change instead of 120 as specified in step 4
   -      during step 5, lost 400 feet while fixating on setting the heading bug for the 180 degree
          heading change, ended up making only a 170 degree heading change




Comments

Task saturation happened almost immediately. This was definitely due lack of proficiency in
BAI and poor technique. The subject, aware of the fact that was getting unorthodox clearances,
seemed preoccupied with setting the heading bug so he wouldn't turn to the wrong heading. This
fixation led to loss of SA in the form of misinterpreting the clearances.

Task 2:

   -      leveled out to execute step 5
   -      worked hard to keep 500 fpm climb rate in step 9
   -      commented on (detected) turbulence in step 13
   -      waited till he was stable before executing step 14
SA technologies                                                                        Page 181


   -   caught step 15 while in step 14's descent, took 2 minutes and 8 secs to detect alternator
       malfunction

Comments

Lack of BAI proficiency and the simulator's sensitivity could account for most of the problems
during this task. The loss of SA in the form of the delay in alternator failure recognition was
probably due to preoccupation with flying the unorthodox clearances. With more practice in this
particular type of simulator, the subject would probably have spotted the alternator quicker.

Task 3
   - caught TC (turn coordinator) malfunction during step 4, used last known bank angle
       instead of calculating correct bank angle for standard rate at his current airspeed
   - leveled out while setting heading bug during step 9
   - caught low oil pressure during step 12 and gained 300 feet while dealing with the event
           o asked for emergency vectors and needed step 12 to be read again
           o increased descent rate in step 12 to 1000 fpm while setting heading bug
           o shot below 2,600 feet as specified in step 12 and went off in his heading
   - lost correct bank and vertical speed while setting heading bug during step 14 and also
       caught fuel imbalance and attempted to fix it using the fuel selector
   - also during step 14, due to fuel imbalance problem and all the activity during step 12,
       only turned 180 degree instead of the specified 360
   - after noticing partial power loss with throttle lever to the stops during step 16, the subject
       leveled out
   - lost 100 feet writing step 20 down, which the subject also need to hear twice
           o fluctuated 200 feet while setting heading bug
           o detected and corrected for pitot ice
           o did not start step 20 till all issues were solved
   - detected AI failure during just after step 22




Comments

The effects of poor BAI proficiency and simulator sensitivity were magnified by the higher
workload in this task. Nearly every emergency event was made worse by the pilot's inability to
keep the aircraft stable while dealing with each emergency event. The subject had barely enough
SA technologies                                                                          Page 182


"residual attention" to pay to the emergency events since most of his focus was on just keeping
the aircraft flying.

   Task 4

   -   tuned and identified
   -   gained 200 feet trying to ascertain location using the VOR
   -   altitude fluctuated while trying to fly the specified clearance to the approach
   -   noted nav flag when nav 1 failed
   -   altitude fluctuated while trying to fly final vector to the approach course
   -   did not catch minimums problem
   -   lost 400 feet after detecting OM problem, but did not tune OCA in ahead of time as
       backup
   -   after all this, subject went missed

Comments

Loss of SA in the form of altitude deviations were again due to poor BAI with was made worse
when the pilot tried to focus both on the approach and on losing the navaids.

Summary

The subject rated a 3 for SA for the entire flight since he kept falling behind the aircraft mostly
due to poor BAI skills.
The subject rated a 3 for Level 1 SA since he had poor BAI.
The subject rated a 3 for Level 2 SA due to poor prioritization, again due to poor BAI.
The subject rated a 4 for Level 3 SA since he was able to stay with the aircraft.
SA technologies                                                                        Page 183


Flight Instructor #3

Instrument Takeoff

   -     did not catch runway discrepancy
   -     did not check turn coordinator during ICC
   -     missed takeoff clearance

Task 1
   - looked for traffic in step 6
   - very steady throughout the task



Comments

The only sign of SA loss during this task was the pilot's response to step 6. He didn't really seem
to think about the peculiarity of a traffic warning while he was in IMC. If he had not been busy
flying and had time to think about it, he probably would have realized his error.

Task 2

   -     missed descent rate when given step 14
   -     caught step 15 and leveled out momentarily to evaluate and correct

Task 3
   -     caught step 3 (failed TC) and used clock to complete std rate turn
   -     caught step 6 in 40 seconds
   -     caught fuel imbalance during step 16 but did think of checking fuel selector
   -     detected step 15 (carburetor ice) after 2:25 seconds
   -     was not affected by step 17 since he turned on pitot heat after hearing temp/dew point
         spread in ATIS
   -     caught AI failure during step 20
   -     turned off carburetor heat after step 21

Comments

The pilot's excellent performance during tasks 2 and 3 was probably due to his excellent BAI
proficiency and which allowed him to stay ahead of the aircraft and prioritize his moves. In
SA technologies                                                                        Page 184


other words, his proficiency provided more "residual attention" for contemplating what was
happening around him.



Task 4

   -     tuned and identified
   -     did not tune AWOS
   -     requested radar vectors after nav 1 dropped out, did not try to use nav 2
   -     gained 200 feet during 360 degree delay vector
   -     after detecting failed OM, asked for radar ID of FAF
   -     started missed approach time late
   -     used 90 knot time while flying approach at 125 knots
   -     did not detect minimum problem
   -     detected wind shift

Comments

Fatigue and lack of proficiency in shooting approaches are what made the difference here. BAI
proficiency is only half the battle during an approach. Things like knowing what your options
are after losing the OM and remembering to check the ATIS against the MDA the first chance
you get are all knowledge and practice related issues. The loss of SA in the form of a mediocre
approach is definitely due to proficiency. In other words, the pilot really had to think hard about
how to respond (considering everything such as regulations and timing) the emergency events.

Summary:
The subject rated a 7 for SA during the entire flight, subject had excellent BAI and was
proactive.
The subject rated a 7 for Level 1 SA due to excellent BAI.
The subject rated a 6 for Level 2 SA since, even though his BAI rarely suffered during
emergencies, prioritization during Task 4 lack a little.
The subject rated a 5 for Level 3 SA due to above average ability to plan ahead.
SA technologies                                                                        Page 185


Flight Instructor #4

Instrument takeoff

   -     missed flaps on preflight checklist
   -     did not do ICC
   -     got flaps on pre-takeoff check
   -     gained 100 feet when told to switch to departure

Task 1

   -     missed heading when given step 4 (had to repeat)
   -     strayed off to 30 degree angle of bank when given step 9 while flying steep turn as
         specified in step 8

Comments

Loss of SA was evident in the form of bad prioritization. The subject lost his bank angle during
the steep turn when he was given the frequency change because he focused his attention on the
frequency change completely rather than dividing it.

Task 2

   -     increased to 1000 fpm during step 9 when given step 8 (transponder code change)
   -     lost 500 fpm descent rate as specified in step 14 when given step 15 (failed alternator);
         subject responded by turning off non-essential instruments and going immediately into a
         power failure procedure; subject eventually decided that recycling the alternator was the
         best way to go

Comments

The descent rate problems were due again to poor division of attention. The unusual reaction to
the alternator failure was probably due more to apprehensiveness on the part of the subject since
he probably figure the alternator would probably be unfixable in a simulator situation.

Task 3

   -     gained 120 feet during step 3 (failed TC) and decided to use last known bank before TC
         failed
SA technologies                                                                        Page 186


   -     had trouble holding 500 fpm descent in step 5
   -     took 2 min and 30 sec to detect low oil pressure (step 6), at which point he also detected
         fuel imbalance (step 11)
   -     gained 250 feet trying to solve fuel imbalance issue, seemed preoccupied with it
   -     noted step 15 (engine ice) during step 14, also had trouble holding 500 fpm climb, which
         clued the subject in on the engine ice
   -     took 2 minutes to detect pitot ice
   -     seemed fatigued during step 20
   -     took 20 seconds to detect step 21, but entered unusual attitude while trying to stay in
         control of the aircraft; subject decided to stop descent and get stable before proceeding



Comments

Time delay before noticing the oil pressure and fuel imbalance problems were probably due to
omission of the engine instrument cluster on the panel (illustrative of the "transition" related
accidents.) The preoccupation with the fuel imbalance issue probably resulted from the subject
being saturated since he now had to include the engine instrument cluster in his scan and adjust
for it. Two minutes to detect the pitot ice was not particularly unusual for this simulator. The
unusual attitude resulted from the 20-second delay in recognizing the attitude failure. The
subject probably took a while to detect the failure since he was fatigued and probably relaxed his
scan.

Task 4

   -     climbed 300 feet while setting up approach
   -     lost heading while getting frequency for AWOS
   -     lost heading while dialing in Jacksonville approach
   -     gained 150 feet following vectors
   -     detected minimums problems
   -     used DME to identify FAF after losing DME
   -     did before landing checklist
   -     tuned and identified
   -     detected wind shift
   -     gained 300 feet while determining what to do after detected nav 1 loss
   -     continued approach even after going full scale
   -     started missed approach time late (appeared as though subject did not even think to set
         time until the nav 1 failed and he decided he might need it)
SA technologies                                                                        Page 187


   -   decided to go down to MAP to "take a peek" without having a way to identify the missed
       approach point
   -   descended below the MDA



Comments

In a nutshell, the approach was unstable and the pilot was more reactive than proactive. Fatigue
and lack of proficiency were definitely contributing factors. These factors resulted in the pilot
being behind the aircraft (very little overall SA.)

Summary

The subject rated a 5 for SA during the entire flight, aside from task 4, the subject had a good
grasp of what was happening and stayed ahead of the aircraft.
The subject rated a 4 for Level 1 SA due to average BAI skills.
The subject rated a 3 for Level 2 SA due to below average prioritization skills.
The subject rated a 3 for Level 3 SA due to poor trend analysis and lack of being proactive.
SA technologies                                                                         Page 188


Flight Instructor #5

Instrument Takeoff
- missed runway discrepancy
- did a prebrief but no run-up
- missed turn direction in takeoff clearance
- set wrong departure frequency initially
- lost 300 feet after frequency change

Comments

This pilot seemed a bit high strung and a little cavalier, this probably contributed to his missing
some of the instructions.

Task 1
- claimed he was keeping his "eyes peeled for the traffic" after step 6
- read back 100 degree for the heading change in step 8, but actually used 180 as specified

Comments

The subject was simply "flying the airplane," he probably knew the traffic warning didn't make
sense, but responded just the same. The bad read back was again probably due to the pilot
rushing through his response so he could get back to flying the clearances.

Task 2

-   good BAI
-   jumbled read back in step 14, but sorted through his own confusion
-   caught alternator quickly and prioritized well

Comments

Overall good BAI proficiency and prioritization (probably due to several hundred hours recent
dual given.)

Task 3

-   caught step 3 (TC failure) and knew the %15 standard rate rule
-   from step 5 through step 10, subject appeared to be with the aircraft, but not ahead of it
SA technologies                                                                           Page 189


-   by step 12, the subject appeared more comfortable and methodical
-   did not hold 200 fpm as specified in step 16
-   caught low oil pressure, fuel imbalance, high engine temperature and partial power loss due
    to step 15 (engine ice) all during step 16; the pilot applied carburetor heat, but not pitot heat
    (there was no pitot ice yet, be he did not anticipate)
-   caught step 17 (pitot ice) and took carburetor heat out during climb, also leveled off during
    climb to determine and fix pitot problem
-   missed heading in step 20, but caught AI failure
-   lost heading when given step 22 (frequency change)

Comments

The pilot caught all engine related problems after the induction ice since this event forced him to
look at his engine instrument cluster. The fact that he had left the engine instrument cluster out
is again illustrative of the common "aircraft transition" error. The beginning of the task went
fairly well as the pilot had fully adjusted to the simulator's sensitivity, which increased his
"residual attention." The heading deviations near the end of the task were probably due to
fatigue.

Task 4

-   tuned and identified, good approach briefing
-   caught minimums problem
-   caught OM failure and used marker beacons to ID FAF
-   caught DME problem related to Nav 1 failure
-   prioritized well and dialed localizer into Nav 2
-   used time to identify missed approach point
-   subject did not catch wind change



Comments

The excellent performance during this approach indicates proficiency and recent practice. The
pilot was ahead of the aircraft, so small emergencies posed little problem.




Summary
SA technologies                                                                      Page 190


The subject rated a 6 for SA during the entire flight since he was proactive and had good BAI
skills.
The subject rated a 6 for Level 1 SA due mainly to his very good BAI skills.
The subject rated a 7 for Level 2 SA since his task prioritization and handling of emergency
situations was superior.
The subject rated a 6 for Level 3 SA due to good trend analysis but did not seem very tuned into
the weather conditions and their possible impact on the flight.
SA technologies                                                                        Page 191


Flight Instructor #6

Instrument Takeoff

-   caught runway discrepancy
-   caught wrong altimeter setting on takeoff roll
-   climbed through 1000 feet (step 1 clearance) by 150 feet
-   climbed to 3000 feet (step 3) at 2500 rpm, subject stated he forgot he was in a Cessna 172

Comments

Pilot appeared very cavalier throughout simulation, as though his poor performance didn't matter
because he wasn't trying hard to begin with.

Task 1

-   noticed TC slightly off after a turn coordinator calibration test during the warm-up
-   kept holding 100 feet above 3000 feet (step 1)
-   responded correctly to traffic alert (step 6), knew he was in IMC
-   used 55 degree angle of bank during steep turn (step 8)

Comments

Aside from some poor BAI (probably due to very little FRASCA time,) overall SA was good.

Task 2

-   subject still held 100 feet high over assigned altitude (3000 feet)
-   when given frequency change in step 5, prioritized BAI first and then changed frequency

Comments

Again, altitude deviation due to poor BAI, but prioritization and overall SA was adequate.




Task 3

-   gained a 150 feet while setting timer for step 4
SA technologies                                                                       Page 192


-   caught TC failure during step 4
-   troubleshot TC after the turn was completed
-   seemed more agitated during step 5
-   subject did not write clearances down at any time
-   subject felt gyro's were "bouncy", possibly due to bad suction
-   detected oil pressure drop (step 6) after 4 min 30 seconds
-   checked temperature in response to low oil pressure
-   gained 100 feet troubleshooting oil pressure problem
-   had trouble leveling off in step 12, inched down the last 100 feet
-   had trouble holding 500 fpm in step 14 and did not reach heading in 2 minutes
-   took 1 min 30 seconds to detect carburetor ice
-   after applying carburetor heat, he turned it off when the power came back up
-   took 5 min 30 seconds to detect fuel imbalance and did not attempt to remedy
-   took 3 min to detect pitot ice and used heat, but turned it off after the problem disappeared
-   held 300 fpm during step 16
-   took 30 seconds to notice AI failure and waited to do frequency change in step 22
-   assumed DG was dead and did not know that the horizontal situation indicator was electric
-   kept applying carburetor heat each time the engine faltered (this happened 3 times)
-   said he had noticed fuel imbalance affects early in the flight but assumed in might have been
    bad rigging or a faulty gauge

Comments

As the emergency events and overall events piled up, the subject's lack of BAI proficiency began
take its toll as he had to devote his attention towards the emergency events and more complex
clearances. This led to the subject becoming upset. His emotional reaction tended to make
things worse as he became more and more frustrated.

Task 4

-   lost 200 feet during approach briefing
-   tuned and identified
-   put the VOR in nav 1 as opposed to the localizer
-   altitude fluctuated a lot while setting frequencies
-   never identified localizer
-   used 2000 fpm descent during first clearance down to approach altitude
-   lost 100 feet tuning frequencies
-   stated that he was 1000 feet higher than he actually was
-   got AWOS late and lost 100 feet while listening to it
SA technologies                                                                         Page 193


-   never slowed down before turning on to the final approach course
-   did not seem to know where he was during the approach
-   identified NDB initially, but not during the approach
-   missed minimums problem
-   missed wind shift
-   went missed after losing nav 1, didn't consider other options for identifying the FAF

Comments

The problems in task 3 carried over into task 4. Lack of BAI proficiency exacerbated all
problems and the already upset emotional state of the subject. Considering the subject's
experience level, it's safe to say that both the lack of BAI proficiency and the subject's emotional
state interfered with his ability to think through and plan the approach.

Summary

The subject rated a 4 for SA during the entire flight, his overall handling of the aircraft was about
average for his experience.
The subject rated a 2 for Level 1 SA due mainly to poor BAI.
The subject rated a 2 for Level 2 SA due very poor task prioritization and inappropriate and/or
slow responses to several of the emergencies.
The subject rated a 3 for Level 3 SA since he seemed be behind the aircraft and slow to pick up
on trends.
SA technologies                                                                           Page 194


Flight Instructor #7

Instrument Takeoff

-   detected backwards ATIS instructions
-   had written down wrong ground frequency
-   no ICC during taxi
-   stopped aircraft on rollout to copy clearance
-   turned at 400 feet as opposed to 1000 feet as specified in step 1

Task 1

-   lost 100 feet copying step 5 clearance
-   declared step 6 traffic "in sight"
-   read back wrong frequency when given step 9 frequency, gained 100 feet while sorting
    through the error

Comments

Overall task prioritization and solid BAI were very good, and expected considering this pilot's
experience level. The BAI went from shaky at first to rock solid. In other words, the pilot
needed just a little warm up. He prioritized his clearances and flew the aircraft first at all times.

Task 2

-   leveled out in step 4 descent while copying step 5 frequency
-   lost 500 feet while setting frequency



Comments

The 500 foot altitude loss was unexpected and quickly corrected, it was the first frequency
change given while the subject was in the middle of a clearance. Though this did not become a
trend, the pilot did have an initial problem with dividing his attention between the frequency
change and flying the clearance. Loss of SA was due to this sudden attention shift.

Task 3

-   took 2 min 30 seconds to notice TC failure
SA technologies                                                                         Page 195


-   turned right instead of left in step 5
-   lost 100 feet copying step 5 clearance
-   did not notify ATC of TC failure, contrary to regulations
-   missed heading in step 12 clearance
-   omitted time in step 14 climb and used 1000 fpm climb rate
-   stated that he forgot he had a clock in the cockpit
-   climbed 200 feet while copying step 16 clearance
-   caught pitot ice, leveled out and declared an emergency
-   caught carburetor ice and used carburetor heat until the power came back and then turned it
    off
-   detected AI failure and lost heading, leveled out at 3,500 on the way up to 5,000 feet while
    troubleshooting

Comments

Almost all problems here are related to aircraft transition. Since the pilot has spent so much time
out of small aircraft cockpits, he left the engine cluster out of his scan. BAI definitely fell off a
little as the workload increased, but the prioritization was still excellent.

Task 4

-   lost 20 degree on the assigned heading while doing approach briefing
-   tuned and identified
-   caught Nav 1 problem
-   caught minimums problem
-   didn't dial DME in for Nav 1
-   assumed he was lost communications since he had dialed in the wrong approach frequency
-   used marker beacon in lieu of failed OM
-   missed wind shift




Comments

Overall, the subject flew a good approach and stayed well ahead of the aircraft. The wrong
frequency could again be traced back to lack of recent experience with the small rudimentary
cockpit. Again, this is a transition issue.
SA technologies                                                                       Page 196


Summary

The subject rated a 7 for SA during the entire flight because he flew a near perfect profile and
was cool and proactive.
The subject rated a 6 for Level 1 SA due to some faulty BAI (i.e. losing 100 feet while copying
clearances.)
The subject rated a 6 for Level 2 SA since he could have been more in tune with the changing
weather conditions, but his prioritization skills were excellent.
The subject rated a 6 for Level 3 SA due lack of adjustment to this simulator. At his experience
level, he was adversely affected by his experience in larger aircraft. In other words, he had a
hard time readjusting to being in a smaller aircraft. However, as an experienced pilot and flight
instructor, he should have been able to adjust faster.
SA technologies                                                                        Page 197


Flight Instructor #8

Instrument Takeoff

-   used wrong transponder setting
-   missed runway discrepancy
-   did complete ICC
-   shot through 1000 feet (step 1) by 250 feet

Comments

Considering overall performance, these seemed to be warm-up errors.

Task 1

-   subject seemed a little apprehensive, declared turbulence when there wasn't any
-   correctly responded to step 6 traffic alert
-   prioritized flying the plane and waited to change frequencies
-   subject rarely, if ever, wrote clearances down

Task 2

-   caught alternator failure and responded smoothly and methodically, excellent BAI




Comments (Tasks 1 and 2)

Excellent performance, probably due to recent experience with BAI, approaches and knowledge
of what we could do to him in the simulator. This subject also has a great deal of time in the
FRASCA. These attributes gave him excellent SA since he was completely ahead of the aircraft
and was able to prioritize and deal with changing situations without it becoming taxing.

Task 3

-   caught TC failure and double checked other gyro's, excellent task prioritization
-   little trouble holding 800 fpm descent in step 12
-   commented that the increase in instructions increased the workload
SA technologies                                                                         Page 198


-   used excessive climb rate in step 14
-   had trouble holding altitude in step 16 due to engine ice, but did not realize engine ice could
    have been a factor
    o subject appeared fatigued
    o declared to ATC that he was unable to climb at 500 fpm as per regulations
    o announced he was unable to climb at all
    o at this point, he caught the pitot ice problem (step 17) and corrected it
-   as the aircraft descended in step 20, the subject caught the engine ice problem and correct it
-   the subject completely missed fuel imbalance and high cylinder temperature

Comments

BAI suffered a little at this point, probably due to fatigue. This seemed to be related to motor
skills only, if he was off, he knew it (no loss of SA.) However, he did omit the engine
instruments, even after he detected the induction ice. This is probably an aircraft transition error
again. In other words, loss of SA (as in omitting the engine cluster) resulted from the subject not
being used to the location of the engine instrument cluster.

Task 4

-   varied 100 feet on approach set up
-   tuned an identified
-   missed AWOS
-   ran checklist
-   lost heading while getting AWOS (when he was told he could get one)
-   caught OM failure (NDB)
-   used DME to identify
-   set missed approach time on descent
-   used nav 2 for localizer
-   noticed the loss of DME when he lost nav 1
-   caught minimums problem
-   caught wind shift



Comments

Small BAI losses due to fatigue. No overall SA loss, probably due to high proficiency in all
areas.
SA technologies                                                                      Page 199


Summary

The subject rated a 7 for SA during the entire flight due to superior BAI.
The subject rated a 6 for Level 1 SA due to good BAI, some marginal deviations near the
beginning and end of simulation kept the rating below 7.
The subject rated a 7 for Level 2 SA due to excellent task prioritization.
The subject rated a 6 for Level 3 SA due to excellent BAI and overall trend analysis, but he was
slightly out of tune with the engine instruments and performance.
SA technologies                                                                       Page 200


Flight Instructor #9

Instrument Takeoff

-   turned pitot heat on after hearing conditions in ATIS
-   no ICC
-   did run-up
-   did not write down clearances (only did so on task 4)

Task 1

-   good BAI
-   stated he was in IMC in response to traffic warning (step 6)

Task 2

-   missed frequency in step 5, needed it read again
-   continued to fly step 14 as he troubleshot alternator

Comments (Tasks 1 and 2)

Overall experience, BAI proficiency and prioritization contributed to the excellent performance
and good SA during these 2 tasks.

Task 3

-   detected TC failure and used speed formula to calculate std rate, but gained 200 feet while
    recognizing failure
-   started step 14 climb before he fully understood the clearance
-   noticed imbalance of airplane, but did not connect that to possible fuel imbalance
-   had trouble holding 200 fpm in step 16
-   took 2:30 seconds to detect carburetor ice and correct it
-   lost descent rate in step 20 until he recognized AI failure and detected an emergency
-   missed fuel imbalance, high engine temperature and oil pressure

Comments

Only real SA problem in this task was related to the omission of the engine instruments. Again,
this is probably an aircraft transition related problem.
SA technologies                                                                       Page 201



Task 4

-   did not write AWOS down and listened 3 times
-   tuned and identified
-   detected OM failure and used marker beacon
-   gained 200 feet during approach briefing and went off 20 degree on heading
-   lost 150 feet doing checklist (approach)
-   used 1000 fpm descent to MDA
-   went missed when he lost Nav 1
-   caught minimums problem
-   missed wind shift

Comments

BAI problems probably fatigue related. No real loss of SA other than the failure to detect the
wind shift. This failure is probably due to the pilot's preoccupation with continuing the approach
after the OM and Nav 1 died. If he had been concentrating fully on the approach, he might have
caught the wind shift.

Summary

The subject rated a 7 for SA during the entire flight due to good BAI and overall awareness
through each phase of the flight.
The subject rated a 6 for Level 1 SA due to excellent BAI.
The subject rated a 7 for Level 2 SA since he was proactive and well ahead of the aircraft.
The subject rated a 7 for Level 3 SA due to his ability to adjust to changing conditions and
response to trends such as ice induced power loss.
SA technologies                                                                     Page 202


Flight Instructor #10

Instrument Takeoff

-   subject missed runway discrepancy

Task 1

-   subject looked for traffic during step 6
-   subject had excellent BAI skills

Task 2

-   subject caught alternator problem and notified ATC

Comments (Tasks 1 and 2)

The subject's overall SA was excellent, due to proficiency and excellent BAI, dealing with
emergencies on the side was no challenge.

Task 3:

-   caught TC failure
-   took 11 minutes to catch oil pressure drop
-   missed increased engine temperature
-   missed fuel imbalance
-   caught engine ice almost immediately
-   caught pitot ice during step 19
-   caught AI failure (step 21) and responded appropriately

Comments

The only loss of SA was related to the omission of the engine instruments. This omission was
probably an aircraft transition issue.

Task 4

-   tuned and identified
-   caught failed OM
SA technologies                                                                        Page 203


-   decided to use DME before nav 1 failed
-   after nav 1 failed, subject decided to use time for missed approach point identification
-   detected wind change
-   caught minimums problem

Comments

The overall approach went very well. Excellent BAI and proficiency in shooting approaches
(especially in a FRASCA) were contributing factors. The subject had plenty of "residual
attention" left over to deal with the emergencies and continue the approach safely.

Summary

The subject rated a 6 for SA during the entire flight due to his methodical approach, which
served him well during emergencies.
The subject rated a 7 for Level 1 SA due to superior BAI skills.
The subject rated a 6 for Level 2 SA due to excellent task prioritization and coolness under
pressure.
The subject rated a 6 for Level 3 SA since he handled emergencies well, but did not see trends
with ice-related engine problems.

								
To top