multitasking - improving CTM by panniuniu

VIEWS: 5 PAGES: 45

									                   Aviate
                    Navigate
                     Communicate
                       Manage Systems




 Improving Cockpit Task
Management Performance:
       The AgendaManager
 Training Pilots to Prioritize Tasks
                                                  Aviate


 Observation: Cockpit Task                         Navigate
                                                   Communicate
                                                     Manage Systems


   Management Errors                             CTM


• Cockpit (flight deck) is a multitask environment
   –   aviate
   –   navigate
   –   communicate
   –   manage systems
• Results of distraction, preoccupation
   – Everglades L-1011 accident
   – many incidents
• Hypotheses:
   – flightcrew must manage as well as perform tasks:
     Cockpit Task Management (CTM)
   – CTM is a significant factor in flight safety
                                                                      2
                                                       Aviate


     Preliminary Normative                              Navigate
                                                        Communicate
                                                          Manage Systems


        Theory of CTM                                 CTM

•   initiate tasks to achieve goals
•   assess status of all tasks
•   terminate completed and „obsolete‟ tasks
•   prioritize remaining tasks based on
    – importance:
        •   aviate
        •   navigate
        •   communicate
        •   manage systems
    – urgency
    – other factors (?)
• allocate resources (attend) to tasks in order of priority
                                                                           3
                                                              Aviate


  Cockpit Task Management                                      Navigate
                                                               Communicate
                                                                 Manage Systems


          Research                                          CTM


• CTM Errors in Aircraft Accidents (1991)
   – 80 CTM errors in 76 (23%) of 324 accidents
• CTM Errors in Critical, In-Flight Incidents (1993)
   – 349 CTM errors in 231 (49%) of 470 incident reports
• Part-Task Flight Simulator Study (1996)
   – CTM error rate increases with workload
• ASRS Study of CTM and Automation (1998)
   – Task prioritization error rate higher in advanced technology reports
• Findings:
   – CTM is a significant factor in flight safety
   – CTM can potentially be improved
                                                                                  4
                              Aviate
                               Navigate
                               Communicate
                                 Manage Systems




                            CTM




Improving CTM Through Technology:


        The
   AgendaManager
                                                  5
                                                     Aviate


  Statement of Needs and                              Navigate
                                                      Communicate
                                                        Manage Systems


  Requirements Definition                          CTM


• CTM aid shall
  – maintain a current model of aircraft state and current
    cockpit tasks,
  – monitor task state and status,
  – compute task priority,
  – remind the flightcrew of all tasks that should be in
    progress, and
  – suggest that the flightcrew attend to tasks that do not
    show satisfactory progress.
  – leave the pilot in control

                                                                         6
                                               Aviate
                                                Navigate
                                                Communicate

          System Analysis                         Manage Systems



                                              CTM


• Generic, twin-engine transport aircraft
   – major subsystems
      •   power plant
      •   fuel system
      •   electrical system
      •   hydraulic system
      •   adverse weather system
      •   autoflight system
      •   flight management system.
   – state variables of importance to pilot
•  specifications for simulator
                                                                   7
                                                             Aviate

Basic and Detailed Design of                                  Navigate
                                                              Communicate
                                                                Manage Systems



  The AgendaManager                                        CTM

• Object-Oriented Design
   – things & activities from IDEF0 models  objects
• Multi-Agent Approach
   – AMgt functions are complex, cognitive functions  AI
   – AMgt is complex interplay of many entities  DAI
      •   System Agents
      •   Actor Agents
      •   Goal Agents
      •   Function Agents
      •   Agenda Agent
      •   Agenda Manager Interface
• Display Design
   – general display design guidelines  alternative display designs
   – consistency with EICAS  final display design
                                                                                 8
      AMgr Architecture and
           Function
                                                             information flow
                  AMgr display                               satisfactory functions
    Pilot                                                    unsatisfactory functions
                   Verbex ASR                                conflicting goals


                                         reduce to 240 kt Goal & Function Agents

                                             maintain 070 deg G & F Agents
                Flightcrew Agent
                                            descend to 9,000 ft G & F Agents
   Aircraft       Aircraft Agent
                                             reduce to 240 kt G & F Agents
  Autoflight    Autoflight Agent
                                             maintain 070 deg G & F Agents

                                            descend to 8,000 ft G & F Agents
   L Engine      L Engine Agent         extinguish L ENGINE FIRE G & F Agents

  Hyd System    Hyd System Agent          restore C HYD PRESS G & F Agents

 Fuel System    Fuel System Agent        correct FUEL BALANCE G & F Agents
System Models   System Agents               Goal & Function Agents
 Simulator                          AgendaManager
                Aviate


  Simulator      Navigate
                 Communicate
                   Manage Systems


(with EICAS)   CTM




                                    10
                    Aviate


  AMgr Display       Navigate
                     Communicate
                       Manage Systems


(replaced EICAS)   CTM




                                        11
                                            Aviate
                                             Navigate
                                             Communicate

        AMgr Operation                         Manage Systems



                                           CTM


•   simulator runs
•   pilot declares goals via ATC acknowledgements
•   System & Actor Agents instantiate Goal Agents
•   Goal Agents watch for goal conflicts
•   Function Agents assess function status
•   AgendaManager informs pilot via display




                                                                12
AgendaManager Display Design




extinguish L engine fire   not OK -> continuing


descend to 7,000 ft        A/F alt goal conflict
maintain 070 deg
slow to 240 kt             fast -> accelerating

correct fuel balance       L heavy -> increasing
        extremely important,
            urgent goals
                                          trend info
          (highest priority)
                                                               aviate goals
                                                              (high priority)
           extinguish L engine fire   not OK -> continuing


           descend to 7,000 ft        A/F alt goal conflict
           maintain 070 deg                                        system goals
           slow to 240 kt             fast -> accelerating        (lower priority)

           correct fuel balance       L heavy -> increasing



gray = OK
amber = not OK
red = important/urgent not OK
Initial Conditions:
     altitude = 15,000 ft
     heading = 120 deg
     speed = 280 kt
     all systems normal




  maintain15,000 ft
  maintain 120 deg
  maintain 280 kt
ATC: “... descend and maintain 11,000 ft”
pilot: “Roger, “... descend and maintain 11,000 ft”
       sets A/F altitude to 11,000 ft
       descent begins




  descend to 11,000 ft     high -> descending
  maintain 120 deg
  maintain 280 kt
ATC: “... turn left heading 070”
pilot: “Roger, “... turn left heading 070”
       begins turn
       levels off at 11,000 ft




  maintain 11,000 ft
  turn L to 070 deg          right of -> turning L
  maintain 280 kt
pilot: rolls out on 070 deg
AMgr:detects fuel imbalance & displays it




  maintain 11,000 ft
  maintain 070 deg
  maintain 280 kt

  correct fuel balance    L heavy -> unbalancing
pilot: begins fuel crossfeed
ATC: “... descend and maintain 9,000 ft; reduce
        speed to 240 kt”
pilot: “Roger ... descend and maintain 9,000 ft;
        reduce speed to 240 kt”
        sets altitude to 9,000 ft, descent begins
        reduces throttles, aircraft slows



  descend to 9,000 ft       high -> descending
  maintain 070 deg
  slow to 240 kt            fast -> slowing

  correct fuel balance      L heavy -> balancing
AMgr: detects left engine fire
pilot: “... we have a problem ...”
ATC: “... descend and maintain 7,000 ft”
pilot: “Roger ... descend and maintain 7,000 ft”
        mis-sets altitude to 6,000 ft
        speed increases

  extinguish L engine fire   not OK -> continuing


  descend to 7,000 ft        A/F alt goal conflict
  maintain 070 deg
  slow to 240 kt             fast -> accelerating

  correct fuel balance       L heavy -> balancing
fire out
speed controlled
pilot: sets A/F to 7,000 ft
       forgets to secure crossfeed when fuel balanced




  maintain 7,000 ft
  maintain 070 deg
  maintain 240 kt

  correct fuel balance    R heavy -> unbalancing
                                                      Aviate
                                                       Navigate
                                                       Communicate

    Test and Evaluation (1)                              Manage Systems



                                                  CTM


• Objective: compare AMgt performance (AMgr vs EICAS)
• Apparatus
   – flight simulator
   – AMgr
• Subjects: 8 line pilots
• Scenarios:
   – EUG to PDX
   – PDX to Eugene
• Primary factor: monitoring and alerting condition
   – AMgr
   – EICAS
                                                                          22
                                                                    Aviate
                                                                     Navigate
                                                                     Communicate

    Test and Evaluation (2)                                            Manage Systems



                                                                  CTM


• General Procedure
   –   subject introduction
   –   automatic Speech Recognition system training
   –   flight training (using MCP)
   –   subsystem training (fault correction)
   –   EICAS/AMgr training
• Trials
   – Scenario 1 (EICAS/AMgr)
        • experimenter/ATC controller gives clearances, induces faults, induces
          goal conflicts
        • subject acknowledges clearances, flies simulator, corrects faults,
          detects and resolves goal conflicts
   – Scenario 2 (AMgr/EICAS)

                                                                                        23
                                                                 Aviate
                                                                  Navigate
                                                                  Communicate

         Evaluation Results                                         Manage Systems



                                                               CTM



Response variable                               AMgr   EICAS sig. level
within subsystem correct prioritization         100%    100%      NS
susbsystem fault correction time (sec)          19.5    19.6      NS
autoflight system programming time (sec)         7.0     5.9      NS
goal conflicts corrected percentage             100%    70%      0.10
goal conflict resolution time (sec)             34.7    53.6     0.10
subsystem/aviate correct prioritization         72%     46%      0.05
average number of unsatisfactory functions      0.64    0.85     0.05
percentage of time all functions satisfactory   65%     52%      0.05
mean subject effectiveness rating (-5 to 5)      4.8     2.5     0.05



                                                                                     24
                                                   Aviate
                                                    Navigate
                                                    Communicate

           Conclusions                                Manage Systems



                                                  CTM


• CTM is a significant factor in flight safety.
• CTM can be facilitated (e.g., AMgr).
• Future success of knowledge-based avionics depends
  on a systematic approach to development:
   – systematic identification of problems, needs, and
     opportunities
   – appropriate application of appropriate technology
   – evaluation of systems based on operationally relevant
     performance measures

                                                                       25
                             Aviate
                              Navigate
                              Communicate
                                Manage Systems




                           CTM




Improving CTM Through Training:


 Training Pilots to
  Prioritize Tasks
                                                 26
                                               Aviate


       Research                                 Navigate
                                                Communicate
                                                  Manage Systems


Motivation and Objective                     CTM



• Is task prioritization trainable?
• Evidence suggests that voluntary control of
  attention is a trainable skill
   – e.g., Gopher (1992)
• Objective
   – Develop and evaluate a CTM training program to
     improve task prioritization performance.
                                                                   Aviate
                                                                    Navigate
                                                                    Communicate
                                                                      Manage Systems


                            Methodology                          CTM


• Participants
        – 12 General Aviation pilots, IFR rated, with at least 100 hrs “pilot-
          in-command” total time.
        – Recruited through flyers and word of mouth
        – Oregon State (Corvallis, Albany, Salem, Eugene, Portland)

• Apparatus: Microsoft Flight Simulator 2000
        – 3 monitors, Flight Yoke, Throttles, and Rudder Pedals
        – IFR conditions
        – Two flight scenarios
             Aviate


Lab Setup     Navigate
              Communicate
                Manage Systems



            CTM
                       Aviate


Participant Display     Navigate
                        Communicate
                          Manage Systems


    (C-182RG)         CTM
                          Aviate


Experimenter‟s Display     Navigate
                           Communicate
                             Manage Systems



                         CTM
                                        Aviate


    Experimental Groups                  Navigate
                                         Communicate
                                           Manage Systems



                                       CTM




• Control Group: No Training
• Descriptive Group: CTM lecture
      •   Multi-tasking
      •   Attention
      •   CTM
      •   Task Prioritization errors
      •   Accident/Incident examples
      •   What to be aware of.
• Prescriptive Group:
   – CTM lecture
   – “APE” procedure
                                                                 Aviate


             APE:                                                 Navigate
                                                                  Communicate
                                                                    Manage Systems


    Assess Prioritize Execute                                   CTM


•    Let the APE help you                                                 A P E
     – Assess the situation:
         •   aircraft systems, environment, tasks, procedures
         •   “What‟s going on?” “What should I be doing?”
     – Prioritize your tasks:
         1. Aviate: “Is my aircraft in control?”
         2. Navigate: “Do I know where I am and where I‟m going?”
         3. Communicate: “Have I communicated or received important
            information?”
         4. Manage systems: “Are my systems okay?”
     – Execute the high priority tasks Now.
•    Invoke the APE frequently.
•    Think out loud.
                                           Aviate


 Experimental Procedure                     Navigate
                                            Communicate
                                              Manage Systems



                                          CTM



1.   Initial briefing, informed consent
2.   Initial 30-minute simulator training
3.   Pre-training flight
4.   CTM training (break for control group)
5.   Additional 30-minute simulator training
6.   Post-training flight (different scenario)
7.   Post-experiment questionnaire
                                                  Aviate


   Dependent Measures                              Navigate
                                                   Communicate
                                                     Manage Systems



                                                 CTM



• Task prioritization error rate
   – 19 Task prioritization challenges, e.g.
      • clearance near end of climb
      • “bust” altitude? (+/- 200 ft)
• Prospective memory recall rate
   – 5 Memory recall challenges (prospective memory), e.g.,
      • “report crossing SHONE [intersection]”
      • remember to report?
                                        Aviate


        Data Collection                  Navigate
                                         Communicate
                                           Manage Systems



                                       CTM



•   Flight Data Recorder
•   Videotape
•   Observation
•   Data reduction to:
    – task prioritization error rate
    – prospective memory recall rate
                                                           Aviate


      Results: ANOVA                                        Navigate
                                                            Communicate
                                                              Manage Systems


(task prioritization error rate)                         CTM




   Effect          df     SS       df   MS       F-ratio       p-value


  Group            2    .1914125   9 .0799194   2.395               .147
  Flight           1    .2053500   9 .0088806   23.123                 .001
  Group x Flight   2    .0429125   9 .0088806    4.832                 .038
                                                                   Aviate


       Interaction Plot                                             Navigate
                                                                    Communicate
                                                                      Manage Systems


(task prioritization error rate)                                  CTM

                                   Interaction Plot
              0.7                                                                      Group
                                                                                         Co
              0.6   Prescriptive                                                         De
              0.5                                                                        Pre
 Error Rate




              0.4   Descriptive
              0.3
              0.2
                        Control
              0.1
               0
                         Pre Training             Post Training
                                         Flight
                                                      Aviate


       Results: ANOVA                                  Navigate
                                                       Communicate
                                                         Manage Systems


(prospective memory recall rate)                     CTM




   Effect          df   SS     df   MS     F-ratio             p-value


  Group            2    .017   9    .028    .603                  .568
  Flight           1    .074   9    .034   2.181                  .174
  Group x Flight   2    .171   9    .034   5.055                  .034
                                                                Aviate


        Interaction Plot                                         Navigate
                                                                 Communicate
                                                                   Manage Systems


(prospective memory recall rate)                           CTM




                                       Interaction Plot
                   1                                                                Gr
                                                                                     C
                  0.9                                                                D
   Memory Tasks




                                                                                     P
                  0.8      Control


                  0.7   Descriptive


                  0.6
                        Prescriptive

                  0.5
                           Pre T raining       Post T raining
                                           Flight
                                            Aviate


          Paired t-tests                     Navigate
                                                Communicate
                                                 Manage Systems



                                           CTM


• Prescriptive training group improved
     • Task prioritization error rate
     • Prospective memory recall rate
• Descriptive training group improved
     • Task prioritization error rate
• Control group did not significantly improve
                                                    Aviate


           Discussion                                Navigate
                                                     Communicate
                                                       Manage Systems



                                                  CTM



• Task Prioritization Error rate
   – Reduced, perhaps, due to (Prescriptive) CTM training.
   – Significant interaction and post-hoc tests support that
     hypothesis.
• Prospective Memory Recall rate
   – Increased, perhaps, due to (Descriptive & Prescriptive) CTM
     training.
   – Significant interaction and post-hoc tests support that
     hypothesis.
                                                                   Aviate


    Possible Interpretations                                        Navigate
                                                                    Communicate
                                                                      Manage Systems



                                                                 CTM



•    Results may have two interpretations:
    1.   CTM training did improve task prioritization performance.
    2.   CTM training did not improve task prioritization.
         •   Floor effect
         •   MSFS experience
         •   Age
•    Research favors first interpretation
         •   ANOVA results
         •   t-tests
         •   Potential for better control group performance was there.
         •   Additional tests
                                          Aviate


      Final Comments                       Navigate
                                           Communicate
                                             Manage Systems



                                         CTM




• CTM performance significant to flight safety
• Results are encouraging
• Evidence suggests that task prioritization is a
  trainable skill
• Follow-up experiment underway to resolve
  ambiguities
• If successful, would provide evidence that CTM
  training can reduce risk of CTM errors and
  subsequent accidents
                              Aviate


    The Cockpit Task           Navigate
                               Communicate
                                 Manage Systems


   Management Website        CTM




http://flightdeck.ie.orst.edu/CTM/




                                                  45
           The AMgr: a KBS

								
To top