A Simulation Evaluation of a Human Centered Approach to Flight ...

1 A Simulation Evaluation of a Human Centered Approach to Flight Deck Procedures and Automation for En Route Free Maneuvering Walter W. Johnson and Vernol Battiste NASA Ames Research Center Stacey Granada, Quang Dao, and Nancy Johnson San Jose State University NASA Ames Human Factors Research & Technology Division April 20, 2005 2 Outline of Presentation En Route Free Maneuvering Concept Approach Goals Simulation Environment Method Experimental Design and Task AFR Flight Rules Ames Pilot Participants Ames Flight Deck Results Separation Management Meter Fix Conformance Workload & Subjective Opinion Conclusions NASA Ames Human Factors Research & Technology Division April 20, 2005 3 En Route Free Maneuvering Concept The En Route Free Maneuvering concept was designed to extend present airspace capacity by adding „extra‟ “autonomous” aircraft into the present en route airspace. Autonomous aircraft would – Fly according to Autonomous Flight Rules (AFR) – Have tools allowing them to maintain separation from AFR and traditionally managed IFR aircraft. – Manage to meter fix constraints in the presence of traditionally managed IFR aircraft – Have access to adequate surveillance information on the location and intent of all surrounding aircraft NASA Ames Human Factors Research & Technology Division April 20, 2005 4 Approach In June 2004, research teams at the NASA Ames and NASA Langley Research Centers conducted a joint human-inthe-loop test of the feasibility of the En Route Free Maneuvering concept. The test – Simulated flight decks at both centers and simulated ATC at Ames. – Was flow in a modified portion of the airspace in and around ZFW and Dallas/Fort Worth TRACON. This presentation only deals with the results from the flight decks simulated at NASA Ames Research Center NASA Ames Human Factors Research & Technology Division April 20, 2005 5 Goals of The En Route Free Maneuvering Study Evaluate whether autonomous operations in a mixed en route and transition airspace are feasible and scalable • Mixed – Traditional managed aircraft and equipped autonomous aircraft share the same airspace • Feasible – Accommodates basic procedural, workload, and safety considerations • Scalable – The number of en route aircraft can be significantly increased beyond present day limits NASA Ames Human Factors Research & Technology Division April 20, 2005 6 Ames Simulation Environment Air Traffic Control Center and TRACON Simulation IFR Multi-aircraft Pseudopilot Stations AFR High Fidelity Flight Simulator AFR Desktop Pilot Participant Stations AFR Multi-aircraft Pseudopilot Stations NASA Ames Human Factors Research & Technology Division April 20, 2005 7 Ames AFR Aircraft Equipage 3D Cockpit Situational Display • Ownship and traffic • State and 4D intent information • Fast-time predictors • Manipulable 3D views. • Strategic conflict detection and alerting • Automated conflict burdening symbology • FMS integrated • Flight path re-planning tool w/ alerting • RTA management tool ADS-B: • AFR & IFR: Broadcast state and intent (full flight plans) at 0.5 Hz • AFR: Receive state and intent at 0.5 Hz MACS Aircraft Simulator • • • • • 757-like aircraft model 777-like interface FMS MCP with integrated alerting Primary flight display NASA Ames Human Factors Research & Technology Division April 20, 2005 8 Experimental Design Autonomous Managed 4 Conditions: Baseline IFR: L1 traffic, all managed Mixed 1: L1 traffic, mixed operations Mixed 2: L2 traffic, mixed operations Mixed 3: L3 traffic, mixed operations Traffic Level L2 L3 T1 L1 L1 T0 Baseline IFR Mixed 1 Mixed 2 Mixed 3 Experimental Condition T0: Threshold approximating current day monitor alert parameter. T1: Threshold above which managed-only operations may become unmanageable. Traffic counts to achieve these levels were established in simulation. NASA Ames Human Factors Research & Technology Division April 20, 2005 9 Task Meter fix RTA is uplinked automatically to the autonomous aircraft at the freeze horizon. The RTA provides a merge slot for TRACON entry. Autonomous aircraft are responsible for solving all conflicts. Center Center controllers are only alerted to imminent (<= 4 min.) autonomous-managed conflicts. If an autonomous aircraft cannot meet RTA, flight crew must alert controller ASAP. Workload permitting, the controller will accommodate by assigning new RTA, relaxing meter fix altitude or speed constraints, or sending aircraft across different fix. Transfer of communication uplink message is sent automatically to autonomous aircraft. Flight crew is instructed to “monitor” not “contact” next frequency.. Autonomous and managed aircraft merge at the meter fix. All aircraft are managed in the TRACON. TRACON NASA Ames Human Factors Research & Technology Division April 20, 2005 10 Autonomous Flight Rules Autonomous Flight Rules were designed to ensure concept safety, and low impact on ATC operations. The AFR flight rules stated that AFR pilots – May choose their own route and altitude. – Must ensure separation of ownship from all other aircraft (5 NM lateral, 1000 ft vertical). – Must give way to managed (IFR) aircraft. – When burdened, must resolve a predicted conflict prior to 2 minutes to loss of separation (LOS). (AFR Rules of the Road, outlined in a separate presentation, determined which aircraft was burdened.) – Must not create near-term conflicts (< 4 minutes) with any aircraft when maneuvering or changing flight modes. NASA Ames Human Factors Research & Technology Division April 20, 2005 11 Autonomous Flight Rules (cont.) – Must conform to arrival clearance at entry to TRACON • RTA +/- 15 sec • Altitude +/- 300 ft • Speed +/- 10 kts – Must notify ATSP if arrival clearance constraints cannot be met and request an amended clearance. NASA Ames Human Factors Research & Technology Division April 20, 2005 12 Ames Participants • Ten air transport rated pilots with – experience in several previous DAG studies. – 4,500 to 22,000 flight hours (mean ~ 11,000) – 85 to 8,000 glass cockpit hours (mean ~ 4000) • Eight pilots flew eight single pilot workstations. • Two flew the full mission flight simulator, functioning as a two-pilot crew. NASA Ames Human Factors Research & Technology Division April 20, 2005 13 Ames Flight Deck Simulation Results • Separation Management Metrics • Meter Fix Conformance Metrics • Subjective Ratings NASA Ames Human Factors Research & Technology Division April 20, 2005 14 Separation Management Metrics The metrics used to evaluate separation management were based on AFR Flight Rules • Losing Separation • Creating Conflicts • Resolving Conflicts • Burdening (Rules of the Road) • Good Practice • Resolution Time Less than 2 min NASA Ames Human Factors Research & Technology Division April 20, 2005 15 Separation Violations None NASA Ames Human Factors Research & Technology Division April 20, 2005 16 Conflict Detection and Resolution Times -12 -10 -8 -6 -4 -2 0 0 Time to LOS When Conflict Resolved (Min) Arrival Not Burdened Arrival Burdened Overflight Not Burdened Overflight Burdened -2 • 89% of all conflicts resolved within 2 minutes of detection • Almost all near term conflicts (under 4 minutes) were arrivals. • 4 due to AFR subject pilots • 3 software malfunctions • 1 procedural error • 2 AFR failures to resolve by 2 minutes • 1 software malfunction • 1 procedural error -4 -6 -8 -10 -12 • Note immediate resolutions on the main diagonal Time to LOS When Conflict Detected (min) NASA Ames Human Factors Research & Technology Division April 20, 2005 17 Creating Conflicts 60 50 Less than 4 Minutes Greater than 4 Minutes Frequency 40 30 20 10 0 Mixed L1 Mixed L2 Mixed L3 AFR Flight Rules Prohibit Maneuvers which Create a Conflict with Less than 4 Minutes to LOS Ames AFR participant aircraft violated this rule in 4 out of 139 conflicts. Of these • Three violations were due to software errors Condition Times Prior to Projected LOS When Conflict First Detected • One violation was a procedural error NASA Ames Human Factors Research & Technology Division April 20, 2005 18 Resolving Conflicts 60 50 40 Under 2 Minutes Between 2 and 4 Minutes Greater than 4 Minutes Frequency AFR flight rules require that burdened aircraft resolve all conflicts prior to 2 minutes to LOS. AFR participant aircraft violated this rule in 2 out of 123 conflicts where the participant aircraft was burdened. • One violation was due to a conflict detection software flaw. • The second violation was a procedural error associated with the aforementionned late maneuver into conflict incident 30 20 10 0 Mixed L1 Mixed L2 Condition Mixed L3 Time to LOS at Which Conflicts were Resolved NASA Ames Human Factors Research & Technology Division April 20, 2005 19 Burdening 90 80 70 Burdened Resolves Ownship Resolves Frequency 60 50 40 30 20 10 0 Intruder Resolves Non-Burdened Resolves While less than 7% of IFR aircraft resolved IFR-AFR conflicts (IFR was never burdened), it is noteworthy that almost 1/3 of the nonburdened AFR aircraft resolved the AFR-AFR conflicts. AFR-IFR IFR AFR Always Burdened AFR-AFR AFR (Burdened) AFR-AFR AFR (Not Subject Pilot Subject Pilot Burdened) Not Burdened Burdened Intruder Burdening Number of conflict resolutions as a function of burdening NASA Ames Human Factors Research & Technology Division April 20, 2005 20 Meter Fix Conformance Metrics Speed Restriction Altitude Restriction RTA Restriction NASA Ames Human Factors Research & Technology Division April 20, 2005 21 Meter Fix Speed Compliance 300 290 280 270 All but one Ames AFR participant aircraft met the speed restriction. Speed Conformance Zone Airspeed (kts) Airspeed (kts) 260 250 240 The exception asked for and received a revised clearance. 230 220 IFR Multi Pilot CDTI AFR Pilot CDTI IFR Pilot 210 200 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Run Number NASA Ames Human Factors Research & Technology Division April 20, 2005 22 Meter Fix Altitude Compliance 12500 12200 11900 Altitude (ft) 11600 All but one Ames AFR participant aircraft met the meter fix altitude restriction Altitude Conform ance Zone 11300 Altitude (ft) 11000 10700 10400 10100 IFR Multi Pilot CDTI AFR Pilot CDTI IFR Pilot 9800 9500 0 2 4 6 8 10 12 14 16 18 Run Number NASA Ames Human Factors Research & Technology Division April 20, 2005 23 Meter Fix RTA Compliance 40.00 30.00 IFR Multi Pilot CDTI IFR Pilot CDTI AFR Pilot RTA/STA Deviation (sec) 20.00 10.00 All Ames AFR participant aircraft met the meter fix RTA restriction RTA Conformanc e Zone 0.00 -10.00 -20.00 -30.00 0 1 2 3 4 5 6 7 8 Run 9 10 11 12 13 14 15 16 NASA Ames Human Factors Research & Technology Division April 20, 2005 24 Subjective Ratings NASA Ames Human Factors Research & Technology Division April 20, 2005 25 Workload Modified Cooper Harper Ratings OR YES Easy Was workload satisfactory without reduction? Workload is low Very Easy Workload Insignificant 1 2 3.5 Arrival 3.0 Overflight NO Fair Enough sp are capacity for easy attention to additional tasks. Workload Rating 3 2.5 2.0 Mental workload is high and should be reduced. Minor, Annoying Insufficient spare capacity for easy attention to additional tasks. 4 1.5 YES OR Was workload tolerable for this task? Moderately Objectionable Reduced spare capacity. Additional tasks cannot be given the desired attention 5 1.0 C1 Low All Managed C2 C3 Low Moderate Condition Mixed Mixed C4 High Mixed NO Very Objectionable Little spare capacity. Level of effort allows little attention to additional tasks. 6 Condition 7 YES Major deficiencies; redesign is strongly recommended. Major difficulty Very little spare capacity, but maintenance of primary task still possible. Workload was higher for • Mixed vs Managed conditions • Arrivals vs Overflights Major difficulty Was it possible to complete the task? OR NO Major difficulty Very high workload with almost 8 no spare capacity. Difficulty main training level of effort. Extremely high workload. No spare capacity. Ability to main tain effort on primary task doubtful. 9 START Major deficiencies; redesign is mandatory. Impossible Task abandoned. Unable to apply sufficient effort 10 But was well within the satisfactory range. There was also no evidence that workload increased with traffic density. April 20, 2005 NASA Ames Human Factors Research & Technology Division 26 AFR-IFR Operations Preference The majority of pilots preferred AFR over IFR operations for • Safety • Workload • RTA • Situational Awareness NASA Ames Human Factors Research & Technology Division April 20, 2005 27 Conclusions From an airborne perspective this autonomous flight operation is • • • • Feasible Scaleable Safe Efficient The concept provided • Complete intent • Good situational awareness • Low workload decision support tools However, future research is required to determine which of these is truly critical to the concept NASA Ames Human Factors Research & Technology Division April 20, 2005