Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out

AD4 Technical results and achievements

VIEWS: 4 PAGES: 31

									                             Space plus Time
                         Investigations: a 3D Air
                       Situation Display to support
                       Controllers in Approach and
                              Tower sectors


Speaker:
                                                                                        Luigi Mazzucchelli
Antonio Monteleone                                                                      AD4 Project Coordinator
AD4 Technical Manager                                                                   NEXT Ingegneria dei Sistemi
                                                                                        Luigi.Mazzucchelli@next.it
NEXT Ingegneria dei Sistemi
                                                                                        Antonio Nuzzo
Antonio.Monteleone@next.it                                                              CNS/ATM Experimental Centre
                                                                                        ENAV SpA
                                                                                        ANuzzo@enav.it




           Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop
                                                                                                                         1
Summary

 The AD4 Project
   Background and objectives
   The AD4 3D Air Situation Display
 Experimental activities in AD4
   Evaluations in the APPROACH sector
    (APP RTHLS)
   Evaluations RTHLS in the AIRPORT sector
    (TWR RTHLS)
   Experiments in the use of AR in the Control
    Tower
 Conclusions and Future Work


      Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   2
AD4:    4D Virtual Airspace Management System


   European Commission VIth FP,
    DG-Research
   NEXT Coordinator
   Duration: 26 months
   Start: January 2005
   Budget: 4M€
   Partners: ENAV, Vitrociset,
    SICTA, Middlesex University,
    Digital Video, Fraunhofer
    Focus, ObjectSecurity,
    SpaceApplications, ESI
   Applicative areas: Virtual
    Reality 3D/4D, ATM, 3D/4D
    Simulations, Human factors,
    MDA, Security

                                                          AD4 day, December 5 2006
                                                    5th EUROCONTROL Innovative Research
                                                     Workshop & Exhibition - Parallel event

         Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   3
Air Traffic Control: current radar displays

 Air Traffic Management is basically a 4D (3D + time)
  problem.
 Controllers work with radar
  displays (2D displays) that
  provide them with a planar
  representation of airspace
  and traffic.
      Absence of (analogical)
       visual cues on
        aircraft altitude
        vertical separation
        volume constraints



 Controllers build 3D airspace models in their mind and
  perform complicated 4D mental computations (i.e. 3D mental
  representations of foreseen evolutions).
        Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   4
Current investigations

 3D-ATC
    Linköping University - NVIS
    EEC



 3D in 2D Planar Display Project, CARE INO-III
    Middlesex University - IDC
    Space Applications Services
    NEXT Ingegneria dei Sistemi
     (Industrial Advisor)



       Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   5
The AD4 Operational Concept

 The AD4 project proposes
  the use of Virtual Reality
  and Augmented Reality
  techniques to provide a
  Virtual 3D Air-Space
                                                                                          +
  representation for ATM
  systems where 3D real
  time interaction with the air
  traffic/airport space is
  accessible to controllers.


 Focus on
   Approach
   Airport Environment




        Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   6
The AD4 Platform




      Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   7
The AD4 3D Radar Display
 The AD4 project has developed a novel 3D Air
  Situation Display.
 3D perspective
  display of airspace,
  airdromes and traffic
    constraints
     (terrain,
     radar minima,
     military restricted
     airspaces, clouds)
    airspace elements
     (waypoints,
     airways, holding
     stacks,
     landing paths, ...)
    airdromes



          Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   8
The AD4 3D Radar Display




Airdrome 3D representation, with different views of the
   same scenario (i.e. landing path)

        Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   9
Experimental activities in AD4
                                                                                                           Define validation

 Evaluations
                                                                                                         aims, objectives and
                                                                                                             hypotheses.



    Validation objectives                                                                                    Prepare the
                                                                                                          validation plan and

      Safety
                                                                                                            exercise runs.



          Workload and Situational Awareness                                                            Execute the exercise
                                                                                                            runs and take

      Human Factors
                                                                                                           measurements.



          Usability and Acceptability                                                                     Analyse results.



    Real Time Human-in-the-loop small                                                                       Develop and
                                                                                                             disseminate

     scale simulations
                                                                                                             conclusions.
                                                                                                                 VDR


      Controllers actively involved into the experiments in
       a simulated operational environment where traffic
       evolves
          Presence of pseudo-pilots and use of voice-communication
 Proof of concepts

       Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop              10
RTHL simulations
 For the APPROACH 4DHMI
      SICTA simulation room at the
       Airport of Naples from 25th to 27th
       November 2006 (including
       training).
      Four active ENAV air traffic
       controllers from the Naples
       Approach Sector
      Executive controller
      AD4-ACE integrated platform

 For the AIRPORT 4DHMI
      SICTA premises in Naples the 24th
       and 25th January 2007 (including
       training).
      Four active ENAV air traffic
       controllers from the Naples
       Control Tower
      AD4-ATRES integrated platform


         Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   11
Simulated airspace
 Sectors of interest
      Naples APP,
       including LIRN
       Airport and R62
       and R63 military
       areas
      Naples TWR
       (LIRN Airport)




        Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   12
APP RTHLS: Simulation Environment




      Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   13
APP RTHLS: Hypothesis and metrics
Low Level        Hypothesis/                             Metrics                                  Data collection
Objectives       Outstanding design                                                               method or tool
                 questions
Safety
Situational
                 What aspect of 3D
Awareness                                                Subjective Rating Scale                  SASHA questionnaire
                 support SA
(SA)
                 How does 3D affects
Workload         controller cognitive                    ISA rating Scale                         ISA
                 workload
Usability
                 How 3D information
Ease of use      supports controllers                    Subjective Rating Scale                  Interview Guide
                 work
Acceptability
            Controller feel the new
Perceived   system support
                                                         Subjective Rating Scale                  CAR rating scale
Performance adequate or improved
            level of performance

              Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   14
APP RTHLS results: situation awareness

Overall perceived situation awareness was rated
  “good” or “quite good”
   Controllers spent
    more time looking at
    the 2D radar display
    than at the innovative
    3D display
   3D display was mainly
    used to focus on
    specific situations
    identified in the 2D
    radar display
   Why?
      Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   15
APP RTHLS results: situation awareness

 3D displays can difficultly guarantee the overall
  (global) picture provided by 2D radar displays
    Perspective distortion
       i.e. difficult to judge relative distance
    Occlusions
       force to navigate to discover hidden portions




 but ...

       Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   16
APP RTHLS results: situation awareness

 … they can provide more
  details when focusing on
  limited portions of space
 During the experiments 3D
  was mainly used locally
    to check results of
     clearances
    to anticipate deviations
     from the assigned path
 This suggests that 3D
  improves understanding of
  local traffic situation

       Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   17
APP RTHLS results: usability issues

 Side by side display configuration
 Lack of coordination
  among displays is an
  issue
 Extra-effort when
  switching display and
  matching elements

 3D camera view-point and navigation
   Free (birds eye) navigation can be time consuming
   Other forms of navigation and/or display formats
    are mandatory
      Preset 3D views and/or map-based 3D navigation
      In-place 3D displays
       Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   18
APP RTHLS results: acceptability
1    Improvement mandatory. Safe operation could not be maintained using 4D display
2    Major deficiencies. System is barely controllable and only with extreme controller
           compensation
3    Major deficiencies. System is marginally controllable. Considerable compensation is needed
           by the controller
4    Major deficiencies. System is controllable. 4D display does not compromise safety. Some
           compensation needed to maintain safe operation. Adequate performance not
           attainable
5    Very objectionable deficiencies. Maintaining adequate performance requires extensive
           controller compensation
6    Moderately objectionable deficiencies. Use of 4D display requires considerable compensation                            3
          to achieve adequate performance
7    Minor but annoyng deficiencies. Desired performance requires moderate controller                                       1
           compensation
8    Only mildly unpleasant deficiencies. System is acceptable and minimal compensation
          needed to meet desired performance. Use of 4D display enhances performance
9    Negligible deficiencies. System is acceptable and minimal compensation needed to meet
           desired performance. 4D display significantly increases performance
10   Deficiencies are rare. System is acceptable, and controller does not have to compensate to
            achieve desired performance

     This score indicates that improvement of the system is needed. However
          3D system was perceived by the controllers as usable only at the cost of
           time consuming interactions (usability issue)
          Controllers were not confident in using the system because they felt under
           trained
              Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop       19
APP RTHLS results: a summary
Summarizing …
 Experiments in AD4 have highlighted that
    perspective 3D displays helped controllers in improving
     local situation awareness
    classical 2D radar displays provide an overall picture
     (global situation awareness) not guaranteed by 3D
     displays
    controllers were interested in pointing the 3D camera view
     in specific positions (localizer, holding stack, ...)
 Such considerations suggest
    to further explore the use of 3D displays in ATC by
     focusing on the integration of “3D displays in 2D displays”
    to develop an HMI where it is possible to rapidly choose a
     desired 3D point of view and focus from there on specific
     situations (in the 3D display) to get awareness of what is
     happening locally: rapid zooming technique
    … but the training period must be adequate to the amount
     of innovation introduced (THREE days are not sufficient)
        Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   20
3D-2D combined displays in AD4
 Multiple Coordinated Views display technique
   Focus + context problem
   Rapid zooming techniques
                            Combination Display Framework (Rozzi et al. 2007)




      Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   21
TWR RTHLS: Simulation Environment




      Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   22
TWR RTHLS: Hypothesis and metrics

Low Level       Hypothesis                               Metrics                                  Data collection method
Objectives                                                                                             or tool

Safety

Situational
                What aspect of 3D
Awareness                                                Subjective Rating Scale                  SASHA questionnaire
                support SA
(SA)

Usability

                How 3D information
Ease of use                               Subjective Rating Scale                                 Interview Guide
                supports controllers work

Acceptability

                    Controller feel the new
                    system support
Perceived
                    adequate or                          Subjective rating Scale                  CARS rating scale
Performance
                    improved level of
                    performance



              Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   23
TWR RTHLS results: situation awareness

 3D displays offer a global view of the traffic more
  informative than the out-of-the-window view
    3D displays can reproduce tower scenarios in 3D by
     adding synthetic information (i.e. identification labels,
     distance to touch down, visual alerts to, etc.).
    3D representation and navigation capabilities allow
     overcoming visibility (distance, weather conditions,
     lighting …) and occlusion problems.




        Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   24
TWR RTHLS results: situation awareness

 3D displays are potentially
  able to provide a “all-in-
  one” visualization approach
  of the airport environment
      runway status
      runways ends
      distance between aircraft and
       to touch down
      position of hidden traffics
 They allow to emulate out of
  the window visual scan
  (head rotation) with 3D
  camera navigation (rotation)
      It can help in reducing the
       visual effort in overseeing the
       runway
        From 180 degrees to few
         tens of degrees

         Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   25
TWR RTHLS results: acceptability
     Item                                                                                                                   Response
1    Improvement mandatory. Safe operation could not be maintained using 4D display                                         1
2    Major deficiencies. System is barely controllable and only with extreme controller
           compensation
3    Major deficiencies. System is marginally controllable. Considerable compensation is needed
           by the controller
4    Major deficiencies. System is controllable. 4D display does not compromise safety. Some
           compensation needed to maintain safe operation. Adequate performance not
           attainable
5    Very objectionable deficiencies. Maintaining adequate performance requires extensive
           controller compensation
6    Moderately objectionable deficiencies. Use of 4D display requires considerable compensation                            4
          to achieve adequate performance
7    Minor but annoyng deficiencies. Desired performance requires moderate controller
           compensation
8    Only mildly unpleasant deficiencies. System is acceptable and minimal compensation
          needed to meet desired performance. Use of 4D display enhances performance
9    Negligible deficiencies. System is acceptable and minimal compensation needed to meet
           desired performance. 4D display significantly increases performance
10   Deficiencies are rare. System is acceptable, and controller does not have to compensate to
            achieve desired performance

      … but again the training period were not adequate to the
      amount of innovation introduced (TWO days are not sufficient)

              Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop          26
3D Displays and (remote) Virtual Towers
 Virtual Tower is a ground-level facility that is remotely
  located from the airport(s) to which it provides
  arrival/departure and ground movement services.
    ATM personnel have no direct window view from the tower
 Why Interest in Virtual Towers
    Security
    Avoid cost of building towers

 3D Displays can
  provide 3D Virtual
  Window Views
  (synthetic out-of-the
  window view)
  augmented with other
  sorts of information
        Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   27
The AD4 AR experiment

 AR systems provide a
  combination of the real
  scene viewed by the user
  with a virtual scene
  generated by the computer.

 The real scene is
  augmented with
  synthetic elements
  conveying additional
  information: aircraft
  identification, runway
  status, …
       Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   28
The AD4 AR experiment: feedback

 Augmented Reality was positively judged as it provided
  more information compared to the normal (non-
  augmented) “out-of-the-window” view .
  In particular, the following benefits were outlined by
  controllers:
    easier identification of aircraft/vehicles moving on the
     airport surface, thanks to the accompanying labels;
    easier understanding of position of aircraft/vehicles in all
     parts of the airport, regardless of occlusion or other
     visibility factors;
    easier understanding of motion (including direction and
     speed), especially for aircraft/vehicles in very far areas
     (e.g. at the holding bay), thanks to the moving labels.




        Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   29
Conclusions
 In the APP sector
    3D displays alone cannot guarantee the overall picture
     provided by 2D planar displays …
    … but they can provide an added value when integrated
     with 2D displays since they enable an improved local
     situation awareness
    Next step is investigating “ATC 2D-3D combination
     displays”
 In the TWR sector
    3D is potentially able to provide added benefits
       3D displays are able to reproduce tower scenarios by
        adding synthetic information
       Next steps is
                investigating effectiveness of 3D displays compared to last
                 generation Surface Working Positions
                Exploiting 4DHMI displays within the Virtual Tower Concept
    Early feedback from controllers on AR has been positive
       low visibility, occlusions and visual identification of
        vehicles are some of the addressable problems.
        Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   30
                                                  http://www.ad4-project.com


                                          Contact:              Antonio Monteleone
                                                                AD4 Technical Manager
                                                                Antonio.Monteleone@next.it
                                                                NEXT Ingegneria dei Sistemi

                                                                Luigi Mazzucchelli
                                                                AD4 Project Coordinator
                                                                NEXT Ingegneria dei Sistemi
For more details                                                Luigi.Mazzucchelli@next.it
please join us at the                                           Antonio Nuzzo
Exhibition                                                      CNS/ATM Experimental Centre
                                                                ENAV SpA
                                                                ANuzzo@enav.it

      Monteleone, Mazzucchelli, Nuzzo, "Space plus Time Investigations - Paper 34" - 6th Eurocontrol INO Workshop   31

								
To top