Hermes brochure by jlhd32

VIEWS: 127 PAGES: 24

More Info
									                                       2 0 0 6   –   2 0 0 9
                                       F i n a l     P r o j e c t   R e p o r t
     t
     · cf
          ES
         M
       ER




            H

                     .4
                  1:1
                              m
                          50 m




                                  Hermes
                                    Human Expressive Representations of Motion
                                    and their Evaluation in Sequences




                                  http://www.hermes-project.eu




hermes-1.indd 1                                                             10/09/2009 19:37:14
hermes-1.indd 2   10/09/2009 19:37:14
                                       The HERMES Project                                                                                    3



                        “HERMES concentrates on how to extract descriptions of human
                       behaviour from video sequences, in a restricted discourse domain.”




                  „   We confront agent, body and        „   The sensing and reasoning            „   The system has been exposed to
                      face motion evaluation, in sce-        processes are tightly integrated         video recordings from different
                      narios ranging from wide-field-        in a perception-action cycle: co-        parts of Europe. Its explanatory
                      of-view with multiple-agent            operating pan-tilt-zoom sensors          and arguing capabilities assess
                      scenes to specific inferences of       enhance cognition via respons-           its strengths and weaknesses.
                      emotional state.                       es to uncertain/ambiguous in-
                                                             terpretations.




                                                                         Project Nº                         027110
                                                                         Duration                           36 months (+6 months extension)
                                                                         Start Date                         01/03/06
                                                                         End Date                           28/02/09 (extended to 31/08/09)
                                                                         European Commission Funding        2 100 000 , 00 €
                                                                         Hermes homepage                    http://www.hermes-project.eu




                                                                                                 Juan José Villanueva
                                                                                                    Project Manager



                                                                                                                                     H ead of H ermes
                                                                                                             Istanbul arcHaeologIcal museum , turkey




hermes-1.indd 3                                                                                                                         10/09/2009 19:37:21
       4                                     Executive Summary



                                                Hermes Project                             Objectives
                                                HERMES is a consortium project that        The main objective of HERMES is to de-
                                                concentrates on extracting descriptions    velop a cognitive artificial system to al-
                                                of people behaviour from videos in re-     low both recognition and description
                                                stricted discourse domains, such as pe-    of a particular set of human behaviours
                                                destrians crossing inner-city roads, ap-   arising from real-world events. Specifi-
                                                proaching or waiting at stops of buses     cally, we model knowledge about the
                                                and even humans in indoor worlds like      environment to suggest interpretations
                                                halls or a lobbies.                        from motion events, and to communi-
                                                These video recordings allow us to ex-     cate with people using natural language
                                                plore a coherent evaluation of human       texts and synthetic films. The events
                                                movements and facial expressions           are detected in image data-streams
                                                across a wide variation of scale.          obtained from arrays of multiple active
                                                                                           cameras (including zoom, pan and tilt).
                                                A system has been developed that starts
                                                with basic knowledge about pedestrian      The HERMES procedure combines:
                                                behaviour in chosen discourse domains,      „   Detection and tracking of agents
                                                but eventually clusters evaluation re-          while they are some distance away
                                                sults into semantically meaningful sub-         from a location.
                                                sets of behaviours.
                                                                                            „   If they are even closer and their
                                                                                                face can be resolved, facial emo-
                                                                                                tions are checked.




                         “We base on the three stages of human behaviour: motion of people,
                            their posture and gestures, and their facial characterization.”




     H ermes , sculpture of szeroka street
     toruń , poland




hermes-1.indd 4                                                                                                                  10/09/2009 19:37:27
                                                                                                                                    5



                                           “Natural language texts and synthetic animation are
                                                  used to communicate with end-users”




           Approach                                     Animation is accommodated within
                                                        HERMES based on the following consid-
           HERMES fulfils two main objectives: on       erations:
           the one hand, the generation of concep-       „   Analysis-by-synthesis at the three
           tual descriptions based on acquired and           stages of human behaviour, i.e.
           analysed motion patterns. On the other            motion of people, their posture
           hand, the communication using visuali-            analysis, and their face characteri-
           zation of synthetic motion patterns.              zation.
           Natural language text generation is ac-
                                                         „   Animated computer graphics as a
           commodated within HERMES based on
                                                             visual language to quickly commu-
           these considerations:
                                                             nicate essential aspects..
             „    Semantic descriptions enable re-
                                                         „   Animated computer graphics,
                  searchers to check details of the
                                                             again at three motion categories,
                  conceptual knowledge base.
                                                             for checking the underlying con-
             „    They also allow communication              ceptual knowledge base.
                  with end-users in a natural manner.   Using both approaches, quantitative
             „    They support conceptual abstrac-      measures and qualitative descriptions
                  tion, thereby facilitating the com-   have been developed to analyze the ro-
                  munication of short messages or       bustness and efficiency of the proposed
                  essential details.                    cognitive system.




                                                                                                    sandalbInder H ermes untyIng HIs sandal
                                                                                                                    louvre museum , france




hermes-1.indd 5                                                                                                               10/09/2009 19:37:31
       6                                        Contents

                  Executive summary                                                          4–5


                  Contents                                                                     6


                  HERMES Consortium                                                            7


                  HERMES Workpackages                                                        8–19
                  WP0: Project management                                                      10
                  WP1: Definition of requirements                                              11
                  WP2: Distributed camera system                                               12
                  WP3: Evaluation of agent motion in video sequences                           13
                  WP4: Evaluation of body motion in video sequences                            14
                  WP5: Evaluation of face motion in video sequences                            15
                  WP6: Software integration for generation of natural language texts           16
                  WP7: Software integration for generation of virtual environments             17
                  WP8: Validation and demonstration                                            18
                  WP9: Exploitation and dissemination                                          19


                  Showcasing Activity                                                       20–21


                  Expertise of the HERMES partners                                          22–23


                  Finances                                                                    24




                                                              c aduceus (Wand of H ermes)




hermes-1.indd 6                                                                                     10/09/2009 19:37:35
                  HERMES Consortium                                                       7


                              Computer Vision Center
                                        CVC                              Spain
                         Universitat Autònoma de Barcelona

                   Institut für Algorithmen und Kognitive Systeme
                                          IAKS                         Germany
                                  Universität Karlsruhe

                  Computer Vision and Media Technology Laboratory
                                       CVMT                            Denmark
                                 Aalborg University

                            Computer Vision Laboratory
                                      BIWI                            Switzerland
                                   ETH Zürich

                              Active Vision Laboratory
                                         AVL                        United Kingdom
                                University of Oxford


                               Answare Technologies                     Spain




hermes-1.indd 7                                                                      10/09/2009 19:37:44
       8


     WP0                                              Project Management


                                                       Project
                                                       Manager
                                                   Juan José Villanueva


                                     Steering                     Management and
                                    Committee                     Exploitation Office



                                    Technical
                                   Coordinator                               Objectives
                                 Andrew D. Bagdanov                          The objective of WP0 is the project or-
                                                                             ganization and the management of the
                                                                             project activities. The main objectives
                                   Management                                are to initiate the project phases, to
                                   Coordinator                               complete all the necessary administra-
                                                                             tive tasks required by the Commission,
                                     Tonny Velin                             and to provide short progress reports
                                                                             and the agreed deliverable results to the
                                                                             EC on due time.
                                    Scientific
                                   Coordinator
                                   Jordi Gonzàlez

                                                              Work Package leaders




            WP0   WP1       WP2      WP3           WP4         WP5        WP6        WP7         WP8          WP9




                  WP0–1 leader     Juan José Villanueva
                                   juanjo@cvc.uab.cat
                                     WP leader:

                                     WP leader institution:




hermes-1.indd 8                                                                                                   10/09/2009 19:37:44
                                                                                                                            9


          Definition of requirements                                                         WP1
            Objectives
            WP1 synthesizes the requirements that
            define the technology to be developed
            in the context of HERMES. Due to the
            risk inherent to the complexity of the
            developed system, the work in WP1 es-
            tablishes a realistic set of requirements,
            assumptions and restrictions to fulfil the   Agent                    Body                        Face
            project objectives and also to allow po-     Motion                   Motion                      Motion
            tential applications (of interest to part-
            ners) while taking into account the lim-
            ited resources allocated.

            Activities & achievements
            The defined requirements were elabo-
            rated, completed, synthesized, and
            harmonized to assure a complete and
                                                              Human expressive representations of motion...
            consistent output for use in the devel-
            opment, design, and implementation
            phases. An ontology was also defined
            to determine what the system is even-
            tually going to say; which videos were                              Validation
            to be recorded; what had to be select-
            ed, tracked, and converted into natural
            language output; and which questions
            were to be answered by the system.                    From Images                From Texts
                                                                    To Texts                 To Images
            Since the output of this WP was input
            for all other WPs, it detailed and con-
            textualized the global requirements for
            each workpackage scope. The discourse
            domains were specified and also the
            locations for recording human agents.
            The considered scenarios provide a
            common test input for all partners, but
            deliberately allowing them for leeway to
            record additional videos for testing.                  ...and their evaluation in sequences




hermes-1.indd 9                                                                                                        10/09/2009 19:37:44
    10


     WP2                                                                       Distributed
                                                                             Camera System

    Objectives                                     Activities & achievements                      „   We made progress on information
                                                                                                      theoretic means to control a set of
    WP2 provides a platform to acquire video       In addressing these two goals, the con-            cameras. The key idea is that lim-
    data for use in WP3, WP4, and WP5, and         sortium has contributed on three ways:             ited sensing resources should be
    investigates the link between high-level                                                          directed to where they will yield the
                                                    „   We developed an architecture for              highest “information” gain.
    understanding and sensing action. Its               multi-camera systems comprising
    goals are: to provide an architecture from
    which to serve video data for WP3, WP4,
                                                        asynchronously communicating,            Future directions
                                                        heterogeneous cameras. An SQL
    and WP5; and to couple high-level scene                                                      We have just scratched the surface of pos-
                                                        database is used for data archiving
    interpretation with sensing actions.                                                         sibilities in the feedback from high-level
                                                        and inter-camera communication.
                                                                                                 inference to sensing process. An imme-
                                                    „   We interfaced the multi-camera           diate step will be to link our information
                                                        system to WP6’s inference engine         theoretic control into a live system as an
                                                        and collaboratively created Situa-       internediate layer between inference and
                                                        tion Graph Trees to describe inter-      action. We also expect our ideas to ex-
                                                        esting activities in scenes.             tend naturally to other cognitive systems.




     HERMES multi-camera architecture              Collaborative distributed cameras (static and active)




                      WP2–3 leader                 WP Reid
                                                 Ian D.leader:
                                                 ian@robots.ox.ac.uk
                                                   WP leader institution:




hermes-1.indd 10                                                                                                                       10/09/2009 19:37:45
                                                                                                                                           11

       Evaluation of Agent Motion
           in Video Sequences                                                                               WP3


                                                                             Tracking by segmentation

            Objectives                                      reasoning about the scene. This infer-
                                                            ence has, in turn, been used for natural
            WP3 aims to track agents and note their tra-    language generation (WP6), generation
            jectories and other coarse scale features, in   of virtual agents (WP7), and camera selec-
            order to generate conceptual descriptions       tion/control (WP2, WP8).
            about the agents and their relationships
            among them and with other scene objects.        Future directions
            Activities & achievements                       Incorporating high-level knowledge with
                                                            low-level tracking data is challenging.
            At the heart of most surveillance systems       Current tracking systems typically operate
            is the requirement to track targets. We         in states of perpetual surprise; algorithms
            developed new algorithms for figure-            try hard not to lose the target, and little
            ground segmentation-based tracking,             intelligence is used in reacquistion. By con-
            and integrated a real-time implementa-          trast, if an activity pattern emerges, a sys-
            tion into the final demonstrator system.        tem –like a human– can afford periods of
            We have also been at the forefront of           deliberate inattention, having a sufficiently
            tracking-by-detection methods, publish-         rich model to be confident of reacquisition.
            ing at IEEE CVPR, ICCV, and BMVC, and           Enumerating all possible situations in a
            winning a best-paper prize at CVPR 2007.        SGT rapidly becomes infeasible for even
            WP3 is about more than tracking: close          small numbers of agents or vaguely com-
            collaboration allowed us to integrate an        plex scenes. A key challenge will be to
            inference engine with the agent-based           capture scene knowledge by learning, in-        Multiple target tracking using person
            tracking data, to conduct simple real-time      stead of hand-crafting the SGTs.                                    detectors as input




hermes-1.indd 11                                                                                                                        10/09/2009 19:37:45
     12


     WP4                                    Evaluation of Body Motion in
                                                 Video Sequences

                                                                                    Activities & achievements
                                                                                    WP4 activities concentrate on:
                                                                                     „   collecting relevant test data sets,
                                                                                     „   pose estimation, and
                                                                                      „ action recognition.
                                                                                    Pose estimation is relevant on its own
                                                                                    (e.g., attentional focus via head pose),
                                                                                    and also a basis for action recognition.
                                                                                    A multitude of actions such as waving,
     Head-pose estimation                                                           boxing, or running are recognized by
                                        Objectives                                  algorithms that have different foci re-
                                                                                    garding their goal: view-invariance, real-
                                        This WP is the core of action evaluation:   time, recognition rate, etc., leading to
                                        already knowing the rough position of       distinct approaches based on features,
                                        the body, derived from WP2 and WP3, it      silhouettes, or depth. The algorithms
                                        supports both the recognition and the       closely integrate with WP3’s segmenta-
                                        animation of body postures. The pose        tion and tracking, yielding comprehen-
                                        and motion of the body is modelled, es-     sive systems.
                                        timated, and associated to verbs such as
                                        walking, standing, or bending.              Future directions
     Recognized action: ‘Running’                                                   Future methods on segmentation and
                                                                                    tracking should be more robust against
                                                                                    partial occlusion, background clutter,
                                                                                    and problematic lightning conditions.
                                                                                    It is also a challenge to enhance robust-
                                                                                    ness by combining multiple cameras
                                                                                    and using depth data. Lastly, context
                                                                                    in action recognition has shown to be a
                                                                                    promising avenue to follow.




                         WP4 leader   Thomas B. Moeslund
                                       WP leader:
                                      tbm@imi.aau.dk
                                       WP leader institution:




hermes-1.indd 12                                                                                                          10/09/2009 19:37:50
                                                                                                                              13

    Evaluation of Face Motion in
         Video Sequences                                                                              WP5
           Objectives                                   Several tools and methods for 2d and 3d
                                                        face tracking and face alignment have
           The focus of WP5 is on the develop-          been developed that allow for auto-
                                                        matic processing of facial motion in real-
           ment of algorithms for facial expression
                                                        time. The systems are designed to proc-
           processing and the analysis of facial mo-    ess a wide range of input streams from
           tion in continuous image streams. The        low-resolution 2d image data to 2.5d
           main issues that we addressed are au-        data acquired by active or passive stereo
           tomatic face processing, real-time per-      systems. A head pose tracker has been
           formance, and several levels of detail to    developed in collaboration with WP4.
           meet the various requirements of end-        Several levels for the generation of facial
           users. Other aspects are the modeling        motion descriptions have been consid-
           and generation of emotion descriptions       ered, ranging from low-level speaking
           which serve as input for the animation       and non-speaking classification over
           of virtual faces.                            speech recognition to high quality facial
                                                        emotion estimation.
           Activities & achievements                    In cooperation with WP7, a virtual face
                                                        model has been designed based on the
                                                        MPEG-4 standard to visualize the gen-
           A system for emotion modeling has
                                                        erated emotion descriptions. Further-
           been specified for distributed camera        more, a system has been developed that
           systems (WP2) with pan-tilt-zoom sen-        accurately aligns a 3d template face to
           sors controlled by the tracked agent mo-     2.5 data. By pre-recording the basic fa-
           tions (WP3). For evaluation, a database      cial expressions, the system tracks and
           containing image sequences of human          transfers the expressions of a human to
           emotions has been built and made avail-      a facial avatar in real-time.
           able to the scientific community.




                                                                                                      Face alignment for the analysis
                                                                                                                  of 2d face images




            Recognition and transfer of an expression
                       to a facial avatar




                           WP5 leader           Luc Van Gool
                                                 WP leader:
                                                vangool@vision.ee.ethz.ch
                                                 WP leader institution:




hermes-1.indd 13                                                                                                           10/09/2009 19:37:50
     14
                                                                      Software Integration
     WP6                                                                for Generation of
                                                                     Natural Language Texts
                                                                                                        3@630  A pedestrian labelled as ‘actor 3’ appears
                                                                                                               in the field of view.
                                                                                                        3@632 He moves on the southeastern sidewalk.
                                                                                                        3@663 He walks within it.
                                                                                                        3@766  He walks on the zebra.
                                                                                                        3@900 Actor 3 stands near by another pedestrian.
                                                                                                        3@928 Actor 3 moves on the zebra.
                                                                                                        3@1085 Actor 3 stands near by another pedestrian.
                                                                                                        3@1147 Actor 3 moves on the zebra.
                                                                                                        3@1250 He walks on the street.
                                                                                                        3@1254 Actor 3 chases another pedestrian.
                                                                                                        3@1289 Actor 3 runs on the street.


     A pedestrian chases another pedestrian.            Objectives                                    demonstrator system generates natural
                                                                                                      language text on activities of a particu-
                                                        Imagine a user communicating with a set
                                                                                                      lar agent (human or road vehicle) from
                                                        of distributed PTZ-cameras, as if they were
                                                                                                      a schematic conceptual representation
                                                        humans reporting what they see. This re-
                                                                                                      expressed as a set of Fuzzy Metric-Tem-
                                                        quires converting a video into a textual
                                                                                                      poral Horn Logic formulas, based on the
                                                        description of temporal events. The user
                                                                                                      agent’s trajectory data.
                                                        should be able to request summaries of
                                                        recent developents in chosen languages;       Future directions
                                                        to obtain responses to his questions for
                                                        details; and to send commands, e.g., to       The capability to detect, represent, and
                                                        zoom in onto a particular body.               describe more involved agent behavior
                                                                                                      is desirable, including stylistically ad-
                                                        Activities & achievements                     equate formulations for a larger range
                                                        The HERMES-consortium has designed,           of behaviors. Also, a combination of the
                                                        implemented, and tested various pro-          algorithmic rigor associated with a for-
                                                        totypical systems to cover the differ-        mal logic approach and the flexibility
                                                        ent requirements set out. Currently, a        expected from a ML-oriented approach
                                                        slimmed-down, fixed- and PTZ-camera           poses a challenge for future research.
     Pedestrian crossing a road via a traffic island.

0@210 A pedestrian labelled as ‘id 0’ comes in
      from the north sidewalk.
0@220 He stops.
0@222 He walks on the north sidewalk.
0@231 He stops.
0@234 He stops to due a car labelled as ‘id 1’.
0@239 He walks on the north sidewalk.
0@266 He walks on the crosswalk.
0@371 Now he has reached the center island.
0@401 He walks on the crosswalk.
0@503 He walks on the south sidewalk.




                            WP6 leader              Hans–Hellmut Nagel
                                                     WP leader:
                                                    nagel@iaks.uni-karlsruhe.de
                                                     WP leader institution:




hermes-1.indd 14                                                                                                                            10/09/2009 19:37:51
                                                                                                                                            15
                   Software Integration
                     for Generation of
                   Virtual Environments
                                                                                                            WP7


            Objectives                                       Producing virtual human motion im-
                                                             plied animating the characters with
            The goal of WP7 is to visualize the con-         the motion descriptions used thorough
            ceptual descriptions generated for be-           WP3-5. The integrated motion was ex-
                                                             amined for consistency using param-
            haviour (WP3), action (WP4) and emo-
                                                             eters of emotion and style, to find stable
            tion (WP5), joining them into a common
                                                             links between both kinds of animations.
            animation framework. WP7 generates               Particularly, we investigated whether a
            synthetic animations reproducing se-             given emotion may influence the anima-
            quences of predefined semantic primi-            tion of particular actions or behaviours.
            tives, provided beforehand or derived
                                                             Additionally, generating virtual anima-
            from end-user’s textual descriptions.            tions based on conceptual descriptions
                                                             can be seen as inverting WP6’s natural
            Activities & achievements                        language generation: here, the seman-
                                                             tic primitives related to behaviours, ac-
            Producing virtual components includes:           tions, and emotions are visualized us-
                                                             ing computer graphics. Since a virtual
              „    Conception, creation and visualiza-       environment contains all motion and
                   tion of a virtual scene;                  position information on all agents and
              „    Its texturing and illumination;           objects, interactions can be revisited.
                                                             This system was evaluated for accuracy           Virtual actors generated from users’
              „    3D-modeling of virtual objects and
                                                             of visualization and degree of interac-      descriptions and reacting to real scenes.
                   human characters; and
                                                             tion. As a result, behaviour descriptions
              „    Their inclusion into the virtual scene.   are visualized by means of virtual actors.




                             WP7 leader               WP Gonzàlez
                                                     Jordi leader:
                                                     poal@cvc.uab.cat
                                                      WP leader institution:




hermes-1.indd 15                                                                                                                         10/09/2009 19:37:52
    16


     WP8                                                                     Validation and
                                                                             Demonstration

                                                  Objectives                                    an outdoor scenario was designed to
                                                                                                probe the effectiveness of the HERMES
                                                  WP8 focuses the integration, demon-           approach to image visual cognition in
                                                  stration and validation of the other WPs’     multi-sensor networks. An integrated
                                                  outputs. A major goal of HERMES is to         hardware platform was designed, built
                                                  establish how the analysis of agent, face     and installed on the rooftop of the CVC
                                                  and body motion, plus synthesis of these      building in Barcelona. This hardware plat-
                                                  outputs in the form of inferred, high-level   form consists of two high-speed cameras
                                                  interpretation should be integrated into      (one fixed and one pan-tilt-zoom), and
                                                  a cohesive platform for experimentation,      three dedicated servers to host HERMES
                                                  validation and demonstration. In WP8, the     systems for analysis of agent motion,
                                                  outputs of all WPs were pooled and inte-      for active camera control, and for infer-
                                                  grated to build integrated hardware and       ring high-level descriptions of agent be-
                                                  software systems to meet such objectives.     haviour in the scene. To further support
                                                                                                demonstration activity, an integrated
                                                                                                and modular software platform was de-
                                                  Activities & Achievements                     signed, allowing modules to be devel-
                                                                                                oped off-site and later integrated into the
                                                  In the final year of the project, the con-    platform with minimal effort.
                                                  sortium turned its attention in earnest to-   Real-time tracking and analysis of agent
                                                  wards realizing both real-time and offline    motion results from WP3, active camera
                                                  demonstration and validation systems.         control from WP2 and inference of agent
                                                  Building on the prototype indoor dem-         behaviors from WP6 and WP7 have been
                                                  onstrator developed in WP2 and WP3,           incorporated into the demonstratiors.




     Real-time demonstrator interface controlling the fixed camera (top), active
     camera (bottom left), and split view of both cameras (bottom right).




                          WP8 leader           Andrew D. Bagdanov
                                                WP leader:
                                               bagdanov@cvc.uab.cat
                                                WP leader institution:




hermes-1.indd 16                                                                                                                       10/09/2009 19:37:59
                                                                                                          17

                       Exploitation and
                        Dissemination                                                                WP9
            Objectives                                  project. Presentations at national and
                                                        international conferences and forums
            WP9 promotes the dissemination of the       have been to be given (e.g. ICPR, CVPR,
            project results to the wide public, mak-    ECCV, ICCV, and other international and
            ing them well known throughout in the       national conferences and scientific jour-
            scientific community and the European       nals, like IEEE Transactions on Pattern
            industrial companies. It also promotes      Analysis and Machine Intelligence, Com-
            the industrial and market exploitation of   puter Vision and Image Understanding,
            the project results.                        Pattern Recognition Letters, etc).
                                                        The construction of the HERMES web-
            Activities & Achievements                   site, http://www.hermes-project.eu,
                                                        has greatly contributed to disseminate
            Many interchanges of research people        the project results. The website contains
            among the HERMES partners took place        demonstrators of the project highlights
            during the whole project duration. The      and other results supplied by the part-
            research partners have disseminated         ners. It has also been used to archive re-
            the results of the project with publica-    ports among the partners.
            tions and presentations in events and
            conferences. Technical papers and arti-     The project partners will exploit the
            cles in scientific and professional jour-   project results by using the basic princi-
            nals and specialized magazines have         ples of HERMES in commercial projects
            been published demonstrating the work       (e.g. smart surveillance for security
            that has been carried out during the        projects) and other research activities.




                          WP9 leader           Tonny Velin
                                                WP leader:
                                               tvelin@answare-tech.com
                                                WP leader institution:




hermes-1.indd 17                                                                                       10/09/2009 19:38:01
     18                                     Showcasing activity


                   “A hardware and software platform integrates the components of HERMES”




                                                                    HERMES showcase activities have been
                                                                    concentrated in the form of an integrat-
                                                                    ed hardware and software platform that
                                                                    consolidates the various components of
                                                                    the HERMES approach.
                                                                    In concert with the demonstration and
                                                                    integration activities of WP8, two inte-
                                                                    grated graphical user interfaces have
                                                                    been implemented to demonstrate and
                                                                    showcase the performance of HERMES
                                                                    modules:
                                                                     „   An interface for interacting with the
                                                                         real-time demonstrator was built
                                                                         that allows the user to configure,
                                                                         administer and monitor the com-
                                                                         ponents of the HERMES demon-
                                                                         strator platform. This interface also
                                                                         allows the user to visualize stream-
                                                                         ing video from both cameras in re-
                                                                         al-time, as well as tracker output in
                                                                         the form of 3D annotations on live
                                                                         video. Inference results from analy-
                                                                         sis and reasoning about agent mo-
                                                                         tion is also presented to the user in
                                                                         this interface, along with captured
                                                                         detail images from the PTZ camera.
                                                                     „    To complement the initial real-time
                                                                          demonstration system, a second
                                                                          offline demonstrator for anima-
                                                                          tion of virtual environments, based
                                                                          on the outputs of WP7, was con-
                                                                          structed that bases its virtual re-
                                                                          construction of agent behaviour
                                                                          on the outputs of the real-time
                                                                          demonstrator.
                                                                    The end result of these efforts is a flex-
                                                                    ible and modular system for showcasing
                                                                    some of the major developments from
                                                                    the HERMES project.




     H ead of H ermes from a HermaIc stele
     ancIent agora museum of atHens , g reece




hermes-1.indd 18                                                                                             10/09/2009 19:38:04
                                                                                                          19




                   “Two demonstration systems, one real-time and one offline,
                         showcase the major developments of HERMES”




                                                                      H ermes bearIng tHe good person by praxIteles
                                                                 arcHaeologIcal museum of ancIent o lympIa , g reece




hermes-1.indd 19                                                                                       10/09/2009 19:38:15
    20             Expertise of the Partners



                   CVC                                           CVMT
                   The CVC is a non-profit institution           The Computer Vision and Media Tech-
                   founded in 1995 by CIRIT (Consell Inter-      nology Laboratory (CVMT) is part of the
                   departamental de Recerca i Innovació          technical faculty at Aalborg University,
                   Tecnològica), CIDEM (Centre d’Innovació i     Denmark. CVMT conducts research in
                   Desenvolupament Empresarial) and UAB          areas such as 1) computer and robot vi-
                   (Universitat Autònoma de Barcelona). The      sion, 2) computer graphics, 3) motion
                   strategic objective of CVC is to do both      analysis, 4) color vision, 5) autonomous
                   Research and Development of quality           systems and agents, 6) multimedia and
                   on Computer Vision. From a scientific         virtual reality (VR) systems and inter-
                   point of view, the CVC wants to contrib-      faces, and 7) operational vision systems
                   ute to increase the knowledge in this         for both industrial and medical applica-
                   field. From a technological point of view,    tions. CVMT has been involved in about
                   the CVC aims to contribute to innova-         15 European projects and a number of
                   tion and industrial competitiveness col-      national projects. Currently CVMT is a
                   laborating with companies to develop          part of and coordinating one EU project
                   technological projects.                       and three national project.


                   IAKS                                          BIWI
                   The Fakultät für Informatik der Univer-       At the Eidgenoesssiche Technische
                   sität Karlsruhe (TH) is one of the lead-      Hochschule (ETH) Zurich the participat-
                   ing faculties for Computer Science in         ing team is the Computer Vision Labora-
                   Germany. The research group Kognitive         tory (BIWI), which is part of the Dept. of
                   Systeme (KOGS) lead by H.–H. Nagel is af-     Electrical Engineering D-ITET. The work
                   filiated with the ‘Institut für Algorithmen   of the group focuses on scene under-
                   und Kognitive Systeme’ der Universität        standing, medical imaging, and remote
                   Karlsruhe. This group addresses all prob-     sensing. Here mainly the expertise in the
                   lems encountered in the algorithmic           area of scene understanding is relevant.
                   transformation of image sequences into        This includes work on tracking, gesture
                   representations of the semantics of the       analysis, and motion capture. The group
                   depicted scenes and their temporal de-        has developed multiple trackers, rang-
                   velopment, covering data-driven treat-        ing from generic, fast blob trackers up
                   ment of video signals via model-based         to trackers for articulated structures like
                   approaches towards detection and              hands or full bodies. The group has been
                   tracking up to logic-based representa-        involved in many European and Swiss
                   tions and their conversion to natural         projects.
                   language text.




hermes-1.indd 20                                                                                               10/09/2009 19:38:20
                                                                                                                   21




                   AVL
                   The Active Vision Laboratory is part
                   of the Robotics Research Group in the
                   University of Oxford’s Department of
                   Engineering Science. Since its inception
                   in 1991 the lab has established a long
                   record of research in active, real-time
                   and robotic vision. A substantial part of
                   the group’s effort is invested in research-
                   ing the theory of visual sensing and per-
                   ception and its application to a wide
                   variety of problems. The group has had
                   substantial success in both visual track-
                   ing and in 3D geometry. The group has
                   been involved in a numerous previous
                   European projects. Its faculty have con-
                   sulted widely for major companies.


                   ANSWARE
                   Answare Technologies is a Spanish SME
                   specialist in Information and Communi-
                   cation Technology, providing consultan-
                   cy and projects development services
                   and offering a variety of high-tech solu-
                   tions based on state-of-the-art technol-
                   ogies, tailored to the specific needs of
                   customers. Answare operates in nation-
                   al and international market sector for
                   Aerospace, Defence, Telecommunica-
                   tions and Internet. Answare’s staff have
                   participated in the complete life cycle
                   for software systems projects from con-
                   ception through development, delivery
                   and into maintenance.




                                                                   H ermes IngenuI WItH kerykeIon (Herald’s staff), kItHara ,
                                                                 petasus (round Hat), traveller ’s cloak and WInged temples
                                                                                              pIo clementIno , vatIcan cIty




hermes-1.indd 21                                                                                                10/09/2009 19:38:23
    22                                                                   Finances


                                                                Hermes expenditure per year
                                                                         (of the 2.100.000,00€ grant)
                     M€ 800
                        700
                        600
                        500
                        400                                                       729.290 €                 742.869 €
                        300                             627.841 €
                        200
                        100
                          0
                                                         Period 1                 Period 2                   Period 3
                                                   Mar 2006 / Feb 2007         Mar 2007 / Feb 2008       Mar 2008 / Aug 2009




                                                               The HERMES project has been allo-
                                                               cated a budget of 2.100.000 € by the
                                                               European Commission to fulfil the
                                                               objectives of its work programme.
                                                               This grant was distributed over the
                                                               42 months period of the project (36
                                                               months + 6 months extension), from
                                                               February 2006 to August 2009. The
                                                               grant was divided among all the
                                                               partners, according to their activities
                                                               and commitment in the project.




                                           “The HERMES project has been allocated a budget of
                                                2.100.000 € by the European Commission”




      H ermes WearIng tHe petasus , a voyager ’s cloak , tHe
      caduceus and a purse
      museo cHIaramontI , vatIcan cIty




hermes-1.indd 22                                                                                                               10/09/2009 19:38:26
                                        Authors:
                   Written with the contribution of all HERMES Partners

                                     Designed by:
                                 Carles Fernández Tena




hermes-1.indd 23                                                          10/09/2009 19:38:26
                   http://www.hermes-project.eu
                          September 2009




hermes-1.indd 24                                  10/09/2009 19:38:26

								
To top