Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out

Recognizing Human Actions from Video Data - Simon Fraser

VIEWS: 0 PAGES: 34

									     Retrieving Actions in Group
              Contexts
Tian Lan, Yang Wang, Greg Mori, Stephen Robinovitch
               Simon Fraser University
                   Sept. 11, 2010
                    Outline
• Contextual Representation of Actions


• Action Retrieval as Ranking


• Results and Future Work
               Nursing Home
• Fall analysis in nursing home surveillance
  videos
  – a system automatically rank the videos according
    to the relevance to fall action is expected
   Action-Action Context
What other
people are
 doing ?




             Context
         Actions in Group Context

• Motivation
  – human actions are rarely performed in isolation,
    the actions of individuals in a group can serve as
    context for each other.
• Goal
  – explore the benefit of contextual information in
    action retrieval in challenging real-world
    applications
Action Context Descriptor
                                                τ


                                   z



                      +
                                             action
            Focal person           Context




                          action
Action Context Descriptor




                                                        score
             Feature           Multi-class
            Descriptor           SVM
                                                            action class
  e.g. HOG by Dalal & Triggs
 score




     action class
                         max

                                     score
           …
  score




                                         action class
         action class
                    Outline
• Contextual Representation of Actions


• Action Retrieval as Ranking


• Results and Future Work
      Classification or Retrieval
• Previous Work
  – Most work in human action understanding
    focuses on action classification.
       Classification or Retrieval
• Most surveillance tasks are typical retrieval
  tasks
  – retrieve a small video segment contains a
    particular action from thousands of hours of
    videos.
• The “action of interest” is rare event
  – Extremely imbalanced classes
Action Retrieval    Query : fall




      Rank according to
      the relevance to falls
                  Learning
• Input: document-rank pair (xi,yi)



• Optimization




                                  Joachims, KDD 06
               Ranking SVM
• Ranking function h(x)

                             h(x)



                                    Rank r1
                                    Rank r2
                                    Rank r3
Action Retrieval - training
                              irrelevant




                               relevant



                                very
                                relevant
                    Outline
• Contextual Representation of Actions


• Action Retrieval as Ranking


• Results and Future Work
                        Dataset
• Nursing Home Dataset
   • 5 action categories: walking, standing, sitting, bending
     and falling. (per person)
   • 18 video clips.
   • Query: fall
• Collective Activity Dataset (Choi et al. VS. 09)
   • 5 action categories: crossing, waiting, queuing,
     walking, talking. (per person)
   • 44 video clips.
   • Query: each of the five actions
                 Dataset
• Nursing Home Dataset
                    Dataset
• Collective Activity Dataset
           System Overview

              Person                 Person                Rank
Video                                                      SVM
             Detector               Descriptor



                                              u
                                              v


        • Pedestrian Detection   • HOG by Dalal & Triggs
          by Felzenszwalb et al. • LST by Loy et al.
        • Background Subtraction at cvpr 09
                     Baselines
• Context vs No Context
  – Action Context Descriptor
  – Original feature descriptors, e.g. HOG (Dalal & Triggs at
    CVPR 05), LST (Loy et al. at CVPR 09)
• RankSVM vs SVM

• Methods
  –   Context + RankSVM (our method)
  –   Context + SVM
  –   No Context + RankSVM
  –   No Context + SVM
Retrieval Results




          Nursing Home Dataset
Retrieval Results




         Collective Activity Dataset
Retrieval Results




         Collective Activity Dataset
Retrieval Results




         Collective Activity Dataset
1   2




3   4
5   6




7   8
Action Classification




                 [10] Choi et al. in VS. 09

             Collective Activity Dataset
                Conclusion
• A new contextual feature descriptor to
  represent actions
  – action context (AC) descriptor
• Formulate our problem as a retrieval
  task.
                Future Work
• Contextual Feature Descriptors
  – How to only encode useful context?
• Rank-SVM loss, optimize the NDCG score
Thank you!
5   6




7   8

								
To top