Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

Kinect for FRC 2012 by Xn8OxX8g

VIEWS: 36 PAGES: 18

									 Kinect for FRC 2012


ALFRED THOMPSON
MICROSOFT ACADEMIC TEAM
 HTTP://BIT.LY/ALFREDTH
                         Outline
                            2

 Introduction
 Kinect Hardware
 Kinect for Windows
 Skeleton, depth, image and audio data
 Kinect for FRC 2012
 Default FRC gestures
 Programming your own gestures
 Additional Resources
          Kinect Hardware
                   3


             IR Projector

                       RGB Camera

                                     IR Camera




Multi-Array Mic               Motorized Tilt
             Kinect and FRC 2012
                          4



 The Kinect has multiple powerful sensors for depth,
 skeleton, video and audio input

 The Kinect is likely intended for robot
 control via the drivers station for the 2012
 FRC game
Kinect for Windows Architecture
               5
Skeleton Data
                   Kinect Audio
                          7

 Four microphone array with hardware-based
  audio processing
  – Multichannel echo cancellation (MEC)
  – Sound position tracking
  – Other digital signal processing (noise suppression
    and reduction)
Kinect Sensor Data
        8
FIRST Kinect Architecture
                          9

FRC Driver Station

                                             WPI Lib
                                         (Kinect Enabled)
 WPI Kinect Server



Kinect for Windows SDK
                                             cRIO

       Kinect



Driver Station                             Robot


                 Joystick Position and Skeleton points
              Kinect for FRC 2012
                                   10

  Kinect        Driver Station PC          Dashboard
   SDK                                  (Skeletal Display)


FRC Kinect                                Driver Station
Server (C#)     Raw Skeletal Data,        (Kinect Diags)
                 Diagnostic Data,
               Default Gesture Data
                  (Kinect Sticks),
                  Optional Data
                                                             Raw Skeletal Data,
                                                              Diagnostic Data,
                       Default Gesture Data
                                                               Optional Data
                          (Kinect Sticks)
                                                                     Robot Gesture
                                                                    Code (LabVIEW,
                                                                     C/C++ or Java)



                      Scenario 1                                           Scenario 2
                 Programming
                         11

 Teams using C# can change/modify the WPI Kinect
 Server on the Driver Station

 Teams using LabView, Java or C++ can process
 joystick AND skeleton point data on the cRIO robot
 controller
                                                                               Default FRC Gestures
                                                                                                             12
                                     Y= -1.00    Y= -1.00   Y= -1.00 Y= -1.00
                                                  90
                                          70
                                                                  90                                                              -90                                       -90
                                                                          70
                  Y=- 0.50                                                                     Y=- 0.50
                              35                                                     35                               -45                                                              -45




                     0             RIGHT ht
        Y= 0.00
                                      Rig                              LEFT                      0 Y= 0.00                                 RIGHT                   LEFT
                                                                                                                  0                                                                          0



                                                                                          35
                             -35
                                                                                               Y= 0.50

                         Y= 0.50          -70                             70                                          45                                                               45
                                                  -90            -90                                                          NAN / Zero                                  NAN / Zero
                                     Y= 1.00                               Y= 1.00
                                                Y= 1.00                                                                            90                                       90
                                                                Y= 1.00                                                                            Facing Kinect

 Right KinectStick2 Y values                                               Left KinectStick1 Y values                           Fig 2 Wrist/hand to Elbow angle definition of Zero
Map to Wrist/Shoulder Angle                                               Map to Wrist/Shoulder Angle
Fig 1 Human Player facing Kinect Y joystick value / angle interpretation

  Buttons:
  1) Head to the Left                                              4) Left Leg Out                                          7) Left Leg Forward
  2) Head to the Right                                             5) Right Leg Forward                                     8) Left Leg Back
  3) Right Leg Out                                                 6) Right Leg Back                                        9) Kinect Control “Enabled”
            Control Considerations
                          13

 Robot control is achieved via skeletal position


 Gestures/poses relate to skeletal position and
 inform which control is desired

 Consider your gestures carefully!


 Response times will be different with Kinect
 compared to traditional joysticks
             Using the FRC Gestures
                                 14


                                                        KinectStick leftArm(1);
                                                        KinectStick rightArm(2);

Labview                                           C++   leftArm.getY()
                                                        leftArm.getRawButton(#)




                 KinectStick leftArm;
                 KinectStick rightArm;
                 …
                 leftArm = new KinectStick(1);
                 rightArm = new KinectStick(2);
          Java   …
                 leftArm.getY()
                 leftArm.getRawButton(#)
                           Gesture Programming
                                     15

 Teams may modify the Kinect Server code (C#) using
    Visual C# Express.
       Allows access to depth/RGB data
       Easier to display feedback on dashboard
       Keeps cRIO processing burden low
       Gesture examples available from general Kinect development
        community
   Teams may use the raw skeleton data sent to the robot to process their
    own gestures in their chosen language (LabVIEW, C/C++ or Java)
        Language familiarity
                   Getting Help
                          16

1.   FRC Forums

2. Public Kinect for Windows Forums


3. Escalate to Microsoft via official FIRST contacts
             Demo and Questions
                     17

 Go Teams, go!
                          Additional Resources
                                        18

 Official Kinect for Windows Website
 FRC Beta Kinect Forum


 Kinect SDK Quickstart Video Tutorials
 Coding4Fun Kinect Toolkit
 Coding4Fun Kinect Projects
 Visual C# Express
 Kinect for Windows SDK Beta Forums



 Getting Started with Microsoft Kinect for FRC document – released at Kickoff
 Kinect Server Code Walkthough – released at Kickoff

								
To top