Source / Fuente: http://channel9.msdn.com/Series/KinectQuickstart

1. QUICKSTART SERIES
KINECT FOR WINDOWS

2. INSTALLING &
USING KINECT
KINECT FOR WINDOWS

3. INTRODUCTION
• Designed to get you started
• Assume you have some programming
experience
• Managed API - Concepts work in both VB/C#
KINECT FOR WINDOWS

4. QUICKSTART SERIES
• Installing and Using the Kinect Sensor
• Setting up your Development Environment
• Camera Fundamentals
• Working with Depth Data
• Skeletal Tracking Fundamentals
• Audio Fundamentals
KINECT FOR WINDOWS

5. KINECT SENSORS
IR Emitter Color Sensor
IR Depth Sensor
Tilt Motor
Microphone Array
KINECT FOR WINDOWS

6. GET STARTED
• http://kinectforwindows.org
• Order Kinect Hardware
• Download Kinect SDK
KINECT FOR WINDOWS

7. INSTALLING AND USING KINECT
DEMO
KINECT FOR WINDOWS

8. SETTING UP
YOUR DEVELOPMENT
ENVIRONMENT
KINECT FOR WINDOWS

9. QUICKSTART SERIES
• Installing and Using the Kinect Sensor
• Setting up your Development Environment
• Camera Fundamentals
• Working with Depth Data
• Skeletal Tracking Fundamentals
• Audio Fundamentals
KINECT FOR WINDOWS

10. KINECT API BASICS
• Manage Kinect state
• Connected
• Enable Color, Depth, Skeleton
• Start Kinect
• Get Data
• Events - AllFramesReady
• Polling – OpenNextFrame
KINECT FOR WINDOWS

11. RESOURCES
• Install Kinect Explorer
• KinectWpfViewers
• Coding4Fun Toolkit
• Skeletal scaling
KINECT FOR WINDOWS

12. SETTING UP YOUR DEV
ENVIRONMENT
DEMO
KINECT FOR WINDOWS

13. CAMERA
FUNDAMENTALS
KINECT FOR WINDOWS

14. QUICKSTART SERIES
• Installing and Using the Kinect Sensor
• Setting up your Development Environment
• Camera Fundamentals
• Working with Depth Data
• Skeletal Tracking Fundamentals
• Audio Fundamentals
KINECT FOR WINDOWS

15. CAMERA DATA
• Events return ImageFrame
• PixelDataLength
• FrameNumber
• Timestamp
• Dimensions: Height, Width
• Use AllFramesReady event to synchronize
KINECT FOR WINDOWS

16. RESOLUTIONS
• Color
• 12 FPS: 1280X960 RGB
• 15 FPS: Raw YUV 640x480
• 30 FPS: 640x480
• Depth
• 30 FPS: 80x60, 320x240, 640x480
KINECT FOR WINDOWS

17. CAMERA FUNDAMENTALS
DEMO
KINECT FOR WINDOWS

18. WORKING WITH
DEPTH DATA
KINECT FOR WINDOWS

19. QUICKSTART SERIES
• Installing and Using the Kinect Sensor
• Setting up your Development Environment
• Camera Fundamentals
• Working with Depth Data
• Skeletal Tracking Fundamentals
• Audio Fundamentals
KINECT FOR WINDOWS

20. DEPTH DATA
• Returns the distance and player for every pixel
• Ex: 320x240 = 76,800 pixels
• Distance
• Distance in mm from Kinect ex: 2,000mm (6.56 feet)
• Player
• 1-6 players
KINECT FOR WINDOWS

21. MODE
Mode Depth & Player Center Hip Joint Other 19 Joints
Default Yes Yes Yes
Near Yes Yes No, for v1.0
KINECT FOR WINDOWS

22. DISTANCES
Feet 1.3’ 2.6’ 9.8’ 13.1’ 26.2’
Meters .4 .8 3 4 8
Default
Mode
Near
Mode
KINECT FOR WINDOWS

23. FORMULAS
• Distance Formula
int depth = depthPoint >> DepthImageFrame.PlayerIndexBitmaskWidth;
• Player Formula
int player = depthPoint & DepthImageFrame.PlayerIndexBitmask;
KINECT FOR WINDOWS

24. WORKING WITH DEPTH DATA
DEMO
KINECT FOR WINDOWS

25. SKELETAL
TRACKING
FUNDAMENTALS
KINECT FOR WINDOWS

26. QUICKSTART SERIES
• Installing and Using the Kinect Sensor
• Setting up your Development Environment
• Camera Fundamentals
• Working with Depth Data
• Skeletal Tracking Fundamentals
• Audio Fundamentals
KINECT FOR WINDOWS

27. SKELETON DATA
KINECT FOR WINDOWS

28. SKELETAL JOINTS
• Each player with set of <x, y, z> joints in meters
• Each joint has associated state
• Tracked, Not tracked, or Inferred
• Inferred - Occluded, clipped, or low confidence joints
• Use TransformSmoothParameters to smooth joint data to reduce
jitter
KINECT FOR WINDOWS

29. SKELETAL TRACKING
• Maximum two players tracked at once
• Six player proposals per Kinect
• Each skeleton has a unique identifier - TrackingID
• SkeletonStream
• Set AppChoosesSkeletons to false
• Use ChooseSkeletons method to choose which skeletons to track
KINECT FOR WINDOWS

30. SKELETAL TRACKING
DEMO
KINECT FOR WINDOWS

31. AUDIO
FUNDAMENTALS
KINECT FOR WINDOWS

32. QUICKSTART SERIES
• Installing and Using the Kinect Sensor
• Setting up your Development Environment
• Camera Fundamentals
• Working with Depth Data
• Skeletal Tracking Fundamentals
• Audio Fundamentals
KINECT FOR WINDOWS

33. SOUND POSITION
• Sound Source Angle – the angle and confidence level of where
audio is coming from
• Beam Angle – The angle used to record audio that you can set as a
“directional microphone”
-50 0 +50
KINECT FOR WINDOWS

34. RECORDING
• KinectAudioSource audio buffer
• Use Echo Cancellation to not record PC sound
KINECT FOR WINDOWS

35. SPEECH RECOGNITION
• Grammar – What we are listening for
• Code – GrammarBuilder, Choices
• Note: Ensure AutomaticGainControl = false
KINECT FOR WINDOWS

36. GRAMMAR
<!-- Confirmation_Yes._value: string ["Yes"]
-->
<rule id="Confirmation_Yes" scope="public">
<example> yes </example> var grammar = new Choices();
<example> yes please </example> grammar.Add(“yes please");
<one-of>
<item> yes </item>
grammar.Add(“yes");
<item> yeah </item> grammar.Add(“yeah");
<item> yep </item> grammar.Add(“ok");
<item> ok </item>
</one-of>
<item repeat="0-1"> please </item>
<tag> out._value = "Yes";</tag>
KINECT FOR WINDOWS

37. MULTI-MODAL FEEDBACK
• Engagement model
– Mic indicator for speech-enabled menus
– Keyword to engage
• Feedback and confirmation
KINECT FOR WINDOWS

38. AUDIO
DEMO
KINECT FOR WINDOWS