Presentation by @sulagarcia at Munich MeetUp April 10th about Intel´s Perceptual Computing SDK

1. Perceptual SDK
Sulamita Garcia
Technical Marketing Engineer

2. /me
@sulagarcia
Copyright© 2012, Intel Corporation. All rights reserved.
*Other brands and names are the property of their respective owners.

3. Intel® Perceptual Computing SDK
2013
 Gold released
 Hand and Fingers tracking (specific IR-
depth camera required)
 Face recognition, tracking and attribution
 Voice and commands recognition
(dual-array mic required)
 3D objects tracking
 Low and high level APIs
Free download on intel.com/software/perceptual
Copyright© 2012, Intel Corporation. All rights reserved.
*Other brands and names are the property of their respective owners.

4. Intel® Perceptual Computing SDK
Mix of algorithms and modules
• Directly usable
• Can be extended
• Hardware accelerated on Intel® platforms
Nature user interfaces - gestures, voice…
Creating and manipulating 3D content
Games - Navigating 3D works, head-coupled perspective
4
Copyright© 2012, Intel Corporation. All rights reserved.
*Other brands and names are the property of their respective owners.

5. Creative* Camera
 30fps RGB 720p Camera
 IR projector
 60fps IR 320x240
Camera
 Dual-array Mic
 Powered by USB
Order from intel.com/software/perceptual
5
Copyright© 2012, Intel Corporation. All rights reserved.
*Other brands and names are the property of their respective owners.

6. SDK directories
Bin: You can find some executable that show you some
possible applications or features available with the SDK
Doc: The documentation in PDF format
Framework: All you need to use the SDK in C#, Unity, etc.
Include: The header files
Lib: The SDK libraries
Redist: Contains the minimum required installations to be
able to run applications that use the SDK
Sample: Contains the source code of many of the
examples available in the bin folder
6
Copyright© 2012, Intel Corporation. All rights reserved.
*Other brands and names are the property of their respective owners.

7. SDK architecture
SDK Applications SDK Samples
SDK Utilities SDK Frameworks and Language Ports
SDK Interfaces
I/O Modules Algorithm Modules
Core Services:
Context
Module Loading
Synchronization
Interoperability Multiple Multiple
Implementations Implementations
Available Available
7 Copyright© 2012, Intel Corporation. All rights reserved.
*Other brands and names are the property of their respective owners.

8. SDK interfaces
Applications
C# Port Processing* Port openFrameworks* Port Unity* Port
Core Framework
PXCMSession PXCUPipeline (C)
PXCMImage
PXCMAudio
I/O
PXCMCapture
Algorithms
PXCMGesture UtilPipeline (C++)
PXCMFaceAnalysis
PXCMVoiceRecognition
PXCMVoiceTTS
Pipeline
UtilMCapture
UtilMPipeline UtilCapture (C++)
C++
Core Framework I/O Algorithms
PXCSession PXCCapture PXCGesture
PXCImage PXCFaceAnalysis
PXCAudio PXCVoiceRecognition
PXCVoiceTTS
8 Copyright© 2012, Intel Corporation. All rights reserved.
*Other brands and names are the property of their respective owners.

9. Gesture recognition
~30cm
Blob Intermediate images, help separating:
• Background
• Hands
GeoNode Skeleton nodes
• Hand openness
• Open Hand: Fingertips, middle, elbows
• Closed Hand: up, middle, bottom
Gesture Predefined:
• THUMB UP/DOWN, PEACE, BIG5
• WAVE, CIRCLE, SWIPE LEFT/RIGHT/UP/DOWN
Alert Alertes prédéfinies
• FOV_LEFT/_RIGHT/_TOP/_BOTTOM
• FOV_BLOCKED/_OK
• GEONODE_ACTIVE/INACTIVE
9
Copyright© 2012, Intel Corporation. All rights reserved.
*Other brands and names are the property of their respective owners.

10. Hands and fingers tracking
QueryNodeData()
– PXCPoint3DF32 positionWorld;
– PXCPoint3DF32 positionImage;
– pxcU64 timeStamp;
– pxcU32 confidence;
– pxcF32 radius;
– Label body;
– PXCPoint3DF32 normal;
– pxcU32 openness;
QueryBlobData()
– pxcU64 timeStamp;
– pxcU32 labelBackground;
– pxcU32 labelLeftHand
– pxcU32 labelRightHand
QueryBlobImage()
– PXCImage** image;
+ HAND_FINGERTIP
10 Copyright© 2012, Intel Corporation. All rights reserved.
*Other brands and names are the property of their respective owners.

11. Hello World
class GesturePipeline:public UtilPipeline {
public:
GesturePipeline(void):UtilPipeline(), m_render(L"Gesture Viewer") {
EnableGesture();
}
virtual void PXCAPI OnGesture(PXCGesture::Gesture *data) {
if (data->active) m_gdata = (*data);
switch (data->label) {
case PXCGesture::Gesture::LABEL_NAV_SWIPE_LEFT: break; //do something
case PXCGesture::Gesture::LABEL_NAV_SWIPE_RIGHT: break; //do something
default: break;
}
}
virtual bool OnNewFrame(void) {
return m_render.RenderFrame(QueryImage(PXCImage::IMAGE_TYPE_DEPTH),
QueryGesture(), &m_gdata);
}
protected:
GestureRender m_render;
PXCGesture::Gesture m_gdata;
};
11 Copyright© 2012, Intel Corporation. All rights reserved.
*Other brands and names are the property of their respective owners.