ITCamp 2013 - Tim Huckaby - Kinect for Windows - Designing Software for Gesture & Voice Controlled User Interfaces

itcampro@ itcamp13# Premium conference on Microsoft technologies
Kinect for Windows -
Designing Software for
Gesture & Voice Controlled
User Interfaces
Tim Huckaby
InterKnowlogy Chairman/Founder
Actus Interactive Software Founder & CEO
Microsoft® Regional Director – Southern California
Microsoft® MVP – Kinect for Windows
E-mail: TimHuck@InterKnowlogy.com
TimH@Actus-Software.com
Blog:
team.interknowlogy.com/blogs/timhuckaby
FaceBook:
www.facebook.com/Tim.Huckaby.Carlsbad
LinkedIn: www.linkedin.com/in/timhuckaby
Twitter: @TimHuckaby

Breakout
SessionHuge thanks to our sponsors!

A little bit about me and why I am
here…
A little bit about me and why I am here…
Kinect for Windows - Designing Software for Gesture & Voice Controlled User Interfaces

Breakout
SessionAbout Tim, InterKnowlogy & Actus…
InterKnowlogy (www.InterKnowlogy.com)
Tim Huckaby, Founder/Chairman (TimHuck@InterKnowlogy.com)
◦ 13+ year old services firm – Custom Application Development
◦ focused on building NUI solutions
◦ Web, mobile, and desktop
◦ Windows 7/8, PixelSense, WP7/8, Azure, Kinect, XAML stack
- Largest Client: Microsoft
Actus Interactive Software (www.Actus-Software.com)
Tim Huckaby, Founder/CEO (TimHuck@Actus-Software.com)
◦ Software Product Company
◦ Interactive Kiosk and Active Digital Signage
◦ Multi-Touch, Gesture, and Voice Driven Solutions

Relevant Use Cases (And Use Cases that do not Make Sense)
Objectives And Agenda
Runway to get started
Technology Roadmap
Where We have Been and Where we are Going
Demos, Demos, Demos….
Gesture & Voice Driven User Experiences

NUI: The Natural User Interface
Multi-Touch Capable
•Common Gestures of Touch
•Physical Object Interaction
•Virtual Object Interaction
Gesture Capable
•Motion Based Interface
•Tracking of full-body movement, facial expression, and voice with precision
Voice Capable
•Multi-array Microphone
•Uniquely Distinguish Audio & Voices
Nueral Capable
•Conscious and non-conscience interfaces
•Non Invasive

World's Fastest-Selling Consumer Electronics Device
… a Future Beyond Gaming
Combination of RGB camera, depth sensor and multi-array microphone
– RGB camera delivers three basic color components
– Depth sensors “sees” in 3-D
– Microphone locates voices by sound and excludes ambient noise
Consumer Priced: $249.99
Tracks full-body movement, facial expression, and voice with precision
Kinect sensor bar scans and monitors 1 million points on the body
Kinect for Windows SDK
– Skeletal Tracking
– Face, Gesture Recognition
– Audio Echo cancellation
– Audio Beam Forming
– Speech Recognition
– Machine Learning

Breakout
SessionInterKnowlogy – Picture Viewer
Tim Huckaby
Founder / Chairman, InterKnowlogy
Founder / CEO, Actus Software
Microsoft RD & MVP

Breakout
SessionUse Case: Data Visualization & 3D Modeling
Interactive data visualization
3D modeling and design
Hands-off presentations
Kinect Fusion

Breakout
SessionKinect – 3D Anatomical Manipulation
Tim Huckaby
Microsoft RD & MVP

Breakout
Session
A Relatively New Hardware Product
A commercial HW SKU for B2B sales; developers can purchase via Amazon
and similar channels
A Commitment Kinect for Windows gives a customer:
Fully supported hardware for Windows, plus a new “near mode” to allow
innovation at 40cm from the sensor
Ongoing updates based on the software innovation that is part of
Microsoft’s deep investment and commitment to NUI, in both speech and
human tracking
A Future of Innovation
Kinect for Windows opens up a new wave of innovation for Human Computer
Interaction on Windows platform
Puts Natural User Interface (NUI) into the public eye in a broad variety of
venues: from car dealerships to malls to boardrooms to operating rooms
What is Kinect For Windows?

Breakout
Session
Use Cases:
Exercise, Physical Therapy, Education, Training
Exercise and physical therapy
Senior wellness – exercise and
monitoring
Interactive science experiments
Language/speech learning
Home Healthcare / Tele-Medicine

Breakout
Session
• Skeletal Tracking
• Near Mode
• Depth Mode
3 Modes of Operation

Breakout
SessionK4W Skeletal Tracking

Breakout
Session
The Kinect for Windows
InterKnowlogy - Physical Therapy
Tim Huckaby
Microsoft RD & MVP

Breakout
Session
The Good News:
• The Kinect SDK for Windows
The Not So Good News:
• Windows Only Device; You are not legally allowed to build
Windows apps for your Xbox Kinect;
• you cannot build apps for 18 million Xbox Kinect owners
with it.
Kinect for Windows

How to:
United States - $249.99 - SKU: L6M-00001
Amazon Visit the Store
GameStop Visit the Store
Microsoft Store Visit the Store
Newegg Visit the Store
http://www.microsoft.com/en-us/kinectforwindows/purchase/

Breakout
Session
Raw sensor streams
Access to raw data streams from the depth sensor, color camera sensor, and four-element microphone array enables
developers to build upon the low-level streams that are generated by the Kinect sensor.
Skeletal tracking
The capability to track the skeleton image of one or two people moving within the Kinect field of view make it easy
to create gesture-driven applications.
Advanced audio capabilities
Audio processing capabilities include sophisticated acoustic noise suppression and echo cancellation, beam
formation to identify the current sound source, and integration with the Windows speech recognition API.
Sample code and documentation
The SDK includes more than 100 pages of technical documentation. In addition to built-in help files, the
documentation includes detailed walkthroughs for most samples provided with the SDK.
Easy and familiar
The SDK installs quickly, requires no complex configuration, and the complete installer size is less than 100 MB.
Developers can get up and running in just a few minutes with a standard standalone Kinect sensor unit (widely
available at retail outlets).
Windows 7, Windows Embedded Standard, Windows 8 Desktop Apps
C++, C#, or Visual Basic in Microsoft Visual Studio 2010.
Free!
Free SDK download, free run time license!
http://www.microsoft.com/en-us/kinectforwindows/develop/
What’s in the Kinect for Windows SDK?

Breakout
SessionKinect for Windows SDK
Tim Huckaby
Microsoft RD & MVP

Breakout
Session
Launch Kinect
for XBox
K4W
Team
Formed
K4W Non-
commercial
SDK beta
K4W
commercial
SDK TAP
Initiated
New K4W
Hardware
announced
K4W
commercial
Release
Timeline
Nov, 2010 April, 2011 June, 2011 Sept, 2011 Nov, 2011 Feb 1, 2012
K4W v1.5 K4W v1.6 K4W v1.7 K4W Dev Center
May 21, 2012 Oct, 2012 March, 2013 May, 2013

Breakout
SessionUse Case: Interactive Media
Interact with customers in-store
Virtual fitting room – in-store or
shop-at-home
Vending machines
Interactive window exhibits
Augmented Reality web chat

Breakout
Session
Perfect for Kinect: Not designed for Kinect
Windows 8 / Metro / Modern UI

Breakout
SessionActus Interactive Kiosk
Tim Huckaby
Microsoft RD & MVP

Breakout
SessionUse Case: Hands-Free Control
Doctors in surgery
operating medical
imaging devices
through gesture
and voice
Airplane
mechanics/engineer
ing workers viewing
instructions
Cooking and other
household scenarios

Breakout
SessionInterKnowlogy: Touch-less Operating Room
Tim Huckaby
Microsoft RD & MVP

Breakout
Session
Leap Motion
– $70
– https://leapmotion.com
– In developer beta
SoftKinetic
– http://www.softkinetic.com
– $249
– Distance and Near Mode
– Multiple Products
– In the space for over a decade
Panasonic
– Works outside in broad daylight
– $2,600
Kinect for Windows Competitors

Breakout
SessionGesturePak: Carl Franklin’s Gesture Recorder
Tim Huckaby
Microsoft RD & MVP

Breakout
SessionCarl Franklin’s Gesture Recorder
GesturePak is a speech-driven gesture recording WPF app and a
matcher class for .NET developers that uses the Kinect For Windows
device and SDK.

Summary
Kinect for Windows is not great in the majority of Use Cases; where it
is good:
• Hands-Free Control
• Exercise, Physical Therapy, Education, Training
• Data Visualization, 3D Modeling
• Interactive Media, Retail, Digital Signage
The hardware, tools, plumbing and platform for Kinect for Windows
Applications are powerful and available now.
…and will just continue to get better, cheaper, powerful and more available
The concepts of Gesture and Voice Recognition change the way we
design and build software in some Use Cases.
…if the goal is better user experiences

More info on InterKnowlogy:
www.InterKnowlogy.com
www.vimeo.com/InterKnowlogy
More info on Actus Software:
◦www.Actus-Software.com
About Tim Huckaby…
◦Founder/Chairman, InterKnowlogy
◦Founder/Chairman, Actus Software
◦Microsoft® Regional Director – Southern
California
◦Microsoft® .NET Partner Advisory Council
Founder / Member
◦Microsoft® MVP – Kinect for Windows
◦INETA Speaker – International .NET Users Group
Association
◦Board Member: Association for Competitive
Technology
◦Board Member: Microsoft Healthcare Partner
Advisory Board
◦Author / Speaker
Contact me:
E-mail:
TimHuck@InterKnowlogy.com
TimH@Actus-Software.com
Phone:
• Office: 760-444-8640
• Mobile:: 619 990 9200
Blog:
http://team.interknowlogy.com/blogs/timhuckaby
FaceBook:
http://www.facebook.com/Tim.Huckaby.Carlsbad
LinkedIn:
http://www.linkedin.com/in/timhuckaby
Twitter: @TimHuckaby
Tim Huckaby

Q & A

ITCamp 2013 - Tim Huckaby - Kinect for Windows - Designing Software for Gesture & Voice Controlled User Interfaces

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to ITCamp 2013 - Tim Huckaby - Kinect for Windows - Designing Software for Gesture & Voice Controlled User Interfaces

Similar to ITCamp 2013 - Tim Huckaby - Kinect for Windows - Designing Software for Gesture & Voice Controlled User Interfaces (20)

More from ITCamp

More from ITCamp (20)

Recently uploaded

Recently uploaded (20)

ITCamp 2013 - Tim Huckaby - Kinect for Windows - Designing Software for Gesture & Voice Controlled User Interfaces