May 2018 Lancaster AI meeting on Tensorflow.js and OpenPose

1. LANCASTER AI
SKELETON TRACKING WITH TENSORFLOW
Richard Everts
Co-founder, Sugey
www.sugeyone.com
rich@sugeyone.com
@sugeyone

2. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
WHAT ARE WE TALKING ABOUT?

3. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
WHAT ARE WE TALKING ABOUT?

4. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
THE TOOLS

5. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
SIMILAR TO OBJECT DETECTION

6. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
POSENET - TENSORFLOW.JS VARIATION
17 Total Points
0 nose
1 leftEye
2 rightEye
3 leftEar
4 rightEar
5 leftShoulder
6 rightShoulder
7 leftElbow
8 rightElbow
9 leftWrist
10 rightWrist
11 leftHip
12 rightHip
13 leftKnee
14 rightKnee
15 leftAnkle
16 rightAnkle

7. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
RETURNING JSON OBJECT SHAPES
6 - 7 million
120 million
{
"score": 0.32371445304906,
"keypoints": [
{ // nose
"position": {
"x": 301.42237830162,
"y": 177.69162777066
},
"score": 0.99799561500549
},
{ // left eye
"position": {
"x": 326.05302262306,
"y": 122.9596464932
},
"score": 0.99766051769257
},
{ // right eye
"position": {
"x": 258.72196650505,
"y": 127.51624706388
},
"score": 0.99926537275314
},
...
]
}
[
{ // pose #1
"score": 0.42985695206067,
"keypoints": [
{ // nose
"position": {
"x": 126.09371757507,
"y": 97.861720561981
},
"score": 0.99710708856583
},
...
]
},
{ // pose #2
"score": 0.13461434583673,
"keypositions": [
{ // nose
"position": {
"x": 116.58444058895,
"y": 99.772533416748
},
"score": 0.9978438615799
},
...
]
},
...
]

8. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
THE POSENET MODEL
6 - 7 million
120 million

9. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
HEATMAPS AND OFFSETS
6 - 7 million
120 million

10. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
HEATMAPS AND OFFSETS
6 - 7 million
120 million

11. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
STRIDES AND ACCURACY
6 - 7 million
120 million

12. LANCASTER AISKELETON TRACKING WITH TENSORFLOW
6 - 7 million
120 million
- Special thanks to Dan Oved and his Medium article on OpenPose
and using Tensorflow.js