The Soul in The Machine - Developing for Humans
Chris Heilmann @codepo8 - May 2017

PROLOGUE
“VISIONS OF THE FUTURE”

Arthur C
Clarke
Aldous
Huxley
George
Orwell
Alan
Moore

http://www.independent.co.uk/life-style/gadgets-and-tech/news/theresa-may-internet-conservatives-government-a7744176.html

https://www.newscientist.com/article/2113299-
googles-deepmind-ai-can-lip-read-tv-shows-
better-than-a-pro/?sf43896303=1
https://arxiv.org/pdf/1611.05358v1.pdf
46.8% vs 12.4% success rate

Technological progress
has merely provided
us with more efficient
means for going
backwards.
— Aldous Huxley

Duophonix @ Deviantart http://duophonix.deviantart.com/art/Ihal-9000-184254683

CHRIS HEILMANN
@CODEPO8
HTTPS://CHRISTIANHEILMANN.COM

ACT ONE
“CREATURE COMFORTS WE ALL ENJOY”

Things machines and software can do for humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
https://www.bloomberg.com/news/articles/
2017-02-28/jpmorgan-marshals-an-army-of-
developers-to-automate-high-finance

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
https://arxiv.org/abs/1702.00783

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
https://gwmt.nvidia.com/
http://www.trustedreviews.com/news/
nvidia-super-resolution-tool-launch

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
Image may contain: dog,
outdoor and nature

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
https://blogs.office.com/2016/12/02/accessibility-in-
office-365-enabling-greater-digital-inclusion/

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm http://captionbot.ai

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
https://www.youtube.com/watch?v=XMdct-5bERQ

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
https://research.googleblog.com/2016/05/aw-so-cute-
allo-helps-you-respond-to.html

Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

ACT TWO
“ENTER THE ENABLERS”

A few years ago all this was pure science
fiction…

And even today it feels like it is too hard
to do, right?

Wrong. The trick is to understand that we
live in a world of services, clouds,
databases and products.

All players are doing a great job giving you access…
https://www.youtube.com/watch?v=ETeeSYMGZn0

Almost every company offers APIs for this.

REST endpoints with support for Node or
even client-side JavaScript.

Images
Audio
Video
(Foreign) Text
Gestures
Faces
Emotions
…
build-stuff.json

A lot of work went into these APIs, work
you might not want to do yourself…

Machines, corporations, services and
products have harvested tons and tons of
data over the last years.

The biggest problem was to understand
the data – as it was a mess.

So we trained machines in thousands of
iterations to get the best information out
of a small set of fata.

What’s in a sentence?

Word dependencies

Labels

Speech / Language Syntax

Language stems

Morphology / Grammar

Better translation through sentences vs. words

Better translation through sentences vs. words

Better translation through sentences vs. words

Bad results? Use the same platforms to
train your own data according to your
specifications.

Add quality by filtering with context…
https://www.microsoft.com/cognitive-services/en-us/custom-recognition-intelligent-service-cris

The good news for us is that we can now
use these findings to build better
interfaces.

ACT THREE
“GO FORTH AND DO SOME GOOD”

The interfaces we
build today condition
the users of now and
of tomorrow…

The interfaces we
build today condition
the users of now and
of tomorrow…

It is time for us to use and build better, more
human interfaces…

There is no such thing as a perfect user - think inclusive.
https://www.microsoft.com/en-us/Design/inclusive
It isn’t about allowing access but about avoiding barriers.

Our job right now is to
create interfaces that are
simple, human and fun
to use…

What we can do with technology for us and our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users
http://code.visualstudio.com/

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users
https://channel9.msdn.com/Events/Build/2017/T6078

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users
JSON:
{
"language": "en",
"orientation": "Up",
"regions": [
{
"boundingBox": "41,77,918,440",
"lines": [
{
"boundingBox": "41,77,723,89",
"words": [
{
"boundingBox": "41,102,225,64",
"text": "LIFE"
},
{
"boundingBox": "356,89,94,62",
"text": "IS"
},
{
"boundingBox": "539,77,225,64",
"text": "LIKE"
}
. . .
TEXT:
LIFE IS LIKE
RIDING A BICYCLE
TO KEEP YOUR BALANCE
YOU MUST KEEP MOVING
JSON:
{
"language": "en",
"orientation": "Up",
"regions": [
{
"boundingBox": "41,77,918,440",
"lines": [
{
"boundingBox": "41,77,723,89"
"words": [
{
"boundingBox": "41,102,225,6
"text": "LIFE"
},
{
"boundingBox": "356,89,94,62
"text": "IS"
},
{
"boundingBox": "539,77,225,6
"text": "LIKE"
}
. . .
TO KEEP YOUR BALANCE
YOU MUST KEEP MOVING

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users
http://cloudinary.com/blog/
automatically_art_directed_responsive_images

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users
https://www.videobreakdown.com/

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users
🎙"
🔊$

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

EPILOGUE
“OH THE PLACES YOU WILL GO…”

The saddest aspect of
life right now is that
science gathers
knowledge faster than
society gathers wisdom.
— Isaac Asimov

The future is already
here – it’s just not very
evenly distributed.
— William Gibson

Let’s fix that! Right now!

CHRIS HEILMANN
@CODEPO8
CHRISTIANHEILMANN.COM
THANKS!