Danny Lange - Artificial Intelligence - A Game Changer in App Development

1. Artificial Intelligence
A Game-changer in App Development
Danny Lange
Vice President of AI & ML
Unity Technologies

2. Head of Machine
Learning
GM of Amazon ML
Retail / AWS
Development Manager
Analytics / ML Toolkit
Computer Scientist
IBM Research
VP of AI & ML
About Danny Lange

3. About Unity
Leading global game industry platform
• 16 Billion Downloads in 2016
• Over 2.6 Billion Unique Devices
• 1 Billion Monthly Active Players
• 40% of top 1000 free mobile games

4. Windows Phone
Windows Desktop
Windows Store

7. From Computer Programming
to Computer Training

8. Typical Business
Challenges
• Lower prices
• Faster delivery
• Higher customer service expectations
• Demand volatility
• High number of products
• Supply complexities
• More frequent shipments
• Transparency and sustainability

9. A Brief History of Artificial Intelligence
Or the end of programming, rule-based systems, and information curation
Algorithm & Rule-based
Custom HW/SW
Reinforcement Learning
Commodity HW/SW
• Shannon’s Computer Chess (1949) - Minimax Algorithm
• Deep Blue vs Garry Kasparov (1997) - Custom HW/SW
• IBM Watson Wins Jeopardy (2011) - Curated Data + Q&A
• AlphaGo vs Lee Sedol (2016) - Deep Learning + GPU

10. What Was Before Machine Learning?
PROGRAM
“ALL-KNOWING
PROGRAMMER”
DATA RESULTS
FEEDBACK
Clockwork Universe

11. What is Machine Learning
MODEL
LEARNER
DATA PREDICTIONS
HISTORIC DATA
Indeterminism

12. Machine Learning Explained
SQUARE
FOOTAGE
BED-
ROOMS
AGE SCHOOL
RATING
PRICE
2,400 3 12 99 $650,000
3,500 5 7 90 $750,000
1,800 3 23 75 $350,000
2,700 4 1 75 $550,000
GROUND
TRUTH
PRICE(SQUARE FOOTAGE, BEDROOMS, AGE, SCHOOL RATING) =
W1 X SF + W2 X BR + W3 X AGE + W4 X SR
SQUARE
FOOTAG
E
BED-
ROOMS
AGE
SCHOOL
RATING
PRICE
W1
W2
W3
W4
INPUT
OUTPUT

13. Types of Machine Learning
• Unsupervised
• E.g., looking for geometrical closeness (clustering)
• Supervised
• Ground truth driven learning (most common)
• Online
• Learning happens as predictions are made
• Active
• Learner queries for specific examples to improve
• Reinforcement
• Learning from rewards function (“good dog / bad dog”)

14. OODA Loop (John Boyd)
Heisenberg's Uncertainty Principle
• A limit on our ability to observe reality with precision.
Gödel's Incompleteness Theorem
• Any model of reality is incomplete and must be continuously
refined in the face of new observations.
Second Law of Thermodynamics (Entropy) - Ludwig Boltzman
• Any given system is continuously changing even as we try to maintain order

15. Making Computers Learn
From Exploration to Exploitation

16. Multi-armed Bandit & Reinforcement Learning
Objective: Maximize winnings
Exploration vs Exploitation
• Gaining knowledge
• Max payout with current knowledge
Reinforcement Learning
• Actions
• Rewards
X% Y% Z%
AGENT
ENVIRONMENT
ACTION
STATE &
REWARD

18. How did the Chicken Cross the Road?
Actions
Rewards
• Fatal penalty (being hit by a car)
• Positive reward (collecting gift packet)
Exploration Exploitation

20. Let us move onto the real world

22. Learning to Become a Gymnast
Unity Physics Engine

24. DeepMind Playing Atari

26. Learning a Complex Skill
Developing an Asset

27. Navigating by Reading Maps
(No GPS)

30. Network Architecture
• Deep Recurrent Q-Network
• Dual-stream input
• Train network with additional
auxiliary losses
Actions
Inputs
Auxiliary signals

31. Unity Reinforcement Learning API
Action
State & Reward

32. Self-driving Vehicles

35. Shift from Programming to Learning

36. Why Now?
Shift from programming to training
• The great AI paradigm shift – Moving from programming to training
• Computing power – Cloud Computing and Graphical Processing Units (GPU)
• Abundance of data – Big Data, Cloud Storage, and synthetic data
• Change is accelerating – Only question of scale
• Automation of high-value jobs – Attractive ROI
• No turning back – Ahead of the curve; looking around corners

37. AI/ML in Every Corner of the Organization
Some Key Tenets of Applied AI/ML
Machine Learning is Becoming an Integral Part of Engineering Infrastructure
• Rich set of proven learning algorithms optimized for your organization
• Scalable infrastructure that can handle large data sets
• Built-in integration with your organization’s data
• Easy to use with lots of automation - UI/API, tools, & pipelines
• Hosted models with low-latency access
• Model management (WWW of data & models)
• Experimentation: integrated decision making - A|B testing

38. Applied AI/ML Architecture
MACHINE LEARNING
AS-A-SERVICEPIPELINES
WEB USER
INTERFACE
REST API / SDK
PREDICTIONSYOUR DATA
BATCH
REAL-TIME
YOUR APPS
MODEL
MANAGEMENT

39. AI/ML Cloud Services
How to get started
• Low hanging fruit: Business problem - “If we just knew…”
• Start supervised: Historic data with ground truth
• Do not start with Big Data
• Use cloud-based offerings
• Amazon Machine Learning on AWS
• Microsoft Azure Machine Learning
• Google Cloud Machine Learning

40. Bridging from Virtual to Real

43. Danny Lange
Vice President, AI & Machine Learning
dlange@unity3d.com
@danny_lange
dannylange