Forrest Iandola is the co-founder and CEO of DeepScale, a company that develops perception systems for autonomous vehicles using deep learning. The document outlines his background and experiences that led him to start DeepScale, including his childhood interest in vehicles, PhD research on neural networks, and student startup Visiblend. It then discusses DeepScale's work on developing small and efficient neural network models and implementations that can run on embedded hardware for real-time perception tasks in autonomous driving.

1. F O R R E S T
I A N D O L A
Co-­‐founder
and
CEO,
DeepScale
My
Adventures
in
Artificial
Intelligence
and
Entrepreneurship

2. DeepScale
PERCEPTION
SYSTEMS
FOR
AUTONOMOUS
VEHICLES
http://deepscale.ai

3. THE
STACK
All
levels
of
vehicle
automation
require
this
flow
to
work.
DeepScale
specializes
in
Real-­‐Time
Perception.
SENSORS
LIDAR
ULTRASONICCAMERA
RADAR
OFFLINE
MAPS
REAL-­‐TIME
PERCEPTION
PATH
PLANNING
&
ACTUATION

4. Outline
1. What
I
wanted
to
be
when
I
grow
up
2. My
experience
as
a
PhD
student
and
a
student-­‐entrepreneur
at
UC
Berkeley
3. The
founding
of
DeepScale
4. Lessons
that
I've
learned
along
the
way

5. For
as
long
as
I
can
remember,
I
have
been
obsessed
with
cars
and
trucks

6. Fo's
Truck
Company

7. Illinois
Math
and
Science
Academy
(2005)

8. Mechanical
Engineering
Building
at
University
of
Illinois
(2008)

9. Siebel
Center
for
Computer
Science
at
University
of
Illinois
(2008)

10. What
my
classmates
planned
to
do
after
finishing
their
undergraduate
degree
in
CS
at
University
of
Illinois
(2012)
Entry-­‐level
jobs
at
Big
Companies
Starting
Companies
…and
countless
smartphone
app
startups Grad
School

11. Starting
my
PhD
in
EECS
at
UC
Berkeley
(2012)

12. My
early
PhD
research
(2013-­‐2014):
Creating
deep
neural
networks
that
are
fast
and
lightweight

13. My
summer
at
Microsoft
Research
(2014)
Dr.
John
Platt
Machine
Learning
Dr.
Meg
Mitchell
Natural
Language
Processing
Dr.
Piotr
Dollar
Computer
Vision
Dr.
Li
Deng
Speech
Recognition
Dr.
Xiaodong He
Deep
Learning
M E N T O R S

14. My
summer
at
Microsoft
Research
(2014)
H.
Fang,
S.
Gupta,
F.
Iandola,
R.
Srivastava,
L.
Deng,
P.
Dollar,
J.
Gao,
X.
He,
M.
Mitchell,
J.C.
Platt,
C.L.
Zitnick,
G.
Zweig.
"From
Captions
to
Visual
Concepts
and
Back."
Computer
Vision
and
Pattern
Recognition
(CVPR),
2015.
Deep
Neural
Networks
"a
group
of
people
riding
bikes
down
a
street"
D E E P
N E U R A L
N E T W O R K S
F O R
I M A G E
C A P T I O N I N G

15. Visiblend
MY
NIGHTS-­‐AND-­‐WEEKENDS
PROJECT
WHILE
AT
MICROSOFT
RESEARCH
Download
YouTube
Videos
Use
deep
learning
to
detect
products
in
videos
Place
relevant
ads
on
videos

16. The
time
I
almost
dropped
of
grad
school
(2014)
Forrest
Iandola:
"I'm
going
to
drop
out
of
the
PhD
program
to
focus
on
Visiblend."
Prof.
Kurt
Keutzer:
1. "Have
you
thought
about
who
would
pay
to
use
Visiblend's
product?"
2. "How
about
you
focus
your
PhD
research
on
practical
problems
that
you
might
face
while
using
deep
learning
at
Visiblend?"

17. Lean
Launchpad
Class
@
UC
Berkeley
(2015)
www.berkeleyleanlaunchpad.com

18. Visiblend
(2015)BUT
WE
LEARNED…
As
of
2015,
brand
advertisers
weren't
willing
to
pay
extra
for
additional
user
engagement

19. The
time
I
almost
dropped
of
grad
school
Forrest
Iandola:
"I'm
going
to
drop
out
of
the
PhD
program
to
focus
on
Visiblend."
Prof.
Kurt
Keutzer:
1. "Have
you
thought
about
who
would
pay
to
use
Visiblend's
product?"
2. "How
about
you
focus
your
PhD
research
on
practical
problems
that
you
might
face
while
using
deep
learning
at
Visiblend?"

20. FireCaffe
(2015)
RAPID
TRAINING
OF
DEEP
NEURAL
NETWORKS
F.N.
Iandola,
K.
Ashraf,
M.W.
Moskewicz,
and
K.
Keutzer.
"FireCaffe:
near-­‐linear
acceleration
of
deep
neural
network
training
on
compute
clusters."
Computer
Vision
and
Pattern
Recognition
(CVPR),
2016.
height: 1
serialized
munication: 2
serialized
communication: p
height:
log2(p) height: 1
serialized
communication: 2
serialized
communicatio
Traditional
approach
(2012-­‐2015):
Central
parameter
server
Our
approach
in
FireCaffe
(2015):
Reduction
tree
Up
to
150x
speedup
on
256
GPUs
(from
weeks
to
hours)

21. DeepScaleTHE
BEGINNING
(2015)
50x
speedup

22. DeepScaleWHAT
WE
LEARNED
FROM
CUSTOMERS
(2016)
First
we
learned…
• Automakers
have
gathered
lots
of
data
for
training
computer
vision
models
– for
use
in
autonomous
driving
• Training
deep
neural
nets
on
these
data
volumes
is
really
slow
(months
or
even
years)
• DeepScale's
distributed
training
system
can
help
After
more
discussions,
we
learned…
• Automakers
would
prefer
to
just
buy
the
"right"
deep
neural
networks,
already
trained
• For
mass-­‐production,
automakers
have
really small
processors
(100x
less
computation
than
full-­‐size
NVIDIA
GPUs)

23. Our
full-­‐stack
approach
TO
DEEP
NEURAL
NETWORKS
Train
rapidly
using
multiprocessor
scaling
Collect
and
annotate
adequate
training
data
Develop
small
and
efficient
deep
neural
network
architectures
Create
efficient
implementations
for
embedded
hardware
F IREC AF F E T H E
D E E P S C A L E
C A R
SQ U EEZEN ET
B O DA

24. DeepScale's
Data
Collection
Car
(2016)

25. M.
Moskewicz,
F.
Iandola,
and
K.
Keutzer.
"Boda-­‐RTC:
Productive
Generation
of
Portable,
Efficient
Code
for
Convolutional
Neural
Networks
on
Mobile
Computing
Platforms."
WiMob,
2016
Boda
(2016)
EMBEDDED
IMPLEMENTATIONS
OF
DEEP
NEURAL
NETWORKS
Latency/Throughput Portability
CPU GPU ASIC

26. SqueezeNet
(2016)
A
SMALL DEEP
NEURAL
NETWORK
FOR
EMBEDDED
APPLICATIONS
SqueezeNet
is
built
out
of
"Fire
modules:"
F.N.
Iandola,
S.
Han,
M.
Moskewicz,
K.
Ashraf,
W.
Dally,
K.
Keutzer.
"SqueezeNet:
AlexNet-­‐level
accuracy
with
50x
fewer
parameters
and
<0.5MB
model
size."
arXiv,
2016.
http://github.com/DeepScale/SqueezeNet
1x1
conv
1x1
conv 3x3
conv
"squeeze"
"expand"

27. Compression
Approach
DNN
Architecture
Original
Model Size
Compressed
Model Size
Reduction in
Model Size
vs. AlexNet
Top-­1
ImageNet
Accuracy
Top-­5
ImageNet
Accuracy
None
(baseline) AlexNet
[1] 240MB 240MB 1x 57.2% 80.3%
SVD
[2] AlexNet 240MB 48MB 5x 56.0% 79.4%
Network
Pruning
[3] AlexNet 240MB 27MB 9x 57.2% 80.3%
Deep
Compression
[4] AlexNet 240MB 6.9MB 35x 57.2% 80.3%
None SqueezeNet
[5]
(ours)
4.8MB 4.8MB 50x 57.5% 80.3%
Deep
Compression
[4] SqueezeNet
[5]
(ours)
4.8MB 0.47MB 510x 57.5% 80.3%
[1]
A.
Krizhevsky,
I.
Sutskever,
G.E.
Hinton.
ImageNet
Classification
with
Deep
Convolutional
Neural
Networks.
NIPS,
2012.
[2]
E.L
.Denton,
W.
Zaremba,
J.
Bruna,
Y.
LeCun,
R.
Fergus.
Exploiting
linear
structure
within
convolutional
networks
for
efficient
evaluation.
NIPS,
2014.
[3]
S.
Han,
J.
Pool,
J.
Tran,
W.
Dally.
Learning
both
Weights
and
Connections
for
Efficient
Neural
Networks,
NIPS,
2015.
[4]
S.
Han,
H.
Mao,
W.
Dally.
Deep
Compression…,
arxiv:1510.00149,
2015.
[5]
F.N.
Iandola,
M.
Moskewicz,
K.
Ashraf,
S.
Han,
W.
Dally,
K.
Keutzer.
SqueezeNet:
AlexNet-­‐level
accuracy
with
50x
fewer
parameters
and
<1MB
model
size.
arXiv,
2016.
SqueezeNet
(2016)
A
SMALL DEEP
NEURAL
NETWORK
FOR
EMBEDDED
APPLICATIONS

28. Our
full-­‐stack
approach
TO
DEEP
NEURAL
NETWORKS
Train
rapidly
using
multiprocessor
scaling
Collect
and
annotate
adequate
training
data
Develop
small
and
efficient
deep
neural
network
architectures
Create
efficient
implementations
for
embedded
hardware
F IREC AF F E T H E
D E E P S C A L E
C A R
SQ U EEZEN ET
B O DA

29. WHAT
WE
LEARNED
ABOUT
PERCEPTION
SYSTEM
APPROACHES
IN
AUTOMOTIVE
(2017)

30. TRADITIONAL
COMPUTER
VISION
• Dedicated
processor
bundled
with
specific
camera
in
a
closed
module
• Pre-­‐dates
Deep
Neural
Networks
à narrow
capability
based
on
hard-­‐coded
algorithms
(e.g.
only
detect
cars
from
certain
angles)
• Major
revisions
dictated
by
hardware
development
cycles
of
2-­‐3
years
(an
eternity
given
how
fast
AI
is
changing)
Approach
#1

31. OPEN-­‐SOURCE
DEEP
NEURAL
NETWORKS
[1]
S
Ren,
K
He,
R
Girshick,
J
Sun.
Faster
R-­‐CNN.
NIPS,
2015.
[2]
J
Redmon,
A
Farhadi.
YOLO9000.
CVPR,
2016.
[3]
W
Liu,
et
al.
SSD:
Single
shot
multibox detector.
ECCV,
2016
• Modern
Deep
Neural
Networks
(DNNs)
have
brought
order-­‐of-­‐magnitude
improvements
in
perception
accuracy
…but,
real-­‐time
DNNs
for
object
detection
require
250W+
of
GPU
computing
[1,2,3]
• This
leads
to
a
trunk
full
of
hot,
expensive,
power-­‐hungry
servers
Approach
#2

32. DEPLOYING
DEEP
NEURAL
NETWORKS
IN
MASS-­‐PRODUCED
VEHICLES
• Server
cost:
$20,000+
• Power:
250W
to
2000W+
Without
DeepScale With
DeepScale
• Chip
Cost:
$20
-­‐ $100
• Power:
10W
Approach
#3

33. My
story
in
one
slide
1993:
Planned
to
start
"Forrest's
Truck
Company"
when
I
grow
up
2008:
Went
to
college
at
University
of
Illinois
with
the
plan
to
study
Mechanical
Engineering
and
then
go
into
automotive
industry
2009:
Switched
to
computer
science;
gave
up
my
dreams
of
working
in
automotive
2012:
Considered
starting
a
startup,
but
choose
grad
school
instead
2013:
Started
doing
research
in
deep
learning
2015:
Started
DeepScale
2016:
Graduated
with
PhD
in
EECS
2016:
Focused
DeepScale
entirely
on
the
automotive
industry
2018:
Today,
DeepScale
is
on
track
to
supply
lifesaving
deep
learning
software
to
automakers
You
can't
connect
the
dots
looking
forward;
you
can
only
connect
them
looking
backwards.
So,
you
have
to
trust
that
the
dots
will
somehow
connect
in
your
future."
– Steve
Jobs

34. Conclusions
and
advice
• You
don't
have
a
business
until
you
know
what
you're
going
to
build,
and
who
will
pay
for
it.
• At
DeepScale,
we
spent
a
lot
of
time
optimizing
our
product-­‐market
fit,
and
it
has
paid
off.
• As
an
entrepreneur,
it's
energizing
to
have
a
mission
that
you
and
your
team
believe
in.
• For
us
at
DeepScale,
it's
about
using
deep
neural
networks
to
save
lives
on
the
road.
• It's
ok
if
you
don't
know
your
mission
right
away.
Keep
looking
for
it.