Telco: Voice-Command Personal Agent Service with AWS Cloud (MBL202) | AWS re:Invent 2013

DOCOMO CLOUD
Mick Etoh, Ph.D.
November 13, 2013
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Friday, November 15, 13

1

Who am I?

2

Who am I?
•

Mick Etoh, SVP, NTT DOCOMO

2

Who am I?
•


•

Role: R&D Strategy

2

Who am I?
•


•

Role: R&D Strategy

•

Expertise:

2

Who am I?
•


•

Role: R&D Strategy

•

Expertise:
Pattern Recognition & Machine Learning,
Signal Processing & Multimedia,
Mobile Networks

2

Who am I?
•


•

Role: R&D Strategy

•

Expertise:
Mobile Networks
Fun:

2

Who am I?
•


•

Role: R&D Strategy

•

Expertise:
Mobile Networks
Fun:
Road Bike, Fly Fishing, and Wine Lover
2


Who is DOCOMO?

3

Who is DOCOMO?
•

$61 million subscribers in Japan

3

Who is DOCOMO?
•


•

$45 billion revenue (2013)

3

Who is DOCOMO?
•


•

Next to China Mobile, Verizon, and
AT&T

3

Who is DOCOMO?
•


•

Next to China Mobile, Verizon, and
AT&T

•

“i-mode” innovator (1999)

3

Revenue Structure (FY2012)
Feature Phone Era
Data Stage
Voice Stage
Service
Cloud

Packet
Network
Voice
Network

Device

4

Revenue Structure (FY2012)
Feature Phone Era
Data Stage
Voice Stage

Smart Phone

Service
Cloud

Service $5.5 Billion

Packet
Network

Data: $19 Billion

Voice
Network

Voice: $13 Billion

Device

4

Wireless Carrier + Service Provider

5

DOCOMO CLOUD Services

6

DOCOMO CLOUD Services
docomo Phonebook

docomo Mail

Photo Collection

databox

dmarket

Schedule and Memo

ToruCa

Hanashite Hon'yaku

Utsushite Hon'yaku

dmenu

Mail-Honyaku-Concier

Shabette-Concier

docomo DriveNet

i-concier

Sugotoku Containt

6

Service Delivery Platform RENEWED

7

Service Delivery Platform RENEWED

DOCOMO CLOUD
7

Enterprises using Public Cloud in Japan

8


2.5% (2011)

Yano Research

8


7.3% (2012)

Yano Research

8

DOCOMO as a Big AWS User

9

DOCOMO as a Big AWS User
Four-Digit
EC2 Instances

9

A Telco Story About Launching
Voice-Command Personal Agent
Service with AWS Cloud
10

Siri

11

“Shabette-Concier” Voice agent service

12

“Shabette-Concier” Voice agent service
Concier

=

=

How may I help you?

Shabette
Voice

Concierge

•
•
•
•

Launched Mar. 1, 2012
Over 40 services in it
Including chatting
10 million users
12


An Entrance to Services

16

An Entrance to Services

37

22

20

search types

applications

functions

Train transfer / Last train alarm / Weather alarm / Traffic information / Receipt search / Map
search / Weather search / Image search / QA search / Movie search / Music search / News
search / Twitter search / Game search / App search / Book/Comic search / Area guide /
Browser search / Artist search / TV program search / Dictionary / Horoscope / Result search
for soccer game / Result search for baseball game / Result search for golf tournament /
Shopping / Calculate calorie of food / Sightseeing search / Call taxi / Knowledge of life /
Q&A for DOCOMO / Phone call / Launch camera / Timer / Alarm / Scedular / Memo / Music
player / Email / i-Concirge / i-Channel / DOCOMO Map / Disaster kit / Ir communication /
Post Twitter / Photo collection / Launch Galary / Translation / Cheat sheet / User support /
Flight mode / Day and time / Terminal information / Display setting / Keyboard and input /
Storage setting / Language setting / Location service setting / Application listup / Security
setting / Sound setting / User dictionary / Wi-Fi setting / Wall paper setting / Battery setting /
Communication service setting

16

Monetization

“d-shopping” : real marketplace

Just say, wanna buy!
17


Monetization
How may I help you?

I want to buy
water!

“d-shopping” : real marketplace

OK !

Just say, wanna buy!
17


“Status Quo”
Shrine

18

“99.9999%”
Temple

“Status Quo”
Shrine

18

“99.9999%”
Temple

“Status Quo”
Shrine

Office of
Inspirations

18

“99.9999%”
Temple
Wannabe
Skunkworks

“Status Quo”
Shrine

Office of
Inspirations

18

In Status Quo Shrine,
so many superstitions....
19

Is this ‘Cloud’?
In Status Quo Shrine,
so many superstitions....
19

In 99.9999% Temple,

20

In 99.9999% Temple,
Belief: High Availability Forever

20

In 99.9999% Temple,
Belief: High Availability Forever

Downtime per Year
31.5 seconds
20

Three Major Concerns to Public Cloud
Security
Performance
Switching Cost
21

At Office of Inspirations

22

At Office of Inspirations

Make a Magic Lamp
right now
with Genie (2009)

22

In Wannabe Skunkworks (2010)

23

Maybe, it’s
high time to make
a voice agent.
But, as a stealth product.

23

Maybe, it’s
high time to make
a voice agent.
Yep, we have lot of expertise & data for
speech recognition.
23


Just out of
curiosity,
I’d like to use Public
Cloud.

Maybe, it’s
high time to make
a voice agent.

Yep, we have lot of expertise & data for
speech recognition.
23

Distributed Speech Recognition (2006)

First Implementation of ETSI AURORA Project (2000-2003)
16 Khz Sampling MFCC feature vectors --> 5.6 Kbps
24

Distributed Speech Recognition (2006)
Mobile Terminal

Dictionary
Server

Feature
Extraction

Encode

Decode

Rec.
Engine

Result

First Implementation of ETSI AURORA Project (2000-2003)
16 Khz Sampling MFCC feature vectors --> 5.6 Kbps
24

In Wannabe Skunkworks
Data Mining Team
(2009)

25

In Wannabe Skunkworks
Data Mining Team
(2009)

Web 2.0 Believers
25

The paradigm shift from ‘Search’

26

The paradigm shift from ‘Search’
Microsoft Bing

Google Voice Search
26

A Small Team in Wannabe Skunkworks (2010-2012)
Product owner
Development
promoter
Core engineer
Agile
Development
Core engineer

Development
promoter
27


Basic Architecture 2010

28


(inspired by Microsoft Bing, and Google Voice Search)

28


(inspired by Microsoft Bing, and Google Voice Search)
text

Text to speech

contents

Voice

text

text

contents

Voice
Recognition

Logging

Task
Recognition

Logging

Service
Providers’ DB

28

History of Voice Agent
5/2011
Stealth Product

Data Center
29

（Hybrid with other Cloud）

5/2011
Stealth Product

3/2012
Version 1

Mar.∼
Data Center

Public
Cloud

29

5/2011
Stealth Product

3/2012
Version 1

Mar.∼
Data Center

Public
Cloud

June
AWS
NC-reg

29

5/2011
Stealth Product

3/2012
Version 1

Mar.∼
Data Center

Public
Cloud

11/2012
Version 2

June
AWS
NC-reg

Sept.
AWS
Tokyo-reg.

29

Voice Recognition
I want...

Speech Recognition

Text

Speech
Acoustic
Model

Machine Learning

Speech
Data

Lexicon

Language
Model

Machine Learning

Language
Data

30

Voice Recognition
I want...

Speech Recognition

Text

Speech
Acoustic
Model

Machine Learning

Speech
Data

Lexicon

Language
Model

Machine Learning

Language
Data

BELIEF: More data usually beats better algorithms.
30

Task Recognition and Service Flow
Restaurant

Lexicon

Task Corpus

Dictionary
I want...

Tokenizer

Text
Abstractor

Contents
Feature
extractor

Abstraction
Dictionary

MC-SVM
Classifier

Query ext.

Find a good Italian restaurant in Palo Alto
31

Task Recognition and Service Flow
Restaurant

Lexicon

Task Corpus

Dictionary
I want...

Tokenizer

Text
Abstractor

Contents
Feature
extractor

Abstraction
Dictionary

MC-SVM
Classifier

Service
Launcher

Search Engine A

Query ext.
Search Engine B
Search Engine C

Find a good Italian restaurant in Palo Alto

PRINCIPLE: Machine learning enhances the service quality.
A better quality service acquires more data.

31

Version 1 Implementation (Mar. 2012)
Started with two local cloud
providers in Japan, and soon faced
two difficulties:

32

two difficulties:

LB

A

B
32


two difficulties:

LB

A

B

• Scalability in

server count

32

two difficulties:

LB

A

B

• Scalability in

server count
• Inflexible monthly
payment
32


Scrambling
•Mar 2012
0.5M accesses/day
•Apr 2012
1.5M accesses/day
•Jul 2012
2.5M accesses/day
33

Moved to AWS (June, 2012)

34


Scale!

34

Technology
Platform
Leader

AWS Partners
Innovative
Ecosystem

Scale!

Global User
Footprint

34

System Architecture (June 2012)
SmartPhone

VPC

ELB
Management
Server

Log Server for VR

(across multiple zones)

Same as
AZ #1

TR Servers

VR Servers

Voice Recognizer(VR)

Availability Zone #1

(across multiple
zones)

LB

ELB

ELB
(across multiple
zones)
Task Recognizer(TR)


Tokenizer

Access Log Servers
Log management system


35

Killer Design Pattern: Multi-Data Center
Voice Recognition Part

Voice

VPC

Route 53

LB

VR servers


HAProxy

LB

m2.4xlarge
x ~300 !!

VR servers

36

DOCOMO Skunkworks: Cloud Natives
• For another service,
adopted 21 out of 48
design patterns.
• Among them, typically
used ‘Queuing
Chain’

and
‘Scale
Out’
Pa4erns.

37

Other Crucial Design Patterns

38

•Scheduled Scale Out

38

•Scheduled Scale Out
•Clone Server

38

Traffic Pattern
Normal traffic

0:00

1:00

2:00

3:00

4:00

5:00

6:00

7:00

8:00

9:00

10:00

11:00

12:00

13:00

14:00

15:00

16:00

17:00

18:00

19:00

20:00

21:00

22:00

23:00

39

Bursty traffic for which Auto-Scale fails
TV commercial causes a spike traffic.

40

30x traffic
# of servers

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

40

10min

30x traffic
# of servers

20:38

0

20:40

1

2

20:43

3

4

5

20:45

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

40

Unexpected Traffic
Scheduled
–TV Commercial
–Introduction by TV program
fortunately pre-scheduled!
Suddenly happened
–Earthquake
–New Feature Release

burst

burst,
but gradually tapering
41


Prepare for Launching Servers
Refresh AMI every day
–VR model has been
changed everyday
Launch AMI for bursty
traffic
–10 min - 30 min to launch
•Loading VR model is
taking time....
42

More intelligence

24hours

43

More intelligence

24hours

Optimized server deployment using
Elastic Map Reduce (EMR) for pattern analysis
43

Moved twice

• Apr 2012
–Start to plan for moving-out to AWS
• Jun 2012
–Moved to AWS Northern California
• Tokyo did not have enough
instances at that time
• Deploy mixture instance types
(c1.xlarge and m2.4xlarge)
• Sep 2012
–Moved to AWS Tokyo
• Unified to m2.4xlarge
–Improved latency


44

October, 2012

“Status Quo”
DOCOMO
“99.9999%”
Shrine
Skunkworks
Temple
Office of
Inspirations
(not ‘Wannabe’ anymore)
45

October, 2012

Kid, your game
is over.

“Status Quo”
DOCOMO
“99.9999%”
Shrine
Skunkworks
Temple
Office of
Inspirations
45

October, 2012

Use our
stable system.

Kid, your game
is over.

“Status Quo”
DOCOMO
“99.9999%”
Shrine
Skunkworks
Temple
Office of
Inspirations
45

October, 2012

Use our
stable system.

Kid, your game
is over.

Move on
quickly!

“Status Quo”
DOCOMO
“99.9999%”
Shrine
Skunkworks
Temple
Office of
Inspirations
45

October, 2012
Catch me
if you can
SCALE!

Use our
stable system.

Kid, your game
is over.

Move on
quickly!

“Status Quo”
DOCOMO
“99.9999%”
Shrine
Skunkworks
Temple
Office of
Inspirations
45

In DOCOMO Skunkworks (2013)

46


They are SLOW and
pricy. Keep Restless
improvement and our leanstartup culture.

46


They are SLOW and
Yep, let’s
continue our system
improvement on the fly.
46


Hey, leader,
how about multi-lingual
translation services?

They are SLOW and
Yep, let’s
continue our system
improvement on the fly.
46


Now, we have …

17-Nov-16 27-Nov-16 7-Dec-16 17-Dec-16 27-Dec-16 6-Jan-17 16-Jan-17 26-Jan-17 5-Feb-17 15-Feb-17 25-Feb-17 7-Mar-17 17-Mar-17 27-Mar-17 6-Apr-17 16-Apr-17 26-Apr-17 6-May-17 16-May-17 26-May-17 5-Jun-17 15-Jun-17 25-Jun-17

17-Nov-12

5-Jul-17

15-Jul-17 25-Jul-17

4-Aug-17 14-Aug-17 24-Aug-17

Number of Speech

• 10 million terminals pre-installed
• More than 4M distinct user accesses
• 1M accesses / day
47

Other Tips
• Instance Management
✓Simple DB as a key-value
store

• Log management

✓Enormous logs from servers
✓Direct write to S3

• Multi-AZ

✓HA and act-act configuration
48


Thanks to the people who gave great
lessons to ‘DOCOMO Skunkworks.’

49

in changing our development style to

49

“Deploy first, think later,”

49

in bringing us a high performance culture,
and

49

in bringing us a high performance culture,
and
in becoming “Cloud Natives.”
49

Please give us your feedback on this
presentation

MBL202
As a thank you, we will select prize
winners daily for completed surveys!

Thank You

50

Telco: Voice-Command Personal Agent Service with AWS Cloud (MBL202) | AWS re:Invent 2013

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Telco: Voice-Command Personal Agent Service with AWS Cloud (MBL202) | AWS re:Invent 2013

Semelhante a Telco: Voice-Command Personal Agent Service with AWS Cloud (MBL202) | AWS re:Invent 2013 (20)

Mais de Amazon Web Services

Mais de Amazon Web Services (20)

Último

Último (20)

Telco: Voice-Command Personal Agent Service with AWS Cloud (MBL202) | AWS re:Invent 2013