Learning Objectives: - Learn how to easily get started with Deep Learning on AWS using the AWS Deep Learning AMI - Learning how to use Apache MXNet and Gluon to start and scale deep learning projects - Learn how to build an LSTM network for sentiment analysis

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vikram Madan, Sr. Product Manager
November 14, 2017
Sentiment Analysis Using
Apache MXNet and Gluon

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Sentiment analysis
Sentiment analysis is a natural language processing (NLP) task for classifying
text or speech as some specified sentiment
Brand Sentiment Political Sentiment Customer Satisfaction

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Performing sentiment analysis
Labelled Data
for Training
Word
Embedding
Train Deep
Neural Network
Deploy Model
for Prediction
Focus of Today’s Discussion

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
IMBD dataset for movie reviews
Positive
“The impetuous pace of the film
is at one with its moral
shamelessness, and, without
thinking, we sign up for both”
“What a magical movie”
“It is a film so awe-inspiringly
wooden that it’s basically a fire-risk”
“An explosion in a stupid factory”
“Love him, hated Hur”
Negative
25,000 movie reviews: 12,500 positive & 12,500 negative

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Converting words to vectors
Two Major Approaches: word2vec & GloVe

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Converting words to vectors
Training Co-Occurrence Data Output Vectors

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Converting words to vectors
Example: 3-Dimensional Word Vectors

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
GloVe: Global Vectors for Word Representation

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deep Neural Networks

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Recurrent Neural Networks (RNNs)

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Long Short-Term Memory (LSTM) networks

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Illustrative Example (Part 1 of 2)
Edwin Chen Blog (http://blog.echen.me/)

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Illustrative Example (Part 2 of 2)
Edwin Chen Blog (http://blog.echen.me/)

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Gluon API Specification
Gluon is a technical specification providing a programming
interface that: (1) simplifies development of deep learning models,
(2) provides greater flexibility in building neural networks, and (3)
offers high performance
Simplicity Flexibility Performance

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Gluon API is Available in Apache MXNet
Others…
Development of the Gluon API specification was
a collaboration between AWS and Microsoft
Available Now
Coming Soon
Gluon API is implemented in Apache
MXNet and soon will be in CNTK

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Advantages of the Gluon API
Simple, Easy-to-
Understand Code
Flexible, Imperative
Structure
Dynamic Graphs
High Performance
 Neural networks can be defined using simple, clear, concise code
 Plug-and-play neural network building blocks – including predefined layers,
optimizers, and initializers
 Eliminates rigidity of neural network model definition and brings together
the model with the training algorithm
 Intuitive, easy-to-debug, familiar code
 Neural networks can change in shape or size during the training process to
address advanced use cases where the size of data fed is variable
 Important area of innovation in Natural Language Processing (NLP)
 There is no sacrifice with respect to training speed
 When it is time to move from prototyping to production, easily cache neural
networks for high performance and a reduced memory footprint

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Simple, Easy-to-Understand Code
Use plug-and-play neural network building blocks, including
predefined layers, optimizers, and initializers
# First step is to initialize your model
net = gluon.nn.Sequential()
# Then, define your model architecture
with net.name_scope():
net.add(gluon.nn.Dense(256, activation="relu")) # 1st layer (256 nodes)
net.add(gluon.nn.Dense(256, activation="relu")) # 2nd hidden layer
net.add(gluon.nn.Dense(num_outputs)) # Output layer

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Flexible, Imperative Structure
Prototype, build, and debug neural networks more easily with a
fully imperative interface
epochs = 10
for e in range(epochs):
for i, batch in enumerate(train_data):
with autograd.record(): # Start recording the derivatives
output = net(data) # the forward iteration
loss = softmax_cross_entropy(output, label)
loss.backward()
trainer.step(data.shape[0])

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Dynamic Graphs
Build neural networks on the fly for use cases where neural
networks must change in size and shape during model training
def forward(self, F, inputs, tree):
children_outputs = [self.forward(F, inputs, child)
for child in tree.children]
#Recursively builds the neural network based on each input sentence’s
#syntactic structure during the model definition and training process

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
High Performance
Easily cache the neural network to achieve high performance
when speed becomes more important than flexibility
net = nn.HybridSequential()
with net.name_scope():
net.add(nn.Dense(256, activation="relu"))
net.add(nn.Dense(128, activation="relu"))
net.add(nn.Dense(2))
net.hybridize()