25. @elbruno
MNIST dataset
• The MNIST database of
handwritten digits,
• Has a training set of 60,000
examples, and a test set of 10,000
examples.
• It is a subset of a larger set
available from NIST.
• The digits have been size-
normalized and centered in a
fixed-size image.
32. @elbruno
A machine learning subfield of learning representations of data. Exceptional effective at
learning patterns.
Deep learning algorithms attempt to learn (multiple levels of) representation by using a
hierarchy of multiple layers
If you provide the system tons of information, it begins to understand it and respond in
useful ways.
Deep Learning (DL)
https://www.xenonstack.com/blog/static/public/uploads/media/machine-learning-vs-deep-learning.png
35. @elbruno
Open source software library for numerical computation using data
flow graphs
Developed by Google Brain Team for machine learning and deep
learning and made open-source
TensorFlow provides an extensive suite of functions and classes that
allow users to build various models from scratch
What is TensorFlow?
These slides are adapted from the following Stanford lectures:
https://web.stanford.edu/class/cs20si/2017/lectures/slides_01.pdf
https://cs224d.stanford.edu/lectures/CS224d-Lecture7.pdf
36. @elbruno
import tensorflow as tf
a = tf.add(2, 3)
TF automatically names nodes
if you do not
x = 2
y = 3
print a
>> Tensor("Add:0", shape=(), dtype=int32)
Note: a is NOT 5
TensorFlow Graphs
a
37. @elbruno
Keras is a layer on top of TensorFlow that makes it much easier to create neural
networks.
It provides a higher level API for various machine learning routines.
Unless you are performing research into entirely new structures of deep neural
networks it is unlikely that you need to program TensorFlow directly.
Keras is a separate install from TensorFlow. To install Keras, use pip install keras
(after installing TensorFlow).
TensorFlow and Keras
38. @elbruno
Keras Sequential model is used to
create a feed-forward network, by
stacking layers (successive ‘add’
operations).
Shape of the input layer is specified
in the first hidden layer (or the
output layer if network had no
hidden layer).
- Input 2D image is flattened to 1D
vector.
- Dropout (with the rate 0.2) is
applied to the first hidden layer
TensorFlow for Classification: MNIST
40. @elbruno
Pooling layer
Pooling layers are a form of downsampling which
usually follow convolution layers in the neural network.
Applying pooling to a feature map transforms the map
into a smaller representation, and it loses some of the
exact positional information of the features. Therefore it
makes our network more invariant to small
transformations and distortions in the input image by
asking whether a feature appears in a given region of
an image (the pooling region) rather than at a specific
location.
The most common method of pooling is max-pooling,
where the the maximum value of a given region in a
feature map is taken. The example below shows max
pooling being applied with a 2×2 pooling region.
https://shafeentejani.github.io/assets/images/pooling.gif
41. @elbruno
Activation: ReLU
Takes a real-valued number
and thresholds it at zero
𝑅 𝑛 → 𝑅+
𝑛
Most Deep Networks use ReLU nowadays
� Trains much faster
• accelerates the convergence of SGD
• due to linear, non-saturating form
� Less expensive operations
• compared to sigmoid/tanh (exponentials etc.)
• implemented by simply thresholding a matrix at
f 𝑥 = max(0, 𝑥)
http://adilmoujahid.com/images/activation.png
43. @elbruno
Deep Neural Network: Cat vs Dog
https://becominghuman.ai/building-an-image-classifier-using-deep-learning-in-python-totally-from-a-beginners-perspective-be8dbaf22dd8
44. @elbruno
Deep Neural Network: Cat vs Dog
https://becominghuman.ai/building-an-image-classifier-using-deep-learning-in-python-totally-from-a-beginners-perspective-be8dbaf22dd8