Exploring Artificial Intelligence and in practice, with several network architectures implemented on the fly in a live coding session. In this talk we went through Shallow / Intermediate / Deep / Convolutional and Residual Networks. Showing the differences in hyperparameters tuning, which to use and when. In addition, also how they came to life though the mind of computer scientists like Geoffrey Hinton, Yoshua Bengio, Xavier Glorot and Adam Coates. The code presented in this talk can be found here: https://github.com/ekholabs/DLinK

1. Artificial Intelligence
A Practical Approach

2. Who am I?
Software & Machine Learning
Engineer;
City.AI Ambassador;
IBM Watson AI XPRIZE contestant;
Kaggler;
Guest attendee at AI for
Good Global Summit at the UN;
X-Men geek;
family man and father of 5 (3
kids and 2 cats).
@wilderrodrigues

3. Machine Learning
without a Ph.D.

4. What is it About?
Source: World Wide Web Foundation, June 2017

5. Points to Focus On
Number and Types of Layers;
Initialisation;
Regularisation;
Cost Functions;
Optimisation;
Data Augmentation.

6. Shallow Neural Networks
Random Initialisation
Mean Squared Error
Stochastic Gradient Descent
Sigmoid

7. Initialisation and Loss Function
Zero initialisation ☹
W1 = np.zeros((layers_dims[l],
layers_dims[l - 1]))
Random initialisation 😐
W1 =
np.random.randn(layers_di
ms[l], layers_dims[l - 1])
* 10
Mean Squared Error ☹
1/m.sum(Y_hat, Y)**2

8. Xavier Glorot
L2 regularisation
Cross Entropy
Adam
Tanh
Intermediate Neural Networks

9. Initialisation and Loss Function
Xavier Glorot 😄
W1 = np.random.randn(layers[l], layers[l - 1]) *
np.sqrt(2 / (layers[l - 1] + layers[l]))
L2 regularisation ☺
(ƛ/2.m).sum(W**2)
Adam
Exponentially Weighted Averages
Vt = βVt-1 + (1 - β)𝛳t
RMSProp
Cross Entropy 😄
1/m.sum(Y.log(a)+(1-Y).log(1-a))

10. Deep Neural Networks
He
Cross Entropy
Dropout
Adam
ReLU

11. Initialisation and Cost Function
He 😄
W1 = np.random.randn(layers[l],
layers[l - 1]) * np.sqrt(2 / layers[l - 1])
Dropout 😄
❌
✅

12. Convolutional Networks
Xavier Glorot
Layers:
Convolutional Layers
Max Pooling Layers
Fully Connected Layers
Cross Entropy
Dropout
Adam
ReLU

13. Convolutional Networks
* =
6x6x3
3x3x16
4x4x16
4x4x16
2x2x16 2x2x16
* =

14. Residual Networks
Xavier Glorot
Layers:
Convolutional Layers
Bottleneck Layers
Max Pooling Layers
Fully Connected Layers
Cross Entropy
Dropout
Adam
ReLU

15. Residual Networks using Inception
5x5
Same
32
28x28x192 28x28x32
28x28x192 * 5x5x32 = 120m
1x1
16
1x1x192
28x28x192
28x28x16
5x5
Same
32
28x28x32
28x28x16 * 192 = 2.4m 28x28x32 * 5x5x16 = 10m

16. Capsule Networks
The is no pose
(translational and
rotational) relationship
between simpler features.
Successive convolutional or
max pooling layers to
reduce spacial size.

17. Capsule Networks

18. Resources and References
https://github.com/ekholabs/DLinK
Machine Learning: Andrew Ng, Stanford University, Coursera.
Neural Networks for Machine Learning: Geoffrey Hinton, University of Toronto, Coursera.
Computational Neuroscience: Rajesh Rao & Adrienne Fairhall, Washington University, Coursera.
Neural Networks and Deep Learning: Andrew Ng, DeepLearning.ai, Coursera.
Structuring Machine Learning Projects: Andrew Ng, DeepLearning.ai, Coursera.
Improving Deep Neural Networks with Hyperparameter Tuning, Regularisation and Optimisation: Andrew
Ng, DeepLearning.ai, Coursera.
Convolutional Neural Networks: Andrew Ng, DeepLearning.ai, Coursera.
Calculus I: Jim Fowler, Ohio State University, Coursera.

19. Thank you!