SlideShare uma empresa Scribd logo

Intro To Convolutional Neural Networks

•
•
M
Mark Scully

Slides from Portland Machine Learning meetup, April 13th. Abstract: You've heard all the cool tech companies are using them, but what are Convolutional Neural Networks (CNNs) good for and what is convolution anyway? For that matter, what is a Neural Network? This talk will include a look at some applications of CNNs, an explanation of how CNNs work, and what the different layers in a CNN do. There's no explicit background required so if you have no idea what a neural network is that's ok.

Dados e análise
1 de 78
Baixar para ler offline
Intro  To  Convolu,onal  Neural   Networks   Mark  Scully   datapraxis.com  
Why  CNNs?  
h@ps://papers.nips.cc/paper/4824-­‐imagenet-­‐classifica,on-­‐with-­‐deep-­‐convolu,onal-­‐neural-­‐networks   Image  Classifica,on  
Object  Recogni,on   h@ps://research.googleblog.com/2014/09/building-­‐deeper-­‐understanding-­‐of-­‐images.html  
h@p://cs.stanford.edu/people/karpathy/deepimagesent/  
Automa,c  Cap,oning   h@ps://research.googleblog.com/2014/11/a-­‐picture-­‐is-­‐worth-­‐thousand-­‐coherent.html  
Facial  Recogni,on     Y.  Taigman,  M.  Yang,  M.  Ranzato,  L.  Wolf,  DeepFace:  Closing  the  Gap  to  Human-­‐Level  Performance  in  Face  Verifica,on,  CVPR   2014  
Terminator  Vision  
Colorize  Black  &  White  Images   h@p://richzhang.github.io/coloriza,on/  
Style  Transfer   h@p://genekogan.com/works/style-­‐transfer/   Mona  Lisa  restyled  by  Picasso,  van  Gough,  and  Monet  
Generate  An  Image  From  A  Sketch   h@ps://affinelayer.com/pixsrv/  
ImageNet  Challenge   Alexnet   Li  Fei-­‐Fei:  ImageNet  Large  Scale  Visual  Recogni,on  Challenge,  2014  
ImageNet  Challenge   ILSVRC+ ImageNet Classification error throughout years and groups Li  Fei-­‐Fei:  ImageNet  Large  Scale  Visual  Recogni,on  Challenge,  2014  
Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   FC  1000  
Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012    
ImageNet  Challenge   Alexnet   Li  Fei-­‐Fei:  ImageNet  Large  Scale  Visual  Recogni,on  Challenge,  2014  
Tradi,onal  Approach  To  Image   Classifica,on   Input  Image   Hand   Extracted   Features   Classifier   Object  Label  
Issues   •  Who  makes  the  features?   – Need  an  expert  for  each  problem  domain   •  Which  features?   – Are  they  the  same  for  every  problem  type?   •  How  robust  are  these  features  to  real  images?   – Transla,on,  Rota,on,  contrast  changes,  etc.  
Are  these  pictures  of  the  same  thing?  
Features  Are  Hierarchical     •  A  squirrel  is  a  combina,on  of  fur,  arms,  legs,   &  a  tail  in  specific  propor,ons.   •  A  tail  is  made  of  texture,  color,  and  spa,al   rela,onships   •  A  texture  is  made  of  oriented  edges,   gradients,  and  colors  
Image  Features   •  A  feature  is  something  in  the  image  or  derived   from  it  that’s  relevant  to  the  task   •  Edges   •  Lines  at  different  angles,  curves,  etc.   •  Colors,  or  pa@erns  of  colors   •  SIFT,  SURF,  HOG,  GIST,  ORB,  etc  
Edges  
Ideally  We’d  Learn  Features   Input   Image   Output   Label  
Ideally  We’d  Learn  Features   Input   Image   Output   Label   CNNs  
What  is  a  Neural  Network?   •  Perceptron  is  biologically  inspired   •  A  mental  model  for  interpre,ng  the  math   h@p://cs231n.stanford.edu/index.html    
Perceptron   1   x1   x2   x3   xm   Σ   Output   Ac,va,on   Func,on   Sum   w0   w1   w2   w3   wm   Weights   Inputs  
Perceptron   1   x1   x2   x3   xm   Σ   Output   Ac,va,on   Func,on   Sum   w0   w1   w2   w3   wm   Weights   Inputs   wi xi i=0 m ∑ = w0 x0 + w1x1 + w2 x2 +...+ wm xm
Ac,va,on  Func,ons  
Training:  Upda,ng  Weights   1   x1   x2   x3   x4   Σ   Output   Ac,va,on   Func,on   Sum   w0   w1   w2   w3   w4   Weights   Inputs   Error  =  Output  -­‐  Target  
Perceptron  Decision  Boundary  
Deep  (Mul,-­‐Layer)  Neural  Network  
Backpropaga,on   •  Error  propagates  backward  and  it  all  works  via   (normally  stochas,c)  gradient  descent.   •  (wave  hands)  
Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012    
CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
Input:  Pixels  Are  Just  Numbers   h@ps://medium.com/@ageitgey/machine-­‐learning-­‐is-­‐fun-­‐part-­‐3-­‐deep-­‐learning-­‐and-­‐convolu,onal-­‐neural-­‐networks-­‐ f40359318721  
CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
Goals   •  Need  to  detect  the  same  feature  anywhere  in   an  image   •  Reuse  the  same  weights  over  and  over   •  What  we  really  want  is  one  neuron  that   detects  a  feature  that  we  slide  over  the  image  
Neuron  =  Filter   •  Act  as  detectors  for  some  specific  image   feature   •  Take  images  as  inputs  and  produce  image  like   feature  maps  as  outputs  
Convolu,on   •  Like  sliding  a  matrix  over  the  input  and   performing  dot  products   •  It’s  all  just  matrix  mul,plica,on  
Convolu,on  
Convolu,on  
Filters  (or  Kernels)   Sharpen  
Filters  (or  Kernels)   Box  Blur  
Filters  (or  Kernels)   Edge  Detec,on   Feature  Map  
Alexnet  Architecture   Convolu,ons  
CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
Nonlinearity  
CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
Max  Pooling  Example  
Alexnet  Architecture   3x3  stride  2  Max  Pooling  
Pooling   •  Allows  us  to  look  at  more  of  the  image   •  Max,  sum,  and  L2  pooling   •  A  type  of  downsampling  
CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   FC  1000   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   Dropout  
Dropout   h@p://cs231n.github.io/neural-­‐networks-­‐2/   •  Randomly  disable  some  neurons  on  the   forward  pass   •  Prevents  overfiong    
Let’s  Predict  Something!   •  We  have  all  these  features,  how  do  we  learn   to  label  something  based  on  them?  
Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   FC  1000   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   Fully  Connected  
Fully  Connected  Layers   •  Each  neuron  is  connected  to  all  inputs   •  Standard  mul,layer  neural  net   •  Learns  non-­‐linear  combina,ons  of  the  feature   maps  to  make  predic,ons  
Alexnet  Architecture  
Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   FC  1000  
Which  Class  Is  It  Again?   •  FC-­‐1000  gives  us  1000  numbers,  one  per  class,   how  do  we  compare  them?  
Soqmax   •  Mul,-­‐class  version  of  logis,c  func,on   •  Outputs  normalized  class  “probabili,es”   •  Takes  m  inputs  and  produces  m  outputs   between  zero  and  one,  that  sum  to  one   •  Cross-­‐entropy  loss   •  Differen,able  
h@ps://papers.nips.cc/paper/4824-­‐imagenet-­‐classifica,on-­‐with-­‐deep-­‐convolu,onal-­‐neural-­‐networks   Image  Classifica,on  
Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012     Layer  1  
Learned  Filters  –  Layer1  
Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012     Layer  2  
Learned  Filters  –  Layer2   Visualizing  and  Understanding  Convolu,onal  Networks  -­‐  Zeiler  &  Fergus,  ECCV  2014    
Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012     Layer  3  
Learned  Filters  -­‐  Layer3   Visualizing  and  Understanding  Convolu,onal  Networks  -­‐  Zeiler  &  Fergus,  ECCV  2014      
Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012     Layer  4   Layer  5  
Learned  Features  –  Layers  4  &  5  
Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012    
Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   FC  1000  
VGG16   h@ps://blog.heuritech.com/2016/02/29/a-­‐brief-­‐report-­‐of-­‐the-­‐heuritech-­‐deep-­‐learning-­‐meetup-­‐5/  
Google’s  Incep,on  Module  
To  Learn  More   •  h@p://colah.github.io/posts/2014-­‐07-­‐ Understanding-­‐Convolu,ons/   •  h@ps://adeshpande3.github.io/ adeshpande3.github.io/The-­‐9-­‐Deep-­‐Learning-­‐ Papers-­‐You-­‐Need-­‐To-­‐Know-­‐About.html   •  h@p://cs231n.github.io/   •  h@p://course.fast.ai/  
Ques,ons?  

Recomendados

Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
Chanuk Lim
 
Convolution Neural Network (CNN)
Convolution Neural Network (CNN)Convolution Neural Network (CNN)
Convolution Neural Network (CNN)
Basit Rafiq
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
Gaurav Mittal
 
Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)
Muhammad Haroon
 
LeNet-5
LeNet-5LeNet-5
LeNet-5
佳蓉 倪
 
Convolutional Neural Network and Its Applications
Convolutional Neural Network and Its ApplicationsConvolutional Neural Network and Its Applications
Convolutional Neural Network and Its Applications
Kasun Chinthaka Piyarathna
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
Christian Perone
 
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
Fellowship at Vodafone FutureLab
 

Recomendados

Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
Chanuk Lim
 
Convolution Neural Network (CNN)
Convolution Neural Network (CNN)Convolution Neural Network (CNN)
Convolution Neural Network (CNN)
Basit Rafiq
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
Gaurav Mittal
 
Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)
Muhammad Haroon
 
LeNet-5
LeNet-5LeNet-5
LeNet-5
佳蓉 倪
 
Convolutional Neural Network and Its Applications
Convolutional Neural Network and Its ApplicationsConvolutional Neural Network and Its Applications
Convolutional Neural Network and Its Applications
Kasun Chinthaka Piyarathna
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
Christian Perone
 
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
Fellowship at Vodafone FutureLab
 
Convolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningConvolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep Learning
Mohamed Loey
 
Neural networks and deep learning
Neural networks and deep learningNeural networks and deep learning
Neural networks and deep learning
Jörgen Sandig
 
Batch normalization presentation
Batch normalization presentationBatch normalization presentation
Batch normalization presentation
Owin Will
 
Rnn & Lstm
Rnn & LstmRnn & Lstm
Rnn & Lstm
Subash Chandra Pakhrin
 
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNN
Shuai Zhang
 
Deep learning
Deep learningDeep learning
Deep learning
Pratap Dangeti
 
Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...
Universitat Politècnica de Catalunya
 
Deep Learning - Overview of my work II
Deep Learning - Overview of my work IIDeep Learning - Overview of my work II
Deep Learning - Overview of my work II
Mohamed Loey
 
Resnet
ResnetResnet
Resnet
ashwinjoseph95
 
GoogLeNet Insights
GoogLeNet InsightsGoogLeNet Insights
GoogLeNet Insights
Auro Tripathy
 
Google net
Google netGoogle net
Google net
Brian Kim
 
Training Neural Networks
Training Neural NetworksTraining Neural Networks
Training Neural Networks
Databricks
 
Modern Convolutional Neural Network techniques for image segmentation
Modern Convolutional Neural Network techniques for image segmentationModern Convolutional Neural Network techniques for image segmentation
Modern Convolutional Neural Network techniques for image segmentation
Gioele Ciaparrone
 
CVPR 2018 Paper Reading MobileNet V2
CVPR 2018 Paper Reading MobileNet V2CVPR 2018 Paper Reading MobileNet V2
CVPR 2018 Paper Reading MobileNet V2
Khang Pham
 
Transfer Learning and Fine-tuning Deep Neural Networks
Transfer Learning and Fine-tuning Deep Neural Networks Transfer Learning and Fine-tuning Deep Neural Networks
Transfer Learning and Fine-tuning Deep Neural Networks
PyData
 
An introduction to Deep Learning
An introduction to Deep LearningAn introduction to Deep Learning
An introduction to Deep Learning
Julien SIMON
 
Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)
Abdulrazak Zakieh
 
Dcgan
DcganDcgan
Dcgan
Brian Kim
 
Image segmentation with deep learning
Image segmentation with deep learningImage segmentation with deep learning
Image segmentation with deep learning
Antonio Rueda-Toicen
 
Machine learning with graph
Machine learning with graphMachine learning with graph
Machine learning with graph
Ding Li
 
Teaching machines to see the process of designing (datasets) with ai
Teaching machines to see the process of designing (datasets) with aiTeaching machines to see the process of designing (datasets) with ai
Teaching machines to see the process of designing (datasets) with ai
DevFest DC
 
Evolution of Deep Learning and new advancements
Evolution of Deep Learning and new advancementsEvolution of Deep Learning and new advancements
Evolution of Deep Learning and new advancements
Chitta Ranjan
 

Mais conteúdo relacionado

Mais procurados

Transfer Learning and Fine-tuning Deep Neural Networks
Transfer Learning and Fine-tuning Deep Neural Networks Transfer Learning and Fine-tuning Deep Neural Networks
Transfer Learning and Fine-tuning Deep Neural Networks
PyData
 

Mais procurados (20)

Convolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningConvolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep Learning
 
Neural networks and deep learning
Neural networks and deep learningNeural networks and deep learning
Neural networks and deep learning
 
Batch normalization presentation
Batch normalization presentationBatch normalization presentation
Batch normalization presentation
 
Rnn & Lstm
Rnn & LstmRnn & Lstm
Rnn & Lstm
 
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNN
 
Deep learning
Deep learningDeep learning
Deep learning
 
Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...Faster R-CNN: Towards real-time object detection with region proposal network...
Faster R-CNN: Towards real-time object detection with region proposal network...
 
Deep Learning - Overview of my work II
Deep Learning - Overview of my work IIDeep Learning - Overview of my work II
Deep Learning - Overview of my work II
 
Resnet
ResnetResnet
Resnet
 
GoogLeNet Insights
GoogLeNet InsightsGoogLeNet Insights
GoogLeNet Insights
 
Google net
Google netGoogle net
Google net
 
Training Neural Networks
Training Neural NetworksTraining Neural Networks
Training Neural Networks
 
Modern Convolutional Neural Network techniques for image segmentation
Modern Convolutional Neural Network techniques for image segmentationModern Convolutional Neural Network techniques for image segmentation
Modern Convolutional Neural Network techniques for image segmentation
 
CVPR 2018 Paper Reading MobileNet V2
CVPR 2018 Paper Reading MobileNet V2CVPR 2018 Paper Reading MobileNet V2
CVPR 2018 Paper Reading MobileNet V2
 
Transfer Learning and Fine-tuning Deep Neural Networks
Transfer Learning and Fine-tuning Deep Neural Networks Transfer Learning and Fine-tuning Deep Neural Networks
Transfer Learning and Fine-tuning Deep Neural Networks
 
An introduction to Deep Learning
An introduction to Deep LearningAn introduction to Deep Learning
An introduction to Deep Learning
 
Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)
 
Dcgan
DcganDcgan
Dcgan
 
Image segmentation with deep learning
Image segmentation with deep learningImage segmentation with deep learning
Image segmentation with deep learning
 
Machine learning with graph
Machine learning with graphMachine learning with graph
Machine learning with graph
 

Semelhante a Intro To Convolutional Neural Networks

Teaching machines to see the process of designing (datasets) with ai
Teaching machines to see the process of designing (datasets) with aiTeaching machines to see the process of designing (datasets) with ai
Teaching machines to see the process of designing (datasets) with ai
DevFest DC
 
Evolution of Deep Learning and new advancements
Evolution of Deep Learning and new advancementsEvolution of Deep Learning and new advancements
Evolution of Deep Learning and new advancements
Chitta Ranjan
 
20141003.journal club
20141003.journal club20141003.journal club
20141003.journal club
Hayaru SHOUNO
 
Yann le cun
Yann le cunYann le cun
Yann le cun
Yandex
 
Fcv learn yu
Fcv learn yuFcv learn yu
Fcv learn yu
zukun
 
Modeling perceptual similarity and shift invariance in deep networks
Modeling perceptual similarity and shift invariance in deep networksModeling perceptual similarity and shift invariance in deep networks
Modeling perceptual similarity and shift invariance in deep networks
NAVER Engineering
 
Mit6870 orsu lecture11
Mit6870 orsu lecture11Mit6870 orsu lecture11
Mit6870 orsu lecture11
zukun
 

Semelhante a Intro To Convolutional Neural Networks (20)

Teaching machines to see the process of designing (datasets) with ai
Teaching machines to see the process of designing (datasets) with aiTeaching machines to see the process of designing (datasets) with ai
Teaching machines to see the process of designing (datasets) with ai
 
Evolution of Deep Learning and new advancements
Evolution of Deep Learning and new advancementsEvolution of Deep Learning and new advancements
Evolution of Deep Learning and new advancements
 
Deep Learning with Python (PyData Seattle 2015)
Deep Learning with Python (PyData Seattle 2015)Deep Learning with Python (PyData Seattle 2015)
Deep Learning with Python (PyData Seattle 2015)
 
Image Translation with GAN
Image Translation with GANImage Translation with GAN
Image Translation with GAN
 
20141003.journal club
20141003.journal club20141003.journal club
20141003.journal club
 
Yann le cun
Yann le cunYann le cun
Yann le cun
 
[Mmlab seminar 2016] deep learning for human pose estimation
[Mmlab seminar 2016] deep learning for human pose estimation[Mmlab seminar 2016] deep learning for human pose estimation
[Mmlab seminar 2016] deep learning for human pose estimation
 
Fcv learn yu
Fcv learn yuFcv learn yu
Fcv learn yu
 
Scalable image recognition model with deep embedding
Scalable image recognition model with deep embeddingScalable image recognition model with deep embedding
Scalable image recognition model with deep embedding
 
Deep Learning: concepts and use cases (October 2018)
Deep Learning: concepts and use cases (October 2018)Deep Learning: concepts and use cases (October 2018)
Deep Learning: concepts and use cases (October 2018)
 
CNN Algorithm
CNN AlgorithmCNN Algorithm
CNN Algorithm
 
Talk from NVidia Developer Connect
Talk from NVidia Developer ConnectTalk from NVidia Developer Connect
Talk from NVidia Developer Connect
 
Artificial Intelligence, Machine Learning and Deep Learning
Artificial Intelligence, Machine Learning and Deep LearningArtificial Intelligence, Machine Learning and Deep Learning
Artificial Intelligence, Machine Learning and Deep Learning
 
Modeling perceptual similarity and shift invariance in deep networks
Modeling perceptual similarity and shift invariance in deep networksModeling perceptual similarity and shift invariance in deep networks
Modeling perceptual similarity and shift invariance in deep networks
 
Intro to Neural Networks
Intro to Neural NetworksIntro to Neural Networks
Intro to Neural Networks
 
Big Data Intelligence: from Correlation Discovery to Causal Reasoning
Big Data Intelligence: from Correlation Discovery to Causal Reasoning Big Data Intelligence: from Correlation Discovery to Causal Reasoning
Big Data Intelligence: from Correlation Discovery to Causal Reasoning
 
MLIP - Chapter 3 - Introduction to deep learning
MLIP - Chapter 3 - Introduction to deep learningMLIP - Chapter 3 - Introduction to deep learning
MLIP - Chapter 3 - Introduction to deep learning
 
Mit6870 orsu lecture11
Mit6870 orsu lecture11Mit6870 orsu lecture11
Mit6870 orsu lecture11
 
Deep neural networks
Deep neural networksDeep neural networks
Deep neural networks
 
A brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsA brief introduction to recent segmentation methods
A brief introduction to recent segmentation methods
 

Último

CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Delhi Call girls
 
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts ServiceCall Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
amitlee9823
 
Best VIP Call Girls Noida Sector 39 Call Me: 8448380779
Best VIP Call Girls Noida Sector 39 Call Me: 8448380779Best VIP Call Girls Noida Sector 39 Call Me: 8448380779
Best VIP Call Girls Noida Sector 39 Call Me: 8448380779
Delhi Call girls
 
Vip Model Call Girls (Delhi) Karol Bagh 9711199171✔️Body to body massage wit...
Vip Model Call Girls (Delhi) Karol Bagh 9711199171✔️Body to body massage wit...Vip Model Call Girls (Delhi) Karol Bagh 9711199171✔️Body to body massage wit...
Vip Model Call Girls (Delhi) Karol Bagh 9711199171✔️Body to body massage wit...
shivangimorya083
 
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 nightCheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Delhi Call girls
 
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfSchema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdf
Lars Albertsson
 
Determinants of health, dimensions of health, positive health and spectrum of...
Determinants of health, dimensions of health, positive health and spectrum of...Determinants of health, dimensions of health, positive health and spectrum of...
Determinants of health, dimensions of health, positive health and spectrum of...
shambhavirathore45
 
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAl Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
AroojKhan71
 

Último (20)

CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Saket (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
 
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
Call Girls in Sarai Kale Khan Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts S...
 
BigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptxBigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptx
 
CebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptxCebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptx
 
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts ServiceCall Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
 
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
Call Girls Bannerghatta Road Just Call 👗 7737669865 👗 Top Class Call Girl Ser...
 
Smarteg dropshipping via API with DroFx.pptx
Smarteg dropshipping via API with DroFx.pptxSmarteg dropshipping via API with DroFx.pptx
Smarteg dropshipping via API with DroFx.pptx
 
Best VIP Call Girls Noida Sector 39 Call Me: 8448380779
Best VIP Call Girls Noida Sector 39 Call Me: 8448380779Best VIP Call Girls Noida Sector 39 Call Me: 8448380779
Best VIP Call Girls Noida Sector 39 Call Me: 8448380779
 
Vip Model Call Girls (Delhi) Karol Bagh 9711199171✔️Body to body massage wit...
Vip Model Call Girls (Delhi) Karol Bagh 9711199171✔️Body to body massage wit...Vip Model Call Girls (Delhi) Karol Bagh 9711199171✔️Body to body massage wit...
Vip Model Call Girls (Delhi) Karol Bagh 9711199171✔️Body to body massage wit...
 
Carero dropshipping via API with DroFx.pptx
Carero dropshipping via API with DroFx.pptxCarero dropshipping via API with DroFx.pptx
Carero dropshipping via API with DroFx.pptx
 
Discover Why Less is More in B2B Research
Discover Why Less is More in B2B ResearchDiscover Why Less is More in B2B Research
Discover Why Less is More in B2B Research
 
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 nightCheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night
 
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfSchema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdf
 
Edukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFxEdukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFx
 
Determinants of health, dimensions of health, positive health and spectrum of...
Determinants of health, dimensions of health, positive health and spectrum of...Determinants of health, dimensions of health, positive health and spectrum of...
Determinants of health, dimensions of health, positive health and spectrum of...
 
Week-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interactionWeek-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interaction
 
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAl Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
 
Sampling (random) method and Non random.ppt
Sampling (random) method and Non random.pptSampling (random) method and Non random.ppt
Sampling (random) method and Non random.ppt
 
Introduction-to-Machine-Learning (1).pptx
Introduction-to-Machine-Learning (1).pptxIntroduction-to-Machine-Learning (1).pptx
Introduction-to-Machine-Learning (1).pptx
 
Ravak dropshipping via API with DroFx.pptx
Ravak dropshipping via API with DroFx.pptxRavak dropshipping via API with DroFx.pptx
Ravak dropshipping via API with DroFx.pptx
 

Intro To Convolutional Neural Networks

  • 1. Intro  To  Convolu,onal  Neural   Networks   Mark  Scully   datapraxis.com  
  • 7. Facial  Recogni,on     Y.  Taigman,  M.  Yang,  M.  Ranzato,  L.  Wolf,  DeepFace:  Closing  the  Gap  to  Human-­‐Level  Performance  in  Face  Verifica,on,  CVPR   2014  
  • 9. Colorize  Black  &  White  Images   h@p://richzhang.github.io/coloriza,on/  
  • 10. Style  Transfer   h@p://genekogan.com/works/style-­‐transfer/   Mona  Lisa  restyled  by  Picasso,  van  Gough,  and  Monet  
  • 11. Generate  An  Image  From  A  Sketch   h@ps://affinelayer.com/pixsrv/  
  • 12. ImageNet  Challenge   Alexnet   Li  Fei-­‐Fei:  ImageNet  Large  Scale  Visual  Recogni,on  Challenge,  2014  
  • 13. ImageNet  Challenge   ILSVRC+ ImageNet Classification error throughout years and groups Li  Fei-­‐Fei:  ImageNet  Large  Scale  Visual  Recogni,on  Challenge,  2014  
  • 14. Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   FC  1000  
  • 15. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012    
  • 16. ImageNet  Challenge   Alexnet   Li  Fei-­‐Fei:  ImageNet  Large  Scale  Visual  Recogni,on  Challenge,  2014  
  • 17. Tradi,onal  Approach  To  Image   Classifica,on   Input  Image   Hand   Extracted   Features   Classifier   Object  Label  
  • 18. Issues   •  Who  makes  the  features?   – Need  an  expert  for  each  problem  domain   •  Which  features?   – Are  they  the  same  for  every  problem  type?   •  How  robust  are  these  features  to  real  images?   – Transla,on,  Rota,on,  contrast  changes,  etc.  
  • 19. Are  these  pictures  of  the  same  thing?  
  • 20. Features  Are  Hierarchical     •  A  squirrel  is  a  combina,on  of  fur,  arms,  legs,   &  a  tail  in  specific  propor,ons.   •  A  tail  is  made  of  texture,  color,  and  spa,al   rela,onships   •  A  texture  is  made  of  oriented  edges,   gradients,  and  colors  
  • 21. Image  Features   •  A  feature  is  something  in  the  image  or  derived   from  it  that’s  relevant  to  the  task   •  Edges   •  Lines  at  different  angles,  curves,  etc.   •  Colors,  or  pa@erns  of  colors   •  SIFT,  SURF,  HOG,  GIST,  ORB,  etc  
  • 23. Ideally  We’d  Learn  Features   Input   Image   Output   Label  
  • 24. Ideally  We’d  Learn  Features   Input   Image   Output   Label   CNNs  
  • 25. What  is  a  Neural  Network?   •  Perceptron  is  biologically  inspired   •  A  mental  model  for  interpre,ng  the  math   h@p://cs231n.stanford.edu/index.html    
  • 26. Perceptron   1   x1   x2   x3   xm   Σ   Output   Ac,va,on   Func,on   Sum   w0   w1   w2   w3   wm   Weights   Inputs  
  • 27. Perceptron   1   x1   x2   x3   xm   Σ   Output   Ac,va,on   Func,on   Sum   w0   w1   w2   w3   wm   Weights   Inputs   wi xi i=0 m ∑ = w0 x0 + w1x1 + w2 x2 +...+ wm xm
  • 29. Training:  Upda,ng  Weights   1   x1   x2   x3   x4   Σ   Output   Ac,va,on   Func,on   Sum   w0   w1   w2   w3   w4   Weights   Inputs   Error  =  Output  -­‐  Target  
  • 32. Backpropaga,on   •  Error  propagates  backward  and  it  all  works  via   (normally  stochas,c)  gradient  descent.   •  (wave  hands)  
  • 33.
  • 34. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012    
  • 35. CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
  • 36. CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
  • 37. Input:  Pixels  Are  Just  Numbers   h@ps://medium.com/@ageitgey/machine-­‐learning-­‐is-­‐fun-­‐part-­‐3-­‐deep-­‐learning-­‐and-­‐convolu,onal-­‐neural-­‐networks-­‐ f40359318721  
  • 38. CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
  • 39. Goals   •  Need  to  detect  the  same  feature  anywhere  in   an  image   •  Reuse  the  same  weights  over  and  over   •  What  we  really  want  is  one  neuron  that   detects  a  feature  that  we  slide  over  the  image  
  • 40. Neuron  =  Filter   •  Act  as  detectors  for  some  specific  image   feature   •  Take  images  as  inputs  and  produce  image  like   feature  maps  as  outputs  
  • 41. Convolu,on   •  Like  sliding  a  matrix  over  the  input  and   performing  dot  products   •  It’s  all  just  matrix  mul,plica,on  
  • 44. Filters  (or  Kernels)   Sharpen  
  • 45. Filters  (or  Kernels)   Box  Blur  
  • 46. Filters  (or  Kernels)   Edge  Detec,on   Feature  Map  
  • 48. CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
  • 50. CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
  • 52. Alexnet  Architecture   3x3  stride  2  Max  Pooling  
  • 53. Pooling   •  Allows  us  to  look  at  more  of  the  image   •  Max,  sum,  and  L2  pooling   •  A  type  of  downsampling  
  • 54. CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
  • 55. Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   FC  1000   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   Dropout  
  • 56. Dropout   h@p://cs231n.github.io/neural-­‐networks-­‐2/   •  Randomly  disable  some  neurons  on  the   forward  pass   •  Prevents  overfiong    
  • 57. Let’s  Predict  Something!   •  We  have  all  these  features,  how  do  we  learn   to  label  something  based  on  them?  
  • 58. Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   FC  1000   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   Fully  Connected  
  • 59. Fully  Connected  Layers   •  Each  neuron  is  connected  to  all  inputs   •  Standard  mul,layer  neural  net   •  Learns  non-­‐linear  combina,ons  of  the  feature   maps  to  make  predic,ons  
  • 61. Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   FC  1000  
  • 62. Which  Class  Is  It  Again?   •  FC-­‐1000  gives  us  1000  numbers,  one  per  class,   how  do  we  compare  them?  
  • 63. Soqmax   •  Mul,-­‐class  version  of  logis,c  func,on   •  Outputs  normalized  class  “probabili,es”   •  Takes  m  inputs  and  produces  m  outputs   between  zero  and  one,  that  sum  to  one   •  Cross-­‐entropy  loss   •  Differen,able  
  • 65. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012     Layer  1  
  • 66. Learned  Filters  –  Layer1  
  • 67. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012     Layer  2  
  • 68. Learned  Filters  –  Layer2   Visualizing  and  Understanding  Convolu,onal  Networks  -­‐  Zeiler  &  Fergus,  ECCV  2014    
  • 69. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012     Layer  3  
  • 70. Learned  Filters  -­‐  Layer3   Visualizing  and  Understanding  Convolu,onal  Networks  -­‐  Zeiler  &  Fergus,  ECCV  2014      
  • 71. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012     Layer  4   Layer  5  
  • 72. Learned  Features  –  Layers  4  &  5  
  • 73. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012    
  • 74. Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   FC  1000  
  • 77. To  Learn  More   •  h@p://colah.github.io/posts/2014-­‐07-­‐ Understanding-­‐Convolu,ons/   •  h@ps://adeshpande3.github.io/ adeshpande3.github.io/The-­‐9-­‐Deep-­‐Learning-­‐ Papers-­‐You-­‐Need-­‐To-­‐Know-­‐About.html   •  h@p://cs231n.github.io/   •  h@p://course.fast.ai/