SlideShare uma empresa Scribd logo
1 de 47
Baixar para ler offline
Introduction   MLCV1: SD           MLCV2: OR           ML Resources          CV Resources        Q&A




               Machine Learning for Computer Vision
                           Applications

                                      Duc-Hieu Tran
                                    tran0066@ntu.edu.sg

                               School of Computer Engineering
                          Nanyang Technological University, Singapore


                             APAC PyCon, 09/June/2012




               SCE, NTU                    ML for CV                  APAC PyCon, 09/June/2012   1 / 34
Introduction         MLCV1: SD    MLCV2: OR            ML Resources          CV Resources        Q&A



Outline


       1       Introduction

       2       ML/CV example 1: Skin Detection in Images

       3       ML/CV example 2: Object Recognition by Bag of Word model

       4       Machine Learning Resources

       5       Computer Vision Resources

       6       Q&A




                     SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   2 / 34
Introduction         MLCV1: SD    MLCV2: OR            ML Resources          CV Resources        Q&A



Outline


       1       Introduction

       2       ML/CV example 1: Skin Detection in Images

       3       ML/CV example 2: Object Recognition by Bag of Word model

       4       Machine Learning Resources

       5       Computer Vision Resources

       6       Q&A




                     SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   3 / 34
Introduction      MLCV1: SD     MLCV2: OR          ML Resources          CV Resources        Q&A



What is Machine Learning?



         From Wikipedia (http://goo.gl/cl42O)
             A branch of Artificial Intelligence (AI).
               Algorithms that allow computers to
               evolve behaviors based on emperical
               data.
               Main focus: to automatically learn to
               recognize patterns and make decisions
               based on data.




                 SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   4 / 34
Introduction      MLCV1: SD     MLCV2: OR          ML Resources          CV Resources        Q&A



What is Computer Vision?



         From Wikipedia (http://goo.gl/vnKJb)
             Methods to acquire, process, analysis
             and understand image data from the
             real world.
               Goal: to produce numerical/symbolic
               information.


                In short: to extract useful information from images.




                 SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   5 / 34
Introduction   MLCV1: SD   MLCV2: OR          ML Resources          CV Resources        Q&A



Machine Learning and Computer Vision




                                                                    From Wikipedia




               SCE, NTU           ML for CV                  APAC PyCon, 09/June/2012   6 / 34
Introduction        MLCV1: SD     MLCV2: OR          ML Resources          CV Resources        Q&A



Machine Learning and Computer Vision (cont.)



               The talk title is too big: two active/large research areas – ML
               and CV.
               Goal:
                   A general idea on what is ML.
                   How to apply it in a CV application: to extract and make use
                   of information from an image.
                   Simple examples of application ML.
                   Resources to learn more about ML and CV.




                   SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   7 / 34
Introduction      MLCV1: SD      MLCV2: OR             ML Resources          CV Resources        Q&A



Everyday applications of ML/CV




               Face detection in digital
               camera.




                  SCE, NTU                 ML for CV                  APAC PyCon, 09/June/2012   8 / 34
Introduction      MLCV1: SD    MLCV2: OR          ML Resources          CV Resources        Q&A



Everyday applications of ML/CV




               Google visual image
               search: upload an image
               and search for
               similar-content images.




                 SCE, NTU             ML for CV                  APAC PyCon, 09/June/2012   8 / 34
Introduction      MLCV1: SD     MLCV2: OR          ML Resources          CV Resources        Q&A



Everyday applications of ML/CV




               Google (Web) Picasa face
               detection and recognition:
               image group of the same
               person.




                  SCE, NTU             ML for CV                  APAC PyCon, 09/June/2012   8 / 34
Introduction      MLCV1: SD     MLCV2: OR          ML Resources          CV Resources        Q&A



Everyday applications of ML/CV




               Google Picasa face movie:
               make face-aligned movies
               – http://goo.gl/bUjqT




                 SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   8 / 34
Introduction      MLCV1: SD       MLCV2: OR          ML Resources          CV Resources        Q&A



Everyday applications of ML/CV




               Youtube image
               stabilization function.




                  SCE, NTU               ML for CV                  APAC PyCon, 09/June/2012   8 / 34
Introduction      MLCV1: SD    MLCV2: OR           ML Resources          CV Resources        Q&A



Everyday applications of ML/CV




               Microsoft Photosynth:
               view 3D world from an
               image collection –
               http://photosynth.net




                 SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   8 / 34
Introduction         MLCV1: SD    MLCV2: OR            ML Resources          CV Resources        Q&A



Outline


       1       Introduction

       2       ML/CV example 1: Skin Detection in Images

       3       ML/CV example 2: Object Recognition by Bag of Word model

       4       Machine Learning Resources

       5       Computer Vision Resources

       6       Q&A




                     SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   9 / 34
Introduction    MLCV1: SD      MLCV2: OR          ML Resources           CV Resources        Q&A



Skin detection in images
       Problem: skin detection in images or video stream (webcam).
       One possible solution:
           Read pixel colors (RGB) from the image: each image pixel – a
           vector of 3 components.
           Manually pre-define an RGB range of skin color.
           Detect image pixels that are in the pre-defined range.
       Advantage:
           Easy and simple.
       Disadvantages:
           The accuracy depends on how well we define the RGB range
           of skin color.
           Not illumination invariant (i.e., light change leads to the skin
           color change).
           Need to manually define various ranges.
           ...

                SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   10 / 34
Introduction        MLCV1: SD     MLCV2: OR          ML Resources           CV Resources        Q&A



Skin detection in images (cont.)


       A solution based on ML:
               Manually annotate skin regions in one or several images.
               Extract pixel colors (RGB) from the images: each image pixel
               – a vector of 3 components (extract features.)
               Label pixel colors from skin regions as 1, pixels from non-skin
               as 0.
               Train a classifier: SVM, Linear Regression, Neural Networks,
               etc.
               Use the classifier to detect skin region in novel images.
       Use opensource Python ML library: scikit-learn




                   SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   11 / 34
Introduction        MLCV1: SD     MLCV2: OR          ML Resources           CV Resources        Q&A



Skin detection in images (cont.)


       A solution based on ML:
               Manually annotate skin regions in one or several images.
               Extract pixel colors (RGB) from the images: each image pixel
               – a vector of 3 components (extract features.)
               Label pixel colors from skin regions as 1, pixels from non-skin
               as 0.
               Train a classifier: SVM, Linear Regression, Neural Networks,
               etc.
               Use the classifier to detect skin region in novel images.
       Use opensource Python ML library: scikit-learn




                   SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   11 / 34
Introduction        MLCV1: SD     MLCV2: OR          ML Resources           CV Resources        Q&A



Skin detection in images (cont.)


       A solution based on ML:
               Manually annotate skin regions in one or several images.
               Extract pixel colors (RGB) from the images: each image pixel
               – a vector of 3 components (extract features.)
               Label pixel colors from skin regions as 1, pixels from non-skin
               as 0.
               Train a classifier: SVM, Linear Regression, Neural Networks,
               etc.
               Use the classifier to detect skin region in novel images.
       Use opensource Python ML library: scikit-learn




                   SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   11 / 34
Introduction        MLCV1: SD     MLCV2: OR          ML Resources           CV Resources        Q&A



Skin detection in images (cont.)


       A solution based on ML:
               Manually annotate skin regions in one or several images.
               Extract pixel colors (RGB) from the images: each image pixel
               – a vector of 3 components (extract features.)
               Label pixel colors from skin regions as 1, pixels from non-skin
               as 0.
               Train a classifier: SVM, Linear Regression, Neural Networks,
               etc.
               Use the classifier to detect skin region in novel images.
       Use opensource Python ML library: scikit-learn




                   SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   11 / 34
Introduction        MLCV1: SD     MLCV2: OR          ML Resources           CV Resources        Q&A



Skin detection in images (cont.)


       A solution based on ML:
               Manually annotate skin regions in one or several images.
               Extract pixel colors (RGB) from the images: each image pixel
               – a vector of 3 components (extract features.)
               Label pixel colors from skin regions as 1, pixels from non-skin
               as 0.
               Train a classifier: SVM, Linear Regression, Neural Networks,
               etc.
               Use the classifier to detect skin region in novel images.
       Use opensource Python ML library: scikit-learn




                   SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   11 / 34
Introduction               MLCV1: SD                    MLCV2: OR                     ML Resources                CV Resources        Q&A



Skin detection in images (cont.)

  1    import s c i p y as sp
  2    from s c i p y i m p o r t m i s c
  3
  4    img = m i s c . i m r e a d ( ’ . / hand1 . j p g ’ )
  5    mask = m i s c . i m r e a d ( ’ . / hand mask . j p g ’ )
  6
  7    s k i n = img [ mask == 0 ]
  8    n o n s k i n = img [ mask != 0 ]
  9
 10    # a r r a y o f s i z e NxF : N − #s a m p l e s , F − f e a t u r e d i m e n s i o n
 11    X = sp . v s t a c k ( ( skin , nonskin ) )
 12
 13    # labels
 14    y = sp . h s t a c k ( ( sp . ones ( s k i n . shape [ 0 ] , sp . u i n t 8 ) , 
 15    sp . z e r o s ( n o n s k i n . shape [ 0 ] , sp . u i n t 8 ) ) )
 16
 17    from s k l e a r n i m p o r t svm
 18    c l f = svm . SVC ( )
 19    c l f . f i t (X , y )
 20
 21    i m g t e s t = m i s c . i m r e a d ( ’ . / hand2 . j p g ’ )
 22    z = c l f . p r e d i c t ( s p . r e s h a p e ( i m g t e s t , (−1, 3 ) ) )
 23    zimg = s p . r e s h a p e ( z , ( i m g t e s t . s h a p e [ 0 ] , i m g t e s t . s h a p e [ 1 ] ) )
 24    m i s c . imshow ( zimg )




                          SCE, NTU                                    ML for CV                           APAC PyCon, 09/June/2012   12 / 34
Introduction         MLCV1: SD        MLCV2: OR           ML Resources           CV Resources        Q&A



Skin detection in images (cont.)
       Note about this example:
               Classify each image pixel, the result is very poor.
               Feature: pixel color.
               The size of training data - size of the image.
       Advantages:
               Automatically recognize the pattern of skin color.
               Work well with many kind of color skins if there are enough training data
               (i.e., images of different skin colors).
               More complicated ML algorithm can detect skin region more accurately.
                    Detect skin as an image region, not just by classifying one
                    pixel at a time (e.g., CRF or MRF).
                    Combine detect skin with detecting other human body parts
                    (e.g., use high-level features other than just pixel colors.)
       Disadvantages:
               Need to collect skin image samples and manually annotate.
               More training data, more accurate.

                    SCE, NTU                  ML for CV                  APAC PyCon, 09/June/2012   13 / 34
Introduction         MLCV1: SD    MLCV2: OR            ML Resources           CV Resources        Q&A



Outline


       1       Introduction

       2       ML/CV example 1: Skin Detection in Images

       3       ML/CV example 2: Object Recognition by Bag of Word model

       4       Machine Learning Resources

       5       Computer Vision Resources

       6       Q&A




                     SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   14 / 34
Introduction      MLCV1: SD      MLCV2: OR           ML Resources           CV Resources        Q&A



Object recognition

       Bag of word model, overall idea
            A local feature: the local
            information of the object
            image.
               An object image: a set of
               local features.
               A visual vocabulary.
               An object image:
               histogram of visual
               vocabulary – a numerical
               vector of D dimensions.
               Use a ML algorithm to
               classify object images.


                  SCE, NTU               ML for CV                  APAC PyCon, 09/June/2012   15 / 34
Introduction      MLCV1: SD      MLCV2: OR           ML Resources           CV Resources        Q&A



Object recognition

       Bag of word model, overall idea
            A local feature: the local
            information of the object
            image.
               An object image: a set of
               local features.
               A visual vocabulary.
               An object image:
               histogram of visual
               vocabulary – a numerical
               vector of D dimensions.
               Use a ML algorithm to
               classify object images.


                  SCE, NTU               ML for CV                  APAC PyCon, 09/June/2012   15 / 34
Introduction      MLCV1: SD      MLCV2: OR           ML Resources           CV Resources        Q&A



Object recognition

       Bag of word model, overall idea
            A local feature: the local
            information of the object
            image.
               An object image: a set of
               local features.
               A visual vocabulary.
               An object image:
               histogram of visual
               vocabulary – a numerical
               vector of D dimensions.
               Use a ML algorithm to
               classify object images.


                  SCE, NTU               ML for CV                  APAC PyCon, 09/June/2012   15 / 34
Introduction      MLCV1: SD      MLCV2: OR           ML Resources           CV Resources        Q&A



Object recognition

       Bag of word model, overall idea
            A local feature: the local
            information of the object
            image.
               An object image: a set of
               local features.
               A visual vocabulary.
               An object image:
               histogram of visual
               vocabulary – a numerical
               vector of D dimensions.
               Use a ML algorithm to
               classify object images.


                  SCE, NTU               ML for CV                  APAC PyCon, 09/June/2012   15 / 34
Introduction      MLCV1: SD      MLCV2: OR           ML Resources           CV Resources        Q&A



Object recognition

       Bag of word model, overall idea
            A local feature: the local
            information of the object
            image.
               An object image: a set of
               local features.
               A visual vocabulary.
               An object image:
               histogram of visual
               vocabulary – a numerical
               vector of D dimensions.
               Use a ML algorithm to
               classify object images.


                  SCE, NTU               ML for CV                  APAC PyCon, 09/June/2012   15 / 34
Introduction              MLCV1: SD                  MLCV2: OR                   ML Resources           CV Resources        Q&A



Object recognition (cont.)



       Extract local features (e.g., SURF) from an object image.
  1    import cv2
  2    import s c i p y as sp
  3    ...
  4
  5    def e x t r a c t f e a t u r e ( img filename , f e a t u r e e x t r a c t o r ) :
  6      # e x t r a c t l o c a l f e a t u r e (SURF) from one image
  7      img = c v 2 . i m r e a d ( i m g f i l e n a m e , c v 2 . CV LOAD IMAGE GRAYSCALE )
  8      k p o i n t s , d e s c = f e a t u r e e x t r a c t o r . d e t e c t ( img , None )
  9
 10    ...
 11    # e x a m p l e : SURF d e t e c t o r
 12    s u r f = c v 2 . SURF ( 5 0 0 )
 13    kp , d e s c = e x t r a c t f e a t u r e ( ’ animage . j p g ’ , s u r f )




                         SCE, NTU                                 ML for CV                     APAC PyCon, 09/June/2012   16 / 34
Introduction               MLCV1: SD                     MLCV2: OR                 ML Resources             CV Resources        Q&A



Object recognition (cont.)


       Build a visual vocabulary.
                 Extract local features from a number of images.
                 In the following code, each row of flist – a local feature.
  1    # e x t r a c t a l l l o c a l f e a t u r e s from a number o f f i l e s
  2    def c r e a t e f l i s t ( f i l e l i s t , f e a t u r e e x t r a c t o r ) :
  3      desc list = []
  4      for f in f i l e l i s t :
  5          kpoints , desc = e x t r a c t f e a t u r e ( f , f e a t u r e e x t r a c t o r )
  6
  7            d e s c l i s t . append ( d e s c )
  8
  9       # a r r a y o f f e a t u r e s from a l l f i l e
 10       f l i s t = sp . v s t a c k ( t u p l e ( d e s c l i s t ) )
 11
 12       return       flist




                          SCE, NTU                                     ML for CV                    APAC PyCon, 09/June/2012   17 / 34
Introduction                MLCV1: SD                    MLCV2: OR                     ML Resources                CV Resources            Q&A



Object recognition (cont.)
       Build a visual vocabulary.
                  Quantize the local features using a clustering algorithm (e.g., k-means)
                  In the code, cluster the rows of flist.
  1    import s c i p y as sp
  2    from s k l e a r n i m p o r t c l u s t e r
  3    from s k l e a r n i m p o r t m e t r i c s
  4
  5    class VisualVocabulary ( object ):
  6      def    init   ( s e l f , v s i z e =0):
  7        self . vsize = vsize
  8
  9       def f i t ( self , f l i s t ) :
 10         s e l f . centroids , labels , i n e r t i a = 
 11                 c l u s t e r . k means ( f l i s t , s e l f . v s i z e )
 12
 13       def quantize ( self , f l i s t ) :
 14         dist = metrics . pairwise distances ( s e l f . centroids ,                              flist )
 15         n e a r e s t c e n t r o i d s = s p . a r g m i n ( d i s t , a x i s =0)
 16
 17            # generate histogram
 18            h i s t , dummy = s p . h i s t o g r a m ( n e a r e s t c e n t r o i d s ,   s e l f . vsize , [0 ,   self . vsize ])
 19
 20            return hist




                           SCE, NTU                                   ML for CV                          APAC PyCon, 09/June/2012         18 / 34
Introduction               MLCV1: SD                   MLCV2: OR                   ML Resources                  CV Resources    Q&A



Object recognition (cont.)


       Represent an object image as a histogram of visual vocabulary.
  1    c l a s s BOW( o b j e c t ) :
  2        def      init      ( self , f i l e l i s t , feature extractor , vsize ):
  3            self . feature extractor = feature extractor
  4            self . vsize = vsize
  5
  6            f l i s t = create flist ( f i l e l i s t , self . feature extractor )
  7            s e l f . vv = V i s u a l V o c a b u l a r y ( s e l f . v s i z e )
  8            s e l f . vv . f i t ( f l i s t )
  9
 10       def represent ( img filename ) :
 11         # extract local features
 12         kpoints , desc = e x t r a c t f e a t u r e ( img filename ,                 self . feature extractor )
 13
 14            # r e p r e s e n t t h e o b j e c t image a s h i s t o g r a m o f v i s u a l v o c a b u l a r y
 15            o b j r e p r = s e l f . vv . q u a n t i z e ( d e s c )
 16
 17            return obj repr




                          SCE, NTU                                  ML for CV                        APAC PyCon, 09/June/2012   19 / 34
Introduction               MLCV1: SD                    MLCV2: OR                 ML Resources                  CV Resources    Q&A



Object recognition (cont.)

       Combine together into a BOW model framework.
  1    s u r f = c v 2 . SURF ( 5 0 0 )
  2
  3    bow = BOW( v v f i l e l i s t , s u r f , 4 0 0 )
  4
  5    # r e p r e s e n t e a c h t r a i n i n g image a s h i s t o g r a m o f t h e v i s u a l v o c a b u l a r y
  6    X, y = [ ] , [ ]
  7    for f , label in zip ( train img list , l a b e l l i s t ):
  8      X . append ( bow . r e p r e s e n t ( f ) )
  9      y . append ( l a b e l )
 10
 11    X = s p . a r r a y (X)
 12    y = sp . a r r a y ( y )
 13
 14    from s k l e a r n i m p o r t svm
 15    c l f = svm . SVC ( )
 16    c l f . f i t (X , y )
 17
 18    # t e s t an o b j e c t image
 19    t e s t o b j = bow . r e p r e s e n t ( ” t e s t i m g . j p g ” )
 20    test label = clf . predict ( test obj )




                          SCE, NTU                                    ML for CV                     APAC PyCon, 09/June/2012   20 / 34
Introduction       MLCV1: SD      MLCV2: OR          ML Resources           CV Resources        Q&A



ML application



       Common steps to apply a ML algorithm:
               Represent data instances in numerical form (e.g., high
               dimensional vectors).
               Use samples to learn a model.
               Apply the model on novel data instances.
       The sample code and related material will be posted to
       http://pythonme.wordpress.com




                   SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   21 / 34
Introduction         MLCV1: SD    MLCV2: OR            ML Resources           CV Resources        Q&A



Outline


       1       Introduction

       2       ML/CV example 1: Skin Detection in Images

       3       ML/CV example 2: Object Recognition by Bag of Word model

       4       Machine Learning Resources

       5       Computer Vision Resources

       6       Q&A




                     SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   22 / 34
Introduction        MLCV1: SD       MLCV2: OR          ML Resources           CV Resources        Q&A



Machine Learning Resources

       ML textbooks
               Pattern Recognition and Machine Learning, Bishop, 20006
               Pattern Classification, Duda et al., 2001
               The Elements of Statistical Learning, Hastie et al., 2010 (Free
               open book!)
               Research papers from ML/CV conferences:
               http://cvpapers.com/
       A lot of free open courses on online universities with lecture
       videos, quizzes and programming assignments.
               http://coursera.org
               http://udacity.com
               A lot of fun, not scary as in offline universities!


                   SCE, NTU                ML for CV                  APAC PyCon, 09/June/2012   23 / 34
Introduction       MLCV1: SD       MLCV2: OR          ML Resources           CV Resources        Q&A



Machine Learning Resources (cont.)
       Free Online Open Courses on ML
            Machine Learning (Coursera)
                   http://goo.gl/wqZii
                   A free, open course by Stanford University.
                   Introduction course to ML, basic algorithms.
                   Video lectures, quizzes, programming assignments every week
                   (Matlab/Octave).
               Probabilistic Graphical Models (Coursera)
                   http://goo.gl/oSOIi
                   Advanced course on ML (Stanford University)
                   Quizzes, programming assignments (Matlab/Octave)
               Introduction to AI (Udacity)
                   http://goo.gl/UlM4C
                   No programming assignment, just quizzes.
               AI: Programming A Robotic Car. (Udacity, CS373)
                   http://goo.gl/FJ8ml
                   Probabilistic approaches to AI/robotics.
                   Quizzes, programming assignments (in Python!)

                   SCE, NTU               ML for CV                  APAC PyCon, 09/June/2012   24 / 34
Introduction       MLCV1: SD       MLCV2: OR          ML Resources           CV Resources        Q&A



Machine Learning Resources (cont.)




         Programming Collective Intelligence:
         Building Smart Web 2.0 Applications,
         Segaran, 2007
               ML algorithms in Python.
               Real world examples on Web 2.0 applications.




                  SCE, NTU                ML for CV                  APAC PyCon, 09/June/2012   25 / 34
Introduction       MLCV1: SD     MLCV2: OR          ML Resources           CV Resources        Q&A



Machine Learning Resources (cont.)

       Opensource Python ML libraries
           scikit-learn: a general purpose ML library in Python.
                   Active community, large development team.
                   Up to date documentation.
               Orange: data visualization and analysis, visual programming.
               LibSVM: specific on Support Vector Machine, Python
               interface.
               mlpy, PyBrain, PyML, mdp-toolkit, etc.
               http://mloss.org – up-to-date list of ML software projects.
               Scipy, NumPy – scientific programming libraries – the core of
               almost Python ML libraries.



                   SCE, NTU             ML for CV                  APAC PyCon, 09/June/2012   26 / 34
Introduction         MLCV1: SD    MLCV2: OR            ML Resources           CV Resources        Q&A



Outline


       1       Introduction

       2       ML/CV example 1: Skin Detection in Images

       3       ML/CV example 2: Object Recognition by Bag of Word model

       4       Machine Learning Resources

       5       Computer Vision Resources

       6       Q&A




                     SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   27 / 34
Introduction       MLCV1: SD        MLCV2: OR          ML Resources           CV Resources        Q&A



Computer Vision Resources

       CV textbooks
               Digital Image Processing, Gonzalez and Woods, 2010.
               Computer Vision: A modern approach, Forsyth and Ponce,
               2003.
               Computer Vision: Algorithms and Applications, Szeliski, 2011.
               (Free open book!)
                   http://goo.gl/t4Znn
                   Comprehensive collection of the state of the art algorithms in CV.
               Computer Vision: Models, Learning and Inference, Prince,
               2012. (Free open book!)
                   http://goo.gl/AqgO5
                   ”A book on statistics with vision examples”.
                   Online pseudo-code of many ML algorithms specified for CV
                   application.



                   SCE, NTU                ML for CV                  APAC PyCon, 09/June/2012   28 / 34
Introduction       MLCV1: SD         MLCV2: OR          ML Resources           CV Resources        Q&A



Computer Vision Resources (cont.)




         Programming Computer Vision with
         Python, Solem, 2012. (Free open book!)

               http://goo.gl/cHQIn
               CV algorithms in Python.
               Real world CV examples.




                  SCE, NTU                  ML for CV                  APAC PyCon, 09/June/2012   29 / 34
Introduction       MLCV1: SD         MLCV2: OR          ML Resources           CV Resources        Q&A



Computer Vision Resources (cont.)



         Learning OpenCV: Computer Vision
         with the OpenCV library, Bradski and
         Kaehler, 2008.
               Learn CV algorithms by practice.
               Code examples in C.
               Online document on Python API:
               http://goo.gl/q0GTa




                  SCE, NTU                  ML for CV                  APAC PyCon, 09/June/2012   30 / 34
Introduction       MLCV1: SD       MLCV2: OR           ML Resources           CV Resources        Q&A



Computer Vision Resources (cont.)
       Opensource Python CV libraries
           OpenCV
                   http://goo.gl/cqYyw
                   Active community, large development team.
                   Large collection of CV algorithm.
                   C++, Python API.
                   ML libraries, but in C++.
               PyGame
                   http://www.pygame.org
                   Basic CV/IP support.
               Scipy/Numpy
                   http://scipy.org
                   General scientific programming libraries.
                   Images/videos as multi-dimensional arrays.
               Python Image Library (PIL)
                   http://goo.gl/FS3xN
                   Image and graphics processing functions.

                   SCE, NTU                ML for CV                  APAC PyCon, 09/June/2012   31 / 34
Introduction       MLCV1: SD       MLCV2: OR          ML Resources           CV Resources        Q&A



Computer Vision Resources (cont.)


               Free online course on coursera.org: Computer Vision (UC,
               Berkeley)
                   http://goo.gl/LNhCR
                   Just quizzes, no programming assignment.
               Research papers from ML/CV (top-rank) conferences:
               http://cvpapers.com/
               The list of companies developing CV products (up to date,
               maintained by David Lowe, author of SIFT):
               http://goo.gl/oUPVp




                   SCE, NTU               ML for CV                  APAC PyCon, 09/June/2012   32 / 34
Introduction         MLCV1: SD    MLCV2: OR            ML Resources           CV Resources        Q&A



Outline


       1       Introduction

       2       ML/CV example 1: Skin Detection in Images

       3       ML/CV example 2: Object Recognition by Bag of Word model

       4       Machine Learning Resources

       5       Computer Vision Resources

       6       Q&A




                     SCE, NTU              ML for CV                  APAC PyCon, 09/June/2012   33 / 34
Introduction   MLCV1: SD   MLCV2: OR          ML Resources           CV Resources        Q&A



Q&A




                               Thank you!




               SCE, NTU           ML for CV                  APAC PyCon, 09/June/2012   34 / 34

Mais conteúdo relacionado

Semelhante a APA PyCon 2012: Machine Learning for Computer Vision Applications

A Literature Survey on Image Linguistic Visual Question Answering
A Literature Survey on Image Linguistic Visual Question AnsweringA Literature Survey on Image Linguistic Visual Question Answering
A Literature Survey on Image Linguistic Visual Question AnsweringIRJET Journal
 
Real Time Sign Language Detection
Real Time Sign Language DetectionReal Time Sign Language Detection
Real Time Sign Language DetectionIRJET Journal
 
Gender Classification using SVM With Flask
Gender Classification using SVM With FlaskGender Classification using SVM With Flask
Gender Classification using SVM With FlaskAI Publications
 
Resume_embedded_systems_Onkar_Gulavani_01-05-2017
Resume_embedded_systems_Onkar_Gulavani_01-05-2017Resume_embedded_systems_Onkar_Gulavani_01-05-2017
Resume_embedded_systems_Onkar_Gulavani_01-05-2017Onkar Gulavani
 
Seamless MLOps with Seldon and MLflow
Seamless MLOps with Seldon and MLflowSeamless MLOps with Seldon and MLflow
Seamless MLOps with Seldon and MLflowDatabricks
 
Image Detection and Count Using Open Computer Vision (Opencv)
Image Detection and Count Using Open Computer Vision (Opencv)Image Detection and Count Using Open Computer Vision (Opencv)
Image Detection and Count Using Open Computer Vision (Opencv)IJERA Editor
 
YOLOv4: A Face Mask Detection System
YOLOv4: A Face Mask Detection SystemYOLOv4: A Face Mask Detection System
YOLOv4: A Face Mask Detection SystemIRJET Journal
 
IRJET- Wearable AI Device for Blind
IRJET- Wearable AI Device for BlindIRJET- Wearable AI Device for Blind
IRJET- Wearable AI Device for BlindIRJET Journal
 
Modelling Framework of a Neural Object Recognition
Modelling Framework of a Neural Object RecognitionModelling Framework of a Neural Object Recognition
Modelling Framework of a Neural Object RecognitionIJERA Editor
 
Semantic Concept Detection in Video Using Hybrid Model of CNN and SVM Classif...
Semantic Concept Detection in Video Using Hybrid Model of CNN and SVM Classif...Semantic Concept Detection in Video Using Hybrid Model of CNN and SVM Classif...
Semantic Concept Detection in Video Using Hybrid Model of CNN and SVM Classif...CSCJournals
 
Big-Data Analytics for Media Management
Big-Data Analytics for Media ManagementBig-Data Analytics for Media Management
Big-Data Analytics for Media Managementtechkrish
 
Efficient Point Cloud Pre-processing using The Point Cloud Library
Efficient Point Cloud Pre-processing using The Point Cloud LibraryEfficient Point Cloud Pre-processing using The Point Cloud Library
Efficient Point Cloud Pre-processing using The Point Cloud LibraryCSCJournals
 
Efficient Point Cloud Pre-processing using The Point Cloud Library
Efficient Point Cloud Pre-processing using The Point Cloud LibraryEfficient Point Cloud Pre-processing using The Point Cloud Library
Efficient Point Cloud Pre-processing using The Point Cloud LibraryCSCJournals
 
Automatic License Plate Recognition using OpenCV
Automatic License Plate Recognition using OpenCVAutomatic License Plate Recognition using OpenCV
Automatic License Plate Recognition using OpenCVEditor IJCATR
 
Automatic License Plate Recognition using OpenCV
Automatic License Plate Recognition using OpenCV Automatic License Plate Recognition using OpenCV
Automatic License Plate Recognition using OpenCV Editor IJCATR
 
Fake Note Detection of Bangladesh
Fake Note Detection of BangladeshFake Note Detection of Bangladesh
Fake Note Detection of BangladeshTanvirAhammed22
 
Project Report on High Speed Face Recognitions using DCT, RBF nueral network
Project Report on High Speed Face Recognitions using DCT, RBF nueral networkProject Report on High Speed Face Recognitions using DCT, RBF nueral network
Project Report on High Speed Face Recognitions using DCT, RBF nueral networkSagar Rai
 
Five Minute Speech: An Overview of Activities Developed in Disciplines and Gu...
Five Minute Speech: An Overview of Activities Developed in Disciplines and Gu...Five Minute Speech: An Overview of Activities Developed in Disciplines and Gu...
Five Minute Speech: An Overview of Activities Developed in Disciplines and Gu...Michel Alves
 
Breaking Through The Challenges of Scalable Deep Learning for Video Analytics
Breaking Through The Challenges of Scalable Deep Learning for Video AnalyticsBreaking Through The Challenges of Scalable Deep Learning for Video Analytics
Breaking Through The Challenges of Scalable Deep Learning for Video AnalyticsJason Anderson
 

Semelhante a APA PyCon 2012: Machine Learning for Computer Vision Applications (20)

A Literature Survey on Image Linguistic Visual Question Answering
A Literature Survey on Image Linguistic Visual Question AnsweringA Literature Survey on Image Linguistic Visual Question Answering
A Literature Survey on Image Linguistic Visual Question Answering
 
Real Time Sign Language Detection
Real Time Sign Language DetectionReal Time Sign Language Detection
Real Time Sign Language Detection
 
Gender Classification using SVM With Flask
Gender Classification using SVM With FlaskGender Classification using SVM With Flask
Gender Classification using SVM With Flask
 
Resume_embedded_systems_Onkar_Gulavani_01-05-2017
Resume_embedded_systems_Onkar_Gulavani_01-05-2017Resume_embedded_systems_Onkar_Gulavani_01-05-2017
Resume_embedded_systems_Onkar_Gulavani_01-05-2017
 
Seamless MLOps with Seldon and MLflow
Seamless MLOps with Seldon and MLflowSeamless MLOps with Seldon and MLflow
Seamless MLOps with Seldon and MLflow
 
Image Detection and Count Using Open Computer Vision (Opencv)
Image Detection and Count Using Open Computer Vision (Opencv)Image Detection and Count Using Open Computer Vision (Opencv)
Image Detection and Count Using Open Computer Vision (Opencv)
 
YOLOv4: A Face Mask Detection System
YOLOv4: A Face Mask Detection SystemYOLOv4: A Face Mask Detection System
YOLOv4: A Face Mask Detection System
 
IRJET- Wearable AI Device for Blind
IRJET- Wearable AI Device for BlindIRJET- Wearable AI Device for Blind
IRJET- Wearable AI Device for Blind
 
Modelling Framework of a Neural Object Recognition
Modelling Framework of a Neural Object RecognitionModelling Framework of a Neural Object Recognition
Modelling Framework of a Neural Object Recognition
 
Semantic Concept Detection in Video Using Hybrid Model of CNN and SVM Classif...
Semantic Concept Detection in Video Using Hybrid Model of CNN and SVM Classif...Semantic Concept Detection in Video Using Hybrid Model of CNN and SVM Classif...
Semantic Concept Detection in Video Using Hybrid Model of CNN and SVM Classif...
 
Big-Data Analytics for Media Management
Big-Data Analytics for Media ManagementBig-Data Analytics for Media Management
Big-Data Analytics for Media Management
 
Efficient Point Cloud Pre-processing using The Point Cloud Library
Efficient Point Cloud Pre-processing using The Point Cloud LibraryEfficient Point Cloud Pre-processing using The Point Cloud Library
Efficient Point Cloud Pre-processing using The Point Cloud Library
 
Efficient Point Cloud Pre-processing using The Point Cloud Library
Efficient Point Cloud Pre-processing using The Point Cloud LibraryEfficient Point Cloud Pre-processing using The Point Cloud Library
Efficient Point Cloud Pre-processing using The Point Cloud Library
 
Automatic License Plate Recognition using OpenCV
Automatic License Plate Recognition using OpenCVAutomatic License Plate Recognition using OpenCV
Automatic License Plate Recognition using OpenCV
 
Automatic License Plate Recognition using OpenCV
Automatic License Plate Recognition using OpenCV Automatic License Plate Recognition using OpenCV
Automatic License Plate Recognition using OpenCV
 
Fake Note Detection of Bangladesh
Fake Note Detection of BangladeshFake Note Detection of Bangladesh
Fake Note Detection of Bangladesh
 
Project Report on High Speed Face Recognitions using DCT, RBF nueral network
Project Report on High Speed Face Recognitions using DCT, RBF nueral networkProject Report on High Speed Face Recognitions using DCT, RBF nueral network
Project Report on High Speed Face Recognitions using DCT, RBF nueral network
 
Five Minute Speech: An Overview of Activities Developed in Disciplines and Gu...
Five Minute Speech: An Overview of Activities Developed in Disciplines and Gu...Five Minute Speech: An Overview of Activities Developed in Disciplines and Gu...
Five Minute Speech: An Overview of Activities Developed in Disciplines and Gu...
 
Breaking Through The Challenges of Scalable Deep Learning for Video Analytics
Breaking Through The Challenges of Scalable Deep Learning for Video AnalyticsBreaking Through The Challenges of Scalable Deep Learning for Video Analytics
Breaking Through The Challenges of Scalable Deep Learning for Video Analytics
 
Karthick
KarthickKarthick
Karthick
 

Último

AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxiammrhaywood
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptxSandy Millin
 
Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphNetziValdelomar1
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfYu Kanazawa / Osaka University
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxAditiChauhan701637
 
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxPractical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxKatherine Villaluna
 
The basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxThe basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxheathfieldcps1
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxraviapr7
 
Patterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptxPatterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptxMYDA ANGELICA SUAN
 
The Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsThe Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsEugene Lysak
 
How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17Celine George
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfMohonDas
 
Education and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxEducation and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxraviapr7
 
How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17Celine George
 
UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.EnglishCEIPdeSigeiro
 
CAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptxCAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptxSaurabhParmar42
 
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxPISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxEduSkills OECD
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapitolTechU
 
How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17Celine George
 

Último (20)

AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
 
Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a Paragraph
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptx
 
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxPractical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
 
The basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxThe basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptx
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptx
 
Patterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptxPatterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptx
 
The Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsThe Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George Wells
 
How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdf
 
Education and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxEducation and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptx
 
How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17
 
UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.
 
CAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptxCAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptx
 
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxPISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptx
 
How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17
 

APA PyCon 2012: Machine Learning for Computer Vision Applications

  • 1. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Machine Learning for Computer Vision Applications Duc-Hieu Tran tran0066@ntu.edu.sg School of Computer Engineering Nanyang Technological University, Singapore APAC PyCon, 09/June/2012 SCE, NTU ML for CV APAC PyCon, 09/June/2012 1 / 34
  • 2. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Outline 1 Introduction 2 ML/CV example 1: Skin Detection in Images 3 ML/CV example 2: Object Recognition by Bag of Word model 4 Machine Learning Resources 5 Computer Vision Resources 6 Q&A SCE, NTU ML for CV APAC PyCon, 09/June/2012 2 / 34
  • 3. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Outline 1 Introduction 2 ML/CV example 1: Skin Detection in Images 3 ML/CV example 2: Object Recognition by Bag of Word model 4 Machine Learning Resources 5 Computer Vision Resources 6 Q&A SCE, NTU ML for CV APAC PyCon, 09/June/2012 3 / 34
  • 4. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A What is Machine Learning? From Wikipedia (http://goo.gl/cl42O) A branch of Artificial Intelligence (AI). Algorithms that allow computers to evolve behaviors based on emperical data. Main focus: to automatically learn to recognize patterns and make decisions based on data. SCE, NTU ML for CV APAC PyCon, 09/June/2012 4 / 34
  • 5. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A What is Computer Vision? From Wikipedia (http://goo.gl/vnKJb) Methods to acquire, process, analysis and understand image data from the real world. Goal: to produce numerical/symbolic information. In short: to extract useful information from images. SCE, NTU ML for CV APAC PyCon, 09/June/2012 5 / 34
  • 6. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Machine Learning and Computer Vision From Wikipedia SCE, NTU ML for CV APAC PyCon, 09/June/2012 6 / 34
  • 7. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Machine Learning and Computer Vision (cont.) The talk title is too big: two active/large research areas – ML and CV. Goal: A general idea on what is ML. How to apply it in a CV application: to extract and make use of information from an image. Simple examples of application ML. Resources to learn more about ML and CV. SCE, NTU ML for CV APAC PyCon, 09/June/2012 7 / 34
  • 8. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Everyday applications of ML/CV Face detection in digital camera. SCE, NTU ML for CV APAC PyCon, 09/June/2012 8 / 34
  • 9. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Everyday applications of ML/CV Google visual image search: upload an image and search for similar-content images. SCE, NTU ML for CV APAC PyCon, 09/June/2012 8 / 34
  • 10. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Everyday applications of ML/CV Google (Web) Picasa face detection and recognition: image group of the same person. SCE, NTU ML for CV APAC PyCon, 09/June/2012 8 / 34
  • 11. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Everyday applications of ML/CV Google Picasa face movie: make face-aligned movies – http://goo.gl/bUjqT SCE, NTU ML for CV APAC PyCon, 09/June/2012 8 / 34
  • 12. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Everyday applications of ML/CV Youtube image stabilization function. SCE, NTU ML for CV APAC PyCon, 09/June/2012 8 / 34
  • 13. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Everyday applications of ML/CV Microsoft Photosynth: view 3D world from an image collection – http://photosynth.net SCE, NTU ML for CV APAC PyCon, 09/June/2012 8 / 34
  • 14. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Outline 1 Introduction 2 ML/CV example 1: Skin Detection in Images 3 ML/CV example 2: Object Recognition by Bag of Word model 4 Machine Learning Resources 5 Computer Vision Resources 6 Q&A SCE, NTU ML for CV APAC PyCon, 09/June/2012 9 / 34
  • 15. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Skin detection in images Problem: skin detection in images or video stream (webcam). One possible solution: Read pixel colors (RGB) from the image: each image pixel – a vector of 3 components. Manually pre-define an RGB range of skin color. Detect image pixels that are in the pre-defined range. Advantage: Easy and simple. Disadvantages: The accuracy depends on how well we define the RGB range of skin color. Not illumination invariant (i.e., light change leads to the skin color change). Need to manually define various ranges. ... SCE, NTU ML for CV APAC PyCon, 09/June/2012 10 / 34
  • 16. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Skin detection in images (cont.) A solution based on ML: Manually annotate skin regions in one or several images. Extract pixel colors (RGB) from the images: each image pixel – a vector of 3 components (extract features.) Label pixel colors from skin regions as 1, pixels from non-skin as 0. Train a classifier: SVM, Linear Regression, Neural Networks, etc. Use the classifier to detect skin region in novel images. Use opensource Python ML library: scikit-learn SCE, NTU ML for CV APAC PyCon, 09/June/2012 11 / 34
  • 17. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Skin detection in images (cont.) A solution based on ML: Manually annotate skin regions in one or several images. Extract pixel colors (RGB) from the images: each image pixel – a vector of 3 components (extract features.) Label pixel colors from skin regions as 1, pixels from non-skin as 0. Train a classifier: SVM, Linear Regression, Neural Networks, etc. Use the classifier to detect skin region in novel images. Use opensource Python ML library: scikit-learn SCE, NTU ML for CV APAC PyCon, 09/June/2012 11 / 34
  • 18. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Skin detection in images (cont.) A solution based on ML: Manually annotate skin regions in one or several images. Extract pixel colors (RGB) from the images: each image pixel – a vector of 3 components (extract features.) Label pixel colors from skin regions as 1, pixels from non-skin as 0. Train a classifier: SVM, Linear Regression, Neural Networks, etc. Use the classifier to detect skin region in novel images. Use opensource Python ML library: scikit-learn SCE, NTU ML for CV APAC PyCon, 09/June/2012 11 / 34
  • 19. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Skin detection in images (cont.) A solution based on ML: Manually annotate skin regions in one or several images. Extract pixel colors (RGB) from the images: each image pixel – a vector of 3 components (extract features.) Label pixel colors from skin regions as 1, pixels from non-skin as 0. Train a classifier: SVM, Linear Regression, Neural Networks, etc. Use the classifier to detect skin region in novel images. Use opensource Python ML library: scikit-learn SCE, NTU ML for CV APAC PyCon, 09/June/2012 11 / 34
  • 20. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Skin detection in images (cont.) A solution based on ML: Manually annotate skin regions in one or several images. Extract pixel colors (RGB) from the images: each image pixel – a vector of 3 components (extract features.) Label pixel colors from skin regions as 1, pixels from non-skin as 0. Train a classifier: SVM, Linear Regression, Neural Networks, etc. Use the classifier to detect skin region in novel images. Use opensource Python ML library: scikit-learn SCE, NTU ML for CV APAC PyCon, 09/June/2012 11 / 34
  • 21. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Skin detection in images (cont.) 1 import s c i p y as sp 2 from s c i p y i m p o r t m i s c 3 4 img = m i s c . i m r e a d ( ’ . / hand1 . j p g ’ ) 5 mask = m i s c . i m r e a d ( ’ . / hand mask . j p g ’ ) 6 7 s k i n = img [ mask == 0 ] 8 n o n s k i n = img [ mask != 0 ] 9 10 # a r r a y o f s i z e NxF : N − #s a m p l e s , F − f e a t u r e d i m e n s i o n 11 X = sp . v s t a c k ( ( skin , nonskin ) ) 12 13 # labels 14 y = sp . h s t a c k ( ( sp . ones ( s k i n . shape [ 0 ] , sp . u i n t 8 ) , 15 sp . z e r o s ( n o n s k i n . shape [ 0 ] , sp . u i n t 8 ) ) ) 16 17 from s k l e a r n i m p o r t svm 18 c l f = svm . SVC ( ) 19 c l f . f i t (X , y ) 20 21 i m g t e s t = m i s c . i m r e a d ( ’ . / hand2 . j p g ’ ) 22 z = c l f . p r e d i c t ( s p . r e s h a p e ( i m g t e s t , (−1, 3 ) ) ) 23 zimg = s p . r e s h a p e ( z , ( i m g t e s t . s h a p e [ 0 ] , i m g t e s t . s h a p e [ 1 ] ) ) 24 m i s c . imshow ( zimg ) SCE, NTU ML for CV APAC PyCon, 09/June/2012 12 / 34
  • 22. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Skin detection in images (cont.) Note about this example: Classify each image pixel, the result is very poor. Feature: pixel color. The size of training data - size of the image. Advantages: Automatically recognize the pattern of skin color. Work well with many kind of color skins if there are enough training data (i.e., images of different skin colors). More complicated ML algorithm can detect skin region more accurately. Detect skin as an image region, not just by classifying one pixel at a time (e.g., CRF or MRF). Combine detect skin with detecting other human body parts (e.g., use high-level features other than just pixel colors.) Disadvantages: Need to collect skin image samples and manually annotate. More training data, more accurate. SCE, NTU ML for CV APAC PyCon, 09/June/2012 13 / 34
  • 23. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Outline 1 Introduction 2 ML/CV example 1: Skin Detection in Images 3 ML/CV example 2: Object Recognition by Bag of Word model 4 Machine Learning Resources 5 Computer Vision Resources 6 Q&A SCE, NTU ML for CV APAC PyCon, 09/June/2012 14 / 34
  • 24. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Object recognition Bag of word model, overall idea A local feature: the local information of the object image. An object image: a set of local features. A visual vocabulary. An object image: histogram of visual vocabulary – a numerical vector of D dimensions. Use a ML algorithm to classify object images. SCE, NTU ML for CV APAC PyCon, 09/June/2012 15 / 34
  • 25. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Object recognition Bag of word model, overall idea A local feature: the local information of the object image. An object image: a set of local features. A visual vocabulary. An object image: histogram of visual vocabulary – a numerical vector of D dimensions. Use a ML algorithm to classify object images. SCE, NTU ML for CV APAC PyCon, 09/June/2012 15 / 34
  • 26. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Object recognition Bag of word model, overall idea A local feature: the local information of the object image. An object image: a set of local features. A visual vocabulary. An object image: histogram of visual vocabulary – a numerical vector of D dimensions. Use a ML algorithm to classify object images. SCE, NTU ML for CV APAC PyCon, 09/June/2012 15 / 34
  • 27. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Object recognition Bag of word model, overall idea A local feature: the local information of the object image. An object image: a set of local features. A visual vocabulary. An object image: histogram of visual vocabulary – a numerical vector of D dimensions. Use a ML algorithm to classify object images. SCE, NTU ML for CV APAC PyCon, 09/June/2012 15 / 34
  • 28. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Object recognition Bag of word model, overall idea A local feature: the local information of the object image. An object image: a set of local features. A visual vocabulary. An object image: histogram of visual vocabulary – a numerical vector of D dimensions. Use a ML algorithm to classify object images. SCE, NTU ML for CV APAC PyCon, 09/June/2012 15 / 34
  • 29. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Object recognition (cont.) Extract local features (e.g., SURF) from an object image. 1 import cv2 2 import s c i p y as sp 3 ... 4 5 def e x t r a c t f e a t u r e ( img filename , f e a t u r e e x t r a c t o r ) : 6 # e x t r a c t l o c a l f e a t u r e (SURF) from one image 7 img = c v 2 . i m r e a d ( i m g f i l e n a m e , c v 2 . CV LOAD IMAGE GRAYSCALE ) 8 k p o i n t s , d e s c = f e a t u r e e x t r a c t o r . d e t e c t ( img , None ) 9 10 ... 11 # e x a m p l e : SURF d e t e c t o r 12 s u r f = c v 2 . SURF ( 5 0 0 ) 13 kp , d e s c = e x t r a c t f e a t u r e ( ’ animage . j p g ’ , s u r f ) SCE, NTU ML for CV APAC PyCon, 09/June/2012 16 / 34
  • 30. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Object recognition (cont.) Build a visual vocabulary. Extract local features from a number of images. In the following code, each row of flist – a local feature. 1 # e x t r a c t a l l l o c a l f e a t u r e s from a number o f f i l e s 2 def c r e a t e f l i s t ( f i l e l i s t , f e a t u r e e x t r a c t o r ) : 3 desc list = [] 4 for f in f i l e l i s t : 5 kpoints , desc = e x t r a c t f e a t u r e ( f , f e a t u r e e x t r a c t o r ) 6 7 d e s c l i s t . append ( d e s c ) 8 9 # a r r a y o f f e a t u r e s from a l l f i l e 10 f l i s t = sp . v s t a c k ( t u p l e ( d e s c l i s t ) ) 11 12 return flist SCE, NTU ML for CV APAC PyCon, 09/June/2012 17 / 34
  • 31. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Object recognition (cont.) Build a visual vocabulary. Quantize the local features using a clustering algorithm (e.g., k-means) In the code, cluster the rows of flist. 1 import s c i p y as sp 2 from s k l e a r n i m p o r t c l u s t e r 3 from s k l e a r n i m p o r t m e t r i c s 4 5 class VisualVocabulary ( object ): 6 def init ( s e l f , v s i z e =0): 7 self . vsize = vsize 8 9 def f i t ( self , f l i s t ) : 10 s e l f . centroids , labels , i n e r t i a = 11 c l u s t e r . k means ( f l i s t , s e l f . v s i z e ) 12 13 def quantize ( self , f l i s t ) : 14 dist = metrics . pairwise distances ( s e l f . centroids , flist ) 15 n e a r e s t c e n t r o i d s = s p . a r g m i n ( d i s t , a x i s =0) 16 17 # generate histogram 18 h i s t , dummy = s p . h i s t o g r a m ( n e a r e s t c e n t r o i d s , s e l f . vsize , [0 , self . vsize ]) 19 20 return hist SCE, NTU ML for CV APAC PyCon, 09/June/2012 18 / 34
  • 32. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Object recognition (cont.) Represent an object image as a histogram of visual vocabulary. 1 c l a s s BOW( o b j e c t ) : 2 def init ( self , f i l e l i s t , feature extractor , vsize ): 3 self . feature extractor = feature extractor 4 self . vsize = vsize 5 6 f l i s t = create flist ( f i l e l i s t , self . feature extractor ) 7 s e l f . vv = V i s u a l V o c a b u l a r y ( s e l f . v s i z e ) 8 s e l f . vv . f i t ( f l i s t ) 9 10 def represent ( img filename ) : 11 # extract local features 12 kpoints , desc = e x t r a c t f e a t u r e ( img filename , self . feature extractor ) 13 14 # r e p r e s e n t t h e o b j e c t image a s h i s t o g r a m o f v i s u a l v o c a b u l a r y 15 o b j r e p r = s e l f . vv . q u a n t i z e ( d e s c ) 16 17 return obj repr SCE, NTU ML for CV APAC PyCon, 09/June/2012 19 / 34
  • 33. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Object recognition (cont.) Combine together into a BOW model framework. 1 s u r f = c v 2 . SURF ( 5 0 0 ) 2 3 bow = BOW( v v f i l e l i s t , s u r f , 4 0 0 ) 4 5 # r e p r e s e n t e a c h t r a i n i n g image a s h i s t o g r a m o f t h e v i s u a l v o c a b u l a r y 6 X, y = [ ] , [ ] 7 for f , label in zip ( train img list , l a b e l l i s t ): 8 X . append ( bow . r e p r e s e n t ( f ) ) 9 y . append ( l a b e l ) 10 11 X = s p . a r r a y (X) 12 y = sp . a r r a y ( y ) 13 14 from s k l e a r n i m p o r t svm 15 c l f = svm . SVC ( ) 16 c l f . f i t (X , y ) 17 18 # t e s t an o b j e c t image 19 t e s t o b j = bow . r e p r e s e n t ( ” t e s t i m g . j p g ” ) 20 test label = clf . predict ( test obj ) SCE, NTU ML for CV APAC PyCon, 09/June/2012 20 / 34
  • 34. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A ML application Common steps to apply a ML algorithm: Represent data instances in numerical form (e.g., high dimensional vectors). Use samples to learn a model. Apply the model on novel data instances. The sample code and related material will be posted to http://pythonme.wordpress.com SCE, NTU ML for CV APAC PyCon, 09/June/2012 21 / 34
  • 35. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Outline 1 Introduction 2 ML/CV example 1: Skin Detection in Images 3 ML/CV example 2: Object Recognition by Bag of Word model 4 Machine Learning Resources 5 Computer Vision Resources 6 Q&A SCE, NTU ML for CV APAC PyCon, 09/June/2012 22 / 34
  • 36. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Machine Learning Resources ML textbooks Pattern Recognition and Machine Learning, Bishop, 20006 Pattern Classification, Duda et al., 2001 The Elements of Statistical Learning, Hastie et al., 2010 (Free open book!) Research papers from ML/CV conferences: http://cvpapers.com/ A lot of free open courses on online universities with lecture videos, quizzes and programming assignments. http://coursera.org http://udacity.com A lot of fun, not scary as in offline universities! SCE, NTU ML for CV APAC PyCon, 09/June/2012 23 / 34
  • 37. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Machine Learning Resources (cont.) Free Online Open Courses on ML Machine Learning (Coursera) http://goo.gl/wqZii A free, open course by Stanford University. Introduction course to ML, basic algorithms. Video lectures, quizzes, programming assignments every week (Matlab/Octave). Probabilistic Graphical Models (Coursera) http://goo.gl/oSOIi Advanced course on ML (Stanford University) Quizzes, programming assignments (Matlab/Octave) Introduction to AI (Udacity) http://goo.gl/UlM4C No programming assignment, just quizzes. AI: Programming A Robotic Car. (Udacity, CS373) http://goo.gl/FJ8ml Probabilistic approaches to AI/robotics. Quizzes, programming assignments (in Python!) SCE, NTU ML for CV APAC PyCon, 09/June/2012 24 / 34
  • 38. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Machine Learning Resources (cont.) Programming Collective Intelligence: Building Smart Web 2.0 Applications, Segaran, 2007 ML algorithms in Python. Real world examples on Web 2.0 applications. SCE, NTU ML for CV APAC PyCon, 09/June/2012 25 / 34
  • 39. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Machine Learning Resources (cont.) Opensource Python ML libraries scikit-learn: a general purpose ML library in Python. Active community, large development team. Up to date documentation. Orange: data visualization and analysis, visual programming. LibSVM: specific on Support Vector Machine, Python interface. mlpy, PyBrain, PyML, mdp-toolkit, etc. http://mloss.org – up-to-date list of ML software projects. Scipy, NumPy – scientific programming libraries – the core of almost Python ML libraries. SCE, NTU ML for CV APAC PyCon, 09/June/2012 26 / 34
  • 40. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Outline 1 Introduction 2 ML/CV example 1: Skin Detection in Images 3 ML/CV example 2: Object Recognition by Bag of Word model 4 Machine Learning Resources 5 Computer Vision Resources 6 Q&A SCE, NTU ML for CV APAC PyCon, 09/June/2012 27 / 34
  • 41. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Computer Vision Resources CV textbooks Digital Image Processing, Gonzalez and Woods, 2010. Computer Vision: A modern approach, Forsyth and Ponce, 2003. Computer Vision: Algorithms and Applications, Szeliski, 2011. (Free open book!) http://goo.gl/t4Znn Comprehensive collection of the state of the art algorithms in CV. Computer Vision: Models, Learning and Inference, Prince, 2012. (Free open book!) http://goo.gl/AqgO5 ”A book on statistics with vision examples”. Online pseudo-code of many ML algorithms specified for CV application. SCE, NTU ML for CV APAC PyCon, 09/June/2012 28 / 34
  • 42. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Computer Vision Resources (cont.) Programming Computer Vision with Python, Solem, 2012. (Free open book!) http://goo.gl/cHQIn CV algorithms in Python. Real world CV examples. SCE, NTU ML for CV APAC PyCon, 09/June/2012 29 / 34
  • 43. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Computer Vision Resources (cont.) Learning OpenCV: Computer Vision with the OpenCV library, Bradski and Kaehler, 2008. Learn CV algorithms by practice. Code examples in C. Online document on Python API: http://goo.gl/q0GTa SCE, NTU ML for CV APAC PyCon, 09/June/2012 30 / 34
  • 44. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Computer Vision Resources (cont.) Opensource Python CV libraries OpenCV http://goo.gl/cqYyw Active community, large development team. Large collection of CV algorithm. C++, Python API. ML libraries, but in C++. PyGame http://www.pygame.org Basic CV/IP support. Scipy/Numpy http://scipy.org General scientific programming libraries. Images/videos as multi-dimensional arrays. Python Image Library (PIL) http://goo.gl/FS3xN Image and graphics processing functions. SCE, NTU ML for CV APAC PyCon, 09/June/2012 31 / 34
  • 45. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Computer Vision Resources (cont.) Free online course on coursera.org: Computer Vision (UC, Berkeley) http://goo.gl/LNhCR Just quizzes, no programming assignment. Research papers from ML/CV (top-rank) conferences: http://cvpapers.com/ The list of companies developing CV products (up to date, maintained by David Lowe, author of SIFT): http://goo.gl/oUPVp SCE, NTU ML for CV APAC PyCon, 09/June/2012 32 / 34
  • 46. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Outline 1 Introduction 2 ML/CV example 1: Skin Detection in Images 3 ML/CV example 2: Object Recognition by Bag of Word model 4 Machine Learning Resources 5 Computer Vision Resources 6 Q&A SCE, NTU ML for CV APAC PyCon, 09/June/2012 33 / 34
  • 47. Introduction MLCV1: SD MLCV2: OR ML Resources CV Resources Q&A Q&A Thank you! SCE, NTU ML for CV APAC PyCon, 09/June/2012 34 / 34