O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Dog Breed Classification Using Part          Localization     Jiongxin       Liu 1,                    Angjoo             ...
Fine-grained classification                                           [Branso[Nilsback                                    ...
Related work• Dense feature extraction:  – Mine discriminative region with random forests [Yao et al    ’11]  – Multiple K...
Related work• Dense feature extraction:  – Mine discriminative region with random forests [Yao et al    ’11]              ...
Same breed or not?              NO!!Entlebucher Mountain Dog   Greater Swiss Mountain Dog
Key insight: Differences in common parts are              more informative  Entlebucher Mountain Dog                 Great...
“Columbia dogs with parts” dataset       133 breeds, 8351 images
Low inter-breed variation   Norfolk Terrier or Cairn Terrier?
High intra-breed variation      Both labrador retriever
Innumerable Poses
Diverse Appearances
Varying geometry of parts
Overview of the system1. Face Detection    2. Part Detection 3. Feature Extraction and ear localization                   ...
Pipeline 1: Dog Face Detection                            Keep the 5                            highest scoring           ...
Pipeline 2: Localize Parts            Part locations    Detector responses            Idea: From the “fit” to K most      ...
Review: Consensus of Exemplars                               ...Local Part Detectors   Exemplar Selection   Part Localizat...
RANSAC-like Exemplar Selection1. Repeat r times:   a. Choose random exemplar k   b. Choose 2 random modes of local detecto...
Final Part LocalizationFor each face part i:   a. Compute distribution of this part from all M aligned exemplars   b. For ...
Pipeline 2: Localize Parts                          Part locations           Detector responses                           ...
Pipeline 3: Infer ears using detected parts     With r(=10) exemplars from each breed
Pipeline 3: Infer ears using detected parts     With r(=10) exemplars from each breed
Pipeline 4: ClassificationExtract SIFT at part locations for each breed+color   histogram  one vs all linear SVM classifier
Qualitative Results: Successful
Qualitative Results: Failures
Results: ROC curves
Available in iTunes now
Take a Picture                 By tapping                  the nose
Get the breed!
Browse Dog Breeds
Thank you!!
Próximos SlideShares
Carregando em…5
×

Dog Breed Classification Using Part Localization

2.664 visualizações

Publicada em

Presentation for the ECCV 2012 Dog paper http://www.umiacs.umd.edu/~kanazawa/papers/eccv2012_dog_final.pdf

Publicada em: Tecnologia
  • Seja o primeiro a comentar

Dog Breed Classification Using Part Localization

  1. 1. Dog Breed Classification Using Part Localization Jiongxin Liu 1, Angjoo Kanazawa2, David Jacobs 2, and Peter Belhumeur1 1 Columbia University 2 University of Maryland
  2. 2. Fine-grained classification [Branso[Nilsback n et aland ‘10]Zisserman’08][Parkhi etal ’12][Kumar etal ‘12]
  3. 3. Related work• Dense feature extraction: – Mine discriminative region with random forests [Yao et al ’11] – Multiple Kernel Learning [Nilsback and Zisserman ’08] – Post-segmentation [Parkhi and Zisserman ’12]• Pose-normalized appearance: – Birdlets [Farrell et al ’11]
  4. 4. Related work• Dense feature extraction: – Mine discriminative region with random forests [Yao et al ’11] Generic sampling of features – Multiple Kernel Learning [Nilsback and Zisserman ’08] contains more noise than useful – Post-Segmentation [Parkhi and Zisserman ’12] information• Pose-normalizedfine-grained classification! for appearance: – Birdlets [Farrell et al ’11]
  5. 5. Same breed or not? NO!!Entlebucher Mountain Dog Greater Swiss Mountain Dog
  6. 6. Key insight: Differences in common parts are more informative Entlebucher Mountain Dog Greater Swiss Mountain Dog Localize parts based on a non-parameteric method by [Belhumeur et al ‘11]
  7. 7. “Columbia dogs with parts” dataset 133 breeds, 8351 images
  8. 8. Low inter-breed variation Norfolk Terrier or Cairn Terrier?
  9. 9. High intra-breed variation Both labrador retriever
  10. 10. Innumerable Poses
  11. 11. Diverse Appearances
  12. 12. Varying geometry of parts
  13. 13. Overview of the system1. Face Detection 2. Part Detection 3. Feature Extraction and ear localization 4. One vs All classification
  14. 14. Pipeline 1: Dog Face Detection Keep the 5 highest scoring windows
  15. 15. Pipeline 2: Localize Parts Part locations Detector responses Idea: From the “fit” to K most similar exemplars weighted by the detector output, take the most probable part location
  16. 16. Review: Consensus of Exemplars ...Local Part Detectors Exemplar Selection Part Localization Slide from Neeraj Kumar
  17. 17. RANSAC-like Exemplar Selection1. Repeat r times: a. Choose random exemplar k b. Choose 2 random modes of local detector outputs D={d i} on query c. Find similarity transform t that aligns exemplar to these points d. Evaluate match of all i face parts for this (k,t) pair: n Probability of this configuration given P(Xk,t | D) = C Õ P(x i k,t i |d ) Part detector probability at this (aligned) location i detector outputs e. Add (k,t) pair to list of possible exemplars, ranked by score2. Take top M (k,t) pairs for determining global configuration Slide from Neeraj Kumar
  18. 18. Final Part LocalizationFor each face part i: a. Compute distribution of this part from all M aligned exemplars b. For each of the top M aligned exemplars [(k,t) pairs]: Multiply normalized local detector outputs with global distribution of part computed from exemplars to get scores at each pixel location c. Add all scores together to get final scores at each pixel and choose max Slide from Neeraj Kumar
  19. 19. Pipeline 2: Localize Parts Part locations Detector responses Difference between current part location and that of exemplarFrom K most similar exemplars and the detector output, take the most probable part location
  20. 20. Pipeline 3: Infer ears using detected parts With r(=10) exemplars from each breed
  21. 21. Pipeline 3: Infer ears using detected parts With r(=10) exemplars from each breed
  22. 22. Pipeline 4: ClassificationExtract SIFT at part locations for each breed+color histogram  one vs all linear SVM classifier
  23. 23. Qualitative Results: Successful
  24. 24. Qualitative Results: Failures
  25. 25. Results: ROC curves
  26. 26. Available in iTunes now
  27. 27. Take a Picture By tapping the nose
  28. 28. Get the breed!
  29. 29. Browse Dog Breeds
  30. 30. Thank you!!

×