https://telecombcn-dl.github.io/2017-dlcv/ Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.

1. [course site]
Elisa Sayrol Clols
elisa.sayrol@upc.edu
Associate Professor
Universitat Politecnica de Catalunya
Technical University of Catalonia
Face Recognition
(with Ramon Morros)
Day 2 Lecture 5
#DLUPC

2. Face Recognition
•Face Detection
•Face Alignment/ Frontalization
•Face Recognition
•Face Identification (Classification)
•Face Verification(Binary Decision)
•People Recognition in videos (Camomile Project at UPC)
•Speech
•Face
•Text

3. Face Detection and Recognition
Biometrics - Security is a relevant application of Face Recognition
[NEC’s NoeFace]

4. Databases
YouTube Faces: [http://www.cs.tau.ac.il/~wolf/ytfaces/]
621.126 pictures, 1.595 identities (celebrities).
Images come from videos so there is not a lot of variability
between them.
May overlap with other celebrity databases
Available info: Original frames, cropped faces, aligned faces.
Head-pose angles for all the faces
FaceScrub: [http://vintage.winklerbros.net/facescrub.html]
106.863 photos of 530 celebrities, 265 whom are male (55.306 images), and 265 female (51.557 images). Face
bounding boxes provided. Full frame and cropped version available.
MegaFace: [http://megaface.cs.washington.edu/]
1 milion faces, 690.572 unique people
MSRA-CFW [http://research.microsoft.com/en-us/projects/msra-cfw/]
202.792 faces, 1.583 people (celebrities). May overlap with other celebrity databases.

5. Databases
Labeled Faces in the Wild (LFW)
[http://vis-www.cs.umass.edu/lfw/ ]
13.000 images of faces collected from the web,
1.680 of the people pictured have two or more
Labeled Faces in the Wild: A Survey.
In Advances in Face Detection and Facial Image Analysis,
edited by Michal Kawulok, M. Emre Celebi,
and Bogdan Smolka, Springer, pages 189-248, 2016.
CelebFaces
[http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html ]
202.599 face images of 10.177 identities (celebrities).
People in LFW and CelebFaces+ are mutually exclusive.
Other databases: https://deeplearning4j.org/opendata
GoogleUPC !!!

6. DeepFace Architecture
Yaniv Taigman, etc (Facebook) . DeepFace: Closing the Gap to Human-Level Performance in Face Verification, CVPR
2014

7. DeepFace, Verification
*S. Chopra, R. Hadsell, and Y. LeCun. Learning a similarity met-ric
discriminatively, with application to face verification, CVPR,2005.
B) Use of Siamese Networks inspired
in Chopra et al*
A) Weighted χ2 distance
where f1
and f2
are the DeepFace
Representations.
The weights parameters wi
are
learned using a linear SVM
In DeepFace:
αi
are the trainable parameters with
standard cross-entropy loss and
backward propagation

8. FaceNet
Florian Schroff et al. (Google) FaceNet: A Unified Embedding for Face Recognition and Clustering, CVPR 2015
This Face recognition/verification/clustering model learns a mapping from face images to a
compact Euclidean space where distances directly correspond to a measure of face
similarity.
Triplet Loss function: Where f is the embedding

9. Deep ID2
Parameters of the Network:
But you compute parameters
From a Verification loss
function and an Identification
loss Function
Yi Sun, etc. Deep Learning Face Representation by Joint Identification-Verification. NIPS 2014

10. Deep ID2
When you backprop you
backprop gradients of
verification and
identification parameters
and you also update the
weight of the convolutional
layers

11. Deep ID2
DeepID2 Uses a Joint Bayesian model in top of the network for face
verification.
If we model a face as
Verification is achieved through Log-Likelihood Ratio
Test:
Represents the Identity. Interpersonal variations
Intrapersonal variations
Both Gaussian Distributed, estimated during Training
Chen, et al. Bayesian Face Revisited: A Joint Formulation, ECCV 2012
HI
Interpersonal hypothesis
HE
Extrapersonal hypothesis

12. Open Challenges in Face Recognition
Difficult initial detection:
Illumination, False positives, Gender
High intra-class variability:
Apparels, Expression, Rotations, Aging
Multimodality (especially in Video),
Partial Annotations, Incremental Learning

13. Questions?