1. Computer vision:
models, learning and inference
Chapter 18
Models for style and identity
Please send errata to s.prince@cs.ucl.ac.uk

2. Identity and Style
Identity differs,
but images similar
Identity same, but
images quite
different
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 2

3. Structure
• Factor analysis review
• Subspace identity model
• Linear discriminant analysis
• Non-linear models
• Asymmetric bilinear model
• Symmetric bilinear model
• Applications
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 3

4. Factor analysis review
Generative equation:
Probabilistic form:
Marginal density:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 4

5. Factor analysis
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 5

6. Factor analysis review
E-Step:
M-Step:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 6

7. Factor analysis vs. Identity model
• Each color is a different identity
• multiple images lie in similar part of subspace
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 7

8. Subspace identity model
Generative equation:
Probabilistic form:
Marginal density:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 8

9. Subspace identity model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 9

10. Factor analysis vs. subspace identity
Factor analysis Subspace identity
model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 10

11. Learning subspace identity model
E-Step:
Extract moments:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 11

12. Learning subspace identity model
E-Step:
M-Step:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 12

13. Subspace identity model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 13

14. Subspace identity model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 14

15. Inference by comparing models
Model 1 – Faces match (identity shared):
Model 2 – Faces dont match (identities differ):
Both models have standard form of factor analyzer
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 15

16. Inference by comparing models
Compute likelihood (e.g. for model zero)
where
Compute posterior probability using Bayes rule
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 16

17. Face Recognition Tasks
GALLERY PROBE
… ? 1. CLOSED SET
FACE IDENTIFICATION
GALLERY PROBE
… NO ? 2. OPEN SET
MATCH FACE IDENTIFICATION
PROBE
NO
MATCH
? 3. FACE VERIFICATION
? 4. FACE CLUSTERING
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 17

18. Inference by comparing models
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 18

19. Relation between models
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 19

20. Structure
• Factor analysis review
• Subspace identity model
• Linear discriminant analysis
• Non-linear models
• Asymmetric bilinear model
• Symmetric bilinear model
• Applications
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 20

21. Probabilistic linear
discriminant analysis
Generative equation:
Probabilistic form:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 21

22. Probabilistic linear discriminant analysis
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 22

23. Learning
E-Step
– write out all images of same person as system of equations
– Has standard form of factor analyzer
– Use standard EM equation
M-Step
– write equation for each individual data point
– Has standard form of factor analyzer
– Use standard EM equation
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 23

24. Probabilistic linear discriminant analyis
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 24

25. Inference
Model 1 – Faces match (identity shared):
Model 2 – Faces dont match (identities differ):
Both models have standard form of factor analyzer
Compute likelihood in standard way
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 25

26. Example results (XM2VTS database)
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 26

27. Structure
• Factor analysis review
• Subspace identity model
• Linear discriminant analysis
• Non-linear models
• Asymmetric bilinear model
• Symmetric bilinear model
• Applications
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27

28. Non-linear models (mixture)
Mixture model can describe non-
linear manifold.
Introduce variable ci which
represents which cluster
To be the same identity, must also
belong to the same cluster
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 28

29. Non-linear models (kernel)
• Pass hidden variable through non-linear function f[ ].
• Leads to kernelized algorithm
• Identity equivalent of GPLVM
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 29

30. Structure
• Factor analysis review
• Subspace identity model
• Linear discriminant analysis
• Non-linear models
• Asymmetric bilinear model
• Symmetric bilinear model
• Applications
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 30

31. Asymmetric bilinear model
• Introduce style variable sij
• indicates conditions in which data was observed
• Example: lighting, pose, expression face recognition
Asymmetric bilinear model
• Introduce style variable sij
• indicates conditions in which data was observed
• Example: lighting, pose, expression face recognition
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 31

32. Asymmetric bilinear model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 32

33. Asymmetric bilinear model
Generative equation:
Probabilistic form:
Marginal density:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 33

34. Learning
E-Step:
M-Step:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 34

35. Asymmetric bilinear model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 35

36. Inference – inferring style
Likelihood of style
Prior over style
Compute posterior over style using Bayes’ rule
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 36

37. Inference – inferring identity
Likelihood of identity
Prior over identity
Compute posterior over identity using Bayes’ rule
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 37

38. Inference – comparing identities
Model 1 – Faces match (identity shared):
Model 2 – Faces dont match (identities differ):
Both models have standard form of factor analyzer
Compute likelihood in standard way, combine with prior in Bayes rule
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 38

39. Inference – Style translation
• Compute distribution over identity
• Generate in new style
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 39

40. Structure
• Factor analysis review
• Subspace identity model
• Linear discriminant analysis
• Non-linear models
• Asymmetric bilinear model
• Symmetric bilinear model
• Applications
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 40

41. Symmetric bilinear model
Generative equation:
Probabilistic form:
Mean can also depend on style...
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 41

42. Symmetric bilinear model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 42

43. Inference – translating style or identity
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 43

44. Multilinear models
Extension of symmetric bilinear model to more
than two factors
e.g.,
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 44

45. Structure
• Factor analysis review
• Subspace identity model
• Linear discriminant analysis
• Non-linear models
• Asymmetric bilinear model
• Symmetric bilinear model
• Applications
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 45

46. Face recognition
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 46

47. Tensortextures
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 47

48. Synthesizing animation
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 48

49. Discussion
• Generative models
• Explain data as combination of identity and
style factors
• In identity recognition, we build models where
identity was same or different
• Other forms of inference such as style
translation also possible
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 49