1. Eigenfaces Developed in 1991 by M.Turk & A.Pentland Based on PCA Fisherfaces Developed in 1997 by P.Belhumeur et al. Based on Fisher’s LDA Moshe Guttmann

45. Fisherfaces – experiments P.Belhumeur et al. – Fisherfaces vs Eigenface

46. Fisherfaces – experiments P.Belhumeur et al. – Fisherfaces vs Eigenface

47. Fisherfaces – experiments P.Belhumeur et al. – Fisherfaces vs Eigenface

49. Appendix – PCA proof Given a sample of n observations on a vector of p variables λ where the vector is chosen such that define the first principal component of the sample by the linear transformation is maximum

50. Appendix – PCA proof cont’ Likewise, define the k th PC of the sample by the linear transformation where the vector is chosen such that is maximum subject to and to

51. Appendix – PCA proof cont’ To find first note that where is the covariance matrix for the variables

52. Appendix – PCA proof cont’ To find maximize subject to Let λ be a Lagrange multiplier by differentiating… then maximize is an eigenvector of corresponding to eigenvalue therefore

53. Appendix – PCA proof cont’ We have maximized So is the largest eigenvalue of The first PC retains the greatest amount of variation in the sample.

54. Appendix – PCA proof cont’ To find the next coefficient vector maximize then let λ and φ be Lagrange multipliers, and maximize subject to and to First note that

55. Appendix – PCA proof cont’ We find that is also an eigenvector of whose eigenvalue is the second largest. In general The k th largest eigenvalue of is the variance of the k th PC. The k th PC retains the k th greatest fraction of the variation in the sample.