1. Adaptive Biometric Systems based on
Template Update Paradigm
Ajita Rattani
University of Cagliari,
Department of Electrical and Electronic Engineering,
ajita.rattani@ diee.unica.it
Supervisors: Prof. Fabio Roli and Dr. Gian Luca Marcialis
P R A G

2. What is Biometrics?
Automatic recognition of person based on their distinctive
anatomical and behavioral characteristics like face and
fingerprint.
Fingerprint Face Signature Voice Hand geometry
Facial Retinal scan Iris Gait
thermogram
2

3. Biometric Phases
Enrollment Phase
Verification/ Identification Phase
3

4. Enrollment Phase
Enrollment Phase
x, y, theta x, y, theta
Feature x, y, theta
“ x, y, theta
Extraction x, y, theta x, y, theta Storage
Extracted Mr. X
Features
Database
Template
4

5. Verification Phase
Database Template
yes
Feature Matching Score or Score >
extraction m odule distance
threshold
Input Query
no Accepted
Rejected
5

6. Problem: Intra-Class Variations
6

7. Template Representativeness
Enrolled templates: usually captured in controlled
environment
Input Query : Substancial intra-class variation
Effect: Making enrolled templates ‘Un-representative’
7

8. Standard Solutions
Multi-biometric
Storing multiple templates (multi-instance)
Using Multi-modalities
Repeating the process of enrollment over time
8

9. Multibiometric
Super Template Multi-Modality
A. Rattani, D. R. Kisku, A. Lagorio and M. Tistarelli, “Facial Template
A. Rattani, D. R. Kisku, M. Bicego and M. Tistarelli, “Feature Level Fusion
Synthesis Based on SIFT Features”, Automatic Identiffication Advanced of Face and Fingerprint”, Biometrics: Theory, Applications and Systems (BTAS 2007), 1-6,
Technologies (AUTOID) 2007 IEEE Workshop, 69-73, Alghero, Italy, 2007 Washington, USA
9

10. Template Update: Solution to
Representativeness
Standard Solutions: Fails to capture Temporal Intra-class
variations
Novel Solutions : “Template Update” procedure/ Adaptive
biometric systems
Aim: Update enrolled templates to the intra-class variation
of the input data
10

11. State of Art: template update
Not Mature Enough
No mention of the learning methodology involved
No investigation of the pros, cons and open issues
Lack of clear statement of the problem
11

12. Goal of PhD Studies
Formulate the taxonomy of the current state of art
template update methods
Pros and Cons of State of Art Update Methods
Effect of update procedures on different group of
users (‘Doddington Zoo’)
Proposal of Novel solution
12

13. Ajita Rattani, Biagio Freni, Gian Luca Marcialis, Fabio Roli , “Template Update Methods in Adaptive Biometric Systems: A
Critical Review", 3rd IEEE/IAPR International Conference on Biometrics ICB 2009, Alghero (Italy), Springer, 02/06/2009
Template based Adaptive Biometric System
Semi-supervised
Supervised
Multiple
Single Modality
Template Selection Modality
Co-training
Editing Self-training
Clustering based Graph
based Mincut
Online Offline
Feature Selection
13

14. State of the Art (Template Update)
Supervised Learning
(Uludag et al., PR 2004)
Offline process
Limitations:
Tedious, time consuming
Inefficient for repeated
updating task
14

15. ….Contd
Semi-Supervised Learning
Initial labelled + Unlabelled input
data (“Automatic Self Update”)
Online Updating
Jiang and Ser, PAMI 2002;
Ryu et al., ICPR 2006
Offline Updating
Roli and Marcialis, SSPR
2006, Roli et al., ICB 2007
15

16. Template Co-update: A Conceptual Example
Initial template Unlabeled Samples
Roli et al. (ICB2007)
Difficult face sample
ple
16

17. Protocol followed for Experimental
Investigation
For Database of size N x M
One sample : Initial template
Remaining M-1 samples are divided into Unlabelled and Test set
Equal number of impostor samples are added: Unlabelled and
Test Set
Unlabelled set (Du): for updating the templates
Test set: measures the performance enhancement after
updating
17

18. An Experimental Analysis on Pros and
Cons of Self-update and Co-update
Performance comparison of the Co-update with Self update
Representativeness of the enrolled templates
Controlled and Un-controlled environment
Can operation at relaxed threshold help “self-update” to
capture difficult patterns?
• Ajita Rattani, Gian Luca Marcialis and Fabio Roli, Capturing large intra-class variation of the biometric data by template co-
updating,IEEE Workshop on Biometrics, Int. Conference on Vision and Pattern Recognition CVPR 2008, Anchorage (Alaska,
USA), IEEE, pp. 1-6, 23/07/2008
• A. Rattani, G.L. Marcialis, F. Roli, Boosting gallery representativeness by co-updating face and fingerprint verification systems,
Best Paper Award at 5th International School for Advanced Studies on Biometrics for Secure Authentication, June, 9-13,
2008, Alghero (Italy).
18

19. Co-updating vs. Self-update: Un-controlled
Environment; EER point of view
30
Face Self-Update 14
Finger Self-Update face self-update
Face Co-update face co-update
finger self-update
25 Finger Co-update 12
finger co-update
10
20
EER (%)
EER (%)
8
15
6
10 4
2
5 0 50 100 150 200 250 300
0 50 100 150 200 250 300 350 # No. of unlabelled data added
# No. of unlabelled data added
Shows EER on the test set as a function of the amount of unlabelled data exploited by template self and
co-update algorithms at each iteration. The curve of the self update is shorter due to non-exploitation of
much unlabelled data because of operation at high threshold.
19

20. Galleries Images as captured by Self-
update and Co-update
Differences with Self-update:
More Unlabelled samples added
Larger intra-class variations
introduced even at initial stages
Initial
19
template initial accuracy
face self-update at varying threshold
18
17
16
EER (%)
15
Initial 14
template 13
12
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
%FAR used for selecting threshold for unlabelled data
20

21. Local Update Behaviour of Self-update
21

22. Remarks
Template Co-update:
Non-Representative templates: Can capture large intra-class variations
Representative templates: Comparable performance of Self-update and Co-
update
Self-updating : very much dependent on the initial templates.
Un-representative initial templates: Results in poor capture of difficult
samples due to operation at stringent threshold
However, operation at relaxed threshold results in counter -productive effect
Ajita Rattani, Gian Luca Marcialis and Fabio Roli, Capturing large intra-class variation of the biometric data by template co-
updating,IEEE Workshop on Biometrics, Int. Conference on Vision and Pattern Recognition CVPR 2008, Anchorage (Alaska,
USA), IEEE, pp. 1-6, 23/07/2008
22

23. Open Issues Unexplored
Effect of Creep in errors (‘impostor introduction’)
Effect of different types of updating threshold
Analysis of the effect of user population on template
update procedure
23

24. Difficult Clients and “Doddington’s zoo”
Doddington et al. (1998) introduced some terms to indicate clients
wrongly classifiable even at high thresholds
Lambs: “easy-to-imitate” clients
High FAR when attacked
Wolves: they can easily imitate other clients
A wolf into a client’s gallery may attract other wolves
Goats: difficult to be recognized
A goat may not be able to update itself
Sheeps: Well behaved Clients
24

25. User Population Characteristics
Hypothesis:
Apart from basic FAR of the system, impostors may be
introduced due to the presence of wolves and lambs
Effect of template updating may not be same because
of the presence of “Doddington zoo”
25

26. Goal of the work
Experimental evaluation of the impact of impostors introduction in on-
line self update
At different settings of updating threshold
Fixed/Dynamic
Global/User-specific
Stringent/Relaxed
Presence of intrinsically “difficult” clients
Non-uniform effect of update procedures on different charateristic
clients
26

27. EER vs. impostors introduction at 1%
updating threshold
34 25
Fixed Non-user specific Fixed Non-user specific
Updated Non-user specific Updated Non-user specific
Fixed User specific Fixed User specific
32
Updated User-Specific 20 Updated User-Specific
Equal Error Rate (EER)
30
% of impostors
15
28
10
26
24 5
22
0 100 200 300 400 500 600 0
# of Unlabelled data used 0 100 200 300 400 500 600
# of Unlabelled data used
Gian Luca Marcialis, Ajita Rattani and Fabio Roli, Biometric template update: An experimental investigation on the relationship
between update errors and performance degradation in face verification, Joint IAPR Int. Workshop on Structural and Syntactical
Pattern Recognition and Statistical Techniques in Pattern Recognition S+SSPR08, Orlando (Florida, USA), Springer, 04/12/2008
27

28. Performance Evaluation of Self-Update After
Division of Database on the basis of Doddington Zoo
1. Lambs 2. Sheeps
100 100 Ajita Rattani, Gian Luca Marcialis
After Updating After Updating and Fabio Roli, "An Experimental
Before Updating Before Updating Analysis of the Relationship between
Biometric Template Update and the
(%) FRR
(%) FRR
50 50
Doddington’s Zoo in Face
Verification", ICIAP 2009, Salerno
(Italy)
0 0
0 50 100 0 50 100
(%) FAR (%) FAR
3. Goats 4. Wolves
100 100
After Updating After Updating
Before Updating Before Updating
(%) FRR
(%) FRR
50 50
0 0
0 50 100 0 50 100
(%) FAR (%) FAR
28

29. “Attraction” path
Unlabelled samples iteratively added to the gallery
Initial template First impostor Other wolves
(wolf) are added
29

30. Remarks
For first-time the effect of misclassification errors in self
update process
It resulted to be very much dependent on the threshold
type settings and the security level for acceptance of input
data
Impostors inclusion cannot be avoided even at strict
threshold settings (zeroFAR)
The presence of different animals result in different
updating effects
30

31. Open Issues Still Remained!
As Analyzed :
Current state of art methods are capable of capturing only near input
images
Operation at relaxed threshold results in increased probability of
impostors introduction
Need: Investigation of more robust update procedures with the
following characteristics
Capture of large intra-class variations without increasing probability of
impostors
Not increasing the probability of impostors introduction
31

32. Graph based Semi-Supervised Learning
Self-update methods : ‘Local’ update behaviour
Graph based methods to Semi-supervised methods :
Application: Machine Learning literature like Image Segmentation , Pattern
Recognition
These methods can study the global structure of the data manifold
Hypothesis: Graph based learning may capture large intra-class
variations
Mincut based labelling is a binary technique assigning labels by finding
min-cut

33. “Well-connected” and “Separated”
hypothesis
Region as a set of different people
(expressions, lighting, poses)
Graph-mincut can better assign
labels to each region, even with a
small amount of labelled samples
(Blum and Chawla, 2001) by
studing underlying structure in the
form of graph.
A. Rattani, G.L. Marcialis, F. Roli, Biometric template update using the graph-mincut algorithm: a case study in face verification,
IEEE Biometric Symposium BioSymp08, September, 23-25, 2008, Tampa (Florida, USA), IEEE, ISBN 978-1-4244-2567-9, pp. 23-
28.
33

34. Basic Graph based Mincut
Graph G= (V, E) ; V= {L, U, v+, v-}
{v+, v-}: Two classification vertices, null nodes
representing “positive” and “negative” classes.
E : edge defining function, basis on which two nodes are
connected
Aim : partition v+ from v- by finding the cut on the
minimum similarity set of edges.
34

35. Graph Theory: Working of Mincut
V E
V+ V-
1. Given an initial Graph
35

36. ….Contd
2. All possible s-t paths are traversed
36

37. ….Contd
3. Flow is increased by an amount which
different capacity edges can take
37

38. ….Contd
Last step: All the nodes reacheable from source are
classified as positive
38

39. Why Graph Mincut may Work ?
Global structure of manifold is analyzed:
By traversing all s-t paths
Minimum capacity edges are saturated first
Probability of impostor introduction is minimized
39

40. An Hypothetical Example:
40

41. Samples Exploited for Updating : Self
Update and Mincut
% Impostors Encountered
% Samples Encountered
A. Rattani, G.L. Marcialis, F. Roli, Biometric template update
using the graph-mincut algorithm: a case study in face
verification, IEEE Biometric Symposium BioSymp08,
September, 23-25, 2008, Tampa (Florida, USA), IEEE, ISBN
978-1-4244-2567-9, pp. 23-28.
41

42. Concluding Remarks
Critical survey on the template update procedure
Pros and cons of state of art methods
Studied the effect of impostor introduction
Proposed novel solutions
42

43. Future Work
Modeling of probability of impostor introduction
The use of quality information of an input sample:
Quality measures are an array of measurements of
conformance of biometric samples to some predefined
criteria known
Genuine Intra-class
variation?
43

44. …Contd
Modeling of Appropriate Stopping criteria for Template
Updating
Use of Cohort information in template updating
Norman et al. 2009
44

45. …Contd
Robust criteria for selection of input data for updating: F-
Ratio or d-prime
FRatio=(µ Gen-µ Imp) ⁄ (σGen+ σImp)
D-prime=(µ Gen-µ Imp)/(σ)
Evaluation on “Large Scale Databases”
45

46. 46