2. Overview 2/26
• I t d ti
Introduction t the Red Eye problem
to th R d E bl
• Red Eye prevention
• Red Eye detection
• Semi-automatic methods
• Automatic methods
•R dE
Red Eye correction
ti
• Desaturation
• Inpainting techniques
• False positives and unnatural corrections
a po a du au a o o
• Red Eye removal examples
Red eye removal 17/04/2008
3. The Red Eye problem 3/26
The Red E
Th R d Eye phenomenon is a well-
h i ll
known problem which happens
when taking flash-lighted p
g g pictures
of people.
The pupils in the picture appear red
instead of black.
This h
Thi happens more often when
ft h
using compact, consumer-oriented
cameras.
a a
Red eye removal 17/04/2008
4. Why the Red Eye? 4/26
Red Eye Cone
Eye
α β
• The red eye cone shines from the flashed eye
back at the flash with an angle α;
g ;
• Its red color is caused by the reflection of the
flash off the blood vessels o the retina;
a o b ood of a;
• The camera will record this red hue if the angle β
be ee
between the flash and the camera is not greater
e as a d e ca e a s o g ea e
than α.
Red eye removal 17/04/2008
5. Red Eye prevention 5/26
Eye
Red Eye Cone
α
β
• If the equipment allows it, the flash can be spaced
further away from the sensor in order to increase
y
β (not possible on compact devices);
• One o more additional flashes before picture
O or o add o a a b o p u
acquisition make the iris tighten and decrease α;
• This methods reduce the probability of the Red
s e ods educe e p obab y o e ed
Eye phenomenon but don’t remove it entirely.
Red eye removal 17/04/2008
6. Red Eye detection 6/26
Most f th ti
M t of the times, red eyes must be removed
d tb d
during post-processing.
For d
F red eyes to be successfully removed, they must
t b f ll d th t
be first detected then corrected.
Methods are classified according to the detection
phase:
h
• Semi-automatic methods ask the user to
manually l
ll localize th red eyes;
li the d
• Automatic methods detect the red eyes
themselves.
th l
Red eye removal 17/04/2008
7. Semi-automatic methods 7/26
The eyes are manually selected
using a visual interface (
g (Adobe
Photoshop ®, Corel Paint Shop
Pro ®, ACD See ®, etc.)
Pros:
• Eyes are easy to localize for
y y
men.
Cons:
• It may be difficult to have
such an interface on a mobile
device;
• Automatic methods are easier
uo a od a a
to use and more appealing.
Red eye removal 17/04/2008
8. Automatic methods 8/26
Automatic methods attempt to find red eyes on
A t ti th d tt t t fi d d
their own. The task is harder than it may seem:
No “
N “perfect” R d E
f t” Red Eye d t ti
detection method has
th d h
been developed yet.
Automatic methods extract features from images in
A t ti th d t tf t f i i
order to identify red eyes. Different methods work
on different features:
• Face detection • Skin detection
• Eye detection • Flash-noFlash
comparison
Red eye removal 17/04/2008
9. Face-based methods 9/26
•F
Faces are looked for using a multiple feature
l k df i lti l f t
object based approach;
•O
Once th f
the face features have been found then the
f t h b f d th th
research is restricted to red pixels.
Face
Extraction
Extracted Multiple Masks
Area Red-Eye
Red Eye Research
Red eye removal 17/04/2008
10. Eye-based methods 10/26
Similar t face detection, but more complex
Si il to f d t ti b t l
because the features are less evident:
•EEyes are seeked, matching fi d t
k d t hi fixed templates at
l t t
different resolutions with regular eyes present
into the images, or looking for red pixels using
g , g p g
computed color LUTs.
Initial Single Eye
g y Pairingg
Candidate
Verification Verification
Detection
Red eye removal 17/04/2008
11. Skin-based methods 11/26
• Ski i d t t d first by pixel colors;
Skin is detected fi t b i l l
• Red circular patches near the skin are then
looked for.
l k df
This approach is simpler and does not take into
account th presence of more complex features.
t the f l f t
Skin
Detection Red Circular Pairingg
Patches Verification
Research
Optional
Red eye removal 17/04/2008
12. Flash-noFlash methods 12/26
• Two different pictures,
one with flash and one
ith fl h d
without, are acquired
one after the other;;
• Red eyes are detected
as patches whose color is
p
red in the “flash”
image and black in the
“nonflash” i
“ fl h” image.
This approach has several drawbacks:
• The dimension of the buffer must double;
• The two images may be mis-aligned;
e o ages ay s a g ed;
• The subject(s) may move between acquisitions.
Red eye removal 17/04/2008
13. An algorithm in detail 13/26
Red Color
Input Skin Morphological Detection in
Detection Operators Skin Area
Image
Morphological
Operators
Corrected Red-Eye Pairing Single Eye
Verification Verification
Image
g Correction
The Algorithm is Skin Feature Extraction based.
• First the skin is extracted and morphologically
modified to blob the enhanced areas;
• Then a successive red color detection is performed to find
red eyes in the skin areas;
• The red regions are then dilated, eroded and analyzed
to identify the Red-Eyes pairs.
Red eye removal 17/04/2008
14. Example 14/26
Input Image Skin Detection Morphological Red Color Detection
Operators
in Skin Area
Morphological Pairing Corrected
Operators Verification
Image
Red eye removal 17/04/2008
16. Algorithm drawbacks 16/26
• U bl t perform single red eye detection;
Unable to f i l d d t ti
• The skin detection is performed over the whole
image (slow);
• Big (
g (slow) morphological operators p
) p g p permit to g
get
good results only on small images (less than 1
Mpixels): it would require even bigger (and slower)
operators t operate on larger images.
t to t l i
Red eye removal 17/04/2008
17. Red eye correction 17/26
Once red eyes have been detected, they must be
O d h b d t t d th tb
corrected.
Red eye correction is quite simpler than detection,
but there are more diffi lt cases than others.
b t th difficult th th
Correction may vary from simple desaturation to
complete reconstruction of iris and pupil.
Red eye removal 17/04/2008
18. Desaturation 18/26
Desaturation means lowering/zeroing the chrominance
components while mantaining the luminance component
component.
It is the best way to correct “easy” red eyes.
Red eye removal 17/04/2008
19. Washed-out irises 19/26
Washed-out irises Wrong correction
Sometimes irises are totally washed out by reflected light.
In these cases a simple desaturation or color correction is not
enough.
It is necessary to use a more complex method to reconstruct a
realistic image of the eye.
Red eye removal 17/04/2008
20. Inpainting techniques 20/26
Some tools completely reconstruct the irises and the pupils
to
t replace the red eye (Jasc Paint Shop Pro ®).
l th d (J P i t Sh P ®)
The results, however, are often unrealistic and look like glass
eyes.
Red eye removal 17/04/2008
21. False positives 21/26
One of the biggest issue in red eye removal are false
p
positives in the detection phase.
p
Unwanted
corrections are
much less
desirable than
missing ones
ones.
Red eye removal 17/04/2008
22. Unnatural corrections 22/26
• Unnatural corrections are another
important issue.
• The most common ones are:
• Partial correction: only a
Partial correction
portion of the red pupil has
f
been corrected;
• Noisy correction: the presence
of heavy noise or jpeg
compression can introduce false
red pixels around the pupil and Noisy correction
thus a strange correction is
made over the iris;
• Wrong luminance correction:
in this case the disk has been
correctly found but the
correction is unnatural due to Wrong luminance
wrong luminance distribution. correction
Red eye removal 17/04/2008
23. Red Eye removal examples 23/26
StopRedeyes® AutoRemover®
Red eye removal 17/04/2008
24. Red Eye removal examples 24/26
StopRedeyes® AutoRemover®
Red eye removal 17/04/2008
25. Red Eye removal examples 25/26
StopRedeyes® AutoRemover®
Red eye removal 17/04/2008
26. References 26/26
• J.Y. Hardeberg, “Red Eye Removal using Digital Color Image Processing”, Conexant
Systems, Inc. , Redmond, Washington, USA
• M. Gaubatz, R. Ulichney, “Automatic Red-Eye Detection and Correction”, HP Lab, Cornell
University,
University ICIP 2002
• S. Ioffe, “Red Eye Detection With Machine Learning”, Fujifilm Software, California, ICIP
2003.
• F Gasparini, R. Schettini “Automatic Redeye Removal for Smart Enhancement of Photos of
F. Gasparini R Schettini, Automatic
Unknown Origin”, DISCO University of Milano-Bicocca, VIS 2005.
• H. Luo, J. Yen, D. Tretter, “An Efficient Redeye Detection and Correction Algorithm”, HP
labs, Palo Alto, California, ICPR 2004.
• A. Patti, K. Konstantinides, D. Tretter, Q. Lin, “Automatic Digital Redeye Reduction”, HP
Labs, Palo Alto California, ICIP 1998.
• L. Zhang, Y. Sun, M. Li, H. Zhang, “Automated Red-Eye Detection and Correction in Digital
Photographs”,
Photographs” Microsoft Research Asia China ICIP 2004
Asia, China,
• B. Smolka, K. Czubin, J.Y. Hardeberg, K.N. Palataniotis, M. Szczepanski, K. Wojciechowski,
“Towars Automatic Redeye Effect Removal”, Pattern Recognition Letter 2003.
• J S Schildkraut R T Gray “A Fully Automatic Redeye Detection and Correction Algorithm”,
J.S. Schildkraut, R. T. Gray, A Algorithm
Kodak Company, NY, ICIP 2002.
• R. Schettini, F. Gasparini, F. Chazli, ”A Modular Procedure for Automatic Redeye Correction
in Digital Photos”, DISCO University of Milano-Bicocca, SPIE 2004
• PETSCHNIGG, G., AGRAWALA, M., HOPPE, H., SZELISKI, R., COHEN, M., and TOYAMA, K.
2004. Digital photography with flash and no-flash image pairs. ACM Transactions on Graphics
23, 3 (Aug.), 664–672.
Red eye removal 17/04/2008