1. Perceptual Image Distortion
Patrick Teo and David Heeger
Stanford University
First IEEE International Conference on Image Processing, v2, pp 982-986,
November 1994
1

2. Distorted Images with Similar
Mean Squared Errors
Many imaging and image processing
methods are evaluated by how well
the images they output resemble
some given image. Examples include:
image data compression, dithering
algorithms, flat-panel display and
printer design. In all of these cases, the
human visual system is the judge of
image fidelity. Most of these methods
use the mean squared error (MSE) or
root mean squared error (RMSE)
between the two images as a measure
of visual distortion. These measures
are popular largely because of their
analytical tractability. It has long been A commonly used image of Albert Einstein
in image processing.
accepted that MSE (or RMSE) is
inaccurate in predicting perceived
distortion. This is illustrated in the
following paradoxical example.
2

3. Similar Mean Squared Errors
(cont’d)
The top two images on the right were
created by adding different types of
distortions to the original image; the
original image is shown below them.
Root Mean Squared Error of 8.5 The root mean squared error (RMSE)
between each of the distorted images
and the original were computed. The
root mean squared error is the square
root of the average squared difference
between every pixel in the distorted
image and its counterpart in the
original image.
The RMSE between the first distorted
image and the original is 8.5 while the
Root Mean Squared Error of 9.0
RMSE between the second distorted
image and the original is 9.0. Although
the RMSE of the first image is less than
that of the second, the distortion
introduced in the first image is more
visible than the distortion added to
the second. Thus, the root mean
squared error is a poor indicator of
perceptual image fidelity.
Original
3

4. A Computational Model of
Perceptual Image Distortion
We have developed a perceptual distortion measure based on a model
of spatial pattern detection.
It is important to recognize the relevance of these empirical spatial
pattern detection results to developing measures of image integrity. In a
typical spatial pattern detection experiment, the contrast of a visual
stimulus (called the target) is adjusted until it is just barely detectable.
Threshold contrasts of the target are measured over a range of spatial
frequencies, mean luminance, and spatial extents.
In some experiments (called contrast masking experiments), the target
is also superimposed on a background pattern (called the masker). In
other experiments (called luminance masking experiments), the target
is superimposed on a brief, bright, uniform background. In either case
(contrast or luminance masking), the contrast of the target is adjusted
(while the masker is held fixed) until the target is just barely detectable.
Typically, a target is harder to detect (i.e., a higher contrast is required)
in the presence of a masker.
A model that predicts spatial pattern detection is obviously useful in
image processing applications. In the context of image compression, for
example, the target takes the place of quantization error and the
masker takes the place of the original image.
4

5. Perceptual Image Distortion
Our model consists of three main
parts: a retinal component, a cortical
component, and a detection
mechanism. The retinal component is
responsible for contrast sensitivity and
its dependence on mean luminance
masking. The cortical component
accounts for contrast masking. To
compute perceptual image distortion,
the reference and distorted images are
passed through these two stages of
the model independently. At this
point, the images have been
normalized for the differential
sensitivities of the human visual
system. The final (detection
mechanism) component of the model
A model of perceptual image distortion. compares these two normalized
images to give a measure of image
fidelity. The final result is an image
representing the probability of
perceiving a distortion at each position
in the distorted image.
5

6. Model Predictions of
Visible Distortion
The top image on the right is the
original image. The two images
directly below it were created by
adding different types of distortions to
the original image. The root mean
squared error (RMSE) between the left
distorted image and the original (8.5)
is smaller than the root mean squared
error between the right distorted
image and the original (9.0). In spite of
that, the distortion is more visible in
the left image.
The images directly below each
distorted image are the predictions of
the perceptual image distortion
model. Lighter areas indicate regions
where the distortion is more visible
while darker areas indicate regions
where the distortion is less visible. The
model correctly predicts that the left
distorted image is more visibly
distorted than the right.
Model predictions: lighter areas indicate greater visible errors.
6

7. Visible Distortion in JPEG
Compressed Images
To further validate the model's
performance, we applied the model to
JPEG compressed images. The original
image was compressed using the JPEG
algorithm at different quality settings.
The model was then used to predict
the visibility of the distortion between
each compressed image and the
original.
The pairs of images on the left are
images compressed using the JPEG
algorithm along with the model's
predictions of the amount of visible
distortion when compared with the
JPEG qual. setting = 80, RMSE = 9.5, PDM = 1.2 original. The image compressed at a
quality setting of 80 is virtually
indistinguishable from the original.
The model's prediction corroborates
this observation. The average
distortion value computed by the
model is 1.2, which indicates that the
distortion is slightly above threshold
(threshold is set at 1.0).
7

8. JPEG Compressed Images
(cont’d)
The image compressed at a quality
setting of 20 is slightly deteriorated
JPEG qual. setting = 20, RMSE=11.4, PDM = 5.7
while the image compressed at a
quality setting of 10 shows marked
blocking artifacts. The model's
predictions agree with these trends
fairly well.
JPEG qual. setting = 10, RMSE = 12.9, PDM = 9.8
8

9. Error Histograms of JPEG
Compressed Images
The top graph plots a histogram of the
squared error differences for individual
pixels. The bottom graph plots a
histogram of the perceptual distortion
predictions of the model for individual
pixels. Both histograms have been
normalized so that the vertical axis
represents fractions of the total
number of pixels.
The histograms of squared error
differences for the different
compressed images are very similar to
one another. The histograms of
perceptual distortion predictions of
the different compressed images are
dramatically different from one
another. It is clear, for example, that
the model predicts that the images
compressed at quality settings of 20
and 10 (the "middle" and "right"
images) are more severely distorted
than the image compressed at a
Histograms of Squared Error (top) and Perceptual quality setting of 80 (the "left" image).
Distortion Measure (bottom) predicted by the model for
the 3 JPEG compressed images: “left” (JPEG quality 80),
“middle” (JPEG quality 20), “right” (JPEG quality 10).
9