Be aware: this presentation is malformed, putting the slides on top of each other. However the download is OK again!

1. On colour
Some references
A visual list

2. Books

41. On the web

62. Papers

63. Color Vision
(e.g., spatial frequency, orientation, motion, depth) which the stimulus differs perceptually from a purely
within a local cortical region. With respect to color achromatic (i.e., white, gray, black) axis. The third
vision per se, the primary processing involves separ- dimension is brightness or lightness. That our per-
ating color and luminance information, and further ceptual space is three-dimensional reflects the basic
separating changes due to the illuminant from those trichromacy of vision.
due to visual objects, by lateral interactions over large A normal observer can describe the hue of any light
regions. (disregarding surface characteristics) by using one or
To separate luminance and color information, the more of only four color names (red, yellow, green, and
outputs of Pc cells are combined in two different ways. blue). These so-called unique hues form two opponent
When their outputs are summed in one way, the pairs, red–green and blue–yellow. Red and green
luminance components to their responses sum and the normally cannot be seen in the same place at the same
color components cancel. Summed in a different time; if unique red and unique green lights are added in
combination, the color components sum and the appropriate proportions, the colors cancel and one
luminance components cancel. Consider a striate sees a neutral gray. Orange can be seen as a mixture of
cortex cell that combines inputs from one or more red and yellow, and purple as a mixture of red and
jLo and jMo cells in a region. The cortical cell would blue, but there is no color seen as a red–green mixture
respond to luminance variations but not to color (or as a blue–yellow mixture). This perceptual op-
variations, since the neurons providing its inputs both ponency is also reflected in color contrast. Red can
fire to luminance increments in the RF center and to induce the appearance of green into neighboring
decrements in the surround, but the color organi- regions, and after staring at a red surface one sees a
zations of its inputs are opposite to each other (one green after-image. The yellow–blue opponent pair
being L-M and the other M-L). Combined with input produces similar effects. It was these perceptual
from a jSo cell, this would produce a V1 cell that fires characteristics of color that led Ewald Hering in the
to white (light increments) and inhibits to black (light nineteenth century to propose that the various color
decrements) but does not respond to pure color systems were not independent but rather that color
variations. This is represented in the top row of Fig. was processed in a spectrally opponent organization,
1C. However, a V1 cell receiving inputs from both an idea which has since been amply verified in the
jLo and kMo cells, or from both jMo and kLo cells presence, discussed above, of spectrally-opponent cells
(columns in Fig. 1C), would respond to color changes in the path from receptors to the cortex.
but not to luminance variations since their color
responses would add, but their luminance RFs, which See also: Color Vision Theory; Vision, Low-level
are opposite to each other, would cancel. This organi- Theory of; Vision, Psychology of; Visual Perception,
zation by itself would produce L-M color cells that Neural Basis of; Visual System in the Brain
would fire to so-called warm colors (red and yellow)
and inhibit to cool colors (blue and green). M-L cells
would fire to cool colors and inhibit to warm colors. Bibliography
As shown in Fig. 1C, the further addition of jSo or De Valois R L, De Valois R L 1988 Spatial Vision. Oxford
kSo cells can split these classes into separate red University Press, New York
and yellow, and separate blue and green systems, Hurvich L M 1981 Color Vision. Sinauer Press, Sunderland, MA
respectively. Kaiser P K, Boynton R M 1996 Human Color Vision. Optical
All of the primary visual information is passed Society of America, Washington, DC
through V1, but subsequent visual areas are partially Neitz J, Neitz M 1998 Molecular genetics and the biological
specialized for the further analysis of various different basis of color vision. In: Backhaus W G S, Kliegl R, Werner
functional aspects of vision. One later visual area (V4) J S (eds.) Color Vision. Walter de Gruyter, Berlin, pp. 101–19
Spillmann L, Werner J S 1990 Visual Perception: The Neuro-
is crucially involved with color perception. Individuals physiological Foundations. Academic Press, New York
with localized V4 lesions can still discriminate objects
on the basis of their color variations, but they report K. K. De Valois and R. L. De Valois
that the objects now appear to have no hue, as if
viewed on a black-white television screen. There is also
a report of one case with the reverse loss: a patient who
could see colored but not black-white objects.
Color Vision Theory
11. Color Appearance
Color vision is the ability to distinguish and identify
The appearance of a color can be specified by values lights and objects on the basis of their spectral
along just three perceptual dimensions known as hue, properties. This entry presents several key topics that
saturation and brightness. Hue refers to the character- underlie current theories of human color vision. These
istic described by such color names as red, yellow, are trichromacy, color opponency, adaptation, and
green, and blue. Saturation refers to the extent to color constancy.
2256

64. Color Vision Theory
1. Introduction primary intensities until the mixture has the same
color appearance as the test light. The primaries used
Information about color is transformed as it flows in the experiment are chosen to be independent, so
from the stimulus through the initial stages of the that no weighted mixture of any two produces a match
human visual system. At each image location, the to the third.
color stimulus is specified by the amount of power it Because the matching light is constrained to be a
contains at each wavelength. The classic color match- weighted mixture of three primaries, it will not
ing experiment shows that the normal human visual generally be possible for the observer to make the test
system is trichromatic: only three dimensions of and matching lights physically identical. For many
spectral variation are coded by the visual system. The test lights, however, the observer can adjust the
biological basis of normal trichromacy is that the matching light so that it appears identical to the test
retina contains three classes of cone photopigment. light even though the two differ physically. For some
After the initial encoding of light by the cones, further test lights, no choice of primary intensities will afford
processing occurs. Two aspects of this processing are a match. In these cases one or more of the primaries
particularly important. First, signals from three classes can be mixed with the test light and primary intensities
of cones are recombined to form a luminance and two found so that the primarytest mixture matches the
color opponent channels. Second, there is adaptive mixture of the remaining primaries. A useful descrip-
signal regulation that keeps neural signals within their tive convention for the color matching experiment is
operating range and stabilizes the appearance of to assign a negative intensity to any primary that must
objects across changes of illumination. be mixed with the test to make a match. Given this
convention, any test light can be matched by a mixture
of three independent primaries.
2. Trichromacy The color matching experiment is an empirical
system. Given a test light described by a vector b, the
2.1 Color Matching experiment returns a vector
The physical property of light relevant for color vision A C
is the spectral power distribution. A light’s spectral t
"
power distribution specifies the amount of power it tl t (2)
contains at each wavelength in the visible spectrum, #
t
B $ D
often taken to lie roughly between 400 and 700 nm. In
practice, spectral power distributions are measured at
whose entries are the individual primary intensities.
discrete sample wavelengths. Let the measured power
When the primaries are scaled by these intensities and
values be denoted by b , …, bNλ where Nλ denotes the
"
number of sample wavelengths. Then the vector
mixed, a match to the test light is created. The vector
t specifies what are called the tristimulus coordinates
A C of the light b. A theory of color matching should let us
b
" predict t for any test light b, given the spectral power
distributions of the primary lights.
bl < (1) As an empirical generalization, the color matching
system is a linear system (e.g., Wyszecki and Stiles
1982, Brainard 1995, Wandell 1995). That is, if we
bNλ have two test lights b and b with tristimulus
coordinates t and t , then any # weighted mixture
"
B D
provides a compact representation of the spectral " #
(a b ja b ) of the two test lights has tristimulus
power distribution. Use of a vector representation for " " # #
coordinates given by the corresponding mixture
spectral quantities facilitates a variety of colorimetric (a t ja t ). In these vector expressions, multiplication
computations (e.g., Brainard 1995). Wavelength of" a vector (e.g., b ) by a scalar (e.g., a ) consists of
" ##
sample spacings between 1 and 10 nm are typical. " "
multiplying each entry of the vector by the scalar,
Trichromacy is demonstrated by the basic color while addition of two vectors (e.g., a b and a b )
matching experiment (Wandell 1995, Brainard 1995). " " # #
consists of adding the corresponding entries of the two
In this experiment, an observer views a bipartite field. vectors.
One side of the field contains a test light. This light is The linearity of color matching makes it possible to
experimentally controlled and can have an arbitrary predict the match that will be made to any test light on
spectral power distribution. On the other side of the the basis of a relatively small number of measurements.
field is the matching light. This consists of the weighted Consider the set of monochromatic lights with unit
mixture of three primary lights. Each primary has a power. If Nλ wavelength samples are used in the
fixed relative spectral power distribution, but its underlying representation, then there are Nλ such
overall intensity in the mixture can be controlled by lights and we can denote their spectral representations
the observer. The observer’s task is to adjust the by c , c , …, cNλ. Each of the ci has a 1 as its ith entry
" #
2257

65. Color Vision Theory
and zeros elsewhere. Note that any light b may be this hypothesis (see Wandell 1995, Rodieck 1998).
thought of as a weighted mixture of monochromatic First, the responses of individual cones depend only
lights, so that b l bici where bi is the ith entry of b. on the rate at which photopigment molecules are
i
Let the vectors ti specify the tristimulus coordinates isomerized by the absorption of light quanta; once the
of the monochromatic lights ci. The linearity of color intensity of two lights has been adjusted so that they
matching then tells us that the tristimulus coordinates produce the same isomerization rates, the cone re-
of any light b are given by t l biti. sponse does not distinguish the two lights. This idea is
i
A set of tristimulus values ti measured for mono- referred to as the principle of univariance. Second,
chromatic lights ci is referred to as a set of color individual cones may be classified into one of three
matching functions. Although these are often plotted distinct types, each with a characteristic spectral
as a function of wavelength, they do not represent the sensitivity. The spectral sensitivity is proportional to
spectral power distributions of lights. The color the probability that light quanta of different wave-
matching functions may be specified by a single matrix lengths will isomerize a molecule of the cone’s photo-
A C pigment. The three types of cones are often referred to
Tl t t t (tNλ D (3)
B # $ as the long- (L), middle- (M), and short- (S) wave-
whose Nλ columns consist of the individual tristimulus length-sensitive cones. If an observer has only three
coordinate vectors ti. This specification allows com- types of cones, each of which obeys the principle of
putation of tristimulus coordinates from spectral univariance, two physically distinct lights that produce
power distributions through simple matrix multipli- the same isomerization rates for all three classes of
cation: cones will be indistinguishable to the visual system.
Quantitative comparison confirms that color matches
t l Tb. (4) set by a standard observer (defined as the average of
matches set by many individual observers) are well
Both tristimulus values and color matching functions predicted by the equations of isomerization rates in
are defined with respect to the primaries chosen for the the L-, M-, and S-cones.
underlying color matching experiment. The Com- As described above, trichromacy occurs for most
mission Internationale de l’Eclairage (CIE) has stan- observers because their retinas contain cones with
dardized a system for color representation based on three classes of photopigments. Genetic consider-
the ideas outlined above. The CIE system is widely ations, however, indicate that some individuals have
used to specify color stimuli and many sources describe retinas containing four classes of cone photopigments
it in detail (e.g., Wyszecki and Stiles 1982, Brainard (Sharpe et al. 1999). Either these individuals are
1995, Kaiser and Boynton 1996). tetrachromatic (mixture of four primaries required to
The advantage of using tristimulus coordinates to match any light) or else their trichromacy is mediated
describe color stimuli is that they provide a more by information lost after quantal absorption. In
compact and tractable description than a description addition, some human observers are dichromatic (only
in terms of wavelength. Tristimulus coordinates are two primaries must be mixed to make a match to any
compact precisely because they do not preserve physi- light.) Most cases of dichromacy occur because one
cal differences that are invisible to the human visual photopigment is missing (Sharpe et al. 1999, Neitz and
system. The representational simplification afforded Neitz 2000).
by tristimulus coordinates is extremely valuable for An alternative to using tristimulus coordinates to
studying processing that occurs after the initial encod- represent the spectral properties of lights is to use cone
ing of light. On the other hand, it is important to coordinates. These are proportional to the isomeriz-
remember that the standard tristimulus represen- ation rates of the three classes of cone photopigments.
tations (e.g., the CIE system) are based on matches The three dimensional vector
made by a typical observer looking directly at a small
A C
stimulus at moderate to high light levels. These qL
representations are not necessarily appropriate for
applications involving some individual observers, non- q l qM (5)
human color vision, or color cameras (e.g., Wyszecki q
B S D
and Stiles 1982, Brainard 1995).
specifies cone coordinates where qL, qM, and qS denote
the isomerization rates of the L-, M-, and S-cone
2.2 Biological Basis of Color Matching photopigments respectively. It can be shown (e.g.,
The color matching experiment is agnostic about the Brainard 1995) that cone coordinates and tristimulus
biological mechanisms that underlie trichromacy. It is coordinates are related by a linear transformation, so
generally accepted, however, that trichromacy typi- that
cally arises because color vision is mediated by three q l Mtqt (6)
types of cone photoreceptor. Direct physiological
measurements of individual primate cones support where Mtq is an appropriately chosen 3 by 3 matrix.
2258

66. Color Vision Theory
Computation of cone coordinates from light spectra A possible approach to understanding post-absorp-
requires estimates of the cone spectral sensitivities. tion processing is to keep the modeling close to the
For each cone class, these specify the isomerization underlying anatomy and physiology and to character-
rates produced by monochromatic lights of unit ize what happens to signals at each synapse in the
power. The sensitivities may be specified in matrix neural chain between photoreceptors and some site in
form as visual cortex. The difficulty is that it is not presently
possible to cope with the complexity of actual neural
A C
sL processing. Thus many color theorists have attempted
S l sM (7) to step back from the details and develop more
abstract descriptions of the effect of neural processing.
s
B S D Models of this sort are often called mechanistic
models. These models generally specify a transform-
where each row of the matrix is a vector whose entries ation between the quantal absorption rates q elicited
are the spectral sensitivities for one cone class at the by a stimulus and a corresponding visual represen-
sample wavelengths. Given S, cone coordinates are tation u postulated to exist at some central site. The
computed from the spectral power distribution of a idea is to choose a transformation so that (a) the color
light as appearance perceived at a location may be obtained
q l Sb (8) directly from the central representation corresponding
to that location and (b) the discriminability of two
Because Eqns. (4), (6), and (8) hold for any light stimuli is predictable from the difference in their
spectrum b, it follows that central representations.
Most mechanistic models assume that signals from
S l MtqT (9) the cones are combined additively to produce signals
at three postreceptoral sites. Two of these sites carry
Current estimates of human cone spectral sensitivities opponent signals. These are often referred to as the
are obtained from color matching data using Eqn. (9) red-green (RG) and blue-yellow (BY) signals. A third
together with a variety of considerations that put site carries a luminance (LUM) signal, which is not
constraints on the matrix Mtq (Stockman and Sharpe thought to be opponent. If we take
1999).
A C
uLUM
3. Postabsorption Processing u l uRG (10)
u
B BY D
Color vision does not end with the absorption of light
by cone photopigments. Rather, the signals that
originate with the absorption of light are transformed to be a three-dimensional vector with entries given by
as they propagate through neurons in the retina and the LUM, RG, and BY signals, then the additive
cortex. Two ideas dominate models of this post- relation between cone coordinates q and the visual
absorption processing. The first is color opponency: representation u may be expressed in matrix form:
signals from different cone types are combined in an
antagonistic fashion to produce the visual represen- u l Moq (11)
tation at a more central site. The second idea is
Many (but not all) detailed models take LUM to be a
adaptation: the relation between the cone coordinates
weighted sum of L- and M-cone signals, RG to be
of a light and its central visual representation is not
a weighted difference between the L- and M-cone
fixed but depends instead on the context in which the
signals, and BY to be a weighted difference between
light is viewed. Section 3.1 treats opponency, while
the S-cone signal and a weighted sum of the L- and M-
Sect. 3.2 treats adaptation.
cone signals. In these models Mo would have the form
A C
m m 0
3.1 Opponency #
Mo l m km 0 (12)
Direct physiological measurements of the responses of # ##
neurons in the primate retina support the general idea km km m
B $ $# $$ D
of opponency (e.g., Dacey 2000). These measurements
reveal, for example, that some retinal ganglion cells where all of the mij are positive scalars representing
are excited by signals from L-cones and inhibited by how strongly one cone class contributes to the signal at
signals from M-cones. One suggestion about why this one post-receptoral site.
occurs is that it is an effective way to code the cone Considerable effort has been devoted to establishing
signals for transmission down the optic nerve (see whether the linear form for the mapping between q
Wandell 1995). and u is appropriate, and if so, what values should be
2259

67. Color Vision Theory
used for the mij. Several types of experimental evidence
have been brought to bear on the question.
As an example, one line of research argues that four
color perceptions, those of redness, greenness, blue-
ness, and yellowness, have a special psychological
status, in that any color experience may be intuitively
described in terms of these four basic perceptions.
Thus orange may be naturally described as reddish-
yellow and aqua as greenish-blue. In addition, both
introspection and color scaling experiments suggest
that the percepts of redness and greenness are mutually
exclusive so that both are not experienced simul- Figure 1
taneously in response to the same stimulus, and A color context effect. The figure illustrates the color
similarly for blueness and yellowness (e.g., Hurvich context effect known as simultaneous contrast. The two
and Jameson 1957, Abramov and Gordon 1994). central disks are physically the same but appear
Given these observations, it is natural to associate the different. The difference in appearance is caused by the
RG signal with the amount of redness or greenness fact that each disk is seen in the context of a different
perceived in a light (redness if the signal is positive, annular surround. This figure is best viewed in color.
greenness if it is negative, and neither red nor green if A color version is available in the on-line version of the
it is zero) and the BY signal with the amount of Encyclopedia
blueness or yellowness. Judgments of the four fun-
damental color perceptions, obtained either through at other locations and at preceding times. To help fix
direct scaling (e.g., Abramov and Gordon 1994) or ideas, it is useful to restrict attention to the disk-
through a hue cancellation procedure (e.g., Hurvich annulus configuration. For this configuration, the
and Jameson 1957), are then used to deduce the visual representation of the disk may be written as
appropriate values of the mij in the second and third
rows of Mo. When this framework is used, the entries ud l f (qd; qa, ) (13)
for the first row of Mo, corresponding to the LUM
signal, are typically established through other means where ud is the visual response to the disk, qd and qa are
such as flicker photometry (e.g., Kaiser and Boynton the cone coordinates of the disk and annulus re-
1996). spectively, and represents other contextual variables
Other approaches to studying the opponent trans- such as the size of the disk and annulus and any
formation include analyzing measurements of the temporal variation in the stimulus. Clearly, f( ) must
detection and discrimination of stimuli (e.g., Wyszecki incorporate the sort of transformation described by
and Stiles 1982, Kaiser and Boynton 1996, Eskew, the matrix Mo in Sect. 3.1 above.
et al. 1999, Wandell 1999), and measurements of how As was the case with the discussion of opponency
the color appearance of lights is affected by the context above, there is not wide agreement about how best to
in which they are viewed (e.g., Webster 1996). In part model adaptation. A reasonable point of departure is
because of a lack of quantitative agreement in the a cone-specific affine model. In this model, the visual
conclusions drawn from different paradigms, there is representation u of a light is related to its cone
currently not much consensus about the details of the coordinates q through an equation of the form
transformation between q and u. One of the major
open issues in color theory remains how to extend the u l Mo(D qkq ) (14)
simple linear model described above so that it accounts
for a wider range of results. where Mo is as in Eqn. (12) and
A C A C
gL 0 0 qL
D l 0 gM 0 , q l qM
3.2 Adaptation (15)
0 0 gS q
Figure 1 illustrates a case where the same light has a B D B S D
very different color appearance when seen in two
different contexts. The figure shows two disk-annulus
stimulus configurations. The central disk is the same in In this formulation, the g’s on the diagonals of D
each configuration, but the appearance of the two characterize multiplicative adaptation that occurs at a
disks is quite different. To explain this and other cone-specific site in visual processing, before signals
context effects, mechanistic models assume that at any from separate cone classes are combined. The entries
given time and image location, the relation between of the vector q characterize subtractive adaptation.
the quantal absorption rates q and the visual rep-
Equation (14) is written in a form that implies that the
resentation u depends on the quantal absorption rates subtractive adaptation also occurs at a cone-specific
2260

68. Color Vision Theory
site. The entries of D and q depend on the cone 4. Color Constancy
coordinates qa of the annulus as well as on spatial and
temporal variables characterized by . Note that the The discussion so far has focussed on how the visual
cone-specific affine model is a generalization of the system represents and processes the spectrum of light
idea that the visual representation consists of a that enters the eye. This is natural, since light is the
contrast code. proximal stimulus that initiates color vision. On the
Asymmetric matching may be used to test the other hand, we use color primarily to name objects.
adaptation model of Eqn. (14). In an asymmetric The spectrum of the light reflected to the eye from an
matching experiment, an observer adjusts a match object depends both on an intrinsic property of the
stimulus seen in one context so that it appears to have object, its surface reflectance function, and on extrinsic
the same color as a test stimulus seen in another factors, including the spectral power distribution of
context. More concretely, consider Fig. 1. In the the illuminant and how the object is oriented relative
context of this figure, an asymmetric matching ex- to the observer.
periment could be conducted where the observer was Given that the light reflected to the eye varies with
asked to adjust the central disk on the right so that it the illuminant and viewing geometry, how is it that
matched the appearance of the central test disk on the color is a useful psychological property of objects? The
left. Suppose such data are collected for a series of answer is that the visual system processes the retinal
N test disks with cone coordinates qti. Denote the image to stabilize the color appearance of objects
cone coordinates of the matches by qmi. Within the across changes extrinsic to the object (e.g., changes in
mechanistic framework, the corresponding visual rep- the spectrum of the illuminant). This stabilization
resentations uti and umi should be equal. If Eqn. (14) process is called color constancy.
provides a good description of performance then Color constancy is closely linked to the phenom-
enon of adaptation described above (Maloney 1999).
Indeed, quantitative models of color constancy gen-
erally incorporate the same idea that underlies mech-
Mo(Dm qmikqm ) l Mo(Dt qtikqt )
anistic models of visual processing: at some central site
qmi l D− (Dt qtikqt jqm )
m
(16) there is a visual representation u that correlates with
qmi l Dtmqtikqtm color appearance. To stabilize this representation
against changes in illumination, it is supposed that the
relation between the quantal absorption rates q elicited
where Dtm l D− Dt and qtm l D− (qt kqm ). This by the light reflected from an object and the visual
prediction may be checked by finding the diagonal
m m representation u depends on the scene in which the
matrix Dtm and vector qtm that provide the best fit to object is viewed. In the case of color constancy, the
the data and evaluating the quality of the fit. Tests of emphasis has been on how the visual system processes
this sort indicate that the cone specific affine model the retinal image so that the transformation between q
accounts for much of the variance in asymmetric and u has the effect of compensating for the variation
matching data, both for the disk annulus configuration in the light reflected to the eye caused by changes of
(Wandell 1995, 1999) and for more complex stimuli illumination and viewing geometry. Psychophysical
(Brainard and Wandell 1992). Nonetheless, there are data on the color appearance of objects viewed under
clear instances for which Eqn. (16) does not give different illuminants are often well-modeled by trans-
a complete account of asymmetric matching (e.g., formations consistent with Eqn. (14) (e.g., Brainard
Delahunt and Brainard 2000) and other color ap- and Wandell 1992).
pearance data (e.g., Webster 1996, Mausfeld 1998, The central theoretical question of color constancy
D’Zmura and Singer 1999). is how the visual system can start with image data and
The cone-specific affine model may also be tested factor it into an accurate representation of the surfaces
against psychophysical data on the detection and and illuminants in the scene. This question has received
discrimination of colored lights. Here again the model extensive treatment, at least for simple scenes. A brief
provides a reasonable point of departure but fails in introduction to this literature on computational color
detail (e.g., Eskew et al. 1999). constancy follows.
To extend the cone specific affine model, various
theorists have suggested the need for adaptation at a
4.1 Computational Color Constancy
second site (after signals from separate cone classes
have been combined) and for the inclusion of non- Consider a scene consisting of diffusely illuminated
linearities in the relation between q and u (see flat, matte surfaces. For such scenes, the spectrum b
references cited in the previous two paragraphs). An reflected to the eye from each surface is given by the
additional open question concerns how the entries of wavelength-by-wavelength product of the spectral
D and q are determined by the viewing context power distribution of the illuminant e and the surface
(e.g., Brainard and Wandell 1992, Delahunt and reflectance function s. The surface reflectance function
Brainard 2000). specifies, at each sample wavelength, the fraction of
2261

69. Color Vision Theory
incident light reflected to the eye. The information linear model also describes illuminant spectral power
about b coded by the visual system is its cone distributions, so that
coordinates, which may be computed as
e $ B ew e (22)
q l S b l S diag(e) s (17)
The second is that the spatial average of the surface
where the function diag( ) returns a square diagonal reflectance functions (s- ) is the same in all scenes and
matrix with its argument placed along the diagonal. known. These additional constraints imply that
Clearly e and s are not uniquely determined from
knowledge of q: without additional constraints the ` V V
q l [S diag(s` )Be]we l Ls` we (23)
color constancy problem is underdetermined. For-
tunately the spectra of naturally occurring illuminants -
where q is the spatial average of the quantal absorp-
and surfaces are not arbitrary. Although the physical tion rates and Ls- is a known three-by-three matrix.
processes that constrain these spectra are not well Inverting Eqn. 23 yields an estimate for the illuminant
understood, analyses of measurements of both illumi- #
e l BeweV . This estimate is then used to provide the
nants and surfaces shows that their spectra are well matrix L− to be used in Eqn. (21).
eV
described by small-dimensional linear models (see Buchsbaum’s algorithm shows how the addition of
Brainard 1995, Maloney 1999). appropriate assumptions allows solution of the
Consider surface reflectances. It is possible to define computational color constancy problem. The difficulty
three fixed basis functions so that naturally occurring with Buchsbaum’s algorithm is that its assumptions
surface reflectances are reasonably well approximated are too restrictive. In particular, it seems unlikely that
by a linear combination of these basis functions. Thus the spatial average of surface reflectances is constant
for any surface, we have across scenes, nor do real scenes consist of diffusely
illuminated flat, matte surfaces. Subsequent work has
s$ws bs jws bs jws bs (18) focused on ways to provide reasonable estimates of
# # $ $ the illuminant and surface reflectances under other
where bs , bs , and bs are the spectra of the basis sets of assumptions (e.g., Maloney 1999).
functions and ws , ws$ , and ws are scalar weights
#
# $
that provide the best approximation of s within the
linear model. Eqn. (18) may be rewritten as 4.2 Computational Color Constancy and Human
Performance
s$Bsws (19)
How does the computational work relate to human
where the three columns of matrix Bs contain the basis performance? This question has not yet been resolved,
functions and the vector ws contains the scalar but it seems appropriate to close with a few obser-
weights. vations. First, the estimated linear model weights of
When the surface reflectance functions lie within Eqn. (21) may be associated with the mechanism
a three-dimensional linear model Eqn. (17) may responses u discussed in Sect. 2. In both types of
#
inverted, once an estimate e of the illuminant has theory, these quantities represent the visual response
been obtained (see below for discussion of illuminant computed from the quantal absorption rates, and both
estimation.) Start by rewriting Eqn. (17) as: are meant to allow direct prediction of appearance. In
the mechanistic approach, one considers a series of
#
q l [S diag(e)Bs] ws l Le# ws (20) transformations whose form is derived from experi-
ments with simple stimulus configurations. In the
where Le# is a three-by-three matrix that depends on the computational approach, the form of the transform-
illuminant estimate. This matrix may be inverted using ation is derived from consideration of the problem
standard methods to yield an estimate of ws: color vision is trying to solve. In both cases, however,
the emphasis is on finding the appropriate parametric
ws l L− q
V eV
(21) form of the transformation and on understanding how
the parameters are set as a function of the image data.
The estimate may then be used together with Eqn. (19) The connection between the two approaches is dis-
to estimate the surface reflectance function. cussed in more detail by Maloney (1999).
Many computational color constancy algorithms The value of the computational approach to under-
assume a linear model constraint for surface reflec- standing human vision depends on how accurately the
tance functions. This reduces the constancy problem transformations it posits may be used to predict the
to finding an estimate of the illuminant to plug into appearance of stimuli measured in psychophysical
Eqn. (20). For illustrative purposes, an algorithm experiments. There have been only a few empirical
due to Buchsbaum (1980) is described here. In comparisons of this sort to date. These comparisons
Buchsbaum’s algorithm, two additional assumptions do, however, indicate that the computational ap-
are added. The first is that a three-dimensional proach shows promise for advancing our understand-
2262

70. Combinatorial Data Analysis
ing of human color vision (Bloj, Kersten, and Hurlbert Gegenfurtner K, Sharpe L T (eds.) Color Vision: From Genes
1999, Brainard, Kraft, and Longre 2001). to Perception. Cambridge University Press, Cambridge, UK,
pp. 3–51
Stockman A, Sharpe L T 1999 Cone spectral sensitivities and
See also: Color Vision; Psychophysical Theory and
color matching. In: Gegenfurtner K, Sharpe L T (eds.) Color
Laws, History of; Psychophysics; Vision, Low-level Vision: From Genes to Perception. Cambridge University
Theory of; Vision, Psychology of; Visual Perception, Press, Cambridge, UK, pp. 53–87
Neural Basis of; Visual System in the Brain Wandell B A 1995 Foundations of Vision. Sinauer, Sunderland,
MA
Wandell B A 1999 Computational neuroimaging: color repre-
sentations and processing. In: Gazzaniga M (ed.) The New
Bibliography Cogniti e Neurosciences, 2nd edn. MIT Press, Cambridge,
MA, pp. 291–303
Abramov I, Gordon J 1994 Color appearance: on seeing red—or
Webster M A 1996 Human colour perception and its adaptation.
yellow, or green, or blue. Annual Re iew of Psychology 45:
Network: Computation in Neural Systems 7: 587–634
451–85
Wyszecki G, Stiles W S 1982 Color Science—Concepts and
Bloj M G, Kersten D, Hurlbert A C 1999 Perception of three-
Methods. Quantitati e Data and Formulae, 2nd edn. John
dimensional shape influences colour perception through
Wiley, New York
mutual illumination. Nature 402: 877–9
Brainard D H 1995 Colorimetry. In: Bass M (ed.) Handbook of
Optics: Volume 1. Fundamentals, Techniques, and Design. D. H. Brainard
McGraw-Hill, New York, pp. 26.1–26.54
Brainard D H, Brunt W A, Speigle J M 1997 Color constancy in
the nearly natural image. 1. Asymmetric matches. Journal of
the Optical Society of America A 14: 2091–110
'
Brainard D H, Kraft J M, Longere P 2001 Color constancy:
developing empirical tests of computational models. In: Combinatorial Data Analysis
Mausfeld R, Heyer D (eds.) Colour Perception: From Light to
Object. Oxford University Press, Oxford, UK Combinatorial data analysis (CDA) refers to a class of
Brainard D H, Wandell B A 1992 Asymmetric color-matching:
methods for the study of relevant data sets in which
How color appearance depends on the illuminant. Journal of
the Optical Society of America A 9(9): 1433–48 the arrangement of a collection of objects is the
Buchsbaum G 1980 A spatial processor model for object colour absolutely central concept. Characteristically, CDA is
perception. Journal of the Franklin Institute 310: 1–26 involved with either: (a) the identification of arrange-
D’Zmura M, Singer B 1999 Contrast gain control. In: ments that are optimal for a specific representation
Gegenfurtner K, Sharpe L T (eds.) Color Vision: From of a given data set, and where such an exploratory
Genes to Perception. Cambridge University Press, Cambridge, process is typically carried out according to some
UK, pp. 369–85 specific loss or merit function that guides a combina-
Dacey D M 2000 Parallel pathways for spectral coding in torial search over a domain of possible structures
primate retina. Annual Re iew of Neuroscience 23: 743–75
constructed from the constraints imposed by the
Delahunt P B, Brainard D H 2000 Control of chromatic
adaptation: Signals from separate cone classes interact. Vision particular representation selected; or (b) a confirma-
Research 40: 2885–903 tory determination as to whether a specific object
Eskew R T, McLellan J S, Giulianini F 1999 Chromatic de- arrangement given a priori reflects the observed data,
tection and discrimination. In: Gegenfurtner K, Sharpe L T and where such a confirmatory process is typically
(eds.) Color Vision: From Genes to Perception. Cambridge operationalized by comparing the empirically observed
University Press, Cambridge, UK, pp. 345–68 degree of correspondence between some given data set
Hurvich L M, Jameson D 1957 An opponent-process theory of and the specific structure conjectured for it, to a
color vision. Psychological Re iew 64(6): 384–404 reference distribution constructed from the collection
Kaiser P K, Boynton R M 1996 Human Color Vision, 2nd edn.
of all possible structures of the same form that could
Optical Society of America, Washington, DC
Maloney L T 1999 Physics-based approaches to modeling have been conjectured.
surface color perception. In: Gegenfurtner K, Sharpe L T The boundaries of what CDA might encompass are
(eds.) Color Vision: From Genes to Perception. Cambridge somewhat open but generally we would exclude
University Press, Cambridge, UK, pp. 387–416 methods based on the postulation of strong stochastic
Mausfeld R 1998 Color perception: From Grassman codes to a models and their specific unknown parametric struc-
dual code for object and illumination colors. In: Backhaus tures as underlying a given data set. Although CDA
W G K, Kliegl R, Werner J S (eds.) Color Vision—Pers- might use or empirically construct various weighting
pecti es from Different Disciplines. Walter de Gruyter, Berlin, functions, the weights so obtained are not to be
pp. 219–50
interpreted as parameter estimates in some presumed
Neitz M, Neitz J 2000 Molecular genetics of color vision and
color vision defects. Archi es of Ophthalmology 118: 691–700 stochastic model viewed in turn as responsible for
Rodieck R W 1998 The First Steps in Seeing. Sinauer, Sunder- generating the data. Manifest data are emphasized
land, MA solely, and the traditional concern for an assumed
Sharpe L T, Stockman A, Jagle H, Nathans J 1999 Opsin genes, relationship between the data and a restrictively
cone photopigments, color vision, and color blindness. In: parameterized stochastic model is avoided. For
2263
Copyright # 2001 Elsevier Science Ltd. All rights reserved.
International Encyclopedia of the Social Behavioral Sciences ISBN: 0-08-043076-7

71. Implications VOL. 03 ISSUE 5 www.informedesign.umn.edu
A Newsletter by InformeDesign. A Web site for design and human behavior research.
Seeing Color and was developed by a Finnish
Color is the most dominant design ele- astronomer, Aron Sigfrid Forsius and
ment, and ironically, the most relative was soon followed by Newton’s color
aspect of design. The perception of color wheel in 1704. The primary objectives of
involves human physiological and psy- these systems are to give order to the
chological responses. Object, light, eye, variables of color and to concretely rep-
and brain are involved in a complex resent colors, because “words are incom-
process of sensation and perception. plete expression as color” (Munsell,
Color attracts our attention, helps us 1981). Munsell developed a three-
make sense of our environment, and dimensional color tree. The three vari-
affects our behavior. Color plays a cul- ables of color — hue, value, and chroma
tural role, an informational role, and are displayed on plexiglass branches,
even a survival role. It functions on a one for each hue (see Figure 1). Darker
basic level of appeal and can elicit strong values of the hue are toward the bottom;
feelings of like or dislike. Color is a lighter values are toward the top.
source of sensual pleasure (Pentak Brighter hues are seen at the outside
Figure 1: Munsell Color Wheel perimeter; duller hues are toward the
Roth, 2003).
center of the tree. A color wheel made of
IN THIS ISSUE
hats and shoes, featured in an exhibition
Seeing Color Color Order Systems in the Goldstein Museum of Design,
We are familiar with the most common arranged the objects in spectral order
Typography and Color type of color arrangement—a color wheel (see Figure 2).
Related Research arranged in spectral order. Spectral
Summaries order is especially pleasing to the human Michel Eugene Chevreul developed a
perceptual system. The spectrum occurs system to explain how colors affect each
in nature in the refraction of light into other. As director of the Gobelins tapes-
bands of color—red, orange, yellow, try studio (France), he realized that color
green, blue, and violet. One hue gradates systems did not account for perceived
into the next, creating a dynamic color color and that colors tend to tinge adja-
sensation. cent hues with its complementary hue.
In response, he designed a color circle
Theoretical Color Systems that accounted for differences of satura-
Scientists, artists, and color theorists tion and value within each hue family.
have developed variations of the color He also created a framework about the
wheel. The first wheel appeared in 1611 effects of simultaneous contrast.

72. Implications www.informedesign.umn.edu
2
The Effect of Surface Quality on Color
Perception
Surface quality contributes to the variability of color,
“one and the same color evokes innumerable read-
ings” (Albers, 1963, p. 1). This variability is due to
differences in the human visual system, light, and
the material and surface quality of the object. When
we view the color of an object, we are really seeing
reflected light. Objects are typically colored with
either pigment or dyes. Dyes permeate the molecular
structure of the object; pigments lay in a coat of color
on the surface of an object. This difference is evident
in viewing fabric that has been painted versus fabric
that has been dyed.
Figure 2: A color wheel made from hats and shoes that are in the
collection of the Goldstein Museum of Design. Surface materiality also affects the appearance of a
color. A smooth, glossy surface will reflect a hue very
Practical Color Systems
differently than a rough surface, and they tend to
Specialized color systems are used in product design
reflect more light than a matte or rough surface.
and manufacturing. Both the Pantone color system
Matte or rough surfaces reflect light in a scattered,
and the Munsell system are widely used. Pantone
diffuse manner that randomly mixes the wavelengths
has developed color systems and products for the
and tends to soften the color, changing it.
graphic, interior, textile, architectural, and industri-
Transparent materials allow color and light to be
al design fields. Pantone has also recently begun
seen through them (see Figure 3). Reflection from
forecasting color trends in fashion and interior
glossy paper can make reading a menu or a magazine
design. The primary goal of both the Munsell and
difficult just as reflection from a highly polished floor
Pantone systems is to communicate color in a sys-
can create spatial perceptual challenges.
tematic way, leaving little room for error. The CIE
(Commission Internationale de l’Eclairage) chro-
maticity diagram displays a color matching system
based on light, and it is shaped like a luminosity
curve. The system attempts to eliminate differences
of color perception through mechanical measure-
ment of the three variables of a color—luminance,
hue, and saturation. While these practical color sys-
tems help to ensure accurate color specification,
color appearance still varies due to lighting, context,
and surface quality.
Figure 3: Glass designed by Dale Chihuly, Museum of Glass,
Tacoma, WA.
Where Research Informs Design®

73. Implications www.informedesign.umn.edu
3
Albers (1963) discusses the interdependence of color Color Harmony
with form and placement, quantity, and quality. It is There are strategies for creating color harmony:
a constant challenge to predict how a color will look using similar values or hues, using hues with com-
on the designed object when seen under different plementary contrast, or using a large number of hues
light sources. While the typical color wheel repre- in careful proportions. Constrast provides a sense of
sents only two or three dimensions, a color system visual balance. Munsell recommended balancing
developed by Albert-Vanel attempted to include vari- light and dark hues, dull and bright hues, and cool
ations due to surface quality, light, and human per- and warm hues. A sense of color harmony is based
ception. This system, called the Planetary color sys- partially in human perception and partially in color
tem and developed in 1983, includes not only hue, trends (see Figure 4).
value, and chroma, but also accounts for contrast
and material. Human Perception of Color
Color can have a profound effect on humans. It can
affect our brain waves, heart rate, blood pressure,
and respiratory rate. Color also affects us emotional-
ly. Exposure to color has an effect on our biological
systems. Not only does color affect our sense of well-
being, but it also may play a role in medical treat-
ments for depression, cancer, and bacterial infec-
tions.
Visual Perception
Our perception of color is dependent on light, object,
Figure 4: The colorful facades of Burano, Italy.
and our eyes and brain. We know that colors are
influenced by adjacent colors, indeed, it is rare to see
Dyes and Colorants an isolated color or color in its pure state. Chevreul
The color of objects is dependent on the pigments or discussed how colors tend to tinge neighboring hues
dyes used in the production of the product. Color with their complement. Including color opposites
trends often evolve out of technological develop- within close proximity in a particular space can mit-
ments. In the mid-1850s, William Henry Perkins igate this phenomenon. Surgical personnel in hospi-
accidentally developed effective synthetic dyes for tals wear greenish-blue scrubs to counter-balance
wool and silk as he attempted to synthesize quinine the visual effect of afterimages. During surgery, all
from aniline. He named the color mauve. Other eyes focus on the patient and typically see a variety
chemists developed synthetic aniline dyes that were of pink and red hues. The red receptors in the eye
significantly brighter and more saturated than early would become fatigued if not for the color of the
natural dyes. This discovery, along with the develop- scrubs providing the opposite hue and thus balanc-
ment of organic chemistry as a discipline, fueled the ing the visual experience.
development of numerous synthetic dyes. Neon dyes
and pigments that were developed in the mid-1980s Color contrast is essential for our understanding of
resulted in bright fabrics, accessories, and paper form and legibility. At least a 70% contrast between
products. the background and letterforms is ideal for signs and
Where Research Informs Design®

74. Implications www.informedesign.umn.edu
4
painted materials. Conversely, too much contrast in nomena also affect the popularity of colors. Fashion
an environment may increase anxiety and tension. prints in the 1960s used the bright palette of colors
Sharp contrasts of color on flooring may create known as psychedelic. These colors were fully satu-
uncomfortable illusions for walkers as they deter- rated and were intended to mimic the sensation
mine whether the floor is flat or not. Research has caused by drugs (see Figure 5). Most of the informa-
shown that the most visible combinations of colors tion about color meaning is highly subjective and
are yellow and black, white and black, white and based on tacit beliefs, rather than research. There is
blue, and red and white. a significant need for systematic research on color
and human perception.
Psychological Responses to Color
We all react differently to color. We have different Typography and Color
color preferences, and we all have our least favorite Typography, the set of alphabetic characters, numer-
colors. Color response is highly personal. What one als, and symbols used to compose copy, can be
person believes is a restful color, another may find manipulated in any number of ways by a graphic
stimulating. Frequently these color preferences are designer. Size, typeface, letterspacing, leading (the
based on our own personal experience—a fondly space between lines of type), case (upper or lower
case), structure (normal, light, bold, italic, bold ital-
ic, etc.), and—of course—color can all be used to
improve the legibility (how easy the text is to read),
readability (how inviting the text is to a reader), and
the hierarchy or structure of typeset copy.
While each of the previously mentioned characteris-
Figure 5: Fabric samples from 1960s-era clothing. tics can be manipulated by designers setting type,
color is an especially important property. We often
remembered yellow kitchen that belonged to grand- imagine type (or copy) that is set in black on a white
mother. There are also cultural associations that background—this is perhaps the most familiar way
influence our reactions to color. In several cultures, to set type on a printed page. However, when we
blue is seen as peaceful, protecting, and soothing think of typography in signage and the built environ-
color. Red typically signifies passion and revolution. ment, a variety of colors and color combinations,
There are multiple associations for each color. For come to mind. Consider the new, colorful green and
example, black may be seen as sophisticated or as yellow logo signage of BP (British Petroleum) that is
depressing. Orange can be warm or aggressive. employed in the design of gas stations. Or, think of
Yellow can be upbeat or acidic. the familiar white type on a green background of
road signs. Color is employed frequently in environ-
Marketing research attempts to discover what colors mental signage to create a memorable identity that
influence human behavior and how people will act helps users navigate a space, remember the business
when they shop, eat, or travel. Findings by market- or company, and create a pleasant impression.
ing researchers are typically short-lived, however;
trends seem to come and go, and other variables in When creating environmental signage, it is critical to
addition to color affect behavior. While technology consider some of the variables associated with the
contributes to color trends, culture and social phe- application of color. Here are a few ideas and tips:
Where Research Informs Design®

75. Implications www.informedesign.umn.edu
5
This is not an exhaustive list of issues to consider
when applying color to environmental signage and
typography. If possible, it is beneficial to have a
graphic designer who understands the interactions
between typography, color, and the built environ-
ment on a design team when designing environments
with signage. In addition, InformeDesign has
Research Summaries about graphic design for the
built environment.
References
—Albers, J. (1963). Interaction of Color. New Haven,
CT: Yale University Press.
Figure 6: An example of poor and excellent contrast between
—Fehrman, K., Fehrman, C. (2004). Color: The
typography and background.
Secret Influence. Upper Saddle River, NJ: Prentice
• Consider the contrast between the color of the Hall.
typography and the background to ensure that the —Munsell, A. H. (1946). A Color Notation. Baltimore:
type is easy to decipher and read. Type/back- Macbeth.
ground color combinations can cause the text to —Pentak, S., Roth, R. (2003). Color Basics.
either advance or recede (see Figure 6). Stamford, CT: Wadsworth.
• Consider the impact of color on interpretation and —Sharpe, D. (1981). The Psychology of Color and
understanding of the content. What does a red Design. Totowa, NJ: Littlefields, Adams Co.
heading indicate versus a brown heading? Does —Stromer, K. (Ed.). (1999). Color Systems in Art and
setting less important information in a brighter, Science. Edition Farbe/Regenbogen Verlag.
more prominent color impact the order that infor- —Walch, M., Hope, A. (1990). The Color
mation is retrieved? Compendium. New York: Van Nostrand Reinhold.
• Consider the user. Be aware of the cultural context
of the environment and the signage, and consider
About the Authors:
cultural norms for particular colors. For example,
Barbara Martinson, Ph.D.,
in Europe and the US, red typography generally
is the Buckman Professor of
means warning or attention. The application of
Design Education in the
color to type can either play into cultural norms for
Department of Design,
color or can contradict them.
Housing, and Apparel,
• Consider the lighting levels of the environment.
University of Minnesota.
While a color combination may work well when
She has taught founda-
evaluated in your office, the combination may be
tions-level color courses for
inappropriate when the lighting levels are different.
20 years, as well as graphic
• Consider the properties of the signage material.
design, design history, and
How will a surface that is reflective or flat change
human factors courses. She
the legibility of the content? How will lighting levels
recently curated Seeing
interact with the surface properties?
Color, an exhibition at the Goldstein Museum of
Where Research Informs Design®

76. Implications www.informedesign.umn.edu
6
Design. Her research focuses on design education, Bright, Saturated Colors Attract Attention
learning styles, and the use of digital media in teach- —Color Research and Application
ing. Her favorite color is blue.
Determining Color in the Built Environment
Kate Bukoski, author of —Color Research and Application
“Typography and Color,” is a
Ph.D. candidate in graphic Effects of Office Color Scheme on Workers
design and holds teaching —Color Research and Application
and reasearch assisantships
in the Department of Design, Color Aids Wayfinding for Young Children
Housing, and Apparel, —Early Childhood Education Journal
University of Minnesota. Her research interests focus
on the history and state of the profession of graphic Space and Color Affects Cooperation Among Children
design. —Environment and Behavior
Additional Resources Color Judgment is Influenced by the Aging Eye
www.digitalanarchy.com/theory/theory_main.html —Family and Consumer Sciences Research Journal
www.colorsystem.com
www.colormatters.com/colortheory.html Light Source, Color, and Visual Contrast
poynterextra.org/cp/ —Family and Consumer Sciences Research Journal
www.colorcube.com/articles/theory/theory.htm
www.tigercolor.com/ColorLab/Default.htm Color of Light Affects Psychological Processes
www.fadu.uba.ar/sicyt/color/bib.htm —Journal of Environmental Psychology
http://webexhibits.org/colorart/ch.html
www.digitalanarchy.com/theory/theory_main.html Color, Meaning, Culture, and Design
—Journal of Interior Design
Related Research Summaries
InformeDesign has many Research Summaries about Photos Courtesy of:
color and related, pertinent topics. This knowledge Barbara Martinson, University of Minnesota (p. 1, 2,
will be valuable to you as you consider your next 4, 5)
design solution and is worth sharing with your Caren Martin, University of Minnesota (p. 3)
clients and collaborators.
The Mission
The Mission of InformeDesign is to facilitate designers’
use of current, research-based information as
a decision-making tool in the design process, thereby
integrating research and practice.
Created by: Sponsored by:
© 2002, 2005 by the Regents of the University of Minnesota.

81. Chapter 13
How does visual memory work?
Photo courtesy of Ann Cantelow. The multichannel neuron model ascribes numbers to channels. The channel
numbers store and communicate analog data. They can also be used, in a distinct addressing system, to
sequentially query the twigs of visual memory.
Addressing and retrieval
For retrieval, the model requires two types of neurons: 1) an address generating neuron, which drives 2) a data
storage neuron. To activate a memory stored as a thing in a place, a stored datapoint must be addressed at
precisely that place. In the specific case of a stored pattern of three bleached disks imported from a
photoreceptor, a trio of associated datapoints, twigs, must be addressed, one right after the other.
We have a mechanism for generating sequential addresses. The principle is inherent in the multichannel neuron
model. The address generator can be the commutator we have postulated at the axon hillock.

82. To stimulate the first 9 twigs of memory, #1 through #9, each in turn, requires this sort of circuit. The output
lines of the axon driven by the addressing commutator are telodendrions, each corresponding to a channel. In
this illustration of this model, telodendrions are numbered in order of their firing. Each individual channel
synapses to a dendrite. Each dendrite will be stimulated in its turn, in accordance with the ascending circular
order of the addressing commutator.
Each dendrite is a “twig memory”. It stores a channel number that stipulates which channel shall be fired in
response to the addressing signal. The effect can be tabulated:

83. The dendrites, which comprise the twigs of memory in this simple model, are each stimulated in turn. The
pattern of bleached disks that each twig has memorized is fired back into the nervous system – precisely
replicating the pattern originally dispatched from a single photoreceptor’s outer segment at some time and day
in the past. In the table, 9 upticks of the address counter’s commutator correspond to a trio of 3D pixels and 3
frames of a film strip. [A slicker model might use just one address tick to elicit all three datapoints,
characterizing intensity, wavelength, phase -- but the point is, visual memory is sampled and read out by the
ticking of a sequential address counter. It is probably written in the same way.]
All pixels recorded from the retina at the same time, stored in twigs on other photoreceptor antipodal trees
will have identically the same time stamp in their address. So simultaneously, synchronously, one pixel from
every other “tree” or photoreceptor antipode in the retina of memory is being triggered.
The effect is to pump out of memory a stream of past images -- each image made up of millions of 3D pixels.
The system is massively parallel and, therefore, moves whole images all at once. It is lightning fast.
Why don't we see these torrents of images from the past? Why aren't we drowning in images? Because these
are not literal images. They are images of the Fourier plane. Fourier images are invisible to us, except perhaps
in the special case of LSD users. Literal images may impinge on the consciousness as, in effect, search
products, but the search itself is conducted as a Fourier process and is unconscious -- offstage and out of sight.
Numbered synapses -- new evidence, old idea
The idea there might be some sort of detectable ordering or sequencing of synapses on the dendrites is
attributed to Wilfrid Rall, who suggested it in 1964 in support of a wholly different and unrelated model of the
nervous system. In the 24 September 2010 issue of Science there is a featured report that reinforces the notion
there exists some sort of sequentially ordered input pattern in the dendrites.

84. In these experiments, a programmed series of successive stimuli is made to “walk” from synapse to synapse
along the dendrite. If the stimulus series progresses toward the cell body it is more likely to trigger off action
potentials than a programmed series of stimuli that walks the other way, away from the soma, toward the tips of
the dendrites.
The front half of this experiment consists of the selective stimulation of a row of individual dendritic spines,
one after another, using a laser to precisely localize release of glutamate. The basic technology was outlined
here. The back half of the experiment is conventional, and consists of electronic monitoring and tabulation of
the axon’s response.
In terms of the multichannel model electrophysiology is difficult to interpret. However, a significant feature of
the model is a staircase of firing thresholds. One might speculate that as the stimulus is made to approach the
soma, it is finding or ultimately directing a pointer to lower and lower firing thresholds, which is to say, lower
channel numbers. These low numbered channels would be more easily triggered than higher numbered
channels.
Unfortunately there is easy no way to directly measure or guess the channel number associated with an action
potential in passage, if indeed multiple channels exist. Again in terms of the model, a plot of channel numbers
versus synapse position on dendrites (or, using different techniques, on the teledendrions) would produce a
fascinating picture. In any event it is interesting that even conventional electrophysiology suggests there may be
some kind sequential ordering, progression, or directional structuring that underlies a map of dendritic spines.
The model
In modeling this visual memory system I think it would be best to use automated rotating or looping machinery,
just as you would in many familiar recording and playback devices. The rotating machine is the commutator. At
each addressing tree, let the loftiest addressing commutators walk forward through time automatically,
incrementing higher channel by channel. Rough synchronization among trees should suffice. Now, instead of
hardwiring and broadcasting addresses in detail, the retrieval system can simply be given a start date/time and
triggered off. A string of retrieval instructions will ensue. The system will, in effect, read itself out like a disk
drive.
As a practical matter, the model of a retina of memory should probably be constructed in software. Each tree of
memory can be modeled as a disk drive storing analog numbers representing 3D pixels, stacked in serial order,
that is, the order or sequence in which they were originally captured from the eye. Millions of disk drives, then,
each of relatively modest capacity, comprise a retina of memory. In a primitive animal one would expect to find
a single retina of memory. In a sophisticated animal, many.
Let’s say the memory trees pre-exist in a newborn animal and that their twigs are unwritten. Each branch is a
point in a commutator sequence, and identifies time (that is, sequence) ranges.
From the point of view of addressing the visual memory, reading and writing are, as in a disk drive, similar
processes. The writing commutator walks forward through the present moments, guiding incoming 3D pixels
from the eye to a series of novel addresses. To elicit a visual memory a reading commutator, which could be the
self-same machine, walks forward through addresses denoting a film strip of past moments.
In effect, the pointer of the base commutator on the address generator, as it ticks ahead, is the pointer of the
second hand of a system clock. Although the images are recorded at a stately and regular rate, such as one per
second -- the recall can be made to happen as fast as the commutator is made to sweep. And it could scan
backwards as well as forwards.
How is a pixel memory deployed?
This is an unsolved problem in the model. We have to assume it happens but the answer isn't easy or obvious.

85. We have stipulated what a 3D pixel memory is: Three numbers -- integers -- that represent a pattern of light
recorded from three disks in a single photoreceptor at a particular moment in time. The three numbers are
sufficient to specify the instantaneous wavelength, intensity and phase of the incoming light, as read out of a
standing wave in the outer segment of the photoreceptor.
We are suggesting these three numbers are configured and stored in the brain as an addressable twig of
memory -- three dendritic launch pads for three action potentials to be fired down three specific, numbered axon
channels. It is nicely set up, this memory, but how did it happen?
The operation of an initial readout commutator in the addressing neuron seems clear. It simply counts up or
down. Other commutators fan out from the initial or system counter. At the upper tier of the addressing tree, the
commutators, once toggled, can tick forward “on automatic.”
But what about the commutator in the memory neuron?
In the most basic model of the multichannel neuron, developed in Chapter 2, the neuron is functioning as a
sensory transducer. The commutator pointer rotates up to a specific numbered channel in proportion to an input
voltage or graded stimulus.
But in the memory neuron, we want the pointer to go, first, straight to a remembered channel. Then, second, to
another remembered channel. Then, third, to another remembered channel. Hop hop hop. From the address
neuron the memory neuron receives three signals in a sequence, via telodendrions 1, 2, 3. The data neuron fires
channels corresponding to three remembered photoreceptor disk positions: 2, 7, 34.
Instead of responding proportionately to an input voltage, as in a sensory neuron, the commutator in the
memory neuron is responding discontinuously to a memorized set of three channel firing instructions. So the
needle of this commutator must swing, not in response to an analog voltage input, but in response to a pixel
memory.
In the multichannel model synapses connect individual channels, rather than individual neurons. It could be that
the commutator is simply bypassed, so that the appropriate axon channels are hardwired to the dendritic twigs
of memory. Synapses at the soma could suggest a short cut past or a way to overrule the inherent commutator.
Maybe there is some rewiring or cross wiring at the level of the dendritic synapses. To borrow a term of art
from the conventional playbook of memory biochemistry, maybe the synapses are subject to tagging. Maybe
biochemical markers delivered into the dendrites when the memory was originally recorded are specifying in
some way the channel numbers to be fired.
This model suggests a Y-convergence of three neurons, not just two. One delivers addresses. One stores the

86. data. A third neuron delivers original data from the retinal photoreceptor – data to be written in sequential order
into the dendrites of the memory neuron.
Whatever specific mechanism one might choose or invent, the model requires that pixel memory arriving from a
photoreceptor in the eye be stored in an antipodal neuron as a trio or linkage of three distinct channel numbers.
Experiment
One interesting aspect of this memory model is that it suggests an experiment. We are guessing that the
individual channels of an addressing axon are, in effect, split out and made accessible as numbered
telodendrions. If there is indeed a numerical succession – a sequential firing order – of the telodendrions, then
this should be detectable. We were taught that the telodendrions must fire simultaneously. Is this always true? I
bet not.
Superimposed networks
Note that we have assumed there exists a double network. Above the information tree there is a second tree, a
replica of the first, used to individually address each memory twig.
The principle of two superimposed networks, one for content and the other for control, is a technical
commonplace. An early application was the superimposition of a telegraph network as a control system for the
railway network. The egregious present day example is the digital computer, with its superimposed but distinct
networks for information storage and addressing.
We are long in the habit of dividing the nervous system into afferent and efferent, sensory and motor, but
surely there must be other ways to split it, e.g., into an information network and a addressing network. It is
typically biological that one network should be a near replica of the other. Evolution proceeds through
replication and modification.
Arborization and addressing capacity
The first anatomist who isolated a big nerve, maybe the sciatic, probably thought it was an integral structure –
in essence, one wire. Closer scrutiny revealed that the nerve was a bundle of individual neurons. We are
proposing here yet another zoom-down in perspective, this time to the sub-microscopic level . We suspect that
each neuron within a nerve bundle is itself a bundle of individual channels.

87. It follows that the functional wiring of the nervous system is at the level of channels. Synapses connect
channels, not neurons. This is why one might count 10,000 synaptic boutons on a single neuron’s soma. The
boutons were not put there, absurdly, to “make better contact” nor to follow the textbook model of signal
integration. They are specific channel connectors, each with a specific channel number.
The neuroanatomical feature that most interests us at this point is axon branching. This is because branching is
of paramount importance in familiar digital technologies for addressing – search trees and other data structures.
We have proposed a treelike addressing system for the visual memory in the brain. It is reasonable to ask --
where are the nodes?
Not at the branch points.
Photo courtesy of Ann Cantelow
Branching in a nerve axon is just a teasing apart and re-routing of the underlying channels. It is not a branching
marked by nodes or connections in the sense of an T or Y connected electrical branch, or a logical branch in a
binary tree.
For an axon that addresses a dendritic twig of memory, all functional branching occurs at the commutator.
Any anatomical branching downstream of the commutator, such as the sprouting from the axon of
telodendrions , simply marks a diverging pathway – an unwinding or unraveling, rather than a distinct node or
connection. In other words, the tree is a circular data store. The datapoints are stored at twigs mounted on the
periphery of a circle. The twigs are accessible through a circular array of addresses. It is analogous to a disk
drive in which the disk holds still and the read-write head rotates.

88. Photo courtesy of Ann Cantelow
Summary of the technology to this point
The tree in this photograph is a metaphor for the brain structure which corresponds to, and is antipodal to, a
single photoreceptor of the eye. It is one single photoreceptor cell's remote memory warehouse -- a tree of
memory.
Each twig is a destination with an address, a neuronal process narrowed down to just two or three channels.
For example channels 3, 7 and 29, only, might constitute a given twig. Each twig is a 3D pixel frozen in time.
The tree will store as many unique picture elements from the photoreceptor’s past as it has twigs.
As many as 125 million of these trees will constitute a retina of memory. We will look for ways to hack down
this number, but for the moment let it stand. The point is, we are talking about millions of trees.
All these trees must be queried simultaneously with a particular numerical address, probably associated with a
time of storage, to elicit firing from all the right twigs -- just one twig per tree. Properly addressed, a forest of
these trees will recreate, almost instantly, a whole-retina image from memory.
In a primitive animal, it would be sufficient to remember 300 images from the recent past. This could be
accomplished with a single addressing neuron, a single commutator. But in a modern mammal, it will be
necessary to stack the commutators. A bottom commutator can point to any of 300 other commutators. And
each of these can, in turn, point to 300 more commutators. With a simple tree of neurons, which is to say, a
logical tree built with commutators, one can very quickly generate an astronomical number of unique addresses.
We require one unique address for each twig of the data trees.
Are there enough addresses available in this system to organize a mammalian lifetime of visual memories? Yes.
Easily. Are there enough memory neurons to match the addressing capacity of the addressing neurons.
Probably not. The neuronal brain that lights up our scanners is probably running its memory neurons as a
scratchpad memory. It seems likely there is a deeper store.
But will it work?
The memory mechanism we have sketched is probably adequate as a place to start. It would work for a
directional eye in which changes in wavelength are highly significant cues to the position and movement of a
target. It is a visual memory for retaining the just now, a film strip comprising a few recent frames.