Cognitive Psychology and Its Implications, Ch. 2

Our bodies are bristling with sensors that detect sights, sounds, smells, and physical

contact. Billions of neurons process sensory information and deliver what they

find to the higher centers in the brain. This enormous information-processing system

creates a problem for higher-level cognition: how to decide what to attend to from all

the sensory information being processed. Chapter 3 will address this problem. First,

this chapter focus on how our sensory systems identify what is in the world outside

our bodies, concentrating on visual perception and, to a lesser extent, on the perception

of speech—the two most important perceptual systems for the human species.

This chapter will address these questions: • How does the brain extract information from the visual signal? • How is visual information organized into objects? • How are visual and speech patterns recognized? • How does context affect pattern recognition?

•Visual Information Processing

Visual Perception in the Brain

Figure 2.1 shows the cortical regions devoted to processing information from the

senses of vision and hearing. The illustration makes clear that humans have a big

neural investment in processing visual information. This investment in vision is

part of our “inheritance” as primates, who have evolved to devote as much as

50% of their brains to visual processing (Barton, 1998). The enormous amount

of processing in the brain underlies the human ability to recognize what we see.

Damage to certain brain regions results in conditions where one is able to

register visual information but unable to recognize anything. Visual agnosia is

an inability to recognize objects that results neither from general intellectual

loss nor from a loss of basic sensory abilities. One case of visual agnosia involved

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 32

Visual Information Processing | 33

a soldier who suffered brain damage resulting from accidental carbon monoxide

poisoning. He could recognize objects by their feel, smell, or sound, but he

was unable to distinguish a picture of a circle from that of a square or to recognize

faces or letters (Benson & Greenberg, 1969). On the other hand, he was

able to discern light intensities and colors and to tell in what direction an object

was moving. Thus, his sensory system was able to register visual information,

but the damage to his brain resulted in a loss of the ability to transform visual

information into perceptual experience. This case shows

that perception is much more than simply the registering of

sensory information.

Generally, visual agnosia is classified as either apperceptive

agnosia or associative agnosia (for a review, read Farah,

1990). Benson and Greenberg’s patient is described as having

apperceptive agnosia. Such patients are unable to recognize

simple shapes such as circles or triangles or to draw

shapes they are shown. Patients with associative agnosia, in

contrast, are able to recognize simple shapes and can successfully

copy drawings, even of complex objects. However,

they are unable to recognize the complex objects. Figure 2.2

shows the original drawing of an anchor and a copy of it

made by a patient with associative agnosia (Ratcliff &

Newcombe, 1982). Despite being able to produce a relatively accurate drawing,

the patient could not recognize this object as an anchor (he called it an umbrella).

It is generally believed that patients with apperceptive agnosia have problems

with early processing of information in the visual system, whereas patients with

associative agnosia have intact early processing but have difficulties with pattern

recognition, which occurs later. It is common to distinguish between early processing

of information in the visual stream and later processing of the information.

This distinction will be one of the organizing principles of this chapter.

FIGURE 2.1 Some of the cortical

structures involved in vision and

audition: the visual cortex, the

auditory cortex, the “where”

visual pathway, and the “what”

visual pathway. (Ungerleider

& Mishkin, 1982.)

FIGURE 2.2 A patient with

associative agnosia was able

to copy the original drawing of

the anchor at left (his drawing

is at right), but he was unable

to recognize the object as an

anchor. (From Ellis & Young, 1988.

Reprinted by permission of the publisher.

© 1988 by Quarterly Journal of

Experimental Psychology.)

Brain Structures

“Where” visual pathway

“What” visual pathway

Auditory cortex

Visual cortex: Early visual processing

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 33

34 | Perception

Figure 2.3 offers an opportunity for a person with normal perception to appreciate

the distinction between early and late visual processing. If you have not

seen this image before, it will strike you as just a bunch of ink blobs. You will be

able to judge the size of the various blobs and reproduce them, just as Ratcliff

and Newcombe’s patient could, but you will not see any patterns. If you keep

looking at the image, however, you may suddenly see a dog in the center of the

picture. Now your pattern perception has succeeded, and you have interpreted

what you have seen.We will discuss how visual information is processed before

patterns are recognized, and then move on to the discussion of the processes of

pattern recognition.

Visual perception can be divided into an early phase, in which shapes and

objects are extracted from the visual scene, and a later phase, in which the

shapes and objects are recognized.

Early Visual Information Processing

We know a fair amount about the neural underpinnings of very early visual information

processing. Figure 2.4 is a schematic representation of the eye. Light

passes through the lens and the vitreous humor and falls on the retina at the

back of the eye. The retina contains the photoreceptor cells, which are made up

of light-sensitive molecules that undergo structural changes when exposed to

light. Light is scattered slightly in passing through the vitreous humor, so the

image that falls on the back of the retina is not perfectly sharp. One of the functions

of early visual processing is to sharpen that image.

FIGURE 2.3 A scene in which

we initially perceive just lowlevel

visual details (blobs); only

after viewing the scene longer

does the pattern of a dog

emerge. (Courtesy of Ronald James.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 34

Visual Information Processing | 35

A photochemical process converts light into neural

energy. There are two distinct types of photoreceptors in the

eye: cones and rods. Cones are involved in color vision and

produce high resolution and acuity. Less light energy is

required to trigger a response in the rods, but they produce

poorer resolution. As a consequence, they are principally responsible

for the less acute, black-and-white vision we experience

at night. Cones are especially concentrated in a small

area of the retina called the fovea. When we focus on an

object, we move our eyes so that the object falls on the fovea.

This enables us to maximize the high resolution of the cones

in perceiving the object. Foveal vision detects fine details,

whereas the rest of the visual field—the periphery—detects

more global information, including movement.

The receptor cells synapse onto bipolar cells and these

onto ganglion cells, whose axons leave the eye and form the

optic nerve, which goes to the brain. Altogether there are

about 800,000 ganglion cells in the optic nerve of each eye.

Each ganglion cell encodes information from a small region

of the retina. The neural firing on a ganglion axon will typically

encode the amount of light stimulation in that region of

the retina.

Figure 2.5 illustrates the neural pathways from the eyes to

the brain. The optic nerves from both eyes meet at the optic

chiasma, and the nerves from the inside of the retina (the side

nearest the nose) cross over and go to the opposite side of the

brain. The nerves from the outside of the retina continue to

the same side of the brain as the eye. This means that the

Cornea

Pupil

Lens

Vitreous humor

Retina

Optic nerve

Aqueous humor

Fovea

Iris

FIGURE 2.4 A schematic representation of the eye.

Light enters through the cornea; passes through the

aqueous humor, pupil, lens, and vitreous humor; then

strikes and stimulates the retina. (From Lindsay & Norman, 1977.

Reprinted by permission of the publisher. © 1977 by Academic Press.)

Eye

Optic chiasma

Lateral geniculate

nucleus

Superior

colliculus

Visual cortex

FIGURE 2.5 Neural pathways from the eye to the

brain. The optic nerves from each eye meet at the optic

chiasma. Information about the left part of the visual

field goes to the right brain, and information about the

right side of the visual field goes to the left brain. Optic

nerve fibers synapse onto cells in subcortical structures,

such as the lateral geniculate nucleus and superior

colliculus. Both structures are connected to the visual

cortex. (From Keeton, 1980. Reprinted by permission of the publisher.

© 1980 by W. W. Norton.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 35

right halves of both eyes are connected to the right brain. As Figure 2.5 illustrates,

the lens focuses the light so that the left side of the visual field falls on the

right half of each eye. Thus, information about the left part of the visual field

goes to the right brain, and information about the right side of the visual field

goes to the left brain. This is one instance of the general fact, discussed in Chapter

1, that the left hemisphere processes information about the right part of the

world and the right hemisphere processes information about the left part.

Once inside the brain, the fibers from the ganglion cells synapse onto cells in

various subcortical structures. (“Subcortical” means that the structures are located

below the cortex.) Two of these structures are the lateral geniculate nucleus and

the superior colliculus (see Figure 2.5). It is thought that the lateral geniculate

nucleus is important in perceiving details and recognizing objects, whereas the

superior colliculus is involved in locating objects in space. This division is referred

to as the “what-where” distinction. Both neural structures are connected to

the primary visual cortex (Brodmann area 17 in figure on the inside front cover).

The primary visual cortex is the first cortical area to receive visual input, but there

are many other visual areas, including Brodmann areas 18 and 19, surrounding it.

Figure 2.6 illustrates the representation of the visual world in the primary

visual cortex. It shows that the visual cortex is laid out topologically, as discussed

in Chapter 1. The fovea receives a disproportionate representation and

36 | Perception

12

9 11

10

5

6

7

8

12

3

4

Fovea

Visual field

Right

Calcarine

fissure

Calcarine

fissure

Primary visual

cortex

Left

1

2

3

4

5

6

7

8

9

12 10

11

FIGURE 2.6 The orderly

mapping of the visual field

(above) onto the cortex. The

upper fields are mapped below

the calcarine fissure and the

lower fields are mapped

above the fissure. Note the

disproportionate representation

given the to fovea, which is the

region of greatest visual acuity.

(From Figure 29-7 in Kandel et al., 1991.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 36

the peripheral areas, less representation. Figure 2.6 shows that the left

visual field is represented in the right cortex and the right in the left. It

also illustrates another “reversal” of the mapping—the upper part of the

visual field is represented in the lower part of the visual cortex and the

lower part, in the upper region.

As Figure 2.1 illustrates, after visual information is projected to the

primary visual cortex, it tends to follow two pathways. The information

that progresses along the “where” pathway goes to parietal regions of the

brain that are specialized for spatial information and for coordinating

vision with action. Information that progresses along the “what” pathway goes

to the temporal cortex, which is concerned with identifying objects. Monkeys

with lesions in the “where” pathway have difficulty learning to identify specific

locations, whereas monkeys with lesions in the “what” pathway have difficulty

learning to identify objects (Pohl, 1973; Ungerleider & Brody, 1977). Other

researchers (e.g., Milner & Goodale, 1995) have argued that the “where” pathway

is really a pathway specialized for action. They point out that patients with

agnosia because of damage to the temporal lobe, but with intact parietal lobes,

can often take actions appropriate to objects they cannot recognize. For

instance, one patient (see Goodale, Milner, Jakobson, & Carey, 1991) could

correctly reach out and grasp a door handle that she could not recognize.

A photochemical process converts light energy into neural activity. Visual

information progresses by various neural tracks to the visual cortex and

from there, along “what” and “where” pathways through the brain.

Information Coding in Visual Cells

Kuffler’s (1953) research showed how information is encoded by the ganglion

cells. These cells generally fire at some spontaneous rate even when the eyes are

not receiving any light. For some ganglion cells, if light falls on a small region of

the retina, these spontaneous rates of firing will increase. If light falls in the

region just around this sensitive center, however, the spontaneous rate of firing

will decrease. Light farther from the center elicits no change from the spontaneous

firing rate—neither an increase nor a decrease. Ganglion cells that respond in

this way are known as on-off cells. There are also off-on ganglion cells: Light at

the center decreases the spontaneous rate of firing, and light

in the surrounding areas increases that rate. Cells in the lateral

geniculate nucleus respond in the same way. Figure 2.7 illustrates

the receptive fields of such cells.

Hubel andWiesel (1962), in their study of the primary visual

cortex in the cat, found that visual cortical cells respond in a

more complex manner than ganglion cells and cells in the lateral

geniculate nucleus. Figure 2.8 illustrates four patterns that have

been observed in cortical cells. These receptive fields all have

an elongated shape, in contrast to the circular receptive fields

of the on-off and off-on cells. The types shown in Figures 2.8a

and 2.8b are edge detectors. They respond positively to light

on one side of a line and negatively to light on the other side.

Visual Information Processing | 37

On-off cell Off-on cell

FIGURE 2.7 On-off and off-on

receptive fields of ganglion cells

and the cells in the lateral

geniculate nucleus.

(a) (b) (c) (d)

FIGURE 2.8 Response

patterns of cells in the visual

cortex. (a) and (b) are edge

detectors, responding positively

to light on one side of a line

and negatively to light on the

other side. (c) and (d) are

bar detectors; they respond

positively to light in the center

and negatively to light at the

periphery, or vice versa.

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 37

They respond most if there is an edge of light

lined up so as to fall at the boundary point.

The types shown in Figures 2.8c and 2.8d are

bar detectors. They respond positively to light

in the center and negatively to light at the

periphery, or vice versa. Thus, a bar with a

positive center will respond most if there is a

bar of light just covering its center. Figure 2.9

illustrates how a number of on-off and off-on

cells might combine to form a bar or edge

detector. Note that no single cell is sufficient

to stimulate a detector; instead, the detector

responds to patterns of the cells. Even at this

low level, we see the nervous system processing

information in terms of patterns of neural

activation, a theme emphasized in Chapter 1.

Both edge and bar detectors are specific with respect to position, orientation,

and width. That is, they respond only to stimulation in a small area of the

visual field, to bars and edges in a small range of orientations, and to bars and

edges of certain widths. Different detectors seem to be tuned to different widths

and orientations, however, and so some subset of bar detectors would be most

stimulated by any bar or edge in the visual field.

Figure 2.10 illustrates Hubel and Wiesel’s (1977) hypercolumn

representation of cells in the primary visual cortex.

They found that the visual cortex is divided into 2 2 mm

regions, which they called hypercolumns. Each hypercolumn

represents a particular region of the receptive field. As noted

in Chapter 1, the organization of the visual cortex is topographic,

and so adjacent areas of the visual field are represented

in adjacent hypercolumns. Figure 2.10 shows that

each hypercolumn itself has a two-dimensional (2-D) organization.

Along one dimension, alternating rows receive input

from the right and left eyes. Along the other dimension,

the cells vary in the orientation to which they are most sensitive.

Adjacent regions represent similar orientations. This organization

should impress upon us how much information

about the visual array is encoded. Hundreds of regions of space are represented

separately for each eye, and within these regions many different orientations are

represented. Also, although this aspect of visual coding is not shown by the illustration,

different cells will code for different sizes and widths of line. Thus, an

enormous amount of information has been extracted from the visual signal by

the time it reaches the first cortical areas.

In addition to this rich representation of line orientation, the visual system

extracts other information from the visual signal. For instance, we can also perceive

the colors of objects and whether they are moving. Livingstone and Hubel

(1988) proposed that the visual system processes these various dimensions

(form, color, and movement) separately. Many different visual pathways and

38 | Perception

(a) (b)

FIGURE 2.9 Hypothetical

combinations of on-off and

off-on cells to form (a) bar

detectors and (b) edge

detectors.

R

L

R

L

FIGURE 2.10 Representation

of a hypercolumn in the visual

cortex. The hypercolumn is organized

in one dimension according

to whether input is coming

from the right eye or left eye. In

the other dimension, it is organized

according to the orientation

of lines to which the

receptive cells are most sensitive.

Adjacent regions represent similar

orientations. (After Horton, 1984.

Adapted by permission of the publisher.

© 1984 by the Royal Society of London.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 38

many different areas of the cortex are devoted to visual processing (32 visual areas

in the count by Van Essen & DeYoe, 1995). Different pathways have cells

that are differentially sensitive to color, movement, and orientation. It seems

that the visual system analyzes a stimulus into many independent features and

represents the locations of these features. Such spatial representations of visual

features are called feature maps (Wolfe, 1994). Thus, if a vertical red bar is

moving at a particular location, there are separate feature maps representing

that it is red, vertical, and moving in that location, and these maps may be in

different visual areas of the brain.

The ganglion cells encode the visual field by means of on-off and off-on cells,

which are combined by higher visual processing to form various features.

Perceiving Depth and Surfaces

Even after the visual system has identified edges and bars in the environment,

a great deal of information processing must still be performed before that

system is able to perceive the world. One of the problems it must solve is

deciding where those edges and bars are located in space. The fundamental

problem is that the information laid out on the retina is inherently 2-D,

whereas we need to construct a three-dimensional (3-D) representation of the

world. The visual system processes a number of cues to infer distance. One of

these cues is texture gradient, which is the tendency of elements to appear more

closely packed together as the distance from the viewer increases. Consider

Gibson’s (1950) classic examples, shown in Figure 2.11. Even though the lines

Visual Information Processing | 39

FIGURE 2.11 Examples of texture gradient. Elements appear to be further away when they

are more closely packed together. (From Gibson, 1950. Reprinted by permission of the publisher. © 1950 by

Houghton Mifflin.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 39

and ovals are rendered on a flat page, the change in the texture gives the

appearance of distance. Another cue to depth is stereopsis, the ability to perceive

3-D depth because each eye receives a slightly different view of the world.

Stereopsis is utilized by the 3-D glasses found in a few movies and some

exhibits such as at Disney World. They filter the light coming from a single

2-D source (say, a movie screen) so that different light information reaches

each eye. The perception of a 3-D structure resulting from stereopsis can be

quite compelling.

A third source of information about 3-D structure comes from motion

parallax. As more distant points move, they move across the retina more

slowly than closer points. Similarly, as one moves one’s head, objects that are

more distant will move across the retina more slowly than closer objects. For

an interesting demonstration, look at the leaves of a nearby tree or bush with

one eye closed. Denied stereopic information, you will have the sense of a very

flat image in which it is hard to see the position of the many leaves relative to

one another. Then move your head. Suddenly, the 3-D structure of the tree

becomes clear, and you can readily perceive the relative positions of leaves

and branches.

Although it is easy to demonstrate the importance of such cues as texture

gradient, stereopsis, and motion parallax to depth perception, it has been a

challenge to understand how the brain actually processes such information. A

number of researchers in the area of computational vision have worked on the

problem. For instance, David Marr (1982) has been influential in his proposal

that these various sources of information work together to create what he calls

a 21/2-D sketch that identifies where various visual features are located relative

to the viewer. He recognized, however, how far this representation is from an

actual perception of the world. In particular, such a sketch represents only parts

of surfaces and does not yet identify how these parts

go together to form images of objects in the environment

(the problem we had with Figure 2.3). He used

the term 3-D model to refer to a representation of

objects in a visual scene.

Cues such as texture gradient, stereopsis, and motion

parallax combine to create a representation of the

locations of surfaces in 3-D space.

Object Perception

A major problem in calculating a representation of the

world is object segmentation. Knowing where the lines

and bars are located in space is not enough; we need to

know which ones go together to form objects. Consider

the scene in Figure 2.12. Many lines go this way

and that, but somehow we put them together to come

up with the perception of a set of objects.

40 | Perception

FIGURE 2.12 An example

of how we aggregate the

perception of many broken lines

into the perception of solid

objects. (From Winston, 1970. Reprinted

by permission of the publisher. © 1970 by

Massachusetts Institute of Technology.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 40

We tend to organize objects into units

according to a set of principles called the

gestalt principles of organization after the

Gestalt psychologists who first proposed

them (e.g.,Wertheimer, 1912/1932). Consider

Figure 2.13. In Figure 2.13a, we perceive four

pairs of lines rather than eight separate lines.

This picture illustrates the principle of proximity:

Elements close together tend to organize

into units. Figure 2.13b illustrates the

principle of similarity. We tend to see this

array as rows of o’s alternating with rows of

x’s. Objects that look alike tend to be grouped

together. Figure 2.13c illustrates the principle

of good continuation. We perceive two lines,

one from A to B and the other from C to D,

although there is no reason why this sketch

could not represent another pair of lines, one

from A to D and the other from C to B. However, the line from A to B displays

better continuation than the line from A to D, which has a sharp turn. Figure

2.13d illustrates the principles of closure and good form.We see the drawing

as one circle occluded by another, although the occluded object could have

many other possible shapes.

These principles will tend to organize even completely novel stimuli into

units. Palmer (1977) studied the recognition of shapes such as the ones shown

in Figure 2.14. He first showed participants stimuli (e.g., Figure 2.14a) and then

asked them to decide whether the fragments depicted in Figures 2.14b through

2.14e were part of the original figure. The stimulus in Figure 2.14a tends to

organize itself into a triangle (principle of closure) and a bent letter n (principle

of good continuation). Palmer found that participants could recognize the

parts most rapidly when they were the segments predicted by the gestalt principles.

So the stimuli in Figures 2.14b and 2.14c were recognized more rapidly

than those in Figures 2.14d and 2.14e. Thus, we see that recognition depends

critically on the initial segmentation of the figure. Recognition can be impaired

Visual Information Processing | 41

(a)

A

C B

D

(c) (d)

(b)

FIGURE 2.13 Illustrations

of the gestalt principles of

organization: (a) the principle

of proximity, (b) the principle of

similarity, (c) the principle of

good continuation, (d) the

principle of closure.

(a) (b) (c) (d) (e)

FIGURE 2.14 Examples of stimuli used by Palmer (1977) for studying segmentation of novel

figures. (a) is the original stimulus that participants saw; (b) through (e) are the subparts of the

stimulus presented for recognition. Stimuli shown in (b) and (c) were recognized more rapidly

than those shown in (d) and (e).

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 41

when this gestalt-based segmentation contradicts the actual pattern

structure. FoRiNsTaNcEtHiSsEnTeNcEiShArDtOrEaD. The

reasons for the difficulty are that the gestalt principle of similarity

makes it hard to perceive adjacent letters of different case as units

and that removing the spaces between words has eliminated the

proximity cues.

These ideas about segmentation can be extended to describe

how more complex 3-D structures are divided. Figure 2.15

illustrates a proposal by Hoffman and Richards (1985) for how

gestalt-like principles can be used to segment an outline representation

of an object into subobjects. They observed that where one segment

joins another, there is typically a concavity in the line outline. Basically, people

exploit the gestalt principle of good continuation: The lines at the points of

concavity are not good continuations of one another, and so viewers do not

group these parts together.

The current view is that the visual processing underlying the ability to identify

the position and shape of an object in 3-D space is largely innate. Young

infants appear to be capable of recognizing objects and their shapes and of

appreciating where these objects are in 3-D space (e.g., Granrud, 1986, 1987).

Gestalt principles of organization explain how the brain segment visual

scenes into objects.

•Visual Pattern Recognition

We have now discussed visual information processing to the point where we

organize the visual world into objects. There still is a major step before we see

the world, however: We also must identify what these objects are. This task is

pattern recognition. Much of the research on this topic has focused on the

question of how we recognize the identity of letters. For instance, how do we

recognize a presentation of the letter A as an instance of the pattern A? We will

first discuss pattern recognition with respect to letter identification and then

move on to a more general discussion of object recognition.

Template-Matching Models

Perhaps the most obvious way to recognize a pattern is by means of template

matching. The template-matching theory of perception proposes that a retinal

image of an object is faithfully transmitted to the brain, and the brain attempts

to compare the image directly to various stored patterns. These patterns are

called templates. The basic idea is that the perceptual system tries to compare the

image of a letter to the templates it has for each letter and then reports the template

that gives the best match. Figure 2.16 illustrates various examples of successful

and unsuccessful template matching. In each case, an attempt is made to

achieve a correspondence between the retinal cells stimulated and the retinal

cells specified for a template pattern for a letter.

42 | Perception

FIGURE 2.15 Segmentation of

an object into subobjects. The

part boundary can be identified

with a contour that follows

points of maximum concave

curvature (dashed line). (From

Stillings et al., 1987; based on Hoffman &

Richards, 1985. Reprinted by permission

of the publisher. © 1987 by Massachusetts

Institute of Technology.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 42

Figure 2.16a shows a case in which a correspondence is achieved and an A is

recognized. Figure 2.16b shows a case in which no correspondence is reached

between the input of an L and the template pattern for an A. But L is matched

in Figure 2.16c by the L template. However, things can very easily go wrong

with a template. Figure 2.16d shows a mismatch that occurs when the image

falls on the wrong part of the retina, and Figure 2.16e shows the problem when

the image is the wrong size. Figure 2.16f shows what happens when the image is

in a wrong orientation, and Figures 2.16g and 2.16h show the difficulty when

the images are nonstandard A’s.

Although there are these difficulties with template matching, it is one of the

methods used in machine vision (see Ullman, 1996). Procedures have been

developed for rotating, stretching, and otherwise warping images to match.

Template matching is also used in fMRI brain imaging (see Chapter 1). Each

human brain is anatomically different, much as each human body is different.

When researchers claim regions like those in Figure 1.15 display activation patterns

like those in Figure 1.16 they typically are claiming that the same region

in the brains of each of their participants displayed that pattern. To determine

Visual Pattern Recognition | 43

Input

(a) (b) (c)

(d)

(g) (h)

(e) (f)

Template

Input Template

Input Template Input Template

Input Template Input Template

Input Template Input Template FIGURE 2.16 Examples of

attempts to match templates

to the letters A and L. (a) and

(c) are successful templatematching

attempts; (b) and

(d) through (h) are failed

attempts. (After Neisser, 1967. Adapted

by permission of the publisher. © 1967 by

Appleton.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 43

that it is the same region, they map the individual brains to a reference brain by

a sophisticated computer-based 3-dimensional template-matching procedure.

Although template matching has enjoyed some success, there seem to be limitations

to the abilities of computers to use template matching to recognize

patterns, as suggested in this chapter’s Implication Box on CAPTCHAS.

44 | Perception

Implications

Separating humans from BOTS

The special nature of human visual perception has been used in the development

of CAPTCHAs (von Ahn, Blum, & Langford, 2002). CAPTCHA stands for “Completely

Automated Public Turing Test to Tell Computers and Humans Apart.” The motivation

for CAPTCHAs comes from such real world problems as faced by YAHOO,

which offers free email accounts. The problem is that automatic BOTs will sign up

for such accounts and then use them to send SPAM. To test that it is a real human,

the system can present a figure like that in Figure 2.17. Although template-based

approaches may fail on recognizing such figures, one should not count computers

out. Some computer scientists have taken their appearance as an intellectual

challenge, and an “arms race” is appearing in which more sophisticated recognition

programs are appearing and more challenging CAPTCHAs are being developed.

The successful methods being used have little to do with template matching,

however, and attempt to extract defining features of these items (e.g., Mori &

Malik, 2003). You can visit the CAPTCHA Web site and contribute to the research

at http://recaptcha.net/captcha.html.

FIGURE 2.17 Examples of CAPTCHAs that humans can read but template-based

computer programs have great difficulty with. Courtesy of Luis von Ahn.

Template matching is a way to identify objects by aligning the stimulus to a

template of a pattern.

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 44

Feature Analysis

Partly because of the difficulties posed by template matching, psychologists

have proposed that pattern recognition occurs through feature analysis. In this

model, stimuli are thought of as combinations of elemental features. Table 2.1

from Gibson (1969) shows her proposal for the representation of the letters of

the alphabet in terms of features. For instance, the capital letter A can be seen as

consisting of a horizontal, two diagonals in opposite orientations, a line intersection,

symmetry, and a feature she called vertical discontinuity. She considered

some of these features like the straight lines to be outputs of the edge and

bar detectors in the visual cortex (discussed earlier—see Figure 2.8).

You might wonder how feature analysis represents an advance beyond the

template model. After all, what are the features but minitemplates? The featureanalysis

model does have a number of advantages over the template model,

however. First, because the features are simpler, it is easier to see how the system

might try to correct for the kinds of difficulties faced by the template-matching

model in recognizing full patterns as in Figure 2.16. Indeed, to the extent that

features are just line strokes, the bar and edge detectors we discussed earlier can

extract such features. A second advantage of feature analysis is that it is possible

to specify those relationships among the features that are most important to

Visual Pattern Recognition | 45

Features A

Straight

Horizontal

Vertical

Diagonal/

Diagonal\

Curve

Closed

Open V

Redundancy

Cyclic change

Symmetry

Discontinuity

Vertical

Horizontal

Open H

Intersection

E F H I L T K M N V W X Y Z B C D G J O P R Q S U

+ + + + +

+ + + + + + + + + + + + + + + +

+ + + + + + + + + + +

+ + + + +

+ + + + + + + +

+ + + + + + + + + + + + + +

+ + + + + + + +

+ + + + + + + + + +

+ + + + + +

+ +

+ + + +

+ + + + + + + + + + +

TABLE 2.1

Gibson’s proposal for the features underlying the recognition of letters

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 45

the pattern. Thus, for A, the critical point is that there are three lines, two diagonals

(in different directions) and one horizontal, and that these intersect. Many

other details are unimportant. Thus, all the following patterns are A’s:

Finally, the use of features rather than larger patterns reduces the number of

templates needed. In the feature-analysis model, we would not need a template

for each possible pattern but only for each feature. Because the same features

tend to occur in many patterns, the number of distinct entities to be represented

would be reduced considerably.

There is a fair amount of behavioral evidence for the existence of features as

components in pattern recognition. For instance, if letters have many features

in common—as C and G do, for example—evidence suggests that people are

particularly prone to confuse them (Kinney, Marsetta, & Showman, 1966).

When such letters are presented for very brief intervals, people often misclassify

one stimulus as the other. So, for instance, participants in the Kinney et al.

experiment made 29 errors when presented with the letter G. Of these errors,

there were 21 misclassifications as C, 6 misclassifications as O, 1 misclassification

as B, and 1 misclassification as 9. No other errors occurred. It is clear that participants

were choosing items with similar feature sets as their responses. Such a

response pattern is what we would expect if a feature-analysis model were used.

If participants could extract only some of the features in the brief presentation,

they would not be able to decide among stimuli that shared these features.

Another kind of experiment that yields evidence in favor of a feature-analysis

model involves stabilized images. The eye has a very slight tremor, called psychological

nystagmus, which occurs at the rate of 30 to 70 cycles per second.

Also, the eye’s direction of gaze drifts slowly over an object. Consequently, the

retinal image of the object on which a person tries to focus is not perfectly constant;

its position changes slightly over time. There is evidence that this retinal

movement is critical for perception. When techniques are used to keep an image

on the exact same position of the retina regardless of eye movement, parts

of the object start to disappear. If the exact same retinal and nervous pathways

are used constantly, they become fatigued and stop responding.

The most interesting aspect of this phenomenon

is the way the stabilized object disappears. It does

not simply fade away or vanish all at once. Instead,

different portions drop out over time. Figure 2.18

illustrates the fate of one of the stimuli used in

an experiment by Pritchard (1961). The leftmost

item was the image presented; the four others are

various fragments that were reported. Two points

are important. First, whole features such as a vertical bar seemed to be lost. This

finding suggests that features are the important units in perception. Second, the

stimuli that remained tended to constitute complete letter or number patterns.

This result indicates that these features are combined to define the recognized

patterns. Thus, even though our perceptual system may extract features, what we

perceive are patterns composed from these features. The feature-extraction and

feature-combination processes that underlie pattern recognition are not available

to conscious awareness; all that we are aware of are the resulting patterns.

, , , .

46 | Perception

FIGURE 2.18 The

disintegration of an image

that is stabilized on the eye.

At far left is the original image

displayed. The partial outlines to

the right show various patterns

reported as the stabilized image

began to disappear. (From Pritchard,

1961. Reprinted by permission of the publisher.

© 1961 by Scientific American.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 46

Feature analysis involves recognizing first the separate features that make up

a pattern and then their combination.

Object Recognition

Although feature analysis does a satisfactory job of describing how we recognize

such simple objects as the letter A, one might wonder how it explains our recognition

of more complex objects that might seem to defy description in terms of a

few features. There is evidence that the same processes might underlie the recognition

of familiar categories of objects such as horses or cups. The basic idea is that a

familiar object can be seen as a known configuration of simple components. Figure

2.19 illustrates a proposal by Marr (1982) about how familiar objects can be

seen as configurations of simple pipelike components. For instance, an ostrich has

a horizontally oriented torso attached to two long legs and a long neck.

Biederman (1987) put forward the recognition-by-components theory. It

proposes that there are three stages in our recognition of an object as a configuration

of simpler components:

1. The object is segmented into a set of basic subobjects. This process reflects

the output of early visual processing, discussed earlier in the chapter.

2. Once an object has been segmented into basic subobjects, one can

classify the category of each subobject. Biederman (1987) suggested that

Visual Pattern Recognition | 47

Horse

Giraffe Ape Dove

Human Ostrich

FIGURE 2.19 Segmentation of some familiar objects into basic cylindrical shapes. Familiar

objects can be recognized as configurations of simpler components. (After Marr & Nishihara, 1978.

Adapted by permission of the publisher. © 1978 by the Royal Society of London.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 47

there are 36 basic categories of subobjects, which he called geons (an

abbreviation of geometric ions). Figure 2.20 shows some examples.We

can think of the cylinder as being created by a circle as it is moved along

a straight line (the axis) perpendicular to its center.We can generalize

this basic cylinder shape to other shapes by varying some of the properties

of its generation.We can change the shape of the object we are moving.

If it is a rectangle rather than a circle that is moved along the axis, we

get a block instead of a cylinder.We can curve the axis and get objects

that curve.We can vary the size of the shape as we are moving it and get

objects like the pyramid or wine glass. Biederman proposed that the

36 geons that can be generated in this manner serve as an alphabet for

composing objects,much as letters serve as the alphabet for building up

words. Recognizing a geon involves recognizing the features that define

it, which describe elements of its generation such as the shape of the

object and the axis along which it is moved. Thus, recognizing a geon

is like recognizing a letter.

3. Having identified the pieces from which the object is composed and

their configuration, one recognizes the object as the pattern composed

from these pieces. Thus, recognizing an object is like recognizing

a word.

As in the case of letter recognition, there are many small variations in the

underlying geons that should not be critical for recognition. For example, one

need only determine whether an edge is straight or curved (in discriminating,

say, a brick from a cylinder) or whether edges are parallel or not (in discriminating,

say, a cylinder from a cone). It is not necessary to determine precisely

how curved an edge might be. Only edges are needed to define geons. Color,

texture, and small detail should not matter. If this hypothesis is correct,

schematic line drawings of complex objects that allow the basic geons to

be identified should be recognized just as

quickly as detailed color photographs of the

objects. Biederman and Ju (1988) confirmed

this hypothesis experimentally: Schematic line

drawings of such objects as telephones provide

all the information needed for quick and accurate

recognition.

The crucial assumption in this theory is that

object recognition is mediated by recognition

of the components of the object. Biederman,

Beiring, Ju, and Blickle (1985) performed a test

of this prediction with objects such as those

shown in Figure 2.21. Whole components of

some objects were deleted; in other objects,

all the components were present, but segments

of the components were deleted. They presented

these two types of degraded figures to participants

for various brief intervals and asked them

48 | Perception

Cylinder Cone Pyramid

Football Horn Wine glass

FIGURE 2.20 Examples of

Biederman’s (1987) proposed

geons, or basic categories of

subobjects. In each object, the

dashed line represents the

central axis of the object. The

objects can be described in

terms of the movement of a

cross-sectional shape along an

axis. Cylinder: A circle moves

along a straight axis. Cone: A

circle contracts as it moves along

a straight axis. Pyramid: A square

contracts as it moves along a

straight axis. Football: A circle

expands and then contracts as

it moves along a straight axis.

Horn: A circle contracts as it

moves along a curved axis. Wine

glass: A circle contracts and

then expands, creating concave

segmentation points, marked

by arrows. (From Biederman, Beiring,

Ju, & Blickle, 1985. Reprinted by permission

of the authors.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 48

to identify the objects. The results are shown in Figure 2.22. At very brief presentation

times (65 to 100 ms), participants recognized figures with component

deletion more accurately than figures with segment deletion; the opposite was

true for the longer, 200-ms presentation. Biederman et al. reasoned that at the

very brief intervals, participants were not able to identify the components with

segment deletion and so had difficulty in recognizing the objects.With 200-ms

exposure, however, participants were able to recognize all the components in

Visual Pattern Recognition | 49

Complete Component

deletion

Midsegment

deletion

FIGURE 2.21 Sample stimuli

used by Biederman et al. (1985)

to test the theory that object

recognition is mediated by

recognition of components

of the object. Equivalent

proportions either of whole

components or of contours at

midsegments were removed.

Results of the experiment are

shown in Figure 2.22. (After

Biederman, 1987. Adapted by permission

of the publisher. © 1987 by the

American Psychological Association.)

Midsegment deletion

Component deletion

40

30

20

10

65 100 200

Exposure duration (ms)

Mean (%) error

FIGURE 2.22 Results from the

test conducted by Biederman,

Beiring, Ju, and Blickle (1985)

to determine whether object

recognition is mediated by

recognition of components of

the object. Mean percentage

of errors of object naming is

plotted as a function of the

type of contour removal

(deletion of midsegments or

of entire components) and of

exposure duration. (After Biederman,

1987. Adapted by permission of the

publisher. © 1987 by the American

Psychological Association.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 49

either condition. Because there were more components in the condition with

segment deletion, they had more information about object identity.

Objects such as horses and cups are recognized as configurations of a set of

subobjects defined by simple features.

Face Recognition

Faces make up one of the most important categories of visual stimuli, and it has

been argued that we have special mechanisms to recognize faces. Special cells that

respond preferentially to the faces of other monkeys have been found in the temporal

lobes of monkeys (Baylis, Rolls, & Leonard, 1985; Rolls, 1992). Damage to

the temporal lobe in humans can result in a deficit called prosopagnosia, in

which people have selective difficulties in recognizing faces. Brain-imaging studies

using fMRI have found a particular region of the temporal lobe, called the

fusiform gyrus, that responds when faces are present in the visual field (e.g.,

Ishai, Ungerleider, Martin, Maisog, & Haxby, 1997; Kanwisher, McDermott, &

Chun, 1997;McCarthy, Puce, Gore,& Allison, 1997).

One piece of evidence that the processing of faces is special comes from

research that examined the recognition of faces turned upside down. In one of the

original studies,Yin (1969) found that people aremuch better at recognizing faces

when the faces are presented in their upright orientation than they are at recognizing

other categories of objects, such as houses, presented in the upright orientation.

When a face is presented upside down, however, there is a dramatic decrease

in its recognition; this is not true of other objects. Thus, it appears that we are specially

attuned to recognizing faces. Studies have also found somewhat reduced

fMRI response in the fusiformgyrus when inverted faces are presented (Haxby et al.,

1999; Kanwisher, Tong, & Nakayama, 1998). In addition, we are much better at

recognizing parts of a face (a nose, say) when it is presented in context, whereas

there is not the same context dependency in recognizing parts of a house (for

example, a window) (Tanaka & Farah, 1993). All this evidence leads some researchers

to think that we are specifically predisposed to identify whole faces, and

it is sometimes argued that this special capability was acquired through evolution.

Other research has not supported the conclusion that there is something

special about face recognition and instead presents evidence that the fusiform

gyrus specializes in making fine-grained distinctions. Because of our great

familiarity with faces, we are good at making such fine-grained judgments in

recognizing them, but similar effects can be found with other stimuli with

which we have had a lot of experience. For instance, Gauthier, Skudlarski, Gore,

50 | Perception

FIGURE 2.23 “Greeble

experts” use the face area

when recognizing these objects.

(From Gauthier, Tarr, Anderson, Skudlarski,

& Gore, 1999. Reprinted by permission

of the publisher. © 1999 by Nature

Neuroscience.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 50

and Anderson (2000) found that bird experts or car experts showed high activation

in the fusiform gyrus when they made judgments about birds or cars. In

another study, people given a lot of practice at recognizing a set of unfamiliar

objects called greebles (Figure 2.23) showed activation in the fusiform gyrus.

The fusiform gyrus, located in the temporal lobe, becomes active when people

recognize faces.

•Speech Recognition

Up to this point, we have considered only visual pattern recognition. An interesting

test of the generality of our conclusions is whether they extend to speech

recognition. Although we will not discuss the details of early speech processing, it

is worth noting that similar issues arise. A major problem for speech recognition

is segmentation of the objects to be recognized. Speech is not broken into discrete

units the way printed text is. Although well-defined gaps between words seem to

exist in speech, these gaps are often an illusion. If we examine the actual physical

speech signal, we often find undiminished sound energy at word boundaries.

Indeed, gaps in sound energy are as likely to occur within a word as between

words. This property of speech is particularly compelling when we listen to

someone speaking an unfamiliar foreign language. The speech appears to be a

continuous stream of sounds with no obvious word boundaries. It is our familiarity

with our own language that leads to the illusion of word boundaries.

Within a single word, even greater segmentation problems exist. These intraword

problems involve the identification of phonemes. Phonemes are the basic

vocabulary of speech sounds; we recognize words in terms of phonemes.1 A

phoneme is defined as the minimal unit of speech that can result in a difference

in the spoken message. To illustrate, consider the word bat. This word is composed

of three phonemes: /b/, /a/, and /t/. Replacing /b/ with the phoneme /p/,

we get pat; replacing /a/ with /i/ we get bit; replacing /t/ with /n/, we get ban.

Obviously, a one-to-one correspondence does not always exist between letters

and phonemes. For example, the word one consists of the phonemes /w/, /e/, and

/n/; school consists of the phonemes /s/, /k/, /ú/, and /1/; and knight consists of

/n/, /¯ı/, and /t/. It is the lack of perfect letter-to-sound correspondence that

makes English spelling so difficult.

A segmentation problem arises when the phonemes composing a spoken

word need to be identified. The difficulty is that speech is continuous, and

phonemes are not discrete in the way letters are on a printed page. Segmentation

at this level is like recognizing a written (not printed) message, where

one letter runs into another. Also, as in the case of writing, different speakers

vary in the way they produce the same phonemes. The variation among speakers

is dramatically clear, for instance, when a person first tries to understand a

speaker with a strong and unfamiliar accent—as when an American listener

Speech Recognition | 51

1Massaro (1996) presents an often proposed alternative that the basic perceptual units are consonant-vowel

and vowel-consonant combinations.

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 51

tries to understand an Australian speaker. Examination of the speech signal,

however, will reveal that even among speakers with the same accent, considerable

variation exists. For instance, the voices of women and children normally

have a much higher pitch than those of men.

A further difficulty in speech perception involves a phenomenon known as

coarticulation (Liberman, 1970). As the vocal tract is producing one sound—

say, the /b/ in bag—it is moving toward the shape it needs for the /a/. As it is

saying the /a/, it is moving to produce the /g/. In effect, the various phonemes

overlap. This means additional difficulties in segmenting phonemes, and it also

means that the actual sound produced for one phoneme will be determined by

the context of the other phonemes.

Speech perception poses information-processing demands that are in many

ways greater than what is involved in other kinds of auditory perception.

Researchers have identified a number of patients who have lost just the ability to

hear speech, as a result of injury to the left temporal lobe (see M. N. Goldstein,

1974, for a review). Their ability to detect and recognize other sounds and to

speak is intact. Thus, their deficit is specific to speech perception. Occasionally,

these patients have some success if the speech they are trying to hear is very

slow (e.g., Okada, Hanada, Hattori, & Shoyama, 1963), which suggests that

some of the problem might lie in segmenting the speech stream.

Speech recognition involves segmenting phonemes from the continuous

speech stream.

Feature Analysis of Speech

Feature-analysis and feature-combination processes seem to underlie speech

perception, much as they do visual recognition. As with individual letters, individual

phonemes can be analyzed as consisting of a number of features. It turns

out that these features refer to aspects of how the phoneme is generated.

Among the features of phonemes are the consonantal feature, voicing, and the

place of articulation (Chomsky & Halle, 1968). The consonantal feature is the

consonant-like quality of a phoneme (in contrast to a vowel-like quality). Voicing

is the sound of a phoneme produced by the vibration of the vocal cords.

For example, compare the ways you speak the words sip and zip. The /s/ in sip is

voiceless, but the /z/ in zip is voiced. You can detect this difference by placing

your fingers on your larynx as you generate these sounds. The larynx will

vibrate for the voiced consonant.

Place of articulation refers to the location at which the vocal tract is closed

or constricted in the production of a phoneme. (It is closed at some point in the

utterance of most consonants.) For instance, /p/, /m/, and /w/ are considered bilabial

because the lips are closed while they are being generated. The phonemes

/f/ and /v/ are considered labiodental because the bottom lip is pressed against

the front teeth. Two different phonemes are represented by /th/—one in thy and

the other in thigh. Both are dental because the tongue presses against the teeth.

The phonemes /t/, /d/, /s/, /z/, /n/, /l/, and /r/ are all alveolar because the tongue

presses against the alveolar ridge of the gums just behind the upper front teeth.

52 | Perception

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 52

The phonemes /sh/, /ch/, /j/, and /y/ are all palatal because the tongue

presses against the roof of the mouth just behind the alveolar ridge.

The phonemes /k/ and /g/ are velar because the tongue presses against

the soft palate, or velum, in the rear roof of the mouth.

Consider the phonemes /p/, /b/, /t/, and /d/. All share the feature of

being consonants. The four can be distinguished, however, by voicing

and place of articulation. Table 2.2 classifies these four phonemes according

to these two features.

Considerable evidence exists for the role of such features in speech

perception. For instance, Miller and Nicely (1955) had participants try to

recognize phonemes such as /b/, /d/, /p/, and /t/ when presented in noise.2

Participants exhibited confusion, thinking they had heard one sound in the

noise when in reality another sound had been presented. The experimenters

were interested in what sounds participants would confuse with which other

sounds. It seemed likely that they would most often confuse consonants that

were distinguished by just a single feature, and this prediction was confirmed. To

illustrate, when presented with /p/, participants more often thought that they

had heard /t/ than that they had heard /d/. The phoneme /t/ differs from /p/ only

in place of articulation, whereas /d/ differs both in place of articulation and in

voicing. Similarly, participants presented with /b/ more often thought they heard

/p/ than /t/.

This experiment is an earlier demonstration of the kind of logic we saw in

the Kinney et al. (1966) study on letter recognition.When the participant could

identify only a subset of the features underlying a pattern (in this case, the pattern

is a phoneme), the participant’s responses reflected confusion among the

phonemes sharing the same subset of features.

Phonemes are recognized in terms of features involved in their production,

such as place of articulation and voicing.

•Categorical Perception

The features of phonemes refer to properties by which they are articulated.

What properties of the acoustic stimulus encode these articulatory features?

This issue has been particularly well researched in the case of voicing. In the

pronunciation of such consonants as /b/ and /p/, two things happen: The closed

lips open, releasing air, and the vocal cords begin to vibrate (voicing). In the

case of the voiced consonant /b/, the release of air and the vibration of the vocal

cords are nearly simultaneous. In the case of the unvoiced consonant /p/, the

release occurs 60 ms before the vibration begins. What we are detecting when

we perceive a voiced versus an unvoiced consonant is the presence or absence of

a 60-ms interval between release and voicing. This period of time is referred to

as the voice-onset time. The difference between /p/ and /b/ is illustrated in

Categorical Perception | 53

TABLE 2.2

The Classification of /b/, /p/, /d/,

and /t/ According to Voicing and

Place of Articulation

Place of Voicing

Articulation Voiced Unvoiced

Bilabial /b/ /p/

Alveolar /d/ /t/

2Actually, participants were presented with the sounds ba, da, pa, and ta.

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 53

Figure 2.24. Similar differences exist in other voiced-unvoiced pairs, such as /d/

and /t/. Again, the factor controlling the perception of a phoneme is the delay

between the release of air and the vibration of the vocal cords.

Lisker and Abramson (1970) performed experiments with artificial (computergenerated)

stimuli in which the delay between the release of air and the onset

of voicing was varied from –150 ms (voicing occurred 150 ms before release)

to +150 ms (voicing occurred 150 ms after release). The task was to identify

which sounds were /b/’s and which were /p/’s. Figure 2.25 plots the percentage

of /b/ identifications and /p/ identifications. Throughout most of the continuum,

participants agreed 100% on what they heard, but there was a sharp

switch from /b/ to /p/ at about 25 ms. At a 10-ms voice-onset time, participants

were in nearly unanimous agreement that the sound was a /b/; at 40 ms, they

were in nearly unanimous agreement that the sound was a /p/. Because of this

sharp boundary between the voiced and unvoiced phonemes, perception of this

feature is referred to as categorical. Categorical perception is the perception of

stimuli as belonging in distinct categories and the failure to perceive the gradations

among stimuli within a category.

54 | Perception

−100 0

Lips released

Time (ms)

Voicing

Voicing

/b/

/p/

+60 +100

FIGURE 2.24 The difference

between the voiced consonant

/b/ and the unvoiced consonant

/p/ is the delay in the case of

/p/ between the release of the

lips and the onset of voicing.

(From H. H. Clark & E. V. Clark, 1977.

Reprinted by permission of the publisher.

© 1977 by Harcourt Brace Jovanovich.)

Voice onset time (ms)

/b/ /p/

Identification (%)

0

20

40

60

80

100

−100 −50 0 +50 +100 +150

FIGURE 2.25 Percentage identification of /b/ versus /p/ as a function of voice-onset time.

A sharp shift in these identification functions occurred at about 25 ms. (From Lisker & Abramson,

1970. Reprinted by permission of the publisher. © 1970 by Academia.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 54

Other evidence for categorical perception of speech comes from discrimination

studies (see Studdert-Kennedy, 1976, for a review). People are very poor at

discriminating between a pair of /b/’s or a pair of /p/’s that differ in voice-onset

time. However, they are good at discriminating between pairs that have the

same difference in voice-onset time but one of the pair is a /b/ and the other is a

/p/. It seems that people can identify only the phonemic category of a sound

and are not able to make acoustic discriminations within that phonemic category.

Thus, people are able to discriminate two sounds only if they fall on different

sides of a phonemic boundary.

There are at least two views of exactly what is meant by categorical perception,

which differ in the strength of their claims about the nature of perception.

The weaker view is that we experience stimuli as coming from distinct categories.

There seems to be little dispute that the perception of phonemes is categorical in

this sense. A stronger viewpoint is that we cannot discriminate among stimuli

within a category.Massaro (1992) has taken issue with this viewpoint, and he has

argued that there is some residual ability to discriminate within categories. He

further argues that findings of poor discrimination within a category may reflect

a bias of participants to say that stimuli within a category are the same even when

there are discriminable differences.

Another line of research that provides evidence for use of the voicing feature

in speech recognition involves an adaptation paradigm. Eimas and Corbit

(1973) had their participants listen to repeated presentations of the sound da.

This sound involves a voiced consonant /d/. The experimenters reasoned that

the constant repetition of the voiced consonant might fatigue, or adapt, the feature

detector that responds to the presence of voicing. Then they presented participants

with a series of artificial sounds that spanned the acoustic continuum—

such as the range between ba and pa (as in the Lisker & Abramson, 1970, study

mentioned earlier). Participants were to indicate whether each of these artificial

stimuli sounded more like ba or like pa. (Remember that the only feature difference

between ba and pa is voicing.) Eimas and Corbit found that some of the

stimuli participants would normally have called the voiced ba, they now called

the voiceless pa. Thus, the repeated presentation of da had fatigued the voiced

feature detector and raised the threshold for detecting voicing in ba, making

many former ba stimuli sound like pa.

Although there is general consensus that speech perception is categorical in

some sense, there is considerable debate about what the mechanism is behind

this phenomenon. Anticipating a theme that will occupy much of Chapters 12

and 13, some researchers (e.g., Liberman & Mattingly, 1985) have argued that

this reflects special speech perception mechanisms that enable people to perceive

how the sounds were generated. Consider, for instance, the categorical distinction

between how voiced and unvoiced consonants are produced—either the

vocal cords vibrate during the consonant or they do not. This has been used to

argue that we perceive voicing by perceiving how the consonants are spoken.

However, there is evidence that categorical perception is not tied to humans

processing language but rather reflects a general property of how certain sounds

are perceived. For instance, Pisoni (1977) created nonlinguistic tones that had a

similar distinguishing acoustic feature as present in voicing—a low-frequency

Categorical Perception | 55

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 55

tone that is either simultaneous with a high-frequency tone or lags it by 60 msec.

His participants showed abrupt boundaries like those in Figure 2.24 for speech

signals. In another study, Kuhl (1987) trained chinchillas to discriminate between

a voiced “da” and an unvoiced “ta.” Even though these animals do not have

a human vocal track, they showed the sharp boundary between these stimuli

that humans do. Thus, it seems that categorical perception depends on neither

the signal being speech (Pisoni, 1977) nor the perceiver having a human vocal

system (Kuhl, 1987). Diehl, Lotto, and Holt (2004) have argued that the

phonemes we use are chosen so that they exist across categorical boundaries that

occur in our auditory perception of speech. So it is more a case of our perceptual

system determining our speech behavior than vice versa.

Speech sounds differing on continuous dimensions are perceived as coming

from distinct categories.

•Context and Pattern Recognition

So far, we have considered pattern recognition as if the only information available

to a pattern-recognition system were the information in the physical stimulus

to be recognized. This is not the case, however. Objects occur in context,

and we can use context to help us recognize objects. Consider the example in

Figure 2.26.We perceive the symbols as THE and CAT, even though the specific

symbols drawn for H and A are identical. The general context provided by the

words forces the appropriate interpretation. When context or general knowledge

of the world guides perception, we refer to the processing as top-down

processing, because high-level general knowledge contributes to the interpretation

of the low-level perceptual units. A general issue in perception is how such

top-down processing is combined with the bottom-up processing of information

from the stimulus itself, without regard to the general context.

One important line of research in top-down effects comes from a series of

experiments on letter identification, starting with those of Reicher (1969) and

Wheeler (1970). Participants were presented very briefly with either a letter

(such as D) or a word (such as WORD). Immediately afterward, they were given

a pair of alternatives and instructed to report which alternative

they had seen. (The initial presentation was sufficiently

brief that participants made a good many errors in this identification

task.) If they had been shown the letter D, they

might be presented with D and K as alternatives. If they had

been shown WORD, they might be given WORD and WORK

as alternatives. Note that both choices differed only in the letter

D or K. Participants were about 10% more accurate in identifying the word

than in identifying the letter alone. Thus, they discriminated between D and K

better in the context of a word than as letters alone—even though, in a sense,

they had to process four times as many letters in the word context. This phenomenon

is known as the word superiority effect.

56 | Perception

FIGURE 2.26 A demonstration

of context. The same stimulus is

perceived as an H or an A,

depending on the context. (From

Selfridge, 1955. Reprinted by permission of

the publisher. © 1955 by the Institute of

Electrical and Electronics Engineers.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 56

Figure 2.27 illustrates an explanation given by

Rumelhart and Siple (1974) and Thompson and

Massaro (1973) for why people are more accurate

when identifying the letter in the word context.

The figure illustrates the products of incomplete

perception: Certain parts of the word cannot be

detected—in part (a) just the last letter is obscured,

whereas in part (b) multiple letters are obscured.

If the last letter was all that a participant

was shown, the participant would not be able to

say whether that letter was a K or an R. Thus, the

stimulus information is not enough to identify the

letter. On the other hand, the context is not

enough by itself either—although it is pretty clear

in part (a) that the first three letters are WOR,

there are a number of four-letter words consistent

with a WOR beginning: WORD, WORE, WORK,

WORM, WORN, WORT. However, if the participant

combines the information from the stimulus

with the information from the context, the whole

word must be WORK, which implies K was the

last letter. It is not that participants see the K better in the context of WOR

but that they are better able to infer that K is the fourth letter. The participants

are not conscious of these inferences, however; so they are said to

make unconscious inferences in the act of perception. Note in particular that

the participants in this example do not have conscious access to specific

features such as the target letter having a lower right diagonal or it would

have been possible to choose correctly whether they saw that letter alone and

were given the alternatives D and K. Rather, the participants have conscious

access only to the whole word or whole letter that the perceptual system has

perceived. Note that this analysis is not restricted to the case where the

context letters are unambiguous. In part (b), the second letter could be an O

or a U and the third letter could be a B, P, or R. Still, WORK is the only possible

word.

This example illustrates the redundancy present in many complex stimuli

such as words. These stimuli consist of many more features than are required to

distinguish one stimulus from another. Thus, perception can proceed successfully

when only some of the features are recognized, with context filling in the

remaining features. In language, this redundancy exists on many levels besides

the feature level. For instance, redundancy occurs at the letter level. We do

not need to perceive every letter in a string of words to be able to read it. To

xllxstxatx, I cxn rxplxce xvexy txirx lextex of x sextexce xitx an x, anx yox stxll

xan xanxge xo rxad xt—ix wixh sxme xifxicxltx.

Word context can be used to supplement feature information in the recognition

of letters.

Context and Pattern Recognition | 57

WORK

WORK

(a)

(b)

FIGURE 2.27 A hypothetical

set of features that might

be extracted on a trial in an

experiment of word perception:

(a) when only the last letter is

obscured; (b) when multiple

letters are obscured.

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 57

Massaro’s FLMP Model for Combination of Context

and Feature Information

We have reviewed the effects of context on pattern recognition in a variety of perceptual

situations, but the question of how to interpret these effects conceptually

still remains. Massaro has argued that the stimulus and

the context provide two independent sources of information

about the identity of the stimulus and that they are

just combined to provide a best guess of what the stimulus

might be. Figure 2.28 shows examples of the material

he used in a test of recognition of the letter c versus the

letter e. The four quadrants represent four possibilities in

the amount of contextual evidence: Only an e can make a

word, only a c can make a word, both letters can make a

word, or neither can make a word. As one reads down

within a quadrant, the image of the ambiguous letter

provides more evidence for letter e and less for letter c.

Participants were briefly exposed to these stimuli and

asked to identify the letter. Figure 2.29 shows the results

as a function of stimulus and context information. As the

image of the letter itself provided more evidence for an e,

the probability of the participants’ identifying an e went up. Similarly, the probability

of identifying an e increased as the context provided more evidence.

Massaro argued that these data reflect an independent combination of evidence

from the context and evidence from the letter stimulus. He assumed that

the letter stimulus represents some evidence Lc for the letter c and that the context

also provides some evidence Cc for the letter c. He assumed that these evidences

can be scaled on a range of 0 to 1 and can be thought of basically as probabilities,

which he called “fuzzy truth values.’’ Because probabilities sum to 1, the evidence

for e from the letter stimulus is Le 1 Lc, and the evidence from the context is

Ce 1 Cc. Given these probabilities, then, the overall probability for a c is

The lines in Figure 2.29 illustrate the predictions from his theory. In general,

Massaro’s theory (called FLMP for fuzzy logical model of perception) has done

a very good job of accounting for the combination of context and stimulus

information in pattern recognition.

Massaro’s FLMP model of perception proposes that contextual information

combines independently with stimulus information to determine what

pattern is perceived.

Other Examples of Context and Recognition

Word recognition is one case for which there have been detailed analyses (for

example,Massaro addressed how context influences perception), but contextual

influences are ubiquitous. For instance, equally good evidence exists for the role

Lc Cc p(c) (Lc Cc) (Le Ce)

58 | Perception

FIGURE 2.28 Contextual clues

used by Massaro (1979) to

study how participants combine

stimulus information from a

letter with context information

from the surrounding letters.

(From Massaro, 1979. Reprinted by

permission of the publisher. © 1979 by the

American Psychological Association.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 58

of context in the perception of speech. A nice illustration

is the phoneme-restoration effect, originally

demonstrated in an experiment by Warren (1970). He

asked participants to listen to the sentence “The state

governors met with their respective legislatures convening

in the capital city,’’with a 120-ms tone replacing

the middle s in legislatures. Only 1 in 20 participants

reported hearing the pure tone, and that participant

was not able to locate it correctly.

An interesting extension of this first study was

an experiment by Warren and Warren (1970). They

presented participants with sentences such as the

following:

It was found that the *eel was on the axle.

It was found that the *eel was on the shoe.

It was found that the *eel was on the orange.

It was found that the *eel was on the table.

In each case, the * denotes a phoneme replaced by

nonspeech. For the four sentences above, participants

reported hearing wheel, heel, peel, and meal, depending

on context. The important feature to note about each

of these sentences is that they are identical through the

critical word. The identification of the critical word is

determined by what occurs after it. Thus, the identification

of words often is not instantaneous but can depend on the perception of

subsequent words.

Context also appears to be important for the perception of complex visual

scenes. Biederman, Glass, and Stacy (1973) looked at the perception of objects

in novel scenes. Figure 2.30 illustrates the two kinds of scenes presented to their

participants. Figure 2.30a shows a normal scene; in Figure 2.30b, the same scene

is jumbled. Participants viewed one or the other scene briefly on a screen, and

Context and Pattern Recognition | 59

1.0

.9

.8

.7

.6

.5

.4

.3

.2

.1

0

2 3 4 5

Stimulus value

Only c

Probability of e response

1c

6

e

Both e and c

Only e

Neither

e nor c

FIGURE 2.29 Probability of

an e response as a function of

the stimulus value of the test

letter and of the orthographic

context. The lines reflect the

predictions of Massaro’s FLMP

model. The leftmost line is for

the case where the context

provides evidence only for e.

The middle line is the same

prediction when the context

provides evidence for both e

and c or when it provides

evidence for neither e nor c.

The rightmost line is for the

case where the context

provides evidence only for c.

(From Massaro, 1979. Reprinted by

permission of the publisher. © 1979 by

the American Psychological Association.)

(a) (b)

FIGURE 2.30 Scenes used by Biederman, Glass, and Stacy (1973) in their study of the role of

context in the recognition of complex visual scenes: (a) a coherent scene; (b) a jumbled scene.

It is harder to recognize the fire hydrant in the jumbled scene. (From Biederman, Glass, & Stacy, 1973.

Reprinted by permission of the publisher. © 1973 by the American Psychological Association.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 59

immediately thereafter an arrow pointed to a position on the now-blank screen

where an object had been moments before. Participants were asked to identify

the object that had been in that position in the scene. So, in the example scene,

the arrow might have pointed to the location of the fire hydrant. Participants

were considerably more accurate in their identifications when they had viewed

the coherent picture than when they had viewed the jumbled picture. Thus, as

with the processing of written text or speech, people are able to use context in a

visual scene to help in their identification of an object.

One of the most dramatic examples of the influence of context on perception

involves a phenomenon called change blindness. People are unable to keep track

of all the information in a typical complex scene. If elements of the scene change

at the same time as there is some retinal disturbance (such as an eye movement or

a scene-cut in a motion picture), they often fail to detect the change if it matches

the context. The original studies on change blindness (McConkie & Currie, 1996)

introduced large changes in pictures that participants were viewing while they

were making an eye movement. For instance, the color of a car in the picture

might change. These changes were usually not detected. Figure 2.31 illustrates a

60 | Perception

(a) (b)

(c) (d)

FIGURE 2.31 An example of change blindness. Frames from a video showing how one

experimenter switched places with an accomplice as workers carrying a door passed between

the experimenter and an unwitting participant. Only 7 of the 15 participants noticed the change.

(a) through (c) show the sequence of the switch; (d) shows the two experimenters side by side.

(From Simons & Levin, 1998. Reprinted by permission of the publisher. © 1998 by Psychonomic Bulletin and Review.)

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 60

dramatic instance of change blindness (Simons & Levin, 1998). The experimenter

stopped pedestrians on Cornell University’s campus and asked for directions.

While the unwitting participant was giving the directions, workers carrying a door

passed between the experimenter and the participant. An accomplice took the

place of the experimenter. Only 7 of the 15 participants noticed the change. In the

scene shown in Figure 2.31, the participants thought of themselves as giving instructions

to a student, and as long as the changed experimenter fit that interpretation,

they did not process him as different. In a laboratory study of the ability to

detect changes in people’s faces, Beck, Rees, Frith, and Lavie (2001) found greater

activation in the fusiform gyrus (see the earlier discussion of face recognition)

when face changes were detected than when they were not.

Contextual information biases perceptual processing in a wide variety

of situations.

•Conclusions

This chapter discusses how the neurons process sensory information, deliver it

to the higher centers in the brain, and how the information becomes recognizable

as objects. Figure 2.32 depicts the overall flow of information processing in

the case of vision perception. Perception begins with light energy from the external

environment. Receptors, such as those on the retina, transform this

energy into neural information. Early sensory processing makes initial sense of

the information. Features are extracted to yield what Marr called the primal

sketch. These features are combined with depth information to get a representation

of the location of surfaces in space; this is Marr’s 21/2-D sketch. The

gestalt principles of organization are applied to segment the elements into objects;

this is Marr’s 3-D model. Finally, the features of these objects and the general

context information are combined to recognize the objects. The output of

this last level is what we are consciously aware of in perception, which is a representation

of the objects and their locations in the environment. This information

is the input to the higher level cognitive processes. Figure 2.32 illustrates an

important point: A great deal of information processing must take place before

we are consciously aware of the objects we are perceiving.

Questions for Thought | 61

Light energy

Primal sketch

2 ½-D sketch

3-D model

Recognized

objects

Feature extraction

Depth information

Gestalt principles

of organization

Feature combination,

contextual information

FIGURE 2.32 How information

flows from the environment and

is processed into our perceptual

representation of it. The ovals

represent different levels of

information in Marr’s (1982)

model and the lines are labeled

with the perceptual processes

that transform one level of

information into the next.

1. Figure 2.33a illustrates an optical illusion called Mach

Bands after the Austrian physicist and philosopher,

Ernst Mach, who discovered them. Each band is a

uniform shade of gray and yet it appears lighter on the

right side near the darker adjacent band, and it appears

darker on the left side near the lighter band. Can you

explain why, using on-off cells, edge detectors, and bar

detectors (see Figures 2.7 & 2.8)?

2. Use the Gestalt principles to explain why we tend to see

two triangles in Figure 2.33b.

3. Rather than Biederman’s geon proposal (see

Figure 2.20), which involves recognizing objects

by recognizing abstract features of their components,

Ullman (2006) proposes we recognize objects by

recognizing concrete fragments like those in

Figure 2.33c.What might be the relative strengths

Questions for Thought

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 61

62 | Perception

(a)

(b)

(c)

Key Terms

21⁄2-D sketch

3-D model

apperceptive agnosia

associative agnosia

bar detectors

bottom-up processing

categorical perception

change blindness

consonantal feature

edge detectors

feature analysis

feature maps

fovea

fusiform gryrus

fuzzy logical model of

perception (FLMP)

geons

gestalt principles of

organization

phonemes

phoneme-restoration effect

place of articulation

primal sketch

prosopagnosia

recognition-by-components

theory

template matching

top-down processing

visual agnosia

voicing

word superiority effect

of the geon theory versus the fragment-based

theory?

4. In Figure 2.21, we see that presented with the stimulus

“cdit,” there is an increased tendency for participants

to say that they have seen “edit,” which makes a

word. Some people describe this as a case of context

distorting perception. Do you agree that this is a

case of distortion?

FIGURE 2.33 Figures for Questions for Thought. (a) Mach bands; (b) demonstration of Gestalt principles of organization;

(c) fragments for recognizing a horse from Ullman (2006).

Anderson7e_Chapter_02.qxp 8/20/09 9:39 AM Page 62

3Attention and Performance

Chapter