feedback week 4
Event perception and memory
Jeffrey M. Zacks Washington University in Saint Louis
Abstract
Events make up much of our lived experience, and the perceptual mechanisms that represent
events in experience have pervasive effects on action control, language use, and remembering.
Event representations in both perception and memory have rich internal structure and connections
one to another, and both are heavily informed by knowledge accumulated from previous
experiences. Event perception and memory have been identified with specific computational
and neural mechanisms, which show protracted development in childhood and are affected by
language use, expertise, and brain disorders and injuries. Current theoretical approaches focus
on the mechanisms by which events are segmented from ongoing experience, and emphasize
the common coding of events for perception, action, and memory. Abetted by developments in
eye-tracking, neuroimaging, and computer science, research on event perception and memory is
moving from small-scale laboratory analogs to facing the complexity of events in the wild.
Introduction
Event perception is the set of mechanisms by which an organism represents the activity in
which it is immersed. Research on event perception has roots in Gestalt psychology (Köhler
1929), in Michotte’s (1946) studies of perceptual causality, in the ecological psychology of
human behavior (Barker 1963), and in the social psychology of attribution (Newtson 1976).
Miller and Johnson-Laird (1976), in their broad and original overview of language and
perception, articulated what would come to be some of the central issues in the field of event
cognition: the relationships between objects and events, the fact that events are ephemeral,
disappearing even as they occur, and the tight relationships between goals in human action
and the parts of events in perception.
In the 1970s and early 1980s there was a burst of interest in the visual perception of
simple motion events. In 1980, Gunnar Johansson and his colleagues wrote a chapter for
the Annual Review of Psychology titled “Event perception” (Johansson et al. 1980). Around
the same time, Gibson’s (1979) volume on ecological perception discussed events in some
detail, Cutting (1981) published six tenets for event perception, and the first International
Conference on Event Perception took place (Warren & Shaw 1985). Interest then waned
until the turn of the century, when a new wave of research took a broader view that
encompassed the other sensory modalities, language perception, and the integration of
jzacks@wustl.edu .
HHS Public Access Author manuscript Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
Published in final edited form as: Annu Rev Psychol. 2020 January 04; 71: 165–191. doi:10.1146/annurev-psych-010419-051101.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
perception and action. Since then, interest in event perception and its interaction with other
aspects of psychology has grown steadily.
The mechanisms of event perception are fundamental organizers of conscious experience,
of active behavior, of language use, and of the experience of remembering the past. The
broader term event cognition refers to event perception and its interfaces with action control,
language, and memory. In this review, I will first summarize some of the key empirical
phenomena in event cognition, focusing on perception and memory. I will then give an
overview of current theoretical models and situate them relative to other ideas in psychology
and neuroscience. I will conclude by describing some of the methodological developments
that are shaping the field.
The psychological study of event perception is broad, and the study of event memory
even broader. This review will necessarily be selective, focusing on the understanding of
everyday events across the lifespan.1 Some topics will receive short shrift here but have
been recently been reviewed elsewhere (Radvansky & Zacks 2014): I will say little about
how event perception and memory are affected by neurological and psychological disorders
(for a review, see Zacks & Sargent 2010). I will also say little about the applications of
event cognition to diagnosing and improving comprehension and memory (see Richmond
et al. 2017, and commentaries). Finally, I will say little about event cognition in text
comprehension—though it is important to note that text comprehension research has had a
strong influence on current theories and research programs in event cognition (Radvansky &
Zacks 2014; Zwaan & Radvansky 1998).
The structure of events in the world and in perception
Everyday experience can be described as chaotic—both in the usual sense of the word and
in the technical sense of dynamical systems theory. The specific movements of leaves in
the breeze or of people in a crowd are unpredictable and are sensitive to small fluctuations.
At the same time, everyday experience can be described as highly structured, regular,
and predictable. The larger patterns of blowing leaves or crowds of people are often
quite straightforward; people’s actions can be predicted from their goals, from their stable
characteristics, and from the situation around them. What is special, perhaps, about human
activity is that the variables that organize the activity are removed from the relevant physical
variables—they track actions and intentions rather than movements, and track objects rather
than shapes or surfaces (Richmond et al. 2017). Human perceptual and conceptual systems
are geared to track these regularities even when this requires abstracting considerably from
the sensory input.
Continuous experience is segmented into continuous events
One thoroughgoing form of abstraction in perception is segmentation—dividing a
continuum into parts. (Segmentation is sometimes described as unitization—grouping some
1To identify relevant research, in January 2019 I searched Google Scholar, Web of Science, and PsycInfo with the terms “event perception” and “event memory.” This resulted in 1054 hits. Even after discarding duplicates, ephemera, and material that was off point, a substantial number of items remained. The final list of cited works reflects selection amongst those items and integration of other materials that either were known to me or came up in reading the retrieved papers and chapters.
Zacks Page 2
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
of a continuum into a gestalt. These are two sides of the same coin.) Much research
on event segmentation has utilized variants of a procedure in which observers press a
key to mark boundaries between events while watching a movie or slide show, or while
reading or listening to a story (Newtson 1973). Studies applying this procedure to depictions
of everyday activities have revealed some key features of event segmentation behavior,
summarized in Table 1. Recent studies of event segmentation have extended the range of
presentation media and materials, exploring the segmentation of very simple displays and
quite complex ones. On the simple end, when people viewed animations of flashing objects,
the introduction of new objects is sufficient to induce event boundaries (Tauzin 2015), and
minimal displays of human body motion are sufficient to support robust segmentation, even
when the displays were inverted and object information was removed (Hemeren & Thill
2011). On the complex end, a recent study of the relationship between editing techniques
in commercial film and segmentation revealed that both changes in the narrated situation
(shifts in space, characters, and time) and editing transitions (dissolves, fades, and wipes) are
associated with higher levels of segmentation (Cutting 2014).
Most previous studies of visual event segmentation have presented activity from a third-
person perspective. Two recent studies have explored the first-person perspective. Magliano
and colleagues (2014) presented movies recorded from a first-person video game and found
good segmentation agreement, replicating what is seen with other stimuli. Also replicating
previous results, event boundaries tended to occur when actors’ goals changed. Swallow
and colleagues (2018) directly compared first-person and third-person perspectives, using
two cameras, one head-mounted and one tripod-mounted, to record everyday activities. They
found strong agreement between segmentation from the two perspectives.
Event representations have internal structure
The events that are distilled from the stream of behavior are not themselves undifferentiated
clumps. Rather, they have considerable structure within them. An event unfolds within a
spatiotemporal framework and includes entities such as people and objects and the relations
amongst them. The representation of this structured segment of experience is called a
working event model (Radvansky & Zacks 2014). (I will reserve the term “working event
model” for representations that are immediately accessible, whereas the broader term “event
model” includes immediately accessible representations and representations in long-term
memory.) Working event models are perceptual representations, because they represent
currently-unfolding activity. At the same time, they are also working memory representation,
because they maintain information throughout the duration of an event. There is strong
converging evidence that event models are rapidly established during event comprehension
and are supported by specialized neural mechanisms that are distinct from representations of
event components (Stawarczyk et al. under review).
For events that are the perception of a human action, the configuration of an agent (who is
doing the action), a patient (to whom or what is the action being done), and an action (what
action is being performed) is particularly important. Observers can extract this configuration
from pictures presented as briefly as 37 ms and subsequently masked (Hafri et al. 2012),
and information about the roles of agent and patient is quickly bound to other features of
Zacks Page 3
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
the entities involved (Hafri et al. 2018). Information about an entity’s role (Hafri et al.
2012) and what it might do (Cohn et al. 2017) is often available from its pose: People
acting as agents generally have more stretched-out body poses than those on whom they
are acting, and the windup to a kick appears quite different from the windup to a throw.
There appears to be a regularity to the time course of establishing an event configuration,
with agents usually being identified first (Webb et al. 2010). A key feature of agents is
that they are animate, and humans appear to have specialized perceptual mechanisms to
identify animate agents based on static features and movements. Neurophysiologically, these
mechanisms draw on brain areas including ones in the ventral temporal cortex specialized
for representing face and body features, on areas in the lateral temporo-occipital cortex
specialized for movement processing, and on networks associated with theory of mind
(Heberlein 2008). In the perception of comic strips, viewers look at agents longer than
they do patients, and are more willing to make predictions about what will happen in a
subsequent frame based on pictures of agents than of patients (Cohn & Paczynski 2013).
In short, event perception rapidly establishes the configuration of agent, patient, and action,
with agents having primacy.
When comprehenders process information about the action itself, they appear to focus on
the action’s goal rather than on the particulars of its surface structure. Goal focus is evident
early in infancy; studies using imitation and habituation paradigms have shown that infants
attend to adults’ goals at the expense of the specifics of their movements (Woodward 2009).
By age four, children can better recognize pictures depicting the goal of an action than other
pictures, and this advantage is present in adulthood (Papafragou 2010). Adults are better able
to detect changes in movie clips when those clips change goal-relevant movement features
than when they change features that are not associated with the action’s goal (Loucks &
Pechey 2016). This goal processing contributes to observers’ ability to make predictions
about upcoming action. For example, observers’ eye gaze anticipates which object is the
goal of an actor’s reach, such that the eyes land on the target object well before the actor’s
hand arrives; this is true both for observers and for actors themselves (e.g., Eisenberg et
al. 2018; Flanagan & Johansson 2003; Hayhoe & Ballard 2005). One important aspect of
goals is that they are a component of event structure to which observers and actors have
quite different access. Someone performing an action has access to information about their
goals and plans that observers cannot access (though such information is by no means
without error). Differences in the information available to actors and to observers can lead to
differences in event segmentation (Fournier & Gallimore 2013).
Building a working event model requires cognitive work. This can be seen in slower
processing of event boundaries in reading (e.g., Pettijohn & Radvansky 2016) and self-paced
slide shows (Hard et al. 2011). In language processing, priming studies suggest that reading
a word naming one part of an event primes other components of that event. For example,
verbs prime the agents, patients, and instruments of their associated events (Ferretti et al.
2001), and are in turn primed by those components (McRae et al. 2005). There is evidence
that the effort to build event structures is re-used when possible: Successive sentences
sharing a goal-subgoal decomposition were read more quickly than those that did not
share goal-subgoal structure (Allen et al. 2010). Event representations can have more or
less complex internal structure, and building more complex structures may place unique
Zacks Page 4
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
demands on the system. For example, if one reads that “The chef will weigh the onion.
And then she will smell the onion,” the onion is in the same state throughout. However,
if one reads that “The chef will chop the onion. And then she will smell the onion,” the
onion changes from intact to chopped, requiring the representation of two object states.
Solomon and colleagues (2015) compared the processing of these two types of passages
during fMRI scanning. They focused on a region of the ventrolateral prefrontal cortex that
is selectively active during instances of representational conflict, and found that it was more
active when objects needed to be represented in two states. Data from eye-tracking and
EEG measures show that during the processing of event descriptions, information from the
visual environment is integrated rapidly and continuously (e.g., Altmann & Mirkovic 2009;
Knoeferle et al. 2008). This supports the proposal that event comprehension operates on a
common representational format that integrates information from multiple sources including
language and perception.
Event perception is structured by partonomy and other relations
Event models not only have internal structure; they also form rich connections across events.
Partonomic hierarchy (see Table 1) can be thought of as either an aspect of within-event
structure or in terms of relationships across events: Viewed “downward” from an event
to its sub-events, the parts are aspects of the within-event structure. Viewed “upward”
from an event to the larger event of which it is a part, the sibling and parent events are
cross-event relations. For goal-directed human activity, the part-subpart structure of events
is conditioned strongly on the goal-subgoal structure that generates action. Hierarchical
organization is ubiquitous in event segmentation data and in descriptions of events (Zacks et
al. 2007). It appears to increase with familiarity with a particular event sequence (Hard et al.
2006) and with domain expertise (Bläsing et al. 2009).
Hierarchically organized event models are more complex than simple strings, but they retain
the constraint of being continuous. An activity such as “walking a dog” may consist of
sub-parts such as “putting on shoes,” “opening the door,” and “walking up the block,” but
all of the temporal parts of the activity occupy a continuous stretch of time. Some activities,
however, appear to form discontinuous events—for example “training for a marathon,”
or “taking a calculus class.” Kubovy (2015) proposes that the mental representation such
discontinuous events constitute projects that cohere despite their temporal discontinuity.
Projects in turn are subcomponents of strands, which are large-scale structures such as
“sports” or “school,” which are themselves temporally discontinuous. On this view, projects
and strands run through lives as do strands through a weaving. An important question for
future research is whether such structure is present during online perception of ongoing
activity or is a construction established as part of the act of remembering.
Event perception is heavily informed by knowledge
When someone experiences a particular event, they can bring to bear knowledge distilled
from their many experiences with varying event types. A longstanding proposal is that event schemas or scripts capture knowledge about the temporal structure of events, entities, roles,
and locations (Abelson 1981). How a perceiver understands a particular moment depends on
how they can bring such knowledge to bear on the immediate stimulus. By monitoring eye
Zacks Page 5
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
movements while participants watched excerpts from a feature film, Loschky and colleagues
(2015) showed that viewers’ looking patterns were more systematic when the clip they were
viewing could be placed in a larger context. Knowledge about event categories can affect
perception rapidly and automatically, as can be seen in a recent study of change detection
(Strickland & Scholl 2015). Viewers were shown clips in which one object either went into another object or behind it. Participants were better able to detect a change an object’s
width than its height when the object was shown going into a vertical container; when the
object was going behind the other object there was no difference. This suggests that viewers’
visual systems tracked object width more carefully when viewing the event type—vertical
containment—for which width is relevant.
The opportunity to deploy knowledge during event comprehension is associated with
specific neural mechanisms. A study using multivoxel pattern analysis fMRI identified
brain areas whose activity patterns distinguished between schemas for eating at a restaurant
or flying from an airport (Baldassano et al. 2018). The analysis targeted areas that had
consistently different patterns for the two schemas across multiple movie clips and story
excerpts. Schema information was represented in medial components of the default mode
network: medial PFC, superior frontal cortex, and medial posterior cortex. Another fMRI
study of movie viewing found that presenting clips with a context that allowed for a schema
to influence processing was associated with sustained fMRI responses in the inferior frontal
cortex, middle temporal gyrus, and angular gyrus, all in the left hemisphere (Keidel et al.
2017). In contrast, when encoding under schema-poor conditions the parahippocampal gyrus
and retrosplenial cortex were more strongly activated. One possibility is that the schema
representations are stored in the medial network, and the left lateral network plays a role in
deploying those representations to guide ongoing processing.
Event perception is predictive and inferential—One thing that knowledge does for
event perception is to enable predictions about how events will unfold. If one is familiar
with going to rock concerts, one can predict that after the lights go down the band will
take the stage. Even in situations where perceivers have little relevant schema knowledge,
their brains may make predictions based on more general knowledge such as knowledge
about how bodies and objects move, about the relationships between goals and actions, and
the like. In scene perception, Intraub and colleagues have shown that viewers quickly fill
in information outside the field of vision, leading to systematic errors in picture memory
(Intraub 2010). One kind of knowledge that can support strong inferences is knowledge
about the causal consequences of actions. For example, kicking a soccer ball leads to the ball
moving. When viewers see a sequence of video clips that show a player running up to a ball
and then show the ball in flight, they often falsely recognize a clip showing the moment of
contact that was not presented, after even quite brief delays or even when asked to monitor
for the moment of contact while watching (Papenmeier et al. 2019; Strickland & Keil 2011).
The predictive nature of event perception can be seen in the previously-discussed fact that
viewers look ahead to the goals of reaching motions (Eisenberg et al. 2018; Flanagan &
Johansson 2003). Viewers can learn to predict even arbitrary, randomly-determined behavior
sequences if they are given repeated experience, and this can allow them to look ahead to
the targets of actions (Monroy et al. 2018). Predictive processing in event perception also
Zacks Page 6
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
can be seen in evoked EEG responses, particularly the N400 (for a review, see Amoruso
et al. 2013; but see also Brouwer et al. 2012). In paradigms that use video editing, picture
presentation, or language to tightly control the timing of event stimuli, researchers have
shown that unpredicted happenings in event comprehension produce larger N400 responses.
There is a tight temporal relationship between prediction and event segmentation. The more
unpredictable activity becomes, the more likely participants are to identify event boundaries
(Huff et al. 2014). Making explicit predictions about the near future is more difficult
near event boundaries, and attempting to do so is associated with increased fMRI activity
in midbrain structures associated with signaling prediction error to the rest of the brain
(Zacks et al. 2011). Predictive eye movements are also less prevalent near event boundaries
(Eisenberg et al. 2018).
Events structure consciousness
The predictive nature of event perception illustrates a point highlighted by William James
(1890): The conscious experience of the present is not an infinitesimal point but has
temporal extent. The fact that event segmentation tasks are so easily learned, produce
such reliable data, and correspond with ongoing neural events supports the conclusion that
conscious experience is segmented into events (Zacks et al. 2007). The time course of
other functions provides converging evidence. When participants are asked to monitor a
complex situation for a visual target, the presence of an event boundary transiently disrupts
performance (Huff et al. 2012). Consistent with this result, mind-wandering while watching
a film was less likely at event boundaries than during the middles of events (Faber &
D’Mello 2018). These findings suggest that model updating is resource-intensive: at event
boundaries, those resources are less available for secondary tasks or off-topic wanderings of
the mind.
Summary
In sum, perception is characterized by the segmentation of ongoing experience into
meaningful events. Event representations have rich internal structure, and also rich
relationships between them. The construction of these relationships is strongly influenced
by knowledge, which enables online predictive inference about the unfolding of activity.
The construction of working event models and the predictive nature of event perception
highlight that the experience of “now” is smeared in time—there is no bright line between
now, a moment ago, and a moment hence. This entails that there is no bright line between
perceptual systems and memory systems (Christophel et al. 2017). A more fundamental
distinction than that between perception and memory is that between the “expanded present”
represented by a working model and events outside of that expanded present.
From perception to memory
Rubin and Umanath (2015) define event memory as “the mental construction of a scene,
real or imagined, for the past or the future.” In other words, we can distinguish between
representations of events that one is currently experiencing and representations of events
that are based on other sources; event memories are the latter. One attractive feature of this
Zacks Page 7
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
conception of event memory is its emphasis on the constructive nature of memory. Another
attractive feature is its acknowledgement that when one attempts bring an event to mind,
the result depends on multiple signals and active strategic mechanisms; event memories
are constructions rather than the mere reloading of previous event representations stored
immutably in the brain’s vault.
This approach entails that experiencing a current event and remembering a previous event
are underwritten by a common representational substrate: a working event model. The
difference is that in perception we focus on how one’s current working model is populated
by currently-available perceptual information, whereas in remembering we focus on how
one’s event model is populated by retrieval of information from long-term memory. This
analysis offers a novel take on the relationship between perception, working memory, and
long-term memory.
Working memory representations are updated at event boundaries
Most theories of human memory distinguish between short-term (or primary or working)
memory and long-term memory. An outstanding puzzle has been: what makes a memory
long-term? The data presented in the previous section, in supporting the proposal that the
brain maintains working event models and updates them periodically, suggests that in many
cases the boundary between working memory and long-term memory may be the end of
one event and the beginning of the next. Striking support for this proposal has come from
a series of studies by Radvansky and colleagues on the memorial consequences of walking
through doorways. In the initial experiments (Radvansky & Copeland 2006a; Radvansky
et al. 2010), participants navigated a virtual reality environment in which they picked up
objects and put them in a backpack, and later put down the objects. From time to time,
they were probed to report which object was currently in their backpack. Controlling for
distance traveled and time elapsed, participants were slower to respond if they had walked
through a doorway after picking up an object. This effect has been found not just in virtual
reality but also in real rooms (Radvansky et al. 2011) and in imagined ones (Lawrence &
Peterson 2016). It does not appear to reflect simply a temporary disruption due to crossing
the doorway because it persists when a constant delay is added to the retention interval
(Pettijohn & Radvansky 2015); nor does it seem to be due to the dissimilarity in context
between the encoding and retrieval conditions because it remains when participants return to
the original room (Radvansky et al. 2011). These interactive paradigms converge with results
from studies of narrative reading (e.g., Rinck & Bower 2000), and with studies of memory
for recently-seen objects in movies (Swallow et al. 2009), which show that after a shift in
location or of narrative time, memory retrieval is often less efficient.
Updating event models at boundaries would seem to be a resource-intensive operation. It
may include comparing the contents of the old model with new perceptual information.
Evidence supporting this idea comes from a study showing that, while viewing films,
changes in actors’ clothing were noticed better at event boundaries than event middles
(Baker & Levin 2015). If information that is present at event boundaries receives special
processing, this may have consequences for later retrieval. Using fMRI, Swallow and
colleagues (2011) showed that successful retrieval of visual object information encoded
Zacks Page 8
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
during an event boundary was associated with selective activity in the hippocampus and
parahippocampal gyrus—structures associated with long-term memory in many studies in
humans and other species. The delay in these studies was only 5 s, which supports the notion
that what makes a memory long term may be whether it requires reaching back across an
event boundary.
Thus, an event boundary may involve the segregation of the immediate present—subserved
by working event models—from events in long-term memory. A natural implication of the
hierarchical structure of activity, described in the previous section, is that event model
updating happens on multiple timescales; some updates may only affect finer-grained
event models, whereas others additionally update coarser-grained event models. If so, then
updating at coarser-scale event boundaries should have larger effects on memory retrieval
and brain activity. This hypothesis has yet to be tested.
Event boundaries also may condition what sorts of representations are formed in long-term
memory. Recent studies of memory retrieval with longer delays have provided evidence
that the experience of event structure in the present leaves heavy footprints on subsequent
memory.
The structure of events in experience is mirrored in long-term memory
One idea that has become dominant in the psychology of human memory holds that the
experience of remembering is fundamentally one of intentionally searching for information
in the mind. While there is no doubt that people sometimes go into a mode of deliberate
memory search (Tulving 1983), it may be that a more basic and more frequent mode of
experiencing event memories results from involuntary associative cuing (Berntsen 2010).
Features such as locations, people, or objects may quickly and automatically bring to
mind events that are associated with those features. For example, a child, upon arriving
in a classroom on Tuesday morning, might retrieve events of Monday’s class in that
room, and upon encountering a friend one might retrieve features of the last conversation
had with them. Such associative retrieval is adaptive: If a new event induces memories
of similar previous events, those previous events are likely to provide valid predictive
information about how the new event will proceed. But what if things change? What if
there is a substitute teacher and as a result the events of yesterday’s lesson do not predict
what will happen today? In such cases, event memory may impair rather than facilitate
comprehension. However, if the child can register the discrepancy as part of the memory for
Tuesday, this can result in a highly effective form of memory that embeds what happened
on both days, the temporal relationship between them, and something about the variability
associated with that context. A recent study of memory for changes in events provides
evidence for such mechanisms (Wahlheim & Zacks in press).
Event memory is structured by the same dimensions as perception
What is the relationship between event structure in perception and in memory? There is
strong evidence that the segments that are identified during event perception correspond
to the representational units in subsequent memory. First, the boundaries themselves are
remembered exceptionally well. For example, Huff and colleagues (2014) showed viewers
Zacks Page 9
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
episodes of a situation comedy, asked them to segment them, and tested their memory later.
Event boundaries tended to occur at points in time when many features were changing, and
those points were remembered better. Ezzyat and Davachi (2011) used a narrative priming
paradigm to show that sentences cued memory retrieval of subsequent sentences more
strongly when the two belonged to the same narrative event than when they did not, and
that the binding of sentences within an event was associated with selective activity in brain
areas including the ventromedial prefrontal cortex. Pettijohn and colleagues (2016) showed
that breaking up the experience of studying a word list into two events (by walking through
a doorway) increased the total number of studied words that were remembered later. They
interpreted this manipulation as decreasing the degree to which words from the first half of
the study experience interfered with those from the second half, and vice versa.
Such data support the existence of integrated event models in memory. However, event
memory also depends on other representations—for example, representations of people and
objects. Furthermore, even bound event models can be confused with similar events during
memory retrieval. When one attempts to remember a past event, one constructs a working
model that depends on these confusable event representations and other long-term memory
representations, interference between similar representations can lead to confusions. For
example, when people see a number of actors involved in a number of actions, confusions
about which person performed which action are common (Earles et al. 2008; Kersten et al.
2013).
Just as there are structural relationships across events in perception, there are relationships
across events in memory. One of the strongest organizers of relationships in event memory
appears to be causal connection (Radvansky 2012). In fact, the organization of memory by
causal relations is likely stronger in memory than in perception, because post-encoding
processes elaborate causal relations. When people read narratives of event sequences,
components with more causal connections are remembered better (Trabasso & Stein 1997).
When people are asked to recall multiple events from distant periods in their lives, events
that are causally related tend to cue each other (Brown 2005).
Spatial location is another powerful organizer of events in memory. It is much easier to
remember the association between multiple objects and a single location than the association
between multiple locations and a single object (e.g., Radvansky et al. 2017). For example,
it is easier to remember that a potted plant, an ATM, and a poster are in a lobby than it is
to remember that there is a potted plant in a lobby, a library, and a café. This suggests that
a collection of objects in a location can be represented in a single event model, whereas
representing the same object in multiple locations requires multiple event models.
Time is also a potential organizer of events in memory. Just as it is easier to associate
multiple objects with a single location than multiple locations with a single object, it is
easier to associate multiple objects with a single time period than multiple time periods with
a single object (Radvansky et al. 1998). Hierarchical organization, prevalent in perception,
is also prevalent in autobiographical memory. The same autobiographical cueing procedure
that reveals causal relations in memory also shows clustering of events by membership in
larger events (Brown 2005). If events are organized into part-subpart hierarchies in memory,
Zacks Page 10
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
this entails representing a certain amount of information about temporal distance, because
two events that are both part of the same larger event will generally be closer in time than
two events that are parts of different larger events. In addition, causal relations can scaffold
memory for temporal order because causes precede their effects. However, some aspects of
temporal organization, such as the order of events within a larger unit, can be quite weak
in memory. For example, Wyer and Bodenhausen (1985) found that people recalling stories
showed good memory for the order of actions within an event (possibly because these tend
to be causally linked) but poor memory for the larger order of events.
For both space and time, there are a range of scales on which things can be organized.
Temporal and spatial scales that have natural organization are probably represented—or
reconstructed—better than those that are more arbitrarily organized, because those natural
organizations facilitate prediction. For example, the arrangement of objects within a room
is systematically related to the locations of doors and windows, and to the actions that take
place within the room. However, the order of rooms on a hallway or of buildings on a
street is much less consistently tied to objects and actions. This may be why, after watching
many episodes of a television series, people have relatively good memory for the layout of
rooms but poor memory for the spatial relations amongst rooms (Levin 2010). Similarly,
the order of actions in making a sandwich is structured by causal and conventional relations
that can facilitate predictions, but the order of which larger activities may precede or follow
sandwich-making is less systematic (Wyer & Bodenhausen 1985). Gravina and Sederberg
(2017) suggest that these relationships of systematicity and the predictions they allow for
account for the temporal and spatial similarity gradients seen in memory representations in
the medial temporal lobes (Nielson et al. 2015).
Memory for the duration of events is often important for planning future activities—and
it is fascinatingly affected by features other than actual duration. In spatial navigation,
routes that have more turns are remembered as being longer in space and also are mentally
replayed more slowly (Bonasia et al. 2016). This appears to be an instance of a more general
phenomenon: Intervals more filled with “stuff” are remembered as having taken longer when
looking back on them. Wang and Gennari (2019) showed viewers animations, asked them
to describe the animations, and then asked them to recall their duration. They found that
those animations which elicited more extensive descriptions were remembered as having
taken longer, controlling for actual duration. Similarly, routine events are usually described
less richly than unusual ones, and such events are remembered as having been shorter
(Avni-Babad & Ritov 2003). What is the “stuff” that goes into remembering duration? At
least one source appears to be the number of sub-events in an interval. Jeunehomme and
D’Argembeau (2018) asked participants to perform activities of daily living while wearing
cameras, to segment their recordings, and then to estimate the durations of particular
activities. They found that those activities which were segmented into more events were
remembered as longer, controlling for actual duration. Finally, Bangert and colleagues (in
press) observed a similar effect of event boundaries on judgments of the durations of
intervals within movies of everyday activity.
Zacks Page 11
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
Event memory, like perception, is heavily informed by knowledge
The fact that relationships within and amongst events are systematic means that they can be
the subject of knowledge. We saw previously that knowledge affects event perception. The
effects of knowledge on event memory are even more striking. Inspired by the influence of
scripts and event schemas on memory for narrative text, Brewer and colleagues conducted
several studies showing that memory for filmed events is influenced by knowledge about
event categories (e.g., Brewer & Dupree 1983). Information that is congruent with an event
schema is often remembered better than information that is irrelevant, in part because the
schema can act as a bias during recall. For example, if one were asked to remember whether
at a wedding the couple recited vows, one could respond based on a schema for weddings in
general in addition to representations specific to that event. However, suppose the couple did
not recite their vows, but instead had them tattooed onto their forearms. This might also be
memorable, and indeed under many circumstances such discrepant information is also well
remembered compared to schema-unrelated information—for example, the kind of car one
took to the wedding as a guest. In a recent study, Bonasia and colleagues (2018) found that
movie clips that were either congruent with a schema or discrepant were both remembered
well compared to clips that were not strongly related to a script. Remembering schema-
congruent clips led to more medial prefrontal fMRI activation, whereas remembering
discrepant clips led to more medial temporal activation. This suggests that the influence
of schemas on event memory may be mediated in part by the medial prefrontal cortex, and
that the medial temporal lobe system is particularly taxed when memories need to bind
arbitrary relations. In most studies of event comprehension and knowledge, the schemas in
question are assumed to be well-learned. (We will see exceptions to this in the study of child
development.) However, recent evidence suggests that event knowledge can be acquired
from a few experiences (MacLean et al. 2018). In this study, participants tasted various
foods in the laboratory, with a different experimenter conducting each tasting. After just a
few of these events, a new food tasting experience in which the experimenter’s behavior
deviated from the others was better remembered.
Remembering events is constructive and destructive
If event memory is the mental construction of a scene (Rubin & Umanath 2015), this
suggests that the act of searching one’s memory for a particular episode is not so
different from imagining a novel event or thinking about a counterfactual event. Indeed,
neuroimaging and neuropsychological data strongly support a common mechanism for
constructing events from memory, imagination, and reasoning (Addis et al. 2007; Schacter
et al. 2012). A reasonable proposal is that all of these tasks depend on constructing an
event model and their common neural correlates reflect either the representational medium
of event models, the systems that are needed to construct such representations, or both.
Barbey and colleagues (2009) used fMRI to examine the mechanisms of counterfactual
reasoning and found that multiple dimensions of events were represented in the spatial locus
of activation in the medial prefrontal cortex. This is consistent with the results of Baldassano
et al. (2018), who found schema-specific patterns in this area during movie viewing.
The coupling of this region with other default network regions during event encoding is
associated with better subsequent memory of narrative sequences (Simony et al. 2016); This
Zacks Page 12
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
suggests that the common activations seen in event perception, event imagining, and event
memory tasks reflect a functional, causal relationship.
The view of event memories as constructive also entails that remembering is reconstructive
and destructive. That is, when one constructs an event memory, the operations of retrieval
and the activation of knowledge during that retrieval affects one’s future remembering. For
events, this can sometimes produce straightforward benefits to subsequent memory: For
example, after viewing movies of everyday events, being reminded of them with pictures
or titles reduces forgetting. However the same reminding also can impair memory for
other events that were not reminded (Koutstaal et al. 1999). Eye witness memory studies
show how a given retrieval attempt can produce negative effects on subsequent memory:
For example, retrieving an event while trying to decide whether a mug shot matches
the perpetrator of the event can lead the person pictured to be falsely incorporated into
subsequent memories for the event (e.g., Kersten & Earles 2017). Post-encoding operations
may affect not just the contents of long term memory event models, but also how activity is
segmented in long term memory (Hohman et al. 2013).
Summary
In sum, although there is no bright line between perception and memory, there is
good evidence for a distinction between information that is maintained in one’s current
working event models and event information that is represented in other neural systems.
Event memory can be conceived as the construction of a working model based on
that other information—including episode-specific representations, knowledge, and new
information generated during the construction process. The act of constructing a working
event model creates new representations in these memory systems, which side-affect
subsequent remembering. These operations have specialized neural mechanisms. The
PM/AT framework (Ranganath & Ritchey 2012) summarizes and integrates current
knowledge about these mechanisms. It proposes that one brain network, including the lateral
temporal cortex and perirhinal cortex, supports the use of object knowledge and perceptual
features in memory formation. Another network, including the medial prefrontal cortex,
retrosplenial cortex, and parahippocampal cortex supports the organization of entities within
a spatial framework into an event model, drawing on event schemas. This account fits well
with the neurophysiological data reviewed here.
Group and individual differences in event cognition
There are substantial group and individual differences in event perception and event
memory, which can inform theories of their mechanisms (Zacks & Sargent 2010). Here,
I will focus on group-level effects of healthy aging and early Alzheimer’s disease, and of
language, and on how individual differences in event perception relate to differences in
memory.
Zacks Page 13
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
The experience of events changes through adulthood, and is impaired in Alzheimer’s disease
We saw previously that event segmentation is characterized by agreement across observers
and by hierarchical organization (“Segmentation of continuous experience into events”).
Both of these features can be used to develop measures of group and individual differences.
When asked to segment movies of everyday activities, older adults usually show lower
segmentation agreement than younger adults (e.g., Kurby & Zacks 2018; but see Sargent et
al. 2013). Older adults also show less hierarchical organization (Kurby & Zacks 2011).
Interestingly, a recent study of the segmentation of filmed everyday events found no
difference in segmentation agreement or alignment between young and older adults (Kurby
& Zacks 2018). One possibility is that age differences in event segmentation may be reduced
when older adults can use knowledge to construct richer event representations (Radvansky &
Dijkstra 2007). Consistent with this proposal older adults, like younger adults, show robust
updating at situational changes in narrative memory updating paradigms (Radvansky et al.
2003).
Early Alzheimer’s disease reflects a divergence from the path of healthy aging that is
characterized by impairments in memory and thinking, which can initially be subtle but
increase in severity and scope with disease progression. These include deficits in event
segmentation and memory (e.g., Bailey et al. 2013b). They also include deficits in the
ability to perform everyday activities, in and out of the laboratory (e.g., Giovannetti et al.
2008; Gold et al. 2015). One possibility is that these deficits result from disruption of event
knowledge, or of the ability to use that knowledge effectively.
Language affects some aspects of event perception and memory—especially when using language
The language one speaks is a group difference that is of particular interest for perception
and memory in general due to debates over whether and how language shapes thought.
The effect of language on event perception and memory has attracted sustained attention
because different languages represent aspects of event structure differently. For example,
languages including English and Arabic encode in the form of their verbs whether an
activity is viewed as ongoing over time (“is walking”) or as a whole (“walks”), whereas
German does not. Because of this grammatical difference, German speakers are more likely
to explicitly describe the locations of the endpoints of actions than are speakers of English
or Arabic. (This is because whole actions tend to make less sense without their endpoints,
whereas ongoing actions make sense with or without endpoints; for example “Stacey walks”
sounds a little odd without a destination specified, whereas “Stacey is walking” sounds
fine.) Corresponding with this linguistic difference, German speakers look more at action
endpoints than do Arabic speakers (Flecken et al. 2014), and show larger P3 EEG responses
to unexpected action endpoints than do English speakers (Flecken et al. 2015), even in tasks
not involving language. However, such differences are not always observed. Papafragou
and colleagues conducted several tests comparing Greek, which highlights path information
more, to English, which highlights path information less, in designs similar to those of
Flecken and colleagues. In these studies, no effects of language were found unless language
was being used (Papafragou 2010; Papafragou et al. 2008; see also Gennari et al. 2002).
Zacks Page 14
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
In addition to affecting attentional selection, language can affect segmentation. French verbs
are more likely to represent motion paths more than do German verbs. For example, in a
situation where a French speaker might say “La voiture a traversé le pont” (roughly, “The
car crossed the bridge”), a German speaker would more likely say “Das Auto fuhr über
die Brücke” (“The car drove over the bridge”). Whereas French verbs more often describe
path (traverse/cross), German verbs more often describe manner of motion (fuhr/drive),
conveying path information in a preposition (über/over). Given that information in a verb
is obligatory, one might expect that changes in path would be more likely to lead to event
model updating for speakers of French than of German; this has indeed been observed
(Gerwien & von Stutterheim 2018).
The potential for language to influence event segmentation also can be seen in the gestures
that speakers use to talk about events. For example, Avatime is a language that can use serial
verbs to package a string of smaller actions into one event. For example, the sentence “lɛ̌
be-dzì e-mu-i” (roughly, “then return ascend”) means “Then they climbed up again.” When
Avatime speakers use this construction, if they gesture while uttering a serial verb string the
gesture spans the entire verb string, suggesting they conceive of it as a single unit (Defina
2016).
There are strong relationships between individual differences in event perception and in event memory
In addition to evidence for group differences, there is evidence for substantial individual
differences in event perception—and individual differences in perception predict individual
differences in memory. In a large-scale study of event segmentation and memory across
the lifespan, people with higher segmentation agreement had better subsequent recall and
recognition for events (Sargent et al. 2013). This held after controlling for individual
differences in processing speed, working memory, crystallized knowledge, and laboratory
episodic memory. Event knowledge also was a significant independent predictor of event
memory. Within older adults, including those with early Alzheimer’s disease, better
segmentation is associated with better event memory and with better ability to perform
everyday actions (Bailey et al. 2013a,b; Kurby & Zacks 2011). Experimental interventions
on event segmentation suggest that the relationship between event perception and event
memory is causal: Interventions that improve event segmentation by instruction or by editing
event stimuli improve subsequent event memory (Flores et al. 2017; Gold et al. 2016).
In short, event perception and event memory vary across individuals and groups.
Moreover, differences in these two abilities are related to each other and also to other
domains including everyday action performance. Intervening to improve event segmentation
improves event memory, which is consistent with the previously-discussed evidence
that segmentation during perception shapes the representational units of event memory.
Differences between young and older adults highlight the importance of adult development
in event perception and memory. But of course, these abilities do not emerge fully formed in
18-year-olds. How do they develop in childhood?
Zacks Page 15
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
The development of event cognition reflects growth in knowledge, language use, and deliberate rehearsal
For many years, Piagetian accounts dominated the study of event understanding and event
memory. These held that infants possess little of the conceptual structure to support adult-
like performance, and that this conceptual structure emerges incrementally over years of
experience in the world. Modern views, however, identify early competence in areas of both
event perception and event memory, as well as extended growth. One important aspect of
growth in perception and memory, which was noted by early theorists as well as current
ones, is growth in schematic knowledge about events (Bauer 2006).
Psychology’s picture of the development of event perception is much hazier than its picture
of the development of event memory. There is substantial evidence for early competence
in infancy, but little data on how infant abilities develop into those of adults. Well within
the first year of life, infants can individuate actions within a continuous stream of behavior
(Sharon & Wynn 1998; Wynn 1996), and show evidence of segmenting activity at points
that are identified by adults as event boundaries (Baldwin et al. 2001; Hespos et al.
2009, 2010; Saylor et al. 2007). Infants are sensitive to causal interactions such as in the
Michotte launching effect (Cohen & Amsel 1998), and can use experiences with recent
events (Nakano & Kitazawa 2017) and the statistics of extended experience with event types
(Monroy et al. 2017) to guide their ongoing processing of current events. However, the
infant studies have used almost completely different measures of event perception than have
adult studies, and little is known about the developmental trajectory of event perception
between infancy and young adulthood.
Much more is known about the development of event memory. This is true, in part, because
of a strong applied interest in what children can remember for the sake of legal testimony.
Whereas older views held that children lacked the ability to form event memories before the
advent of language, a new generation of research has found evidence for early competence
in event memory formation and continuity of development through the early phases of
language acquisition (Bauer 2006). By 18–24 months, toddlers can recall elements of an
otherwise-forgotten event if they are cued with features of that event (Sheffield & Hudson
1994); this indicates that the elements are bound into a coherent whole. At the same
age, toddlers show better memory for event boundaries than for event middles, and show
selective memory impairment from occluding event boundaries during encoding (Sonne
et al. 2016, 2017). From ages 2 to 10, the development of event memory is linked to
growth in event knowledge (Fivush 1997; Hudson et al. 1992). Young children tend to
misremember events as having conformed to the schemas they have learned, but by 7
or 8 years, children have more ability to recall deviations from schemas and to recall
schema-irrelevant details, especially with environmental support (Brown & Pipe 2003). One
important aspect of children’s event knowledge is goal relations, which have strong early
and continuing effects on event memory (Loucks et al. 2016). By age 10, children’s event
memory looks qualitatively like that of adults, though encoding efficiency and completeness
continue to increase with age. Together, the limited perceptual data and more extensive
memory data indicate that very young children make use of event models that are in some
Zacks Page 16
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
ways quite adult-like, but that these become more elaborated over development, in part due
to the development of knowledge structures.
Summary
In short, contrary to earlier views, there is good reason to think that infants construct
event representations that are similar in form and content to those of adults, though more
limited in many aspects. The development of event memory is conditioned strongly on the
development of knowledge about event classes. An important issue for future research is the
role of knowledge and other factors in the development of event perception.
Theoretical approaches to event cognition
In previous generations, theories of event cognition tended to deal with perception (Gibson
1979; Johansson et al. 1980; Michotte 1946), memory and inference (Abelson 1981), or
action control (Miller et al. 1960). The memory and action control theories emphasized
structure in mental representations, whereas the perceptual theories emphasized structure in
the environment. In contrast, current theories tend to bridge at least two of these domains,
and to consider structure both in the mind and in the world.
Common, multimodal coding of events and actions
One problem taken on by current theories is the relationship between people’s roles as
perceivers and as actors in the stream of events. One such theory, which has been highly
influential, is the Theory of Event Coding (TEC) proposed by Hommel and colleagues
(2001). TEC is a qualitative account of how perception and action control are integrated
in events on a short time-scale, from tens of milliseconds to a few seconds. It can be
described in terms of a set of proposals about how immediate events are represented, listed
in Table 2. The third and fourth proposals state that event representations are composites of
feature codes. The range of possible codes is determined by previous experience encoded as
knowledge. For example, most animals will have access to codes such as “red” and “short”
to characterize a cup, but as one interacts with a cup one might add codes for its previous
location and the level of liquid inside. The final proposal governs how event representations
can be shaped by current interests: If one intends to pick up a cup to drink from it, shape
features will be highly weighted; if one is selecting a cup to purchase in a gift shop, features
related to attractiveness and desirability will receive more weight.
A key set of findings accounted for by TEC involves situations in which actions or intentions
interact with perception. Planned actions can affect how visual or auditory stimuli are
weighted, and irrelevant features of visual or auditory stimuli can affect action execution.
TEC’s proposal that feature codes are first activated and then bound has the advantage
that it can account for both facilitation from overlapping features and also for interference.
However, in any given situation it is often not clear whether to predict facilitation or
interference. As evidence has accumulated, it appears that additional mechanisms are
needed to account for the complex patterns of facilitation and interference that are observed
(Zwickel & Prinz 2012).
Zacks Page 17
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
One of the most grounded ways to build a model of cognitive representation for perception
and action is to explicitly build a controller for an agent. The REtrospective and PRospective
Inference SchEme (REPRISE) model (Butz et al. 2018) does just that. REPRISE is a
recurrent neural network that uses a bank of contextual neurons to represent which of
a number of potential event states the network is currently experiencing, and to bias
processing in the rest of the network based on the currently-inferred event state. The model
identifies which state it is in by looking backwards at the recent dynamics of the system,
and uses the current state and the values of the physical variables describing the environment
to predict the consequences of its actions. Butz and colleagues applied REPRISE to a
simulated environment in which the model learned to control a set of vehicles, and the
vehicle it was driving could be changed without warning. Thus, at any given time, the
model had to simultaneously infer which vehicle’s dynamics were in play and had to drive
the vehicle. It could learn to identify vehicle changes (event boundaries) and to update its
contextual neurons appropriately, improving driving performance.
Event segmentation and the formation of event memories
The application of REPRISE to detecting state changes illustrates the importance of
modeling temporal structure in activity. Event segmentation theory (EST; Zacks et al.
2007) proposes that event segmentation occurs as a side-effect of constructing working
event models that improve perceptual prediction. EST starts from a perceptual processing
stream that takes a representation of the current state of the world (which may include
perceptual information, language, and other sources) and produces predictions about what
will happen a short time in the future. Such predictive processing is assumed to be an
ongoing component of comprehension, which facilitates more effective and timely behavior.
This predictive processing stream is modulated by a working event model that maintains
a stable representation of “what is happening now.” The architecture retains its current
working model as long as prediction error is low, and updates its working model when
prediction error spikes. A simplified version of EST was implemented as a gated recurrent
network by Reynolds and colleagues (2007). The model was trained using back-propagation
on a series of inputs representing the position of an actor’s body over time while completing
a sequence of goal-directed actions. The model was able to learn to predict the actor’s
motions and to use spikes in prediction error to update its working models, improving
prediction performance. EST proposes that the error-based updating mechanism could be
implemented by phasic activity of the midbrain dopamine system, a system with broad
projections throughout the cortex via direct connections to the prefrontal cortex and via the
basal ganglia. This proposal has received support from neuroimaging (Zacks et al. 2011).
Another neurophysiological hypothesis was that event model maintenance depended heavily
on the lateral prefrontal cortex. This proposal has fared less well empirically; instead,
components of the brain’s default network including the medial prefrontal and posterior
cortex and parts of the lateral inferior parietal cortex are shaping up as better candidates
(Stawarczyk et al. under review).
The event horizon model of Radvansky and colleagues incorporates EST as a front-end
mechanism to account for a number of features in long-term memory for events (Radvansky
2012; Radvansky & Zacks 2014; see Table 3). The model provides an integrated descriptive
Zacks Page 18
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
account of many of the features of event memory described in “Event memory is structured
by the same dimensions as perception” above. Segmentation during perception leads to
units in event memory. The occurrence of similar event features in multiple events leads
to better memory for those features, but to worse memory for identifying which particular
features occurred in a specific event. Across events, causal relations are a major organizing
feature that determines the likelihood of remembering a particular event and which other
events that event will bring to mind. An important question left unresolved by EST and
the event horizon model is how event structure evolves with forgetting, repeated retrieval,
and subsequent experience. A recent study by Hohman and colleagues (2013) found
that, when pressed, people retrieving information from autobiographical memory could
retrieve additional sub-events from a remembered event and that doing so extended the
remembered event’s boundaries. One possibility is that the temporal continuum that is sliced
by perception can be resliced by the actions of memory retrieval. Another possibility is that
such malleability of event boundaries in memory does not reflect the reslicing of a stored
continuous experience, but rather a process in which the originally stored representations are
discrete, and new discrete event representations are constructed through the act of retrieval.
The segmentation of events is one aspect of event structure; other aspects include the
order of events, their hierarchical organization, and the roles that actors and objects play
in an activity. All of these, we saw, are influenced by knowledge about different event
types. Elman and McRae (in press) have recently proposed a model of the acquisition of
event knowledge that takes on all of these aspects of structure. The model is a recurrent
connectionist network that codes for information about agents, actions, patients, instruments,
locations, and recipients. The model learns associations amongst these elements within and
across timepoints. Both kinds of associations are subject to variability in natural experience;
for example, when changing a tire, one may loosen the lug nuts before or after jacking up
the car; and might pull the car into a driveway or not. Trained on a corpus of event sequences
from people’s descriptions, the model learns the most typical patterns and also information
about the alternatives and the degree of variability. It is able to use this information to
make predictions about what will happen next in a novel sequence—a key feature of
comprehension.
Perhaps the broadest-scope model of event cognition to date is the structured event memory (SEM) model (Franklin et al. 2019). SEM uses a hybrid architecture in which the dynamics
within events are represented using recurrent neural networks and relations across events are
modeled as a partially observable discrete process that generates a sequence of persisting
states which correspond to event types. The model learns a library of possible event
dynamics, stored in the weights of a set of recurrent networks, and learns at each point
in time to apply a previously-stored weight set or to create a new one if none of the
existing weight sets fit. The model segments ongoing activity into events, forms online event
representations, and can retrieve event information later. It provides a novel account of the
learning of event schemas: When the model first creates a new neural network to represent
an event that is not well fit by existing learned dynamics, this representation looks like
an episodic memory. As it encounters more instances that activate the same dynamics, the
representation grows more and more schematic. Like the Reynolds et al. (2007) model, SEM
can segment ongoing activity based on a dynamic video (in this case, full motion video
Zacks Page 19
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
rather than a body-tracking recording). It is sensitive to the statistical structure of activity
subunits in a way similar to human observers. It shows memory updating at event boundaries
in an adaptation of the “walking through doorways” paradigm (Radvansky & Copeland
2006b), the long-term memory benefit of splitting encoding across two events (Pettijohn
et al. 2016), and stronger memory cueing within an event than across events (Ezzyat &
Davachi 2011).
The hierarchical organization of activity in time is a central feature to be accounted for.
In the models described thus far, hierarchical organization is addressed only implicitly.
For example, in EST, segmentation on different timescales can be achieved by varying
the time constant of integration of the prediction error signal: Longer time constants do
more smoothing on the error signal and produce less frequent event model updating in
response to large, slow error spikes; shorter time constants produce more frequent event
model updating in response to quicker error spikes that need not be as large. The hierarchical process memory theory (Hasson et al. 2015) explicitly addresses how different time scales of
representation might be implemented in the brain. Rather than view memory as a specialized
system or set of systems, the authors note that all of the brain’s dynamical systems have
characteristic timescales. In the retina and the cochlea, neural activity tracks the current
state of the world closely, and only retains the influence of previous states of the world
for tens or hundreds of milliseconds. This fast temporal fading carries through the earliest
stages of cortical processing, but as sensory information is processed through successive
cortical stages temporal dependencies grow longer and longer. Complementary patterns of
time-dependence are seen in motor control: Cortical systems that are close to synapsing
on muscles show fast-fading temporal integration, whereas earlier stages of motor control
show longer timescales. Brain areas that show the longest temporal dependencies tend to be
multimodal and tend to represent both perceptual and action-related features of activities;
these areas overlap with the default network, and include the angular gyrus and areas in the
medial posterior and medial frontal cortex.
Summary
In sum, recent theoretical approaches to event cognition have focused on two topics: the
multimodal integration of representations for perception and for action, and the processing
of temporal structure in events. Most current theories integrate computational descriptions
with neurophysiological descriptions in accounting for behavior with respect to events.
New methodological developments shaping event cognition
A key to recent progress in event cognition has been the deployment of new methodological
tools and approaches. One such development is eye-tracking techniques for working
with complex, dynamic stimuli. Whereas for decades eye-tracking has made important
contributions to reading, scene perception, and attention, improvements in hardware and
software have now made it tractable to present naturalistic movies or live interactive
experiences, record eye movements, and calculate features including looking to target
objects (Hayhoe & Ballard 2005) and gaze synchronization across viewers (Loschky et
al. 2015). These improvements have also made it tractable to track the characteristics of
Zacks Page 20
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
saccades over time in conjunction with stimulus features (Eisenberg et al. 2018). Eye
tracking is particularly promising for studies of young children and others who are unable to
perform complex tasks under instruction.
The current resurgence of interest in event perception has corresponded with the rise
of fMRI as a means to study brain activity. Many of the key contributions of this
method have already been discussed: fMRI initially provided a noninvasive means to
test hypotheses about the operations of segmentation during ongoing visual event and
narrative event comprehension (Speer et al. 2007; Whitney et al. 2009; Zacks et al.
2001). More recently, multivariate pattern analysis, intersubject synchronization measures,
and functional connectivity analysis have allowed researchers to test sophisticated new
hypotheses (Baldassano et al. 2017, 2018; Hasson et al. 2008). These methods have provided
evidence that the brain segments ongoing experience at a range of temporal scales, with later
perceptual processing stages specializing in longer time-scales. They also have shown that
phasic activity at the boundaries of events is predictive of online memory updating and of
the organization of subsequent long term memory (Baldassano et al. 2017; Ben-Yakov &
Dudai 2011; Ben-Yakov et al. 2013, 2014; DuBrow & Davachi 2016; Ezzyat & Davachi
2011; Hsieh et al. 2014). New developments in EEG theory and methods also have made
key contributions. An important theoretical development is a reinterpretation of the P600
component. Previous accounts held that this component reflected syntactic reanalysis, but
a recent proposal is that it reflects integration of new information into an event model
(Brouwer et al. 2012). New EEG methods include multivariate pattern analysis such as
those that have been applied to fMRI. For example, Sols and colleagues (2017) recently
used pattern-based EEG to provide evidence that sequential structure within an event is
recapitulated at event boundaries, and that this replay predicts subsequent memory (see
also, e.g., Knoeferle et al. 2008). Over time, neurophysiological studies have increasingly
embraced naturalistic materials and experimental designs (Maguire 2012).
Meanwhile, in computer vision, a new generation of neurally-inspired machine learning
models have transformed object and action recognition from naturalistic stimuli (Herath et
al. 2017; see also Google Cloud Vision API; FaceBook Detectron 2019). These methods
provide tools to test behavioral and neural hypotheses about human event perception at
scale. Crucial for these investigations will be the creation of large corpora of coded event
stimuli (McNamara et al. 2017). Finally, event cognition is starting to move outside the
laboratory to confront the richness of perception and memory “in the wild.” One ongoing
development is the use of wearable sensors and lifelogging devices to measure behavioral
structure on the scale not just of seconds or minutes but of hours to weeks (Nielson et al.
2015; Zhuang et al. 2012). These data suggest that events over the course of the day have
segmental structure consistent with that attested by observers’ segmentation of relatively
brief events in the laboratory.
Looking ahead in event perception
So, where do we stand and how do things look? Since Johansson and colleagues (1980)
canvassed the study of event perception in the Annual Review there has been a dramatic
growth and widening in the empirical phenomena encompassed by the field. First, event
Zacks Page 21
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
perception has become more cognitive, embracing the investigation of mental representation
as a complement to characterizing the structure of the stimulus. Second, event perception
and event memory have converged on a number of features of event structure. Third,
new measures—especially neurophysiological ones—have been brought to bear. Finally,
the sheer volume of the empirical database on event perception and memory has grown
dramatically. These developments warrant using the broader term “event cognition” to
describe the new state of the art.
These empirical developments have been accompanied by new generations of theories.
These theories have attempted to account for the interaction of perception, action, and
memory, for the segmentation of ongoing activity, and for the temporal organization
of events on multiple scales. Though the models reviewed here vary in their scope,
assumptions, and format, they share a concern with the structure of event representations
in the mind and brain. That is, like the empirical research programs, the theoretical programs
have become very cognitive.
What comes next? One possibility is that the models will scale up to something more like
the full complexity of event comprehension, as the empirical work has already started to
do. We can look forward to theoretical and computational models that can “experience”
the same environments that our participants confront and mechanistically account for their
comprehension and memory for those environments in terms of representational form and
neurophysiological instantiation. Ideally, a model would take in the same stimuli as a
participant and produce a set of representations that would enable it track activity in real
time, and also have event memories and event knowledge. Such a model should not be a
passive perceiver but should also be able to act on its environment, closing the loop between
perception and action, using event memory and knowledge as guides.
Another encouraging development is that event cognition appears to be drawing together
research from multiple areas of psychology with neurophysiology, linguistics, and computer
science. As the field of cognitive science has grown over the last several decades, its
transdisciplinary strength has dissipated. One may hope that event cognition will develop
into a point of newfound cognitive science consilience.
Acknowledgments
This research was supported by grants R21AG05231401 from the US National Institutes of Health and N00014– 17-1–2961from the US Office of Naval Research. I would like to thank Susan Fiske for thoughtful and incisive editing, and the following for sharing feedback on the manuscript: Aya Ben-Yakov, Dorthe Berntsen, Neil Cohn, James Cutting, Johannes Gerwien, Bernhard Hommel, Helene Intraub, Alan Kersten, Peter Krøgjaard, Michael Kubovy, Christopher Kurby, Jeffrey Loucks, Ken McRae, Frank Papenmeier, Jesse Sargent, Brent Strickland, Martin Takac, Trine Sonne, Christiane von Stutterheim, and the members of the Dynamic Cognition Laboratory.
Terms and Definitions
Event schema Structured knowledge representation about an event type. The term script has a similar
meaning but is sometimes reserved for events with social conventional structure
Event segmentation
Zacks Page 22
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
The mechanisms by which the brain segments ongoing activity into meaningful events. Also
sometimes used to refer to tasks intended to measure these mechanisms
Functional connectivity analysis A collection of techniques for measuring the degree to which different brain regions’ activity
(usually measured with fMRI) rise and fall together
Intersubject synchronization The degree to which a behavioral or neurophysiological measure is correlated across
participants
Launching effect A configuration of two moving objects such that one is perceived to cause the other to start
moving. Michotte (1946) found that the perception of launching depends precisely on spatial
arrangement, speed, and timing of motion onset and offset
Lifelogging Recording ongoing information about one’s life using wearable sensors and software
Medial temporal lobe The hippocampus and surrounding structures in the temporal lobe, on the medial-ventral
surface of the forebrain
Multivoxel pattern analysis A collection of techniques for analyzing the spatiotemporal patterns of fMRI within a brain
region that are associated with task parameters or stimuli; in distinction to techniques that
analyze the overall level of fMRI signal in a region
N400: A negative-going electroencephalographic (EEG) response that peaks near 400 ms after
stimulus onset. The N400 is associated with processing unexpected stimuli or stimuli that
are difficult to integrate semantically
P3, or P300 An early positive-going electroencephalographic (EEG) response associated with
mechanisms of attention and target detection
P600 A late positive-going electroencephalographic (EEG) response associated with syntactic
analysis or building a representation of a discourse
Piagetian The school of thought in developmental psychology pioneered by the Swiss psychologist
Jean Piaget
Prediction error
Zacks Page 23
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
The difference between a system’s prediction about a variable or state and what it
subsequently observes. Prediction errors play key roles in theories of learning of event
perception
Prefrontal cortex The anterior part of the frontal lobes
Recurrent neural networks A family of neurally-inspired computational models consisting of a large number of simple
computing units that influence each other by connections analogous to axons. Whereas
in feed-forward networks information flow goes in only one direction, recurrent networks
include information flow in the opposite direction
Literature Cited
Abelson RP. 1981. Psychological status of the script concept. American Psychologist 36:715–29
Addis DR, Wong AT, Schacter DL. 2007. Remembering the past and imagining the future: Common and distinct neural substrates during event construction and elaboration. Neuropsychologia 45(7):1363–77 [PubMed: 17126370]
Allen K, Ibara S, Seymour A, Cordova N, Botvinick M. 2010. Abstract structural representations of goal-directed behavior. Psychological Science 21(10):1518–24 [PubMed: 20855906]
Altmann GTM, Mirkovic J. 2009. Incrementality and prediction in human sentence processing. Cognitive Science 33(4):583–609 [PubMed: 20396405]
Amoruso L, Gelormini C, Aboitiz FA, Alvarez González Manes F, et al. 2013. N400 ERPs for actions: building meaning in context. Front. Hum. Neurosci 7:
Avni-Babad D, Ritov I. 2003. Routine and the perception of time. Journal of Experimental Psychology: General 132(4):543–50 [PubMed: 14640847]
Bailey HR, Kurby CA, Giovannetti T, Zacks JM. 2013a. Action perception predicts action performance. Neuropsychologia 51(11):2294–2304 [PubMed: 23851113]
Bailey HR, Zacks JM, Hambrick DZ, Zacks RT, Head D, et al. 2013b. Medial temporal lobe volume predicts elders’ everyday memory. Psychological Science 24(7):1113–22 [PubMed: 23630222]
Baker LJ, Levin DT. 2015. The role of relational triggers in event perception. Cognition 136:14–29 [PubMed: 25490125]
Baldassano C, Chen J, Zadbood A, Pillow JW, Hasson U, Norman KA. 2017. Discovering event structure in continuous narrative perception and memory. Neuron 95(3):709–721 [PubMed: 28772125]
Baldassano C, Hasson U, Norman KA. 2018. Representation of real-world event schemas during narrative perception. J. Neurosci 38(45):9689–99 [PubMed: 30249790]
Baldwin DA, Baird JA, Saylor MM, Clark MA. 2001. Infants parse dynamic action. Child Development 72(3):708–17 [PubMed: 11405577]
Bangert AS, Kurby CA, Zacks JM. in press. The influence of everyday events on prospective timing “in the moment.” Psychonomic Bulletin & Review
Bar M 2007. The proactive brain: using analogies and associations to generate predictions. Trends in Cognitive Sciences 11(7):280–89 [PubMed: 17548232] Reviews Bar’s proposal for how predictive processing guides perception.
Barbey A, Krueger F, Grafman J. 2009. Structured event complexes in the medial prefrontal cortex support counterfactual representations for future planning. Philosophical Transactions of the Royal Society B-Biological Science 364(1521):1291–1300
Barker RG, ed. 1963. The Stream of Behavior: Explorations of Its Structure & Content Appleton- Century-Crofts Reviews ecological studies of the structure of human behavior by Barker’s field research station.
Zacks Page 24
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
Bauer PJ. 2006. Event Memory. In Handbook of Child Psychology: Cognition, Perception, and Language, Vol. 2, 6th Ed., eds. Kuhn D, Siegler RS, Damon W, Lerner RM, pp. 373–425. Hoboken, NJ: John Wiley & Sons Inc Describes theory and data on how event memory develops in children.
Ben-Yakov A, Dudai Y. 2011. Constructing realistic engrams: Poststimulus activity of hippocampus and dorsal striatum predicts subsequent episodic memory. The Journal of Neuroscience 31(24):9032–42 [PubMed: 21677186]
Ben-Yakov A, Eshel N, Dudai Y. 2013. Hippocampal immediate poststimulus activity in the encoding of consecutive naturalistic episodes. Journal of Experimental Psychology: General 142(4):1255 [PubMed: 23815458]
Ben-Yakov A, Rubinson M, Dudai Y. 2014. Shifting gears in hippocampus: Temporal dissociation between familiarity and novelty signatures in a single event. The Journal of Neuroscience 34(39):12973–12981 [PubMed: 25253846]
Berntsen D 2010. The unbidden past: Involuntary autobiographical memories as a basic mode of remembering. Current Directions in Psychological Science 19(3):138–42 Proposes that many experiences of remembering events result from involuntary associative retrieval.
Bläsing B, Tenenbaum G, Schack T. 2009. The cognitive structure of movements in classical dance. Psychology of Sport and Exercise 10(3):350–60
Bonasia K, Blommesteyn J, Moscovitch M. 2016. Memory and navigation: Compression of space varies with route length and turns. Hippocampus 26(1):9–12 [PubMed: 26418606]
Bonasia K, Sekeres MJ, Gilboa A, Grady CL, Winocur G, Moscovitch M. 2018. Prior knowledge modulates the neural substrates of encoding and retrieving naturalistic events at short and long delays. Neurobiology of Learning and Memory 153:26–39 [PubMed: 29474955]
Brewer WF, Dupree DA. 1983. Use of plan schemata in the recall and recognition of goal-directed actions. Journal of Experimental Psychology: Learning, Memory, and Cognition JEP:LMC 9(1):117–29
Brouwer H, Fitz H, Hoeks J. 2012. Getting real about Semantic Illusions: Rethinking the functional role of the P600 in language comprehension. Brain Research 1446:127–43 [PubMed: 22361114]
Brown D, Pipe M- E. 2003. Individual differences in children’s event memory reports and the narrative elaboration technique. Journal of Applied Psychology 88(2):195–206
Brown NR. 2005. On the prevalence of event clusters in autobiographical memory. Social Cognition 35–69
Butz MV, Bilkey D, Humaidan D, Knott A, Otte S. 2018. Learning, planning, and control in a monolithic neural event inference architecture. arXiv:1809.07412 [cs]
Christophel TB, Klink PC, Spitzer B, Roelfsema PR, Haynes J- D. 2017. The distributed nature of working memory. Trends in Cognitive Sciences 21(2):111–24 [PubMed: 28063661]
Cohen LB, Amsel G. 1998. Precursors to infants’ perception of the causality of a simple event. Infant behavior and development
Cohn N, Paczynski M. 2013. Prediction, events, and the advantage of Agents: The processing of semantic roles in visual narrative. Cognitive Psychology 67(3):73–97 [PubMed: 23959023]
Cohn N, Paczynski M, Kutas M. 2017. Not so secret agents: Event-related potentials to semantic roles in visual event comprehension. Brain and Cognition 119(Supplement C):1–9 [PubMed: 28898720]
Cutting JE. 1981. Six tenets for event perception. Cognition 10(1):71–78 [PubMed: 7198564]
Cutting JE. 2014. Event segmentation and seven types of narrative discontinuity in popular movies. Acta Psychologica 149:69–77 [PubMed: 24704782]
Defina R 2016. Do serial verb constructions describe single events?: A study of co-speech gestures in Avatime. Language 92(4):890–910
DuBrow S, Davachi L. 2016. Temporal binding within and across events. Neurobiology of Learning and Memory 134, Part A:107–14 [PubMed: 27422018]
Earles JL, Kersten AW, Curtayne ES, ... 2008. That’s the man who did it, or was it a woman? Actor similarity and binding errors in event memory. Psychonomic bulletin & …
Eisenberg ML, Zacks JM, Flores S. 2018. Dynamic prediction during perception of everyday events. Cognitive Research: Principles and Implications 3(1):53 [PubMed: 30594977]
Zacks Page 25
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
Elman JL, McRae K. in press. A model of event knowledge. Psychological Review Presents a recurrent connectionist model of the acquisition and deployment of event knowledge.
Ezzyat Y, Davachi L. 2011. What constitutes an episode in episodic memory? Psychological Science 22(2):243–52 [PubMed: 21178116]
Faber M, D’Mello SK. 2018. How the stimulus influences mind wandering in semantically rich task contexts. Cognitive Research: Principles and Implications 3(1):
FaceBook Detectron 2019. https://github.com/facebookresearch/Detectron
Ferretti TR, McRae K, Hatherell A. 2001. Integrating verbs, situation schemas, and thematic role concepts. Journal of Memory and Language 44(4):516–547
Fivush R 1997. Event memory in early childhood. In The Development of Memory in Childhood, ed. Cowan N, pp. 139–61. Hove England: Psychology Press/Erlbaum (UK) Taylor & Francis
Flanagan JR, Johansson RS. 2003. Action plans used in action observation. Nature 424:769–71 [PubMed: 12917683]
Flecken M, Athanasopoulos P, Kuipers JR, Thierry G. 2015. On the road to somewhere: Brain potentials reflect language effects on motion event perception. Cognition 141:41–51 [PubMed: 25917431]
Flecken M, von Stutterheim C, Carroll M. 2014. Grammatical aspect influences motion event perception: Findings from a cross- linguistic non-verbal recognition task. Language and Cognition: An Interdisciplinary Journal of Language and Cognitive Science 6(1):45–78
Flores S, Bailey HR, Eisenberg ML, Zacks JM. 2017. Event segmentation improves event memory up to one month later. Journal of Experimental Psychology: Learning, Memory, and Cognition 43(8):1183–1202
Fournier LR, Gallimore JM. 2013. What makes an event: Temporal integration of stimuli or actions? Atten Percept Psychophys 75(6):1293–1305 [PubMed: 23661159]
Franklin N, Norman KA, Ranganath C, Zacks JM, Gershman SJ. 2019. Structured event memory: a neuro-symbolic model of event cognition. bioRxiv 541607
Gennari SP, Sloman SA, Malt BC, Fitch WT. 2002. Motion events in language and cognition. Cognition 83(1):49–79 [PubMed: 11814486]
Gerwien J, von Stutterheim C. 2018. Event segmentation: Cross-linguistic differences in verbal and non-verbal tasks. Cognition 180:225–37 [PubMed: 30092460]
Gibson JJ. 1979. The Ecological Approach to Visual Perception Boston: Houghton Mifflin
Giovannetti T, Bettcher BM, Brennan L, Libron DJ, Kessler RK, Duey K. 2008. Coffee with jelly or unbuttered toast: Commissions and omissions are dissociable aspects of everyday action impairment in Alzheimer’s disease. Neuropsychology 22(2):235–45 [PubMed: 18331166]
Gold DA, Park NW, Troyer AK, Murphy KJ. 2015. Compromised naturalistic action performance in amnestic mild cognitive impairment. Neuropsychology 29(2):320–33 [PubMed: 25151114]
Gold DA, Zacks JM, Flores S. 2016. Effects of cues to event segmentation on subsequent memory. Cognitive Research: Principles and Implications 2:
Google Cloud Vision API. Google Cloud. https://cloud.google.com/vision/
Gravina MT, Sederberg PB. 2017. The neural architecture of prediction over a continuum of spatiotemporal scales. Current Opinion in Behavioral Sciences 17(Supplement C):194–202
Hafri A, Papafragou A, Trueswell JC. 2012. Getting the gist of events: Recognition of two-participant actions from brief displays. Journal of Experimental Psychology: General
Hafri A, Trueswell JC, Strickland B. 2018. Encoding of event roles from visual scenes is rapid, spontaneous, and interacts with higher-level visual processing. Cognition 175:36–52 [PubMed: 29459238]
Hard BM, Recchia G, Tversky B. 2011. The shape of action. Journal of Experimental Psychology: General 140(4):586–604 [PubMed: 21806308]
Hard BM, Tversky B, Lang D. 2006. Making sense of abstract events: Building event schemas. Memory & Cognition 34(6):1221–35 [PubMed: 17225504]
Hasson U, Chen J, Honey CJ. 2015. Hierarchical process memory: memory as an integral component of information processing. Trends Cogn. Sci 19(6):304–13 [PubMed: 25980649] Proposes, based
Zacks Page 26
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
on neuroimaging studies using naturalistic materials, that there is a hierarchy of ranges of time- dependence on the human cortex.
Hasson U, Yang E, Vallines I, Heeger DJ, Rubin N. 2008. A hierarchy of temporal receptive windows in human cortex. Journal of Neuroscience 28(10):2539–50 [PubMed: 18322098]
Hayhoe M, Ballard D. 2005. Eye movements in natural behavior. Trends in cognitive sciences 9(4):188–194 [PubMed: 15808501]
Heberlein AS. 2008. Animacy and intention in the brain: Neuroscience of social event perception. In Understanding Events: From Perception to Action, Vol. 4, eds. Shipley TF, Zacks JM, pp. 363–88. New York, NY: Oxford University Press
Hemeren PE, Thill S. 2011. Deriving motor primitives through action segmentation. Front. Psychology 1:
Herath S, Harandi M, Porikli F. 2017. Going deeper into action recognition: A survey. Image and Vision Computing 60:4–21
Hespos S, Saylor M, Grossman S. 2009. Infants’ ability to parse continuous actions. Developmental Psychology 45(2):575–85 [PubMed: 19271840]
Hespos SJ, Grossman SR, Saylor MM. 2010. Infants’ ability to parse continuous actions: Further evidence. Neural Networks 23(8–9):1026–32 [PubMed: 20729035]
Hohman TJ, Peynircioğlu ZF, Beason-Held LL. 2013. Flexibility of event boundaries in autobiographical memory. Memory 21(2):249–60 [PubMed: 22989194]
Hommel B, Muesseler J, Aschersleben G, Prinz W. 2001. The Theory of Event Coding (TEC): A framework for perception and action planning. Behavioral & Brain Sciences 24(5):849–937 Describes TEC and reviews a wide range of empirical support.
Hsieh L- T, Gruber MJ, Jenkins LJ, Ranganath C. 2014. Hippocampal activity patterns carry information about objects in temporal context. Neuron 81(5):1165–78 [PubMed: 24607234]
Hudson JA, Fivush R, Kuebli J. 1992. Scripts and episodes: The development of event memory. Applied Cognitive Psychology 6(6):483–505
Huff M, Meitz TGK, Papenmeier F. 2014. Changes in situation models modulate processes of event perception in audiovisual narratives. Journal of Experimental Psychology: Learning, Memory, and Cognition 40(5):1377–88
Huff M, Papenmeier F, Zacks JM. 2012. Visual target detection is impaired at event boundaries. Visual Cognition 20(7):848–64
Intraub H 2010. Rethinking scene perception: A multisource model. In Psychology of Learning and Motivation, Vol. 52, pp. 231–64. Elsevier
James W 1890. The Principles of Psychology, Vol. 1. New York: Henry Holt
Jeunehomme O, D’Argembeau A. 2018. Event segmentation and the temporal compression of experience in episodic memory. Psychological research
Johansson G, Hofsten C von H, Jansson G. 1980. Event Perception. Annual Review of Psychology 31(1):27–63
Keidel JL, Oedekoven CSH, Tut AC, Bird CM. 2017. Multiscale integration of contextual information during a naturalistic task. Cerebral Cortex 1–9 [PubMed: 28365777]
Kersten AW, Earles JL. 2017. Feelings of familiarity and false memory for specific associations resulting from mugshot exposure. Memory & Cognition 45(1):93–104 [PubMed: 27473580]
Kersten AW, Earles JL, Upshaw C. 2013. False recollection of the role played by an actor in an event. Memory & Cognition 41(8):1144–58 [PubMed: 23722927]
Knoeferle P, Habets B, Crocker M, Munte T. 2008. Visual scenes trigger immediate syntactic reanalysis: Evidence from ERPs during situated spoken comprehension. Cerebral Cortex 18(4):789–95 [PubMed: 17644830]
Köhler W 1929. Gestalt Psychology New York: H. Liveright
Koutstaal W, Schacter DL, Johnson MK, Galluccio L. 1999. Facilitation and impairment of event memory produced by photograph review. Memory & Cognition 27(3):478–93 [PubMed: 10355237]
Kubovy M 2015. the deep structure of lives. Philosophia Scientiæ 19(3):153–76
Zacks Page 27
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
Kurby C, Zacks JM. 2011. Age differences in the perception of hierarchical structure in events. Memory & Cognition 39(1):75–91 [PubMed: 21264613]
Kurby CA, Zacks JM. 2018. Preserved neural event segmentation in healthy older adults. Psychology and Aging 33(2):232–45 [PubMed: 29446971]
Lawrence Z, Peterson D. 2016. Mentally walking through doorways causes forgetting: The location updating effect and imagination. Memory 24(1):12–20 [PubMed: 25412111]
Levin DT. 2010. Spatial representations of the sets of familiar and unfamiliar television programs. Media Psychology 13(1):54–76
Loschky LC, Larson AM, Magliano JP, Smith TJ. 2015. What would Jaws do? The tyranny of film and the relationship between gaze and higher-level narrative film comprehension. PLOS ONE 10(11):e0142474 [PubMed: 26606606]
Loucks J, Mutschler C, Meltzoff AN. 2016. Children’s representation and imitation of events: How goal organization influences 3-year-old children’s memory for action sequences. Cognitive Science
Loucks J, Pechey M. 2016. Human action perception is consistent, flexible, and orientation dependent. Perception 45(11):1222–39 [PubMed: 27251164]
MacLean CL, Coburn PI, Chong K, Connolly DL. 2018. Breaking script: Deviations and postevent information in adult memory for a repeated event. Applied Cognitive Psychology 32(4):474–86
Magliano JP, Radvansky GA, Forsythe JC, Copeland DE. 2014. Event segmentation during first-person continuous events. Journal of Cognitive Psychology 26(6):649–61
Maguire EA. 2012. Studying the freely-behaving brain with fMRI. NeuroImage 62(2):1170–76 [PubMed: 22245643]
McNamara Q, De La Vega A, Yarkoni T. 2017. Developing a comprehensive framework for multimodal feature extraction. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1567–1574. New York, NY, USA: ACM
McRae K, Hare M, Elman JL, Ferretti T. 2005. A basis for generating expectancies for verbs from nouns. Memory & Cognition 33(7):1174–1184 [PubMed: 16532852]
Michotte AE. 1946. The Perception of Causality New York: Basic Books
Miller GA, Galanter E, Pribram KH. 1960. Plans and the Structure of Behavior New York: Holt
Miller GA, Johnson-Laird PN. 1976. Language and Perception Cambridge, MA: Harvard University Press
Monroy CD, Gerson SA, Domínguez-Martínez E, Kaduk K, Hunnius S, Reid V. 2017. Sensitivity to structure in action sequences: An infant event-related potential study. Neuropsychologia
Monroy CD, Gerson SA, Hunnius S. 2018. Translating visual information into action predictions: Statistical learning in action and nonaction contexts. Mem Cogn 1–14
Nakano T, Kitazawa S. 2017. Development of long-term event memory in preverbal infants: an eye-tracking study. Scientific Reports
Newtson D 1973. Attribution and the unit of perception of ongoing behavior. Journal of Personality and Social Psychology 28(1):28–38
Newtson D 1976. Foundations of attribution: The perception of ongoing behavior. In New Directions in Attribution Research, eds. Harvey JH, Ickes WJ, Kidd RF, pp. 223–48. Hillsdale, New Jersey: Lawrence Erlbaum Associates
Nielson DM, Smith TA, Sreekumar V, Dennis S, Sederberg PB. 2015. Human hippocampus represents space and time during retrieval of real-world memories. Proceedings of the National Academy of Sciences 112(35):11078–83
Papafragou A 2010. Source-goal asymmetries in motion representation: Implications for language production and comprehension. Cognitive Science 34(6):1064–92 [PubMed: 20729982]
Papafragou A, Hulbert J, Trueswell J. 2008. Does language guide event perception? Evidence from eye movements. Cognition 108:155–84 [PubMed: 18395705]
Papenmeier F, Brockhoff A, Huff M. 2019. Filling the gap despite full attention: the role of fast backward inferences for event completion. Cogn. Research 4(1):3
Pettijohn KA, Radvansky GA. 2015. Walking through doorways causes forgetting: Event structure or updating disruption? The Quarterly Journal of Experimental Psychology (just-accepted):1–27
Zacks Page 28
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
Pettijohn KA, Radvansky GA. 2016. Narrative event boundaries, reading times, and expectation. Mem Cogn 44(7):1064–75
Pettijohn KA, Thompson AN, Tamplin AK, Krawietz SA, Radvansky GA. 2016. Event boundaries and memory improvement. Cognition 148:136–44 [PubMed: 26780472]
Radvansky G, Copeland D. 2006a. Walking through doorways causes forgetting: situation models and experienced space. Memory & Cognition 34(5):1150–56 [PubMed: 17128613]
Radvansky GA. 2012. Across the event horizon. Current Directions in Psychological Science 21(4):269–272
Radvansky GA, Copeland DE. 2006b. Walking through doorways causes forgetting: Situation models and experienced space. Memory & Cognition 34(5):1150–56 [PubMed: 17128613]
Radvansky GA, Copeland DE, Berish DE, Dijkstra K. 2003. Aging and situation model updating. Aging, Neuropsychology, and Cognition 10:158–66
Radvansky GA, Dijkstra K. 2007. Aging and situation model processing. Psychonomic Bulletin & Review 14(6):1027–42 [PubMed: 18229472]
Radvansky GA, Krawietz S, Tamplin A. 2011. Walking through doorways causes forgetting: Further explorations. The Quart. J. of Expt. Psych 1–14
Radvansky GA, O’Rear AE, Fisher JS. 2017. Event models and the fan effect. Mem Cogn 45(6):1028– 44
Radvansky GA, Tamplin AK, Krawietz SA. 2010. Walking through doorways causes forgetting: Environmental integration. Psychonomic Bulletin & Review 17(6):900–904 [PubMed: 21169587]
Radvansky GA, Zacks JM. 2014. Event Cognition New York: Oxford University Press This book gives a comprehensive review of research on event cognition and describes EST and the Event Horizon model in depth.
Radvansky GA, Zwaan RA, Federico T, Franklin N. 1998. Retrieval from temporally organized situation models. Journal of Experimental Psychology: Learning, Memory, & Cognition 24(5):1224–37
Ranganath C, Ritchey M. 2012. Two cortical systems for memory-guided behaviour. Nature Reviews Neuroscience 13(10):713–26 [PubMed: 22992647] Presents the posterior-medial/ anterior temporal (PMAT) account of how brain systems integrate knowledge-based associations and arbitrary episodic associations in memory.
Reynolds JR, Zacks JM, Braver TS. 2007. A computational model of event segmentation from perceptual prediction. Cognitive Science 31:613–43 [PubMed: 21635310]
Richmond LL, Gold DA, Zacks JM. 2017. Event perception: Translations and applications. Journal of Applied Research in Memory and Cognition 6(2):111–20 [PubMed: 28936393]
Rinck M, Bower G. 2000. Temporal and spatial distance in situation models. Memory and Cognition 28(8):1310–20 [PubMed: 11219959]
Rubin DC, Umanath S. 2015. Event memory: A theory of memory for laboratory, autobiographical, and fictional events. Psychological Review 122(1):1–23 [PubMed: 25330330]
Sargent JQ, Zacks JM, Hambrick DZ, Zacks RT, Kurby CA, et al. 2013. Event segmentation ability uniquely predicts event memory. Cognition 129(2):241–55 [PubMed: 23942350]
Saylor M, Baldwin D, Baird J, LaBounty J. 2007. Infants’ on-line segmentation of dynamic human action. Journal of Cognition and Development 8(1):113–28
Schacter DL, Addis DR, Hassabis D, Martin VC, Spreng RN, Szpunar KK. 2012. The future of memory: Remembering, imagining, and the brain. Neuron 76(4):677–94 [PubMed: 23177955]
Sharon T, Wynn K. 1998. Individuation of actions from continuous motion. Psychological Science 9(5):357–62
Sheffield EG, Hudson JA. 1994. Reactivation of toddlers’ event memory. In Long-Term Retention of Infant Memories. Memory, Vol. 2, No. 4., ed. Fivush R, pp. 447–65. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc
Simony E, Honey CJ, Chen J, Lositsky O, Yeshurun Y, et al. 2016. Dynamic reconfiguration of the default mode network during narrative comprehension. Nature Communications 7(1):
Zacks Page 29
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
Solomon SH, Hindy NC, Altmann GTM, Thompson-Schill SL. 2015. Competition between mutually exclusive object states in event comprehension. Journal of Cognitive Neuroscience 27(12):2324– 38 [PubMed: 26284994]
Sols I, DuBrow S, Davachi L, Fuentemilla L. 2017. Event boundaries trigger rapid memory reinstatement of the prior events to promote their representation in long-term memory. Current Biology 27(22):3499–3504 [PubMed: 29129536]
Sonne T, Kingo OS, Krøjgaard P. 2016. Occlusions at event boundaries during encoding have a negative effect on infant memory. Consciousness and cognition 41:72–82 [PubMed: 26891192]
Sonne T, Kingo OS, Krøjgaard P. 2017. Bound to remember: Infants show superior memory for objects presented at event boundaries. Scandinavian Journal of Psychology 58(2):107–13 [PubMed: 28252194]
Speer NK, Reynolds JR, Zacks JM. 2007. Human brain activity time-locked to narrative event boundaries. Psychological Science 18(5):449–55 [PubMed: 17576286]
Stawarczyk D, Bezdek MA, Zacks JM. under review. Constructing event representations: The role of the midline default network core
Strickland B, Keil F. 2011. Event completion: Event based inferences distort memory in a matter of seconds. Cognition 121(3):409–15 [PubMed: 21917244]
Strickland B, Scholl BJ. 2015. Visual perception involves event-type representations: The case of containment versus occlusion. Journal of Experimental Psychology: General 144(3):570 [PubMed: 25867222]
Swallow KM, Barch DM, Head D, Maley CJ, Holder D, Zacks JM. 2011. Changes in events alter how people remember recent information. Journal of Cognitive Neuroscience 23(5):1052–64 [PubMed: 20521850]
Swallow KM, Kemp JT, Candan Simsek A. 2018. The role of perspective in event segmentation. Cognition 177:249–62 [PubMed: 29738924]
Swallow KM, Zacks JM, Abrams RA. 2009. Event boundaries in perception affect memory encoding and updating. Journal of Experimental Psychology: General 138(2):236–57 [PubMed: 19397382]
Tauzin T 2015. Simple visual cues of event boundaries. Acta Psychologica 158:8–18 [PubMed: 25867112]
Trabasso T, Stein NL. 1997. Narrating, representing, and remembering event sequences. In Developmental Spans in Event Comprehension and Representation: Bridging Fictional and Actual Events., Vol. Developmental Spans in Event Comprehension and Representation: Bridging Fictional and Actual Events, pp. 237–70. Mahwah, NJ, USA: Lawrence Erlbaum Associates, Inc
Tulving E 1983. Elements of Episodic Memory New York: Oxford University Press
Wahlheim CN, Zacks JM. in press. Memory guides the comprehension of event changes for older and younger adults. Journal of Experimental Psychology: General
Wang Y, Gennari SP. 2019. How language and event recall can shape memory for time. Cognitive Psychology 108:1–21 [PubMed: 30453099]
Warren WH, Shaw RE, eds. 1985. Persistence and Change: Proceedings of the First International Conference on Event Perception Hillsdale, N.J: L. Erlbaum Associates
Webb A, Knott A, MacAskill MR. 2010. Eye movements during transitive action observation have sequential structure. Acta Psychologica 133(1):51–56 [PubMed: 19854431]
Whitney C, Huber W, Klann J, Weis S, Krach S, Kircher T. 2009. Neural correlates of narrative shifts during auditory story comprehension. NeuroImage 47(1):360–66 [PubMed: 19376237]
Woodward AL. 2009. Infants’ grasp of others’ intentions. Current Directions in Psychological Science 18(1):53–57 [PubMed: 23645974]
Wyer RS, Bodenhausen GV. 1985. Event memory: The effects of processing objectives and time delay on memory for action sequences. Journal of Personality and Social Psychology 49(2):301–16 [PubMed: 4032223]
Wynn K 1996. Infants’ individuation and enumeration of actions. Psychological Science 7(3):164–69
Zacks Page 30
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
Zacks JM, Braver TS, Sheridan MA, Donaldson DI, Snyder AZ, et al. 2001. Human brain activity time-locked to perceptual event boundaries. Nature Neuroscience 4(6):651–55 [PubMed: 11369948]
Zacks JM, Kurby CA, Eisenberg ML, Haroutunian N. 2011. Prediction error associated with the perceptual segmentation of naturalistic events. Journal of Cognitive Neuroscience 23:4057–66 [PubMed: 21671745]
Zacks JM, Sargent JQ. 2010. Event perception: A theory and its application to clinical neuroscience. In The Psychology of Learning and Motivation: Advances in Research and Theory, Vol. 53, ed. Ross BH, pp. 253–99. San Diego, CA US: Elsevier Academic Press
Zacks JM, Speer NK, Swallow KM, Braver TS, Reynolds JR. 2007. Event perception: A mind/brain perspective. Psychological Bulletin 133(2):273–93 [PubMed: 17338600]
Zhuang Y, Belkin M, Dennis S. 2012. Metric based automatic event segmentation. International Conference on Mobile Computing, Applications, and Services, pp. 129–148. Springer
Zwaan RA, Radvansky GA. 1998. Situation models in language comprehension and memory. Psychological Bulletin 123(2):162–85 [PubMed: 9522683] Reviews work on event representations in discourse comprehension.
Zwickel J, Prinz W. 2012. Assimilation and contrast: the two sides of specific interference between action and perception. Psychological Research 76(2):171–82 [PubMed: 21556808]
Zacks Page 31
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
Zacks Page 32
Table 1:
Characteristics of behavioral event segmentation (see Radvansky & Zacks, esp. Ch. 5).
Intersubjective agreement Observers tend to identify similar boundaries.
Correlated with change Observers tend to identify boundaries when more features of the stimulus are changing.
Varying grain size People can adjust the temporal grain at which they report event boundaries.
Partonomic hierarchy Fine-grained events cluster into larger coarse-grained events.
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
Zacks Page 33
Table 2:
Posits of the theory of event coding (TEC; Hommel et al., 2001).
1. Shared representations Representations for perceiving and action planning are functionally equivalent; both are correspondences between brain states and anticipated interactions of the actor/observer with external events.
2. Distal coding Events are coded in terms of distal features such as objects and their movements, rather than in terms of proximal features such as the feel of touching an object or a sequence of limb movements planned to move it.
3. Feature codes Stimuli and objects are both represented as temporary composites of feature codes.
4. Activation and integration
The formation of an event code is composed of two phases: activation and integration.
5. Intentionality The combination of feature codes is weighted by current goals and intentions.
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
A uthor M
anuscript A
uthor M anuscript
A uthor M
anuscript A
uthor M anuscript
Zacks Page 34
Table 3:
Posits of the Event Horizon model (from Zacks & Radvansky, 2014).
1. Segmentation Continuous ongoing activity is segmented into discrete events, and an event model is constructed for each event.
2. Working models The event model corresponding to the event you are currently experiencing at any particular timescale has special status. It is actively maintained by recurrent neural activity.
3. The causal network Long-term memory links event models by their causal relations.
4. Noncompetitive attribute retrieval
When elements of events are represented in multiple event models, access to those elements is facilitated.
5. Competitive event retrieval When several event models are similar, accessing any specific event model is more difficult.
Annu Rev Psychol. Author manuscript; available in PMC 2021 December 17.
- Abstract
- Introduction
- The structure of events in the world and in perception
- Continuous experience is segmented into continuous events
- Event representations have internal structure
- Event perception is structured by partonomy and other relations
- Event perception is heavily informed by knowledge
- Event perception is predictive and inferential
- Events structure consciousness
- Summary
- From perception to memory
- Working memory representations are updated at event boundaries
- The structure of events in experience is mirrored in long-term memory
- Event memory is structured by the same dimensions as perception
- Event memory, like perception, is heavily informed by knowledge
- Remembering events is constructive and destructive
- Summary
- Group and individual differences in event cognition
- The experience of events changes through adulthood, and is impaired in Alzheimer’s disease
- Language affects some aspects of event perception and memory—especially when using language
- There are strong relationships between individual differences in event perception and in event memory
- The development of event cognition reflects growth in knowledge, language use, and deliberate rehearsal
- Summary
- Theoretical approaches to event cognition
- Common, multimodal coding of events and actions
- Event segmentation and the formation of event memories
- Summary
- New methodological developments shaping event cognition
- Looking ahead in event perception
- References
- Table 1:
- Table 2:
- Table 3: