response essay about payoffs of processing media mindfully?

Expanding

W hen I ask people to define, in one or two sentences, the word literacy—what literacy is and what it enables people to do—the answers I receive are quite similar. To most people, literacy means the ability to read and write, to understand information, and to express ideas both concretely and abstractly. The unstated assumption is that “to read and write” means to read and write text. Although media and computer literacy are occasionally mentioned in these definitions, media literacy is most often defined as the ability to understand how television and film manipulate viewers, and computer literacy is generally defined as the skills to use a computer to perform various tasks, such as accessing the Web. If I also ask people about the nature of language, I usually receive the response that language enables us to conceptualize ideas, to abstract information, and to receive and share knowledge. The underlying assump- tion, so accepted that it is never stated, is that language means words.

Twenty-five years ago, a rather popular book was entitled Four Arguments for the Elimination of Television.1 Clearly, that vision of the world was not realized: television has not been elimi- nated, and screens—from television screens to computer screens—now dominate our lives. This reality needs to be acknowledged. So, in the spirit of the title of the earlier book, I’d like to suggest four arguments for an expanded definition of literacy:

1. The multimedia language of the screen has become the current vernacular. 2. The multimedia language of the screen is capable of constructing complex meanings in-

dependent of text. 3. The multimedia language of the screen enables modes of thought, ways of communicating

33M a r c h / A p r i l 2 0 0 3 � E D U C A U S E r e v i e w32 E D U C A U S E r e v i e w � M a r c h / A p r i l 2 0 0 3

By Elizabeth Daley

I l l u s t r a t i o n b y D e r e k L e a , © 2 0 0 3 © 2 0 0 3 E l i z a b e t h D a l e y

Elizabeth Daley is Executive Director of the Annenberg Center for Communication and Dean of the School of Cinema-Television at the University of Southern California. She holds a Ph.D. in communi- cation arts from the University of Wisconsin. This article is drawn from a paper delivered at the 20 02 A spen Symposium of the Forum for the Future of Higher Education.

Literacy THE CONCEPT OF

most contemporary Americans, although a printed text or oral explanation might well complement and extend their meanings. However, even if we did not know the con- text of their creation, they would each carry strong meanings and convey powerful emotions. Multimedia and cinema, though sometimes enriched by language, embrace many other elements as co-equal—not only image but also sound, duration, color, and design. Think also for a moment of historic cinematic moments: the first moon landing; the planes slamming into the World Trade

Center. What would it be like to try to fully share these and other momentous events without access to the language and power of the screen?

3. The mult imedia language of the screen enables modes of thought, ways of communicating and conducting re- search, and methods of publication and teaching that are essentially differ- ent from those of text. Since the Enlightenment, the intellectual community has valued the rational over the affective, the abstract over the con- crete, the decontextualized over the con- textualized. These values, combined with a deeply ingrained suspicion of practice and of the creation of product, make it difficult to bring the vernacular of con- temporary media into the academy.

Accepting the language of multimedia as co-equal with text will require a major paradigm shift that challenges the domi-

nation of science and rationality, abstrac- tion and theory. I am certainly not alone in thinking that this shift is long overdue. For example Stephen Toulmin, in his highly regarded 1990 book Cosmopolis, eloquently argued for the academy to move b eyond the dominance of the Cartesian model.2 But just how difficult this transition will be was made clear to me a few years ago when a senior aca- demic figure explained why she found it difficult to judge the worthiness of artists for promotion and tenure. She said that

she found their work too specific and lacking an abstract, theoretical base. It seemed that a professor was more valued if he or she wrote about art rather than made art.

The language of multimedia is, no doubt, more closely related to the affec- tive and subjective language of art than to the rational and linear language of sci- ence. Sergei Eisenstein, the great Russian filmmaker, once described the language of art, as opposed to the language of sci- ence, as a language of conflict, a dialecti- cal language as opposed to a linear one. When the language of science is applied to art, he argued, it ossifies art; for exam- ple, a landscape becomes a topographical map, and a painted Saint Sebastian be- comes an anatomical study.3

The grammar of multimedia and the ways in which it creates meaning are only beginning to be systematically articu- lated. By contrast, the language of the cin-

ema offers us an extensive body of theory, providing a starting point from which to think about multimedia. In 1923 Dziga Vertov, a Soviet documentary filmmaker and one of the Russian pioneers of cine- matic language, wrote for the motion- picture camera a fanciful monologue that might well be applied to multimedia today: “I am a mechanical eye, I a ma- chine, show you a world the way only I can see it. Now and forever, I free myself from human immobility. I am in constant motion. I draw near and then away from

objects . . . recording movement . . . of the most complex combinations. Freed from the limits of time and space, I put together any given points in the universe no matter where I’ve recorded them. . . . My path leads to the creation of a fresh perception of the world. I decipher in a new way a world unknown to you.”4

One of the fundamental cine- matic building blocks, which also applies to much of multimedia, is montage, or the juxtaposition of ele- ments, both within and between shots. For the filmmaker, this is the art of editing, and it is core to the cre- ation of cinema. Montage offers a clear and important example of how

differently text and multimedia construct meaning. Through montage, one is able to manipulate time and space and create sequences that could never exist in the physical world but that are thematically and conceptually related. Montage per- mits an interaction between the creator and the receiver, as well as among the ele- ments of the creation. It not only allows but encourages the recombination of ele- ments to create new meanings.

The famous demonstration of mon- tage by Lev V. Kuleshov, another of the Russian pioneers, elucidates the concept. Kuleshov made a short film sequence that juxtaposed a well-known Russian actor’s face against three different shots: a bowl of soup, a dead woman in a coffin, and a girl playing with a toy bear. When asked to describe what they had seen, viewers— according to V. I. Pudovkin, who was then a student in Kuleshov’s workshop —in- sisted that the man was hungry when looking at the soup, joyous when watch- ing the girl, and sad when viewing the

35M a r c h / A p r i l 2 0 0 3 � E D U C A U S E r e v i e w

and conducting research, and meth- ods of publication and teaching that are essentially different from those of text.

4. Lastly, following from the previous three arguments, those who are truly literate in the twenty-first century will be those who learn to both read and write the multimedia language of the screen.

These four statements are the founda- tional principles for the work being done at the Institute for Multimedia Literacy (IML) at the Annenberg Center at the University of Southern California (USC).

1. The multimedia language of the screen has become the current vernacular. I often ask colleagues to imagine that they are living and teaching in Padua around the year 1300. Inside the stone walls of that great university they lecture in Latin, but the people walking on the streets be- neath their windows, including their own students, speak Italian. Eventually that vernacular has to be embraced (and, in- deed, it was embraced) within the Italian academy. The corresponding argument today, simply put, is that for most peo- ple—including students—film, television, computer and online games, and music constitute the current vernacular.

Print first allowed for mass literacy, and it has been very effective, but to priv- ilege a print language often means to ig- nore the success of the technologies— audio recording, radio, cinema , and television—that have come into existence since the primary modes of print were de- veloped. These technologies have be- come, for average citizens, the most com- mon methods of receiving information, communicating with one another, and entertaining themselves. It is not hard to see how the grammar of these technolo- gies has long since invaded our collective thinking. Metaphors from the screen have become common in every aspect of daily conversations. Close-up is synony- mous for “in-depth” and “penetrating.” We speak of flashing back to our earlier lives. We frame events to put them in con- text. We cut to the chase when we are in a hurry. We dissolve or fade out or segue from one topic to another, and we have back-

ground sound. We spend many hours at our computers looking at and sharing screens. Students are accustomed to the direct emotional experience of music as one of the primary factors in creating their iden- tities, and they spend hours playing com- puter games in online communities. In short, our shared experiences as human beings are more often than not derived from the images and sounds that exist on screens.

2. The mult imedia language of the screen is capable of con- structing complex mean- ings independent of text. At USC, the highly ranked School of Cinema-Television is praised, envied, and ad- mired but still held suspect. In the world of a research u n iv e r s it y, it r e m a i n s a n anomaly. No university is ranked nationally on the basis of such a school, nor is such a school considered to be critical, as are the Depart- ments of Physics and English.

I believe the reason for this lack of status is not only because media creators and scholars deal with that “dis- reputable” world of enter- tainment but also, and even more important, because in their work in this discipline, they do not give primacy to print. They believe that images and s o u n d s , i n t e g r a t e d i n a t i m e -b a s e d medium, can be as important in creating knowledge and communicating ideas and information as text. At the most fun- damental level, their work does not en- dorse the premise, widely held for the past two millennia, that comprehension of and expression through the printed medium defines what it means to be liter- ate and, by extension, educated.

A few years ago, a colleague sent me an article that had appeared in the Chronicle of Higher Education. The author, a re- spected art historian, asserted that it was time for the academy to give up a deeply ingrained suspicion of images and realize that the visual could indeed contain intel- lectual content, that it might in some cases be equal to text. I assumed that the article was a reprint, written in the 1930s

perhaps, but it was current. At that mo- ment, I realized that we at the IML would need to defend not only the idea of a ver- nacular that was a cinematic/multimedia- based language but also, more important, the value of that idea. We had a very long way to go to establish that such a language might have clear differences from, and even advantages over, print in some instances.

By arguing for the importance of the language of the screen, I do not intend to attack words or print. But print carries its

own technological bias. Print supports linear argument, but it does not value as- pects of experience that cannot be con- tained in books. Print deals inadequately with nonverbal modes of thought and nonlinear construction.

Like text, multimedia can enable us to develop concepts and abstractions, com- parisons and metaphors, while at the same time engaging our emotional and aesthetics sensibilities. Rich media, with its multiple simultaneous layers, does much more than provide enhancements, illustrations, and tools for enriching, accessing, and transmit- ting the established literacy. Think for a mo- ment of the still images that have defined many important moments in U.S. history: the photo essays of the Great Depression; a sailor kissing a girl in Times Square at the end of World War II; a young Vietnamese girl fleeing napalm; a college student at Kent State kneeling over a body. As icons, they no longer require any explanation for

34 E D U C A U S E r e v i e w � M a r c h / A p r i l 2 0 0 3

Print supports linear argument, but it does not value aspects of experience that cannot be contained in books.

dead woman. They described his various emotions with great detail, as proof of his outstanding talent as an actor. All ver- sions were, however, the same shot of the man. Nothing in his image had changed. The meaning attributed to each sequence came only from the collision of the shots.

With the computer, we can now elec- tronically construct images and sounds and manipulate space and time to create meaning in ways about which our Russ- ian progenitors could only dream. The tools of keying, compositing, and morph- ing are more than ways of misrepresent- ing truth. They enable the construction of higher orders of meaning, nuance, and inference. Synthesizing and sampling, as well as simultaneity, are natural to multi- media, permitting a form of bricolage, a process through which, in the words of John Seely Brown, one is able to “find something (perhaps a tool, some open source code, images, music, text) that can be used or transformed to build some- thing new.”5

Interactivity as a core factor in multi- media is in some ways closely related to performance and can enable the viewer/ reader/user to participate directly in the construction of meaning. It is perhaps worth digressing for a moment to note that whereas performance has long been devalued as “entertainment,” the art of storytelling, always performative, has been a major way of transmitting culture and values throughout human history.

The very vocabulary of multimedia encourages approaches different from those used to write text. One “creates” and “constructs” media rather than writing it, and one “navigates” and “explores” media rather than reading it. The process is ac-

tive, interactive, and often social, allowing for many angles of view.

The physical production techniques used to make multimedia and the prac- tices of distributing it also differ from the models used to produce and publish text. First, and perhaps foremost, the produc- tion of multimedia is most often an act of collaboration. A film, large or small, is rarely made by one person. The “film by” credit is most often the manifestation of a particular business practice and does not reflect the true nature of authorship or the process of creation. This collaborative process may be reduced in scale in multi- media projects, but it is still typical. It lies in the very nature of the creative process and does not fundamentally change be- cause of the difficulty or simplicity of the tools that are required.

Second, the final product—be it a film, a television program, or some other form of multimedia—is most suc- cessful when it emerges in large part during the process of creation. A script or scenario or storyboard provides a guide, but if one wants to go beyond the predictable and formulaic, there needs to b e ro o m f o r d i s c o v e r y a n d e v e n serendipity during the production or creation of a film or multimedia docu- ment. One of the great U.S. filmmakers, Walter Murch, who edited The English Patient, among other well-known films, refers to this process as the “collision of intelligences” that produces something u n f o r e s e e n b y t h e c r e at iv e te a m , a process that allows for and respects in- tuition. In some ways, this process con- stitutes a typ e of active research in which one studies what one is doing while doing it. Such work demands a cli- mate open to experimentation and a w i l l i n g n e s s t o ex p l o r e a n d fa i l — a n “ecology of experimentation,” to borrow a term from a University of Michigan document.6 It allows for rapid iteration and quick changes of direction.

Third, media forms are usually meant for public distribution and presentation. They are intended to be seen in environ- ments beyond that of their creation. In the beginning at IML , both students and faculty felt that student projects were pri- vate and meant for viewing only by the student and the professor. Over the past semesters, however, we have seen the

36 E D U C A U S E r e v i e w � M a r c h / A p r i l 2 0 0 3

Multimedia, so ubiquitous to young people’s experience, often seems to be particularly hard for them to analyze or deconstruct.

nature of authorship change. Students no longer write to please only the professor. They want to be understood by their peers and by others who will see and ex- perience their projects. They consider themselves to be authors who possess expertise in a certain research area. Like- wise, faculty in IML workshops are intent on making their own research projects ac- cessible to those not in their discipline. Faculty in the humanities and in arts and sciences—from disciplines as diverse as quantum physics, art history, and philoso- phy—have found common ground, in- sights, and points of access into the peda- go g i c a l a n d r e s e a r c h i s s u e s i n o n e another’s disciplines. Certainly not all work in a discipline will be understand- able to those without training, but for in- terdisciplinary work, faculty must find a language to speak across the boundaries. Multimedia may well have the potential to provide a much-needed new space in which cross-disciplinary conversation can occur between the humanities and the sciences.

4. Lastly, following from the previous three arguments, those who are truly literate in the twenty-first century will be those who learn to both read and write the multimedia language of the screen. After a hundred years, the language of cinema has been rather well defined, and a large body of critical literature exists. The production methodologies are also quite well understood and articulated, al- though that knowledge often seems to rest primarily within the oral culture of the filmmaking community. Even with this history and ample evidence of the skill required to construct media, the atti- tude widely held by both faculty and ad- ministration is that complex media texts do not deserve classroom or research time, especially if such study might take emphasis away from traditional activities such as essays and research papers.

Since the 1960s, universities and col- leges and even high schools have taught so-called media or visual literacy courses. These courses, however, have had two limitations. First, they often seem to have an underlying assumption that televi- sion, cinema, and related media are infe- rior communication forms that may mis-

represent reality, since media at its worst manipulates us and lies to us and at its best is superficial. These courses, so far as I can see, enforce the belief that real edu- cation remains in books and that real knowledge is rational and linear. Stu- dents are taught to read visual texts in order to defend themselves against the onslaught of visual culture. Second, these courses have been extremely one-sided in the definition of literacy, focusing on a “read only” approach. Full literacy de- mands the ability to write, as well as to read. I was recently told by a very well- known scholar that images are less useful than text because one can interpret im- ages many ways but words are far more precise. I wondered if he had never had the experience that most us have on a daily basis—that of saying, “No, what I re- ally meant to say was . . .”

The current situation is further com- plicated by the widespread assumption that students already have an adequate knowledge of screen language and multi-

media. No doubt, young people today have less fear of the computer and more technical ability with software for rich media; multimedia is indeed their every- day language. However, they have no more critical ability with this language than do their elders—perhaps less. They need to be taught to write for the screen and analyze multimedia just as much as, if not more than, they need to be taught to write and analyze any specific genre in text. Generally, they have had instruction in text at the secondary level, but rarely have they received similar instruction for multimedia. Multimedia, so ubiquitous to their experience, often seems to be particularly hard for them to analyze or deconstruct.

Another consideration is that al- t h o u gh t h e a c a d e m ic st u d y o f f i l m , media, and audiovisual culture has estab- lished pedagogical precedents that pro- vide theoretical building blocks for the critical application of film, television, and multimedia into higher education, it is

37M a r c h / A p r i l 2 0 0 3 � E D U C A U S E r e v i e w

common to see media integrated into dis- ciplines across the curriculum without these important critical tools in place. Films or segments of films are screened in various classes “to engage students” in the subject matter. Often this usage appears to have occurred without appropriate re- gard for the nature of audiovisual media, its inherent meaning and structure, the

cultural context of its creation, or the consequences of dissecting it.

The most glaring examples of this practice have occurred during the past two decades in history depar tments, where film has become an integral part of the curricula. Despite initial resistance, dramatic narrative films are now recog- nized for their ability to bring the past “to

life,” creating an emotional impact that is thought to exceed that of writ- ten texts. However, most history and humanities faculty are not trained to address the rhetorical codes and nar- rative strategies of cinema. Thus, his- torical films are frequently analyzed empirically and are evaluated accord- ing to the same criteria as are conven- tional historical documents. Without a background in film theory and screen language, students and faculty do not read a film as the product of highly developed systems of signifi- cation embedded in a cultural con- text . By contrast , they have been taught these skills for textual narra- tives since their early stages of reading education.

To read or write the language of media and understand how it creates meaning within particular contexts, one needs some understanding of frame composition, color palette, edit- ing techniques, and sound-image rela- tions, as well as of the mobilization of generic and narrative conventions, the context of signs and images, sound as a

conveyor of meaning, and the effects of ty- pography. Such principles as screen direc- tion, the placement of objects in the frame, color choices, morphing, cuts, and dis- solves all do much more than make a s c re e n c o m m u n ic at i o n a e st h e t ic a l ly pleasing. They are as critical to the creation of meaning as adverbs, adjectives, para- graphs, periods, analogies, and metaphors are to text. Multimedia also requires that attention be paid to design, navigation, and interface construction. The mouse, the click, the link, and the database have already taken their place alongside more traditional screen descriptors.

Outside of schools of film, instruction in these formal elements of multimedia and cinematic construction is not pro- vided in the same way that it is in English or foreign languages. In fact, even the most cursory knowledge of media is not included in the general education curric- ula of most colle ges or universities. Higher education institutions require that students learn not only the content but also the formal techniques of such authors as Steinbeck, Hemingway, and Frost well enough to discuss both the content of their work and their creative style in light of an established body of lit- erary theory. Such work is highly unlikely to be required in media of any kind. At best, one course in “the arts” may be required, but it is unlikely to be a course t h a t r e q u i r e s t h e e q u i v a l e n t o f a language lab.

At the IML, we are committed to em- powering faculty and students to choose the best language for the task at hand. In some cases, this language may well be lin- ear text, and in some cases, it may be one or more kinds of multimedia. To make that choice, a faculty member or student must have a command of the elements of multi- media and screen language and must un- derstand how to use that command to cre- ate and disseminate knowledge.

The Institute for Multimedia Literacy My work with multimedia literacy began a few years ago when one of the most fa- mous alumni of the School of Cinema- Television, George Lucas, asked me a very provocative question. “Don’t you think,” he said, “that in the coming decade, stu- dents need to be taught to read and write cinematic language, the language of the

38 E D U C A U S E r e v i e w � M a r c h / A p r i l 2 0 0 3

Even the most cursory knowledge of media is not included in the general education curricula of most colleges or universities. --------------->>

screen, the language of sound and image, just as they are now taught to read and write text? Otherwise, won’t they be as il- literate as you or I would have been if, on leaving college, we were unable to read and write an essay?”

As I flew back to Los Angeles that day, I knew that he was correct. What I did not fully realize, at 25,000 feet over the Pa- cific, was that what he had so casually sug- gested was likely to be highly disruptive within the academy. I was rather new to the university at the time and assumed that the proposal would be of considerable interest to col- leagues. However, when I called a leading fac- ulty member in the Department of Eng- lish to suggest that we explore ways of incorporating teach- ing some basic multi- m e d i a w r i t i n g i n freshman composition, I was greeted with pro- found silence.

My colleagues in the School of Cinema, however, strongly endorsed the idea, and in the fall of 1998, we began what would become the USC Annenberg Center’s Institute for Multimedia Liter- acy. We started with one course in which students who were not cinema majors were required to create a multimedia project , in their own discipline, that would have the intellectual rigor ex- pected in a five-to-seven-page term paper but that could be “explored” (i.e., read) only on the computer screen. This multi- media project could not be printed out as hard copy. It had to employ sound and image; it had to be time-based; and it had to be interactive. Needless to say, we all found this assignment challenging. These first students came from anthropology, history, sociology, and English literature.

Today, we oversee classes that incorpo- rate work in the extended literacy of multi- media throughout the undergraduate cur- riculum at USC, as well as in satellite classes at Berkeley, at Cal Tech, and in two Los Angeles–area high schools. Each se- mester, we offer a group of faculty the op- portunity to participate in our program for the first time, while faculty from previous semesters continue to work with us. So far,

more than two thousand students and forty faculty have been included. The fac- ulty are assisted by teaching assistants and post-docs from the Division of Critical Studies in the School of Cinema. All are re- quired to attend a summer workshop con- ducted by continuing post-docs but also, and most important, by faculty who have taught in the program previously. Peer mentoring operates at all levels. The goal is to help the incoming faculty members re- think the content of their courses. We ask

them to clearly tell us what it is that they want to teach, and then

we try to help them dis- cover how the language

of multimedia might further that goal. We also ask them to as- sess their current r e s e a r c h t o s e e how the language

o f m u l t i m e d i a m i gh t o f fe r f r e s h

p e r s p e c t i v e s o r a t least suggest a new ap-

proach to their current ef- forts. We hope that in doing so,

they will connect their teaching to their re- search in more direct ways. In the work- shop, they must author their own project, which must be specific to their discipline. At the same time, they are engaged in re- designing their syllabi to include multime- dia projects.

Courses drawn into the program have ranged from gender studies to quantum physics, from Slavic studies to philoso- phy. The projects vary just as greatly in approach. Some are structured around Web sites, as in the case of the Slavic course in which the students developed a site enabling visitors to navigate the text of Crime and Punishment through maps of St. Petersburg. Another student site, cre- ated in an Asian American literature class, explicated the novel Woman Warrior by examining the abuse of women, from foot binding to corseting, using popular culture images that inculcated these prac- tices into belief systems. Other classes have worked with databases, as in the “Culture of the 1960s” course in which the students collected oral histories. The faculty member will use this material both in his own research and in future classes. Most recently, we are seeing the

creation of elaborate interactive games. Over the past few years I have become

more convinced than ever that the rap- idly developing language of multimedia— the language of the screen—can bring im- portant new approaches to research, publication, and teaching. Now we sim- ply have to accept the challenge to em- brace the paradigm shift that is required to bring this vernacular into the academy. Fortunately, I have colleagues who agree. For example Dr. Mark Kann, chair of the USC Political Science Department, re- cently stated: “It seems to me that at some point, multimedia expression is going to be like writing: it’s something you don’t leave college without. Kids are very so- phisticated in navigating on computers and surfing the Internet. I think pretty soon they’re going to have to be as sophis- ticated in expressing themselves using the media. And I wouldn’t be too sur- prised if at some point a multimedia pro- gram that is the equivalent of freshman writing will start appearing at universi- ties. It will become a requirement for graduation.”

The concept of a language composed of elements other than word and text is neither fundamentally new nor particu- larly revolutionary. Rather, this concept is an evolutionary development of the ideas and practices that have been with us since people first struggled to leave records and tell stories. Technology is simply enabling these alternative ways o f c o m m u n ic at i n g to p e n e t rate o u r lives more directly and in more powerful ways. e

Notes 1. Jerry Mander, Four Arguments for the Elimination of

Television (New York: Morrow, 1978). 2. Stephen Toulmin, Cosmopolis: The Hidden Agenda of

Modernity (New York: Free Press, 1990). 3. Sergei Eisenstein, Film Form, trans. and ed. Jay

Leyda (New York: Meridian Books, 1957), 46. 4. Kino-eye: The Writings of Dziga Vertov, ed. Annette

Michelson, trans. Kevin O’Brien (Berkeley: Uni- versity of California Press, 1984), 17–18.

5. John Seeley Brown, “Learning in the Digital Age,” in Maureen Devlin, Richard Larson, and Joel Mey- erson, eds., The Internet and the University: Forum 2001 (Boulder, Colo.: EDUCAUSE and The Forum for the Future of Higher Education, 2002), 71–72, <http://www.educause.edu/forum/ffpiu01w.asp> (accessed January 21, 2003).

6. University of Michigan, President’s Information Rev- olution Commission Report, April 2001, <http://www. umich.edu/pres/inforev2/> (accessed January 21, 2003).

40 E D U C A U S E r e v i e w � M a r c h / A p r i l 2 0 0 3