DBA 701 - 3.2
anoobrosea
CHAPTER4LITERATUREREVIEWS.docxCHAPTER 4 LITERATURE REVIEWS Life is complicated, but that often doesn’t stop people from speaking loudly and confidently about it. Radio talk show hosts have a real knack for pontificating at great lengths about a story they read in the news that morning. Sometimes our friends, family members, and coworkers can act equally self-assured and all knowing, even though they may simply be parroting a claim they heard on television or saw online. “Squawk! Have you heard the latest?” Virtually anyone can participate in casual conversations. There is no peer review screening process, nor even an expectation that you have read anything about a topic before you make a claim or formulate an argument. Many people do keep abreast of current events via daily newspapers, monthly magazines, news websites like CNN.com, or (gods help us) Twitter and other random Internet sources. But—as I argued in Chapter 2—even serious journalism is usually a far cry from the scholarship produced by social researchers. The pre-pontificating background reading that laypersons do is usually pretty meager, regardless of whether they bother to peruse reputable news reports. EXERCISE 4.1 The next time a friend or relative expresses a confident opinion about a social issue, try to make a few polite inquiries about the source of their knowledge. You could say, “Don’t take this question the wrong way—this is something I have to do for class. From what sources did you learn about this issue? Have you read any serious books on this topic? Any peer-reviewed articles written by social scientists?” In contrast, social researchers read—a lot. And it’s quality reading, not just quantity. Before (and as) they engage in a project, researchers usually do an extensive review of the scholarly literature. They familiarize themselves with the latest books and articles on a topic, as well as “classic” works that have stood the test of time. Admittedly, reviewing the literature can involve some quick skimming; some books or articles may turn out to be minimally useful to one’s project, after all. On the other hand, doing a lit review sometimes requires a slow, painstaking, line-by-line examination of densely complicated texts as scholars try to unpack the strengths, weaknesses, assumptions, and implications of prior work. (Students who have taken courses in social theory probably understand well what I am saying here. It can take half an hour to comprehend a single page of text written by a profound scholar.) A journal article contains a reference section that lists the prior works that were cited in the paper. The list of books and articles can be quite long, but no matter how long, reference sections rarely include everything an article’s author has read on the topic. Scholars read (and skim) much more than they end up citing. Why read so much? Authors don’t want to simply parrot what an earlier scholar has said—and peer reviewers and editors usually won’t let them. Instead, authors want their journal articles to advance their fields in some way. Authors need to familiarize themselves with the theories, methods, and findings of prior scholarship so that they can fill a neglected “gap” in the literature, overcome limitations in prior work, or make other kinds of positive contributions. The work of a literature review can inform a scholar’s entire research project. From past scholarship, researchers often derive and refine key concepts, hypotheses, sampling strategies, analytical techniques, solutions to ethical dilemmas, and other helpful ideas and practices. Despite their wide-ranging impact, literature reviews are often associated with the introductory sections of a journal article—everything after the abstract and before the methodology section—because that is where scholars summarize prior research and explain how their article will add to it. In these introductory sections, much gets accomplished. Researchers expertly distill key points from the dozens or hundreds of prior studies that they have read; they identify competing perspectives, uncover important contradictions or disagreements that need to be resolved, and show the connections (and disconnects) between disparate studies—studies that may have been conducted decades apart by scholars in different disciplines and subfields. I have a challenge for you: Pick a random journal article, and compare its (probably extensive) literature review with your findings from Exercise 4.1. My guess is that this comparison will give you an appreciation for the amount of reading scholars do, and the work they put into synthesizing it, before they pontificate. FINDING IMPERFECTIONS IN LITERATURE REVIEWS So far, I have argued that social scientists read more and read better than most laypersons. Casual conversationalists (like talk show hosts) are free to espouse opinions based on any tidbits of news they may hear. Researchers, in contrast, tend to read widely and deeply before formulating arguments and presenting them to an audience via journal articles. We’ve thus established the first half of our guiding theme: Social research is superior to ordinary ways of knowing. Now it’s time for the second half: recognizing the inevitable imperfections within social research. While many weaknesses could be discussed, I will focus on three major dilemmas that all scholars face: selecting relevant publications from the literature, identifying what is relevant in any given publication, and interpreting the meaning of prior works. 1) What counts as relevant research? Imagine you are a graduate student who is preparing to write a doctoral dissertation. And—lucky for you—you’ve developed a fairly passionate research interest early in your training. You have decided you want to study something about the factors influencing the migration of women from the Philippines to the United States. This seems a precise enough topic to start with and one that may eventually lead to a successful research project. As you formulated your topic and collected data to analyze, you would want to read the relevant literature. A dilemma would immediately arise for you, and it wouldn’t go away. The problem is that your research topic—like any research topic—is arguably related to literally hundreds of other similar subjects and to thousands of previous studies. Certainly much of the broad literature on gender might be germane because you have expressed an interested the factors influencing the migration of women from the Philippines to the United States. Much research on family and mothering could also be of interest because some Philippine women may be sending money back home to their own children, even while they help care for their employers’ children. Relevant literatures could also be found in the areas of inequality and work because regional disparities and the pursuit of economic opportunity may be influencing Philippine women’s decisions to seek employment outside their own country. Moreover, the Philippines may not prove to be an entirely unique case; as a serious student, you might want to read studies that examine migration to and from other regions (such as Europe or Latin America) in order to benefit from the insights, comparisons, and historical context that earlier research might provide. I could make similar arguments for the relevance of literatures on globalization, identity, race relations, decision making, emotion management, and many other topics. A cornucopia of research on all of these potentially related topics could be found through a rigorous library search. As I write this, the database Sociological Abstracts—an electronic resource for locating journal articles—monitors more than 1,800 serials for content deemed relevant to sociology, with each outlet publishing hundreds or thousands of pages annually. And this list is by no means complete. Many helpful and relevant articles could also be found by searching databases associated with anthropology, economics, history, political science, psychology, women’s studies, and so on. Plus, thousands of books are published each year in the social sciences. The existing literature is massive and constantly evolving; it could be described as a “blooming, buzzing confusion” (cf. James, 1890), especially for a novice researcher. I could continue, but I think I have said enough to make my point: Drawing the line between the relevant and irrelevant literature is an arbitrary act. Researchers must be selective with their attention because they cannot possibly attend to all the previous literature that might be pertinent to their research projects. The manner in which authors exercise their selectivity has consequences: It shapes how they design their studies and interpret their results. It shapes what they think is known, what they try to find, and what they tell their readers.1 2) What is relevant within a publication? Scholars’ selective attention inevitably limits the number of published works that they choose to read and cite in their research projects. In addition, selectivity also operates at another level. Researchers usually can pay attention to and incorporate only a small amount of the information that appears within any given publication that they read. An average article contains several thousand words. A careful reader could search all those words and find, undoubtedly, hundreds of discernible claims. Authors make claims related to the importance and extent of the problem at hand, the nature of the existing literature, the available methods that have been and can be used to study the topic, the implications their findings have and the directions that future research should take, and so on. Any journal article puts forth numerous wide-ranging assertions. Thus, when reviewing a specific article (not to mention a book), scholars must choose what specific ideas to pay attention to and what ideas to filter out. The vast majority of what authors read (or skim) is not reflected in the final draft of their summaries of prior works. Authors must use their judgment—the best subjective judgment they can render—to decide what is relevant about any given article (or any given page of an article) for the purposes of their research projects.
EXERCISE 4.3 Here is another scavenger hunt assignment. First, choose a relatively recent journal article that interests you—it can be the same article you started with in Exercise 4.2. Then, look at the reference section and pick out an interesting publication the author cited. Obtain a copy of this publication (preferably by downloading it from your library’s website), and read it for yourself. Is there any important information that the authors left out when they cited this previous publication? Does the earlier publication contradict the authors’ article in any way that the authors neglect to mention? Are there differences in theoretical orientations, conceptual definitions, measurement strategies, or findings? Can you make an argument that the authors used information from the earlier publication in a selective (or even somewhat misleading) fashion? 3) What does the existing literature mean? In Chapter 1, we discussed various manifestations of ordinary human inquiry as opposed to social research. One of those manifestations was the practice of consulting a sacred text in order to make sense out of social life. Believers cite or quote the Bible, the Koran, the Book of Mormon, or other religious documents not only to discuss otherworldly matters but to make claims about human affairs. What is the best way to form a family? Is homosexuality a healthy sexual orientation? Is abortion, euthanasia, slavery, or warfare ever justifiable? Thus, a 2,000-year-old passage in the Bible can be used to promote marriages where the husband leads and the wife follows, or a 1,400-year-old passage in the Koran can be invoked to justify the need for women to wear a veil in public. Interestingly, religious texts are always open to multiple interpretations. Liberal believers can read the same text and draw different inferences than their conservative counterparts do. Some Christians infer from St. Paul’s letter to the Ephesians that women should be equal partners in marriages, and some Muslims treat the Koran as saying that veiling is not mandatory for women (Bartkowski & Read, 2003). Believers can debate the translation of key words, the meanings of phrases and sentences, the necessity of reading the text literally, figuratively, as a product of its historical context, and so on. Given the theme of this book—that research is better than ordinary human inquiry yet far from perfect—there are two comparisons I would make between the use of texts by the religious and use of literature reviews by social scientists. Any Text Can Be Criticized and Overturned First, here’s a key difference: Social scientists seem somewhat less likely to treat the truth as something eternally established by any key text. In empirical journal articles, social scientists want to test out ideas, reject or revise outdated ideas, develop new theories, and so on. Scholars are not beholden to the eternal legitimacy of any one text; they are encouraged to develop and propose new truths at every turn and to justify those truths via careful examination of data. Thus, rather than simply claiming that the Bible says (in some passages) that marriage should be between one man and one woman, social scientists are more likely to attempt to collect data on testable research questions, such as “Do children raised by heterosexual parents exhibit a smaller or larger number of behavioral troubles than children raised by same-sex parents?” (e.g., see Patterson, 2006). Any social science book or article—even those written by founding figures of the discipline—is liable to be critiqued or overturned without fear of sanction. In fact, it’s quite the opposite: Kudos go to social scientists who radically reshape how their disciplines approach a particular topic. Arguably, the willingness to critically scrutinize, test out, and overturn ideas (rather than accepting them on faith) is something that is much more common in journal articles than in everyday life.2 I would call it a strength of research—another way social science is (arguably) better than ordinary human inquiry. Any Text Can Be Interpreted in Different Ways My second comparison focuses on the imperfections that researchers share with ordinary human inquiry. I have always found it fascinating (even humorous) how debates between social scientists can sound so similar to religious debates over authoritative texts. Just as there are interpretive camps within religious communities, so too are there within the social sciences. Scholars often do not agree with each other on what the previous literature means. A book, an article, a paragraph, a sentence, a concept—all these and more can be interpreted in different ways by different scholars (e.g., see Fine & Kleinman, 1986; Harris, 2010). In my experience as an author, journal editor, and reader of the literature, I have seen many times how social scientists can adopt different and contradictory viewpoints on the same piece of scholarship. It’s not just that psychologists disagree with sociologists or that political scientists disagree with anthropologists. And it’s not just that subfields can be in contention—as when culturally oriented psychologists disagree with biologically oriented psychologists. Scholars may be trained in the same discipline, and they may work in the same subfield, and they may even share the same theoretical and methodological preferences, yet they may still disagree radically on fundamental questions about the existing literature. And by fundamental, I mean questions like “Is that article good, mediocre, or horrible?” “What, if any, cumulative knowledge has been produced on a particular topic over the past few decades?” And, as we already saw in Chapter 3, “What’s the best way to define the central terms of our analyses?” For any of these questions, answers can vary across disciplines, across camps within disciplines, across subcamps within camps, and so on. Disagreement starts in the peer review process. In evaluating a manuscript, editors usually seek out peer reviews from two, three, or four experts. As we discussed in Chapter 2, editors keep reviewers’ identities confidential in order to encourage brutally honest and critical evaluations. Reviewers usually provide two or three pages of typed feedback outlining what they see as the strengths and weaknesses of a manuscript; they also render judgments about whether they think an article should be accepted outright (which rarely happens), rejected outright (which sometimes happens), or considered again after revisions have been made (which often happens). Disagreements can and usually do arise as some reviewers advise editors to reject, while others recommend to accept with minor changes or invite authors to make major changes and resubmit. Even when reviewers adopt the same general stance toward a paper, they disagree on the details: One may think a particular methodological strategy is strong, but another thinks it’s weak, and so on. An editor may also have his or her own thoughts on the paper. All of these diverse views must be considered as an author strives to compose an article that is “good enough” for an editor, ultimately, to accept. It can be difficult, of course, to accommodate commentators who have incompatible evaluations of a manuscript. (Ask your professors about it; they probably have entertaining horror stories to tell.) After articles (and books) survive the rigorous review process, they are still far from perfect—they are not immune to further criticism and disagreement by scholars who read, cite, and discuss them. Consider two renowned works: Foucault’s (1979) Discipline and Punish and Hochschild’s (1983) The Managed Heart, Each book has been cited thousands of times and is widely considered to be profound and groundbreaking, yet each has been subjected to harsh criticism. Von Schriltz (1999, p. 410) claimed that Foucault was wrong “in virtually every major detail,” and Wouters (1989, pp. 112, 119–120) asserted that Hochschild wrote an “irritating” and “exaggerated” book that lacked an adequate theoretical framework. My own, infinitely less-influential research (Harris 2001, 2006) has been praised as beneficial reading for all social scientists (Kettlitz, 2008) and as a prime example of what researchers should not do3 (Kleinman & Kolb, 2011). Not only do scholars sometimes disagree about the quality of prior work, but the message or implications of a text are also matters of interpretation. For example, Herbert Blumer (1969) coined the phrase symbolic interaction to refer to a theoretical perspective that has influenced disciplines of sociology and communication, among others. Compared to most founding figures, Blumer’s writing is exceedingly clear and straightforward. Nevertheless, disagreements arise about his perspective. Some say Blumer disdained quantitative methodologies, whereas others say he likes them just fine (see Ulmer, 2001). Some say Blumer advocated an approach that focuses on the micro level (face-to-face level) of social interaction, whereas others say Blumer dealt rigorously with larger-level, macro issues (see Maines, 1988). In literature reviews, as in most things, opinions vary. CONCLUSION Let’s recap. I started this chapter by arguing that people don’t read very much in everyday life. In comparison, social scientists read more, and what they read is better—peer-reviewed, scholarly research. Scholars should be commended for reading (potentially) hundreds of books and articles as they engage in their own projects. Their research benefits immensely from the intense literature reviews that they conduct. Nevertheless, scholars are imperfect. They cannot read everything. And, when they do read prior publications, they ignore information that does not neatly fit their interests and agendas. Hence, researchers are doubly selective—selecting from the literature at large and selecting what to pay attention to within any given article or book. Alongside imperfections of selection, scholars must interpret what they read. They render subjective judgments about the quality and the implications of past work. Literature reviews are not simply objective portraits of fields but can be seen as stories that authors tell from their own perspectives (Agger, 2000). Researchers (with feedback from editors and peer reviewers) make subjective decisions about how to characterize the prior work that they have chosen to cite in the limited journal space that they are allotted. If you are a newcomer to the world of journal articles, then literature reviews can seem overwhelming: dense prose, dozens of citations to prior research that you probably have not read, objective-sounding summaries written by smart individuals with PhDs. Who are we, mere readers, to disagree with what the experts say about the literatures they have reviewed? Yet, in this chapter, I have suggested that a persistent reader can launch a meaningful critique of virtually any literature review. Even the most technically sophisticated article can be challenged for its selective and interpretive treatment of prior works. Exercise 4.4, along with the three earlier exercises, provides some guidance on how to put this chapter’s ideas into practice.
EXERCISE 4.4 From Exercise 4.3, you should have one relatively recent article (let’s call it RA) and one earlier article (let’s call it EA) that was published several years prior. The authors of RA cited and discussed EA. Your mission—should you choose to accept it—is to critically evaluate what the authors of RA said about EA. You have two options: 1. Make an argument that the authors of RA actually misinterpreted EA. Did the authors of RA characterize EA in a way that you can reasonably disagree with? For example, did the authors of RA describe EA as about topic X, but you think EA is more about topic Y? a solid piece of scholarship but (in your opinion) EA has fatal flaws? a weak piece of scholarship, but (in your opinion) EA has many strengths? These bullet points are merely suggestions; if you are creative, you might come up with additional arguments you could make. 2. Make an argument that different scholars have interpreted EA differently. To do this, use Google Scholar (or another database) to find a few more articles that also cite EA. Download them. Search within these recent articles (electronically, if possible) in order to find the pages on which the authors of EA are mentioned. Read carefully, and determine if EA is characterized inconsistently across RA and all of your other recent articles. Warning: This option may take a few attempts and could be challenging!
