Research again

3/27/2019 Evaluating Research

https://ncuone.ncu.edu/d2l/le/content/122307/viewContent/1252316/View?ou=122307 1/12

CHAPTER 3

Reading a Research Report

People don’t usually do research the way people who write books about research say that people do research.

—A.J. Bachrach (1981)

Overview

The purpose of this chapter is to provide you with an overview of how to read a research report. You will learn about a technique to enhance your critical-reading skills. You may not have enough knowledge about methodology and statistics to understand everything in a research report, but you will learn about general style, as well as the difference between what is typically included, what is sometimes included, and what is generally not included in a research report. You will also learn about the traditional organization of a research report.

INTRODUCTION

Good critical reading requires practice, thought, and awareness. The more critical reading you do, the more your reading will improve. The more you think about what you are reading, and the more you pay attention to what others have written about what they read—good or bad—the more your reading will improve. To become a better reader, read more and write about what you read. Beyond reading more, however, there are specific techniques that can be used to improve your critical-reading skills. One of these is the SQ3R method—Survey, Query, Read, Recite, and Review—a technique designed to enhance understanding while reading.

The SQ3R method of reading (Robinson, 1970; Vacca & Vacca, 1999), slightly adapted for higher-level materials (and readers), will help you to organize what you are reading. It takes a little practice and seems somewhat awkward at first, but it is well worth the effort. It is a very effective method for ensuring that one understands technical material.

The “survey” part of the technique represents obtaining an overview of the material to be read. For research articles, this can be accomplished by reading the title and abstract; you also should examine any tables and figures contained in the article. As you survey, you should “question,” which involves asking as many questions as possible about the title, abstract, tables, figures, section headings, et cetera. Think about what each part of the report might mean, wonder how it fits with the rest of the material, wonder about how the graph or table was constructed, think about alternative words that could be used. The more questions you ask, the more focused your reading will be.

p. 60

p. 59

03/28/2019 - RS0000000000000000000001583532 - Evaluating Research

3/27/2019 Evaluating Research

https://ncuone.ncu.edu/d2l/le/content/122307/viewContent/1252316/View?ou=122307 2/12

you as , t e o e ocused you ead g w be. The first of the three Rs stands for “read,” which is an obvious component of reading a research report,

but in this case the reading should be done actively. Don’t just allow the words to pass in front of your eyes; try to answer the questions you asked during the survey phase of the process. As you encounter something you do not understand, stop to figure it out, look it up in a textbook or search online for information, and use other techniques to ensure you understand what you are reading before continuing. I find that “read” works best when I do it one paragraph at a time; I make sure that I understand each paragraph before attempting to read the next paragraph.

The second R represents “recite,” which stands for articulating in your own words what you just read in the paragraph. I prefer to “recite” using my pen; I try to write, in my own words, the point that was made in the paragraph I read. Whether you actually recite or write, the purpose of this R is to test your knowledge of what you have read. Note that using a highlighter to mark passages you think are important is not sufficient; highlighting only marks the author’s words. Instead, you should express the author’s idea in your own words; either say or write what you think the passage means. If you can express the point in your own words, then you are more likely to understand the material. I used to write in the margin of whatever I was reading, but now I “recite” while taking notes in a document on my computer. My “reciting” forms an outline of what I’ve read, and the notes are very convenient for the final R, “review.” Before you decide that you have read and understood a research article, review the notes you have made to make sure that those notes still make sense in the context of the entire article. You may discover, for example, that some of your earlier notes can be improved as a result of your more comprehensive understanding of the entire article.

That is about all I can include in this chapter about how to improve your reading, for to comment any more would require a one-on-one, in-person session. Instead, the remainder of this chapter deals with why and how to read a research report. There are some general (and a few specific) conventions adopted by those who write reports about research, and we will examine these conventions in this chapter. Obviously, I cannot provide comments about specific reports you may read, nor does it make much sense to predict the kinds of reports you will read. Therefore, the examples from reports that are included herein are from my own writing, which also enables me to provide some indication of the underlying purpose and intent concerning specific passages.

RESEARCH REPORT ORGANIZATION

The purpose of writing a research report is simple: to inform others about the research the investigators conducted. This simple purpose involves the assumption that others want to be informed about the research. Part of what writing a research report is all about is writing the report so that others will want to read it, but the audience for which most researchers write is other researchers. This means that authors of research articles focus on the things that they believe other researchers want to know—the theoretical and empirical background of the research, the procedures used to collect the data, the results of the data analyses and how they were obtained, and the conclusions drawn from the results—and they create that focus using a particular style. You will probably encounter a fair amount of jargon, such as the statement “The interaction effect was reliable,” as you read reports. You need to be familiar with such jargon in order to understand the article. In addition to jargon, you also will encounter passive-voice writing, such as “the subjects were assigned randomly,” which is more difficult to follow than material written in active voice, and most of what you read will be in the third person because researchers consider the primary focus to be the data, not who collected the data. Finally, you will encounter very succinct writing, which is much denser than, say, a novel or even an advanced textbook. In short, you should be prepared for reading much more slowly and much more carefully than you have previously done.

The organization of a report of empirical research is fairly standard among behavioral and social sciences—indeed, most sciences. In psychology, the standard is comprehensively described in the Publication Manual of the American Psychological Association (American Psychological Association, 2010). Slightly different formats may be used to present the organized material, but every research report is likely to contain an introduction, sections for method and/or procedures, results, discussion, and references, and a summary or abstract of the report.

The major sections—introduction, method, results, and discussion—are often described as analogous to an hourglass of information through which the author guides the reader. As depicted in Figure 3.1, the report begins with one or more general statements that place the research in context followed by increasingly

p. 62

p. 61

03/28/2019 - RS0000000000000000000001583532 - Evaluating Research

3/27/2019 Evaluating Research

https://ncuone.ncu.edu/d2l/le/content/122307/viewContent/1252316/View?ou=122307 3/12

begins with one or more general statements that place the research in context, followed by increasingly more specific statements until the reader is informed of the intentions and hypotheses of the research. The method section includes information about how the data were collected, the results section contains what was found, and the discussion is used to bring the reader back to the general context. As the center of the hourglass, the results are the focus of the research report. For this reason, you cannot consider your understanding of the research to be more than a tentative idea until you understand the results.

Figure 3.1

The Hourglass Analogy of the Research Report. The introduction begins leading the reader from very general issues to successively more specific issues until the results, the center point of the article, are presented. Then, the reader is led back from the very specific results to the most general implications of the results.

Introduction

To explain the purpose of the research, most researchers include an introduction, which enables them to explain, and you to learn, how the particular project described in the report fits into the general scheme of inquiry.

General Context

Below is a portion of the first part of the introduction from the report Dane and McPartland (1983) wrote about research on judgments of the propriety of various public displays of affection:

Intimacy between two people is often inferred from the manner in which they relate to each other at a nonverbal level. Since the publication of Hall’s (1959) The Silent Language, research on nonverbal indicators of intimacy has grown at a geometric rate, and touch occupies a primary position within th lit t C f i i d th ti l l (P tt 1976

p. 63

03/28/2019 - RS0000000000000000000001583532 - Evaluating Research

3/27/2019 Evaluating Research

https://ncuone.ncu.edu/d2l/le/content/122307/viewContent/1252316/View?ou=122307 4/12

the literature. Coverage ranges from rigorous reviews and theoretical proposals (Patterson, 1976, 1982) through textbook summaries (Kleinke, 1975; Malandro & Barker, 1983). Throughout this literature, however, authors have focused on people directly involved in interpersonal exchanges. Observers’ reactions have been neglected.

Hall’s volume, for example, is devoted almost entirely to interpreting nonverbal behaviors from the actors’ viewpoints. He offers only a few examples in which the reader is asked to imagine amazement upon observing peculiar behaviors. Kleinke (1975) similarly devotes several pages to touching, but only from the perspective of involvement. Malandro and Barker (1983, p. 270) concluded “the sender and receiver recognize that touch carries messages that are often more dominant than the words we speak,” but failed to mention inferences drawn by observers. A thorough review of the literature yields many more analyses of touch from participants’ viewpoints, but no analysis of observers’ viewpoints. (Dane & McPartland, 1983)

In two paragraphs, we presented the general area into which our research fit (nonverbal behavior), introduced the specific aspect on which we would concentrate (observer’s reactions to touch), and provided several examples to buttress our argument that research is needed in this area. We established the beginning of the hourglass pathway through which we would lead the reader.

As a reader, you now know where to go for more general information about the topic because we provided references. You also know specifically what the research report is about—observers’ reactions to interpersonal touch. Such context is important to understand if you are to understand the remainder of the report.

Literature Review

After presenting the general context of the research, authors next provide information about other research that is relevant to the specific project described in the report. Just as understanding the literature is an important part of designing a research project, understanding the empirical and theoretical context is important to understanding and interpreting the results of a project someone else conducted. Without knowing how this particular project fits into the grand scheme of knowledge, you are not likely to be able to make a judgment about the relative importance of the results for your purposes.

The literature review does not contain everything the authors ever read on a given topic; it is tailored specifically to place the individual project in context. Consider the following portion of a literature review:

A decade ago, Hunt (1974, p. 1) dramatized the frequency with which observers were exposed to public displays of intimacy by noting “the unprecedented spectacle of sexually normal young people stripping off their clothes and having intercourse out in the open.” Indeed, about 70,000 people in the U.S. were arrested for “sexual misconduct” in 1980 (Flanagan & McLeod, 1983). Somewhere along the continuum from holding hands to sexual intercourse, observers’ primary reactions to public touching change from a simple inference about intimacy to marked disapproval. At what point that change takes place is not known. For that matter, we can only assume the existence of general cognitive structures about reactions to interpersonal intimacy.

The paucity of research on observers’ reactions to public touching involves only general attitudes. Excluding research on erotica presented through mass media, we found only one study on the topic. Crawford and Crawford (1978, p. 396) reported individuals’ reactions to the statement “Public displays of affection are in poor taste” were highly correlated with items concerning public nudity, sexually explicit reading material, and so on. They concluded reactions to public intimacy are a subset of general attitudes toward sexuality. Conceptual leaps aside, it seems overly simplistic to assume reactions to observing public touching will mirror, in substance and form, reactions to engaging in the same behavior. (Dane & McPartland, 1983)

These two paragraphs from the center of our introduction (Dane & McPartland, 1983) accomplish three purposes. First, they provide the reader with information about existing research. Second, they provide some indication of the reasons that existing research is not sufficient. Finally, they lead the reader further into the hourglass to the point at which we presented specific information about our study:

p. 65

p. 64

03/28/2019 - RS0000000000000000000001583532 - Evaluating Research

3/27/2019 Evaluating Research

https://ncuone.ncu.edu/d2l/le/content/122307/viewContent/1252316/View?ou=122307 5/12

The purpose of the present study is to assess individual attitudes toward the propriety of specific displays of intimacy in public. Relationships among different displays were explored to determine the extent to which attitudes reflect categorical reactions. Reactions to kissing and hugging, for example, may be considered similarly appropriate but distinct from reactions to mutual fondling or body contact sans clothing. In light of the issues addressed by Paul, Weinrich, Gonsoriek, and Hotvedt (1982), attitudinal differences concerning heterosexual and homosexual couples were considered likely. It would seem straightforward to predict that a male kissing another male in public would be judged less appropriate than a male kissing a female. It may also be the case that categorical reactions differ for heterosexual and homosexual couples. (Dane & McPartland, 1983)

Little by little, we led the reader from a very general statement about human behavior to a specific statement about our research, but we did it in relatively few words. Notice also that the above paragraph— the last paragraph of the introduction—included some predictions about what we expected to find in the research. Of course, those predictions were not really predictions; we had already analyzed the data and knew the outcome. The last paragraph of the introduction serves as a transition to the specific details of the research, just as this sentence serves as a transition to the next section about methods.

The Method Section

The method section of the report is the section wherein the researcher attempts to accomplish two things. First, the researcher provides enough information for you to understand how he or she collected the data and to decide whether or not the data collection is appropriate for your purposes. In one sense, the method section is a secondary introduction, for it serves to guide you to the results. The second purpose involves providing enough information to enable someone to replicate the research. Thus, the method section is also an archive.

Participants

The first subsection of the method section is usually a description of the participants. It contains information about who they were, how they were selected, and any other pertinent information. From Dane and McPartland (1983):

The participants were 73 male and 113 female undergraduate volunteers attending a small state college in upstate New York. The median age was 19, and each received partial course credit for participating.

Brief though it may be, there is much to learn from these two sentences. We included information about how many, gender, regional location, and age. Other than the label “volunteers,” there is no specific information about sampling procedures, but the label says it all. By calling them volunteers and not providing additional information about sampling, you can infer that the sample was an accidental sample; the participants were included in the study because they were conveniently available. Although information by omission may not be the best way to describe our sampling procedures, such is the convention in research reports. Thus, every word, even words not included, may be important to your understanding of the project.

Sampling

When researchers use a procedure other than accidental sampling, they will explain the procedure they used. You might, for example, read “Random digit dialing was used to obtain a random sample of residents in South Carolina. Business and other nonresidential phone numbers were eliminated, which produced the final sample of 600 residents.” There is not a lot of detail, such as describing the computer program that generated the random phone numbers, unless the procedure is so unusual as to require such explanation.

p. 66

03/28/2019 - RS0000000000000000000001583532 - Evaluating Research

3/27/2019 Evaluating Research

https://ncuone.ncu.edu/d2l/le/content/122307/viewContent/1252316/View?ou=122307 6/12

Thus, authors rely a great deal on the knowledge of their readers. You are expected to know enough about research methods and procedures to be able to interpret the two sentences about random digit dialing, so you need to be a very active reader when reading research reports.

Apparatus or Materials

The apparatus or materials subsection is included if the researcher used any special equipment or materials in the study. These descriptions are included so that others may duplicate the research procedures as well as to enable readers to understand the procedures. Of course, common equipment is usually not described; no one really cares what brand of pencil was used to record observations. We wrote:

The face sheet of the questionnaire contained instructions for completing the instrument, and items concerning age, gender, and religious preference. The second and third page contained a list of 26 behaviors, ranging from holding hands to sexual intercourse, with instructions that each behavior was to be rated as to its propriety as a public display of affection. The seven-point rating scale ranged from a −3 (strongly inappropriate) to a +3 (strongly appropriate), including a labeled neutral point. Separate ratings were made for a male-female, male-male, and female-female couple, and the items were listed on the page in a randomly determined order. (Dane & McPartland, 1983)

Included in the above paragraph are examples of the types of items and a very explicit description of the response scale. Rarely is a copy of the actual instrument included. Only those readers interested in replicating the research exactly would need to see the entire questionnaire, so there’s no need to add such lengthy material to the body of the report. If you need to see the exact instrument used in the study, you can always write to the researcher(s) and ask for a copy. Almost all researchers are happy to comply with such requests.

If the questionnaire had been described in a previous report, whether ours or someone else’s, we would have provided a more general description of the instrument and referred the reader to the more detailed published description. Someone else might write, for example, “The questionnaire used by Dane and McPartland (1983) was used to obtain attitudinal information about a variety of public displays of affection.” Such a statement would then be followed by a general description of the questionnaire, a description that would enable the reader to understand what was done without having to look up the Dane and McPartland reference. Until you become familiar with the measurement techniques used in your area of interest, however, I strongly recommend that you look up the original reference so you can be sure you understand the full scope of what was measured in the research.

Procedure

The procedure subsection contains details about the specific manner in which the data were collected. Enough detail is usually included to accomplish the archival purpose of this section, as well as to enable you to empathize with the participants. That is, the procedure is usually written from the participants’ point of view, which is necessary to understand the participants’ reactions (the results). In our report, we wrote:

To ensure anonymity, participants completed the questionnaire in groups of 10 to 20. For each session, a male researcher described the study as an investigation of reactions to public displays of affection. Participants were asked to rate the propriety of specific displays of public affection, for which public was defined as any location where one could reasonably expect to be observed by others. It was explicitly noted that those who believed they would be offended by the task should complete only the face sheet of the questionnaire, but no one exercised this option. (Dane & McPartland, 1983)

Notice how much information we packed into the above paragraph. If you were reading actively, you now know that the questionnaire was administered anonymously and in small groups. You know what instructions were provided and how informed consent was attained. You don’t know the exact wording of

p. 68

p. 67

03/28/2019 - RS0000000000000000000001583532 - Evaluating Research

3/27/2019 Evaluating Research

https://ncuone.ncu.edu/d2l/le/content/122307/viewContent/1252316/View?ou=122307 7/12

the instructions, but if that wording had been important to the data collection process, we would have provided a direct quote. Because the procedure involved completing a questionnaire, there was no need to describe manipulations of an independent variable or to describe graphically the physical environment.

Manipulations

When research involves experimental manipulations, they are described in sufficient detail for the reader to understand them and again from the participant’s viewpoint. For example, you might read something like this:

Three different feedback groups were created. In the Positive Feedback group, we told participants they correctly answered 45 of the 50 test questions. In the Negative Feedback condition, we said they correctly answered only 5 of 50 questions. Participants in the No Feedback group did not learn about their performance.

Notice that the labels given to the various groups are informative labels. Notice also that, as an active reader, you will be expected to remember what those labels mean throughout the remainder of the article. When the procedure in the study is very complicated, researchers will sometimes provide a brief summary of the method and design before moving on to the results section.

The Results Section

As is implied by the name, the results section is where researchers report the results of the data analyses. It is the section around which the entire report is centered. Researchers want to be sure readers understand the results, and so they generally follow the rule of thumb used in oral presentations: Tell them what you are going to tell them, tell them, and tell them what you told them. Following this rule, however, does not mean the researcher will be redundant; instead, it means the information is often presented at three different levels. I’ll provide examples as we move on to the preliminary and main subsections of the results section.

Preliminary Results

Just as the overall research report requires an introduction to prepare the reader for what is to come, the results section may include an introduction to prepare the reader for the main results. Depending on the procedures, there may be any number of things accomplished at the beginning of the results section.

If the researcher used a manipulation or selected participants on the basis of some pretest measure, the researcher will usually present information about the success of the manipulation or selection process. Thus, the first part of the results may include the results of analyses that are used to describe the impact of the manipulation or selection procedure. For example:

Participants rated their level of performance on a ten-point scale ranging from poor (1) to excellent (10). These ratings were used to determine whether the feedback manipulation was effective, and produced a main effect for Feedback when subjected to Analysis of Variance, F (2, 59) = 10.25, p < .05; the manipulation was successful. Participants in the Positive Feedback group rated their performance (μ = 8) better than those in the No Feedback group (μ = 5).

Included is information about the manipulation, the result of the analysis, an interpretation of the result, and a clarification. The quote follows the rule—tell them three times—without writing the same thing three times. An active reader will learn new information from each repetition.

Other material in the preliminary results might include analyses of interrater reliability, descriptive statistics about the participants, information about data eliminated from the analyses, and any other results that move the reader toward the main results. How much information is included in a preliminary results subsection depends on the specific nature of the study. In the public displays of affection report, for example, we used the beginning of the results section to convince the reader that the participants’ ratings

p. 70

p. 69

03/28/2019 - RS0000000000000000000001583532 - Evaluating Research

3/27/2019 Evaluating Research

https://ncuone.ncu.edu/d2l/le/content/122307/viewContent/1252316/View?ou=122307 8/12

were sensible:

Table 1 contains the overall mean propriety rating for each of the 26 behaviors, according to type of couple. The participants clearly discriminated among the items, which are ordered from most to least appropriate within ratings for the male-female couple. Of particular interest is the trend for female couples’ behaviors to be rated consistently less appropriate than the male-female couples and, without exception, the trend for male couples’ behaviors to be least appropriate. These trends are not overly surprising in light of Gross, Green, Storck, and Vanyur’s (1980) findings concerning prevalent negative reactions to overt homosexuality. (Dane & McPartland, 1983)

I have not included the table from the article; it is too long to merit the space as an example here. The table in the report exhibited the properties described in the text. It is evident from the table that the participants rated holding hands, for example, as more appropriate than kissing, and so forth. The results are not surprising, but surprise was not the point. Rather, the purpose was to convince the reader that the data make sense, that the participants did what they were expected to do. In this case, active reading would enable you to make decisions about the extent to which we measured reactions in an appropriate way, that our participants were not atypical, and that the results we obtained are relatively consistent with related results reported by other researchers.

Another part of the preliminary results subsection may contain an overview of the statistical procedures used to analyze the bulk of the data. This part of the results section is usually included because it serves to orient the reader—to enable the reader to develop expectations about the kinds of information to come—but is sometimes neglected. Any transformations performed on the data will be included at this point in the results section. We wrote:

To explore the potential structure underlying the ratings, separate factor analyses were conducted for each type of couple. Because the analysis was exploratory, all factors yielding eigenvalues greater than one were retained prior to orthogonal rotation. For all three couple types, four factors emerged, accounting for 64%, 72%, and 67% of the variance for male-female, female-female, and male-male couples, respectively. The order of emergence was identical across couple type, although specific item loadings differed among couple types for the fourth factor. (Dane & McPartland, 1983)

Active reading enables you to decide whether or not factor analysis is appropriate to the purpose. Factor analysis is a statistical technique for investigating structure among a number of variables (see Chapter 4). Essentially, it can be used to group variables together based on their correlations in much the same way that those who write college admissions tests group items into separate subject categories. Even though this paragraph dealt with preliminary results, we began to discuss some of the main results of the study. We pointed out, for example, that each of the three sets of ratings produced four factors. Beginning to present main findings in the last paragraph of the preliminary results is one way to provide a transition from one subsection to the next. As an active reader, you need to be attuned to such transitions so that you don’t miss some main results that are presented in what appears to be a preliminary results section.

Main Results

When researchers report the main results of the study, they usually present the most central or most important results first, then present decreasingly important results. The same format is typically used for each result: a general description, more specific information, and finally relevant qualifications.

For each set of results, what question the result is supposed to answer is presented. In the above paragraph from our report, for example, we reminded the reader that we wanted to explore the structure of the propriety ratings. Then we restated the result in terms of the specific operational definition used to measure the variable. This is typically followed by a prose statement of the result, such as “The ratings produced four factors for each type of couple.” Results are often interpreted as they are presented, but some researchers will present all of the results before engaging in interpretation. As an active reader, you want to think about the interpretation that is presented and make sure that you understand the interpretation and

p. 71

03/28/2019 - RS0000000000000000000001583532 - Evaluating Research

3/27/2019 Evaluating Research

https://ncuone.ncu.edu/d2l/le/content/122307/viewContent/1252316/View?ou=122307 9/12

agree with it. If you are reading for a specific purpose, such as the preparation of a review of research for policy analysis purposes, you may also want to interpret the results in terms of the policy you are analyzing. The researcher’s interpretation is not the only interpretation that is allowed. The statistical results are presented to support the conclusion, but you can draw a different conclusion if that conclusion makes sense with respect to the methods employed by the researcher.

Qualifications or elaborations of the results are presented using the same format: prose, numbers, and prose. For example, the excerpt below, which did not follow the above paragraph in the results section, concerns analyses conducted on the scores produced by the factor analyses mentioned above:

Additional exploration of the reactions involved determining whether the participants’ characteristics were related to their reactions. Item loadings were used to create weighted factor scores ranging from negative one (inappropriate) to positive one (appropriate). These were then subjected to a 2 (Gender) × (Religion: Catholic, Jewish, Other, None) × 4 (Factor Label) × 3 (Couple Type) Analysis of Variance.

The analysis yielded a reliable main effect for Gender, F (1, 172) = 4.10, p < .05. Males (μ = 0.07) considered the behaviors to be generally more appropriate than did females (μ = −0.05). The only other reliable effect was a Factor Label × Gender interaction, F (3,516) = 8.27, p < .0001, for which the means are presented in Table 6. As is evident from the table, women rated all but the asexual behaviors as less appropriate than did men. (Dane & McPartland, 1983)

Each finding is first presented in prose, then in numbers, and then again in prose. Each statistical analysis is clearly identified, and each effect is accompanied by its level of statistical significance. For the simpler main effect, however, the means were included in the text. The table (labeled Table 6 because it was the sixth one we included in the report) was used to clarify the interaction effect because it was more efficient to present the eight means in tabular form than in prose. Notice, however, that a prose clarification of the interaction was included in the text. The active reader is therefore able to compare the prose to the table and consider what other descriptions (more relevant to the reader’s purpose) might also fit the table. The same point may be made about figures—graphs, pictures, and drawings—that are used to convey information more efficiently than it could be conveyed in prose or a table.

Notice also that each effect was accompanied by a small amount of interpretation, such as “males considered the behaviors to be generally more appropriate than did females.” The active reader should think about such interpretation to make sure it is consistent with the numbers presented. The small amount of interpretation in the results section should not be accepted at face value; read actively and decide for yourself whether the interpretation is consistent with the numerical results.

Conclusions

The conclusions drawn from the results are placed in the discussion section of the report. This is where the researcher begins to expand the hourglass, beginning with a restatement of the main results. In some respects, the discussion section is the reverse of the introduction: Instead of leading the reader to the results (as was the case in the introduction), the researcher is leading you away from the results toward the general points he or she wants to make. The conclusions should logically follow from the results, but some researchers take the reader further from the results than others. It is unlikely that a published article will contain just any old thing in the discussion section, but it’s certainly possible that a research report could contain conclusions that are less than apparent from the results.

Typically, the discussion is begun with a restatement of the main research question and a summary of the relevant results. For example:

In general, the data demonstrate that observers do make attributions about public displays of affection. Specifically, judgments about the propriety of the behaviors conform to expectations based on the level of sexual intimacy for the behavior. More important, the data show that propriety may be judged on the basis of categorical information about groups of behaviors. (Dane & McPartland, 1983)

p. 73

p. 72

03/28/2019 - RS0000000000000000000001583532 - Evaluating Research

3/27/2019 Evaluating Research

https://ncuone.ncu.edu/d2l/le/content/122307/viewContent/1252316/View?ou=122307 10/12

This is a very full first paragraph for a discussion section. It includes a restatement of the research question: Do people react to intimate touching other than registering it as an intimate act? It also includes the point that observers’ reactions may be categorical, and the conclusions are limited to the data collected. All this is done, however, without merely restating what has already appeared in earlier sections of the report. Even though the researcher is continuing along the path defined by previous sections of the report, the discussion section will not include a mere rehash of what has already been written; active reading continues to be a necessary skill. The discussion section will contain information not previously presented in the report, and you need to be prepared to consider critically the new knowledge and understanding of the study it contains.

Although we could not do it in the public displays of affection report, the discussion typically will also include efforts to relate the results to previous research on the topic. (We could not do it because there was no previous research.) The researcher will not mention every study ever done on the same topic but will provide enough information to place the results in a theoretical context. Differences between the researcher’s results and those of previous researchers will usually be noted, but researchers are not likely to dwell on every detailed difference between their results and others’ results. If the result is related to the policy you want to analyze, pay particular attention to noted differences and follow up with the cited article so that you understand those differences and how they may relate to the policy you are analyzing.

Toward the end of the discussion section, the researcher may present what he or she considers questions that remain unanswered despite the results or questions that are raised because of the results:

Furthermore, research and theory that may be stimulated by the current exploration should include consideration of the discrepancies between heterosexual and homosexual behavior. For example, there is a considerably wider range of behavior that our participants considered sexual for homosexuals than for heterosexuals, even though the same behavior was rated for both couple types. It is clearly the case that observers process and react to the public behaviors of others, and it is time we began to include observers’ inferences in our theories about the meaning of touch as a means of nonverbal communication. (Dane & McPartland, 1983)

Because the study was exploratory and the results may be limited to the sample we used, the suggestions about future research are rather general. The more specific the hypotheses with which the researcher has dealt, the more specific will be the recommendations that the researcher will include. The fact that there remain unanswered questions, however, should not be taken as an indication that the research was somehow insufficient, incompetent, or incomplete. A single research study almost always is about a single, very finite issue or question. You, as the active reader and eventual review writer, will be able to put all those little answers together to form a more general answer about the policy you are analyzing.

One last note about the discussion section needs to be made: The discussion section will probably not contain any surprises. Therefore, don’t be lulled into thinking that the researcher has nothing new to present in the discussion section. Active reading is required to catch and process the sometimes subtle additions to what you already know about the project. There may not be any surprises, but if you think there is nothing at all new in the discussion, then you are probably not reading actively.

Other Sections

The introduction, method, results, and discussion sections constitute the bulk of the research report. There are, however, a few other sections that may appear in a research report. Even though they may not be included in the limelight, they are equally important in terms of the information an active the reader can glean from them. They include the abstract and the references. Again, the American Psychological Association’s (2010) publication manual is an excellent source for additional information.

Abstract

Regardless of the length of the report, most reports will contain a summary, usually called an abstract and printed at the beginning. An abstract includes about 100 to 150 words, which is not many words in

p. 75

p. 74

03/28/2019 - RS0000000000000000000001583532 - Evaluating Research

3/27/2019 Evaluating Research

https://ncuone.ncu.edu/d2l/le/content/122307/viewContent/1252316/View?ou=122307 11/12

p g g , y which to summarize a research report that may be as long as 25 pages. More than any other section of a research report, then, an abstract is incredibly succinct and requires active reading to decode. It also requires patience; you may have to reread an abstract several times before you understand all that it has to offer. Consider the abstract from the public displays report:

The purpose of this study was to explore observers’ reactions to public, intimate behaviors. College students rated the propriety of imagined behaviors ranging from holding hands to sexual intercourse for opposite- and same-sex couples. Behaviors for the heterosexual couples were consistently rated as most appropriate. Factor analysis of the ratings yielded four categories: genital contact, other erotic contact, asexual contact, and an inconsistent factor that varied with type of couple. Suggestions for future research included potential underlying cognitive processes of observers and the need to incorporate such processes into current theories about nonverbal behavior. (Dane & McPartland, 1983)

Some people are tempted to read only the abstract of a research report. It does, of course, contain the main points of the complete report, but an abstract does not contain any of the qualifications and elaborations necessary to understand those main points. The abstract is essentially a teaser, but if you stop reading at the abstract, you will miss out on a great deal of information that may be helpful in your policy analysis. You have already read enough of the public displays report to know just how little of our results are contained in the abstract.

On the other hand, you may use the abstract to decide whether or not to read the full report. This is a perfectly legitimate use of the abstract; indeed, it is the purpose for which the abstract was written. If the abstract contains nothing that is relevant to your interests, then the full article probably will not contain relevant material, either. However, if there is one shred of relevance apparent in the abstract, you would do very well to read the entire article.

References

Any published material to which the researcher refers in the report will be referenced in what is called, straightforwardly enough, the references section. Styles for references vary from journal to journal, but the predominant style in the behavioral and social sciences is that recommended by the American Psychological Association (2010) in its publication manual. The references are an excellent source of additional material to read as you prepare your research review.

Enough. The only way to improve your reading skills is to read some research articles. So go and read a research report (but first read the summary).

SUMMARY

Good reading requires practice, thought, and critical thinking. The more you do of any of these things, the more your reading skills will improve and the more efficiently you will be able to read research reports. The purpose of reading a research report is to find out about the results the researcher obtained. The results are the focal point of any research report. Research reports are organized in an hourglass fashion. First is presented information about the general phenomenon, then more specific issues related to the phenomenon, next the details about the research procedures, then the results, and finally interpretations that typically end as generally as the introduction began. Because scientific writing is succinct, active reading is required to obtain as much information from the article as the researcher put into it. Passive voice, third person writing, and past tense make it difficult to read a research report quickly. The SQ3R method of reading will help you read more efficiently. The introduction typically begins with a statement about the behavior of interest; then it should lead

p. 76

03/28/2019 - RS0000000000000000000001583532 - Evaluating Research

3/27/2019 Evaluating Research

https://ncuone.ncu.edu/d2l/le/content/122307/viewContent/1252316/View?ou=122307 12/12

yp y g ; you logically and straightforwardly to the specific hypotheses of the study. The method section should describe the way in which the data were collected. It should contain enough detail to enable you to empathize with the participants and to replicate the study. The results section will sometimes include a preliminary exposition of the major types of analyses used. It will include all results obtained from the data, ordered so that the most important results are presented first. The discussion section usually opens with a brief summary of the major hypotheses and the results relevant to those hypotheses. Further generalization from the results proceeds in the same logical and straightforward organization used in the introduction. The abstract will include the main points of the complete report but will be extremely brief. You should not count on the abstract containing all that you need to know about the research. References may be useful in identifying and obtaining additional material relevant to the policy you are analyzing or background information that will be useful in understanding the specific information contained in the research report. Stop reading this book now, and go read a research report.

EXERCISES

1. If you have not already done so, practice the SQ3R method while reading this chapter. 2. Find a research article relevant to your interests (or use one you already have) and identify the

paragraph(s) corresponding to Figure 3.1. 3. Using an interesting research article, identify parts that you do not understand. Make a list of the

things you need to know in order to understand this article. Do the authors provide references that you can read for additional clarification?

4. Find a popular-press report of one of the articles you have found and compare the popular-press version to the research article.

p. 78

p. 77

03/28/2019 - RS0000000000000000000001583532 - Evaluating Research