2 part paper
,ffid; ,ffiffit#.$ Objectives nL. Describe the three basic
goals of research reports.
; Identify unique problems that must be overcome in writing student papers, theses, applied research reports, and journal articles.
J" List the major sections of a research article.
,- Describe the eiements that should be considered in writing research reports to ensure adherence to the ethical standard of honesty.
') Discuss the motivations for plagiarism and the ways to avoid suspicions of plagiarism.
i- Identify major steps in the review of research reports.
7" Understand the need to display results without redundant information using compressed displays.
Vo, learned in Chapter 2 thatresearch is a circular process, so it is appropri- I ate that we end this book where we began. The stage of reporting research
results is also the point at which the need for new research is identified. It is the time when, so to speak, the rubber hits the road-when we have to make our research make sense to others. To whom will our research be addressed? How should we present our results to them? Should we seek to influence how our research report is used?
The primary goals of this chapter are to guide you in writing worthwhile reports of your own, displaying findings, and communicating with the public about research. This chapter gives particular attention to the writing process itself and points out how that process can differ when writing qualitative ver- sus quantitative research reports. We highlight the goals of research reports, including expanding the discussion of participatory action research (PAR), which we introduced in Chapter 1. We will conclude by considering some of the ethical issues unique to the reporting process, with special attention to the problem of plagiarism.
The goal of research is not only to discover something but to communi- cate that discovery to a larger audience-other social scientists, government officials, your teachers, the general public-perhaps several of these audiences. Whatever the study's particular outcome, if the research report enables the intended audience to comprehend the results and learn from them, the research can be judged a success. If the intended audience is not able to learn about the study's results, the research should be judged a failure no matter how much it cost to conduct it, how sophisticated its design, or how much of yourself you invested in it.
This conclusion may seem obvious and perhaps a bit unnecessary. After all, you may think that all researchers write up their results for other people to read. But the fact is that many research projects fail to produce a research report. Sometimes the problem is that the research is poorly designed to begin with and cannot be carried out in a satisfactory manner; sometimes unantici- pated difficulties derail a viable project. But too often, the researcher just never gets around to writing a report. And then there are many research reports that are very incomplete or poorly written or that speak to only one of several interested audiences. The failure may not be complete, but the project's fuIl potential is not achier.ed.
47:',z
SUMMARIZI NG AND REPORTING RESEARCH
RESEARCH REPORT GOALS
The research report will present research findings and interpretations in a way that reflects some combination of the researcher's goals, the research sponsor's goals, the concerns of the research subjects, and perhaps the con- cerns of a wider anticipated readership. Understanding the goals of these different groups will help the researcher begin to shape the final report even at the start of the research. In designing a proposal and in negotiating access to a setting for the research, com- mitments often must be made to produce a particular type of report or at least cover certain issues in the final report. As the research progresses, feedback about the research from is participants, sponsoring agencies, collaborators, or other interested parties may suggest the importance offocusing on particular issues in the final report. Social researchers tradition- ally have tried to distance themselves from the concerns of such interested parties, payrng attention only to what is needed to advance scientific knowledge. But in recent years, some social scientists have recommended bringing these interested parties into the research and the reporting process itself.
Advance Scientifi c Knowled ge
Most social science research reports are directed to other social scientists working in the area of study so they reflect orientations and concerns that are shared within this community of interest. The traditional scientific approach encourages a research goal to advance scientific knowledge by providing reports to other scientists. This approach also treats value consid- erations as beyond the scope of science: "An empirical science cannot tell anyone what he should do but rather what he can do and under certain circumstances what he wishes to do" (Weber 19+9,5+).
The idea is that developing valid knowledge about how society is organized or how we live our lives does not tell us how society should be organized or howwe should live our lives. There should, as a result, be a srict separation between the determination of empirical facts and the evaluation of these facts as satisfactory or unsatisfactory (Weber 1949). Social scien- tists must not ignore value considerations, which are viewed as a legitimate basis for selecting a research problem to study. After the research is over and a report has been vritten, many scientists also consider it acceptable to encourage govefilment offrcials or private organiza- tions to implement the findings. During a research project, howeveq value considerations are to be held in abeyance.
Shape Social Policy
As we highlighted in our discussion of applied research in Chapter 12, many social scientists seekto influence social policy bywriting abouttheir research. Bynow,you have been exposed to examples ofseveral such studies in this text, including all the evaluation research (Chapter 12). These particular studies, like much policy-oriented social science research, are similar to those that aim stricdy to increase knowledge. In fact, these studies might even be considered
CHAPTER 16 . SUMMARIZING AND REPORTTNG RESEARCH 473
Participatory action
research (PAR):
&i,;p* *i, r***arch in ',vhich th* r***ar*h*r inu*iv*s **m{j fr r t} ilt\ t z*ti rs t e il m * r*ls t r'; *.r, a$i,t * rSarti r:i pa*ls tltr *utrt?t*Ltt i*rt P ro * c s s c f stt: dtiifi fi an rs r #anizati ** ; t"Lt *
U*al i* ntakmg rfisa*u*,t i* tli* rsr*a*iruii*n.
contributions to lnowledge first and to social poliry debate second. What distinguishes the reports of these studies from stricdy academic reports is their attention to policy implications.
Other social scientists who seek to influence social policy explicidy reject the tradi- tional scientific, rigid distinction between facts and values (Sjoberg and Nett 1968). Bellah et al. (1985) have instead proposed a model of "social science as public philosophy," in which social scientists focus explicit attention on achieving a more just society. Social science makes assumptions about the nature of persons, the nature of society, and the relation between persons and society. It also, whether it admits it or not, makes assumptions about good per- sons and a good society and considers how far these conceptions are embodied in our actual society.
Social science as public philosophy, by breaking tlrough the iron curtain between the social sciences and the humanities, becomes a form of social self-understanding or self-interpretation. . . . By probing the past as well as the present, by looking ato'val- ues" as much as at "facts," such a social science is able to make connections that are not obvious and to ask difEcult questions. (301)
This perspective suggests more explicit concern with public poliry implications when reporting research results. But it is important to remember that we all are capable of distorting our research and our interpreations ofresearch results to cor:respond to our own value prefer- ences. The temptation to see what we want to see is enormous, and research reports cannot be deemed acceptable unless they avoid this temptation.
Organize Social Action-Participatory Action Research For the same reasons that value questions are traditionally set apart from the research process, many social scientists consider the application of research a nonscientific concern. Whyte (1943), whose Street Corner Society studyyottencountered in Chapter 9, has criticized this belief and proposed an alternative research and reporting strategy he calls participatory action research (PAR). \ 4ryte (1991) argues that social scientists must get out of the academic rut and engage in applied research to develop a better understanding ofsocial phenomena.
In PAR, the researcher involves some members of the setting studied as active partici- pans. Both the organizational members and the researcher are assumed to lyant to develop valid conclusions, to bring unique insighs, and to desire change, but Whyte (1991) believed that these objectives were more likely to be obained if the researcher collaborated actively with the persons he studied. PAR can bring researchers into closer contact with participans in the research setting through groups that discuss and plan research steps and then ake steps to implement research findings. Kemmis and McTaggart (2005) summarize the key features of PAR research as o'a spiral of self-reflecting rycles" involving
o planning a change, o acting and observing the process and consequences ofthe change, o reflectingon these processes and consequences, . replanning, and . actingandobservingagain.
In contrastwith the formal reporting ofresults at the end of a research project, these rycles make research reporting an ongoing part ofthe research process.
414 sECTroN v . AFTER THE DATA ARE coLLECTED
CASE STUDY
Seeking Higher Education for lnmates
While prison populations in the United States have been significandy increasing, access to college programs within prisons has been essentially eliminated. Primarily because of the "tough on crime" policies of the 1990s, by 1995 , only eight of the previously existing 3 50 col- lege programs in prisons remained open nationwide (Torre and Fine 2005). To remedy this situation, Jbrre and Fine became involved in PAR to facilitate a college and college-bound program at the Bedford Hills Correctional Facility (BHCF), a maximum-security women's prison in NewYork. Jb conduct a study of the effects of the program, Fine was the principal investigator, along with four prisoner researchers and four researchers from the Graduate Center of the City University of Nevr York. This participatory research team asked several questions: (a) Who are the women in the college program? @) What is the impact of the college experience on inmate students and their children? (c) What is the impact of the col- lege experience on the prison environment? (d) What is the impact of the college experience beyond college on recidivism? (e) What is the cost of such a program to taxpayers? The researchers used a triangulated methodology employing quantitative analysis of recidivism rates and costs of the program along with in-depth interviews with the participants; focus groups with inmates, faoalty, children, and college presidents; and surveys of faculty who aught in the program. Although they were not using a randomized experimental design, Torre and Fine, along with their coinvestigators, tracked participants in the college program after release and found that women who had not participated in the program were four times more likely to be returned to custody than women who participated.
The narratives from the interviews with college inmates also illuminated the benefits of the education. One inmate college student said,
Because when you ake somebody that feels that they're not gonna amount to anything, and you put them in an environment, like, when you're in college it takes you away from the prison, . . . it's like you're openingyour mind to a whole different experience. (582)
The positive impact of college on the inmates was also transferred to their children. The cost+enefit analysis of the program indicated that the savings based on decreased recidivism rates for those who attended the college far outweighed the initial cost of the program iself. In sum, with a small grant from a private foundation, the PAR team brought together univer- sities, prisoners, churches, community organizations, and prison administrators to resurrect a college at BHCEThe authors concluded, "Key elements of this program include broad-based community involvement, strong prisoner participation in design and governance, and the support of the prison administration" (591). fu you can see, PAR has the potential to be life changing for all involved!
Dialogue With Research Subjects
Guba and Lincoln (1989) have carried the notion of involving research subjects and others in the design and reporting of research one step firrther. What they call the constnrctivist para- digm is a methodology that emphasizes the importance of exploring how different stakehold- ers in a social setting construct their beliefs. As we noted in Chapter 1, this approach rejecs the assumption that there is a reality around us to be studied and reported on. Instead, social scientists operating in the constructivist paradigm try to develop a consensus among partici- pants in some social process about how to understand the focus of inquiry which is often a
Constructivist paradiUm:
f'!,eth*rful*ilv i: a g * d t n
r*'iy*lir:* *t b*li*f in a* *v.i*r*al r*alit',i; it efirtlha$iies the importitnce
*f eirplori*g ihe lvay in'*ihicli *itt*r**t stak*hfi ld€ r$ i rT fi. scrial scttinU coilstruct thsir hrli*f s.
CHAPTER 16 o SUMMARIZING AND REPORTING RESEARCH 475
social program that the scientist is evaluating. A research report will then highlight different views ofthe social program and explain how a consensus can be reached.
The constructivist approach provides a usefirl way of thinking about how to best make sense of the complexity and subjectivity of the social world. Other researchers write reports intended to influence public policy, and often, their findings are ignored. Such neglect would be less common if social researchers gave more attention to the different meanings attached by participants to the same events in the spirit of constructivist case reports. The philosophlr ofthis approach is also similar to the utilization-based evaluation research approach advanced by Patton (1997; see Chapter 12) that involves all stakeholders in the research process.
WRITING IS NOT EASY!
Perfectionism is the voice of the oppressor, the enemy of the people. It will keep you cramped and insane your whole life and it is the main obstaile b"ttr"".r yoo ,rd , shitty first draft. (I-amott 1994,28)
We often hear lamentations from students such as, "It is impossible to know where to begin" or o'I have a hard time getting started." To this we say "Begin wherever you are most comfortable but begin early!" You do not have to start with the introduction; start in the method section if you prefer. The main point is to begin somewhere and then keep typing, keep typing, and keep typinsl It is always easier to rewrite a paper than it is to write the first draft. The fine art of writing is really in the rewritingl
Those of you who began with a research proposal have a head start; you will find that the final report is much easier to write. It is very disappointing to discover that something important was Ieft out when it is too late to do anything about it. We do not need to point out that students (and professional researchers) often leave final papers (and reports) until the last possible minute (often for understandable reasons, including other course work and job or family responsibilities). But be forewarned: The last-minute approach does not work!
Successful research reporting requires both good writing and a proper publication out- let. We will first review guidelines for successful writing before we look at particular types of research publications.
Consider the following principles formulated by experienced writers @ooth, Colomb, andWlliams 1995):
Respect the complexity of the task and dont expect to write a polished draft in a linear fashion. Your thinking will develop as you write, causing you to reorganize and rewrite.
Leave enough time for dead ends, resafts, revisions, and so on, and accept the fact that you will discard much of what you write.
Write as fast as you comforably can. Don't wory about spelling, grammar, and so on until you are polishing things up.
fuk anyone whom you trust for reactions to what you have written.
Write as you go along, so you have notes and report segments drafted even before you focus on writing the repoft.
It is important to oudine a report before writing it, but neither the report's organization nor the first written draft should be considered fixed. As you write, you will get new ideas about how to orgarizethe report. Try them out. As you review the first draft, you will see many wals to improve your *riting. Focus particularly on how to shorten and clarifi, your statements.
SECTION V . AFTER THE DATA ARE COLLECTED476
Make sure that each paragraph concerns only one topic. Remember the golden rule of good writing: Writing is revisingl
You can ease the burden ofreport writing in several ways:
o .Draw on the research proposal and on project notes. . (Jse a word processing program on a computer to facilitate reorganizing and editing. . Seek criticism from friends, teachers, and other research consumers before turning in
the final product.
We often find it helpfirl to use reverse oudining After you have rritten a complete first draft, oudine it on a paragraph-by-paragraph basis, ignoring the actual section headings you used. See if the paper you vrote actually fits the outline you planned. How could the organization be improved? Most importandy, leave yourself enough time so thatyou can revise, several times if possible, before turning in the final draft. Revision is essential until complete clarity is achieved.
For more suggestions about writing, see Becker (1986), Booth et al. (1995), Cuba (2002), Strunk and White (2000), and Tirabian (1996).
RESEARCH REPORTTYPES
Research projects designed to produce student papers and theses, applied research reports, and academic articles all have unique features thatwill influence the final research report. For example, student papers are written for a particular professor or for a thesis committee and often are undertaken with almost no financial resources and in the face of severe time con- straints. Applied research reports are written for an organization or agency that usually also
fieverse outlining: {1,,ii:^'..,.., iL.-. ^..nrin,,.-
',n. n,, \J 1..1 t, i I 1 i I i I tl i I I A iirj't: I 1 t) i I i I 1 I {; i I
?,ir t: t'h i,, r,,r.,*, tJr iif't t:i it , t\ lltlt\l ti\ti itt.l
i)ii.l.;\-rt \-Jt i \.,1.r'.-ii,. 11.; tittl.)i (iri, :i.:.i
t, i'it !\r.titn ii r,,', i * iL r. r, n..ti ri t't{i1\l,l/1111:/t!tt tli l,r tliJ-,. iri...it lzt(ilt:,)l tj'.AllJLULIUii,ii ii) r' :rL.r.: 1rr LL( r.
CHAPTER 16 . SUMMAR]ZING AND REPORTING RESEARCH 477
Mitchell B e zziln.a, B S, Gyb ers ecurity G on sultant
Nditchell Bezzina has been a computer foren- sic examiner for over 15 years. He has conducted investigations centered on many areas of inquiry including intellectual property theft, employee misconduct, fraud inves- tigations, cross-border investigations, court orders, and regulatory
inquiries. This work has spanned multiple jurisdic- tions and countries. With hands-on experience in security and digital investigations of every kind, he has designed, developed, and implemented operational and procedural policies for digital forensics, e-discov- ery and engineering departments to gain production efficiencies and comply with business requirements.
While knowledge of computers is a must for the kind of work Bezzina does, the cyber world is constantly changing, so the main thing you need to have is curiosity and motivation to learn. Bezzina believes there are several career paths for someone interested in his line of work, includ- irg digital forensics, e-discovery, and incident response/threat intelligence. From there, lour path is defined by your interests, as you will be investing more than 40 hours per week and doing what you love.
His advice to students is that they take as many computer and data analysis courses as possible and never stop learning. If you go into the consultingbusi- ness, you also have to be able to communicate with several different stakeholders in both business and government. For this reason, it's also important to be able to communicate effectively, both verbally and in writing.
7a
Ti
Source: Courtesy of Mitchell Bezztna
li
has funded the research and has expectations for a particular type ofreport.Journal articles are written for the larger academic community and will not be published until they are judged accepable by some representatives of that community (e.g., after the article has gone through extensive peer review).
These unique features do not match up so neady with the specific types ofresearch prod- ucts. For example, a student paper that is based on a research project conducted in collabora- tion with a work organizadonmay face some of the same constraints as a project designed to produce an applied research report. An academic article may stem from an applied research project conducted for a goverlrment agenry. An applied research report often can be followed by an academic article on the same topic. In fact, one research study may lead to all three types ofresearch reports, as students write course papers or theses for professors who write both academic articles and applied research reports.
Student Papers and Theses
What is most distinctive about a student research paper or thesis is the audience for the final product a professor or, for a thesis, a committee of professors. In light of this, it is important for you to seek feedback early and often about the progress ofyour research and about your professor's expectations for the final paper. Securing approval ofa research proposal is usually the first step, but it should not be the last occasion for seeking advice prior to writing the final paper. Do not become too anxious for guidance, however. Professors require research projects in part so that their students can work through, at least somewhat independendy, the many issues they confront. A great deal ofinsight into the research process can be gained this way. So, balance your requests for advice with some independent decision making.
Most student research pro;'ects can draw on few resources beyond the student's own time and effort, so it is important that the research plan not be overly ambitious. Keep the paper deadline in mind when planning the project, and remember that almost every researcher tends to underestimate the time required to carry out a project.
Group Projects
Pooling your resources with those of several students in a group project can make it possible to collect much more data but can lead to other problems. Each student's role should be clarified at the outset and written into the research proposal as a formal commitrnent. Group members should try to help each other out rather than competing to do the least work pos- sible or to receive the most recognition. Complaints about other group members should be made to the professor when things cannot be worked out among group members. Each group member should have a clear area of responsibilitywith respect to writing the final report, and one may want to serve as the final editor.
TheThesis Committee
Students who are preparing a paper for a committee, usually during the senior year ol a bachelor's degree or at the masters or PhD level, must be prepared to integrate the multiple perspectives and comments of committee members into a plan for a coherent final report. (The thesis committee chair should be the primary guide in this process; carefirl selection of faculty to serve on the committee is also important.) As much as possible, committee mem- bers should have complementary areas of expertise that are each important for the research project: perhaps one methodologist, one specialist in the primary substantive area of the the- sis, and one specialist in a secondary area. Theses using data collected by service agencies or other organizations often benefit if an organizational representative is on the commiftee.
It is very important that you work with your committee members in an ongoing man- ner, both individually and collectively.In fact, it is vitally important to have a group meeting
AFTER THE DATA ARE COLLECTED478 SECTION V O
with all committee members at the beginning of the project to ensure that everyone on the committee supports the research plan. Doing this will avoid obstacles that arise due to mis- communication later in the research process.
JournalArticles
It is the peer review process that makes preparation of an academic journal article most unique. In a process similar to a grant review; the journal's editor sends submitted articles to two or three experts (peers) who are asked whether the paper should be accepted more or less as is, revised and then resubmitted, or rejected. Reviewers also provide coilrments, sometimes quite lengthy, to explain their decision and to guide any required revisions. The process is an anonyrnous one at most journals; reviewers are not told the author's name, and the author is not told the reviewers'names. Although the journal editor has the final say, editors' decisions arenormally based on the reviewers' comments.
This peer review process must be anticipated in designing the final report. Peer review- ers are not pulled out ofa hat. They are experts in the field or fields represented in the paper and usually have published articles themselves in that field. It is critical that the author be familiar with the research literature and be able to present the research findings as a unique contribution to that literature. In most cases, this hurdle is extaordinarily hard to jump: Most leading journals have a rejection rate of over 90%. Of course, there is also a certain luck of the draw in peer review. One set of two or three reviewers may be inclined to reject an article that another set of reviewers would accept. But in general, the anonymous peer review process results in higher-quality research reports because articles are revised prior to publication in response to the suggestions and criticisms of the experts.
Criminological and criminal justice research is published in a myriad of journals within several disciplines, including criminology, law; sociology, pqychology, and economics. As a result, there is no one formatting style that all criminological literature abides by. If, for example, you are submitting your paper to a psychology-related journal, you must abide by the formatting style dictated by the Publication Manual of the American Prycbohgical,*sociation (American Psychological Association 2009). The easiest way to determine how to format a paper for a particular journal is to examine recent volumes of the journal and format your paper accordingly. To give you a general idea ofwhat a journal article looks like, an article in its entirety has been reprinted in Appendix B, along with an illustration of how to read a journal article. There are also numerous articles available on the Student Study Site.
Despite the slight variations in style across journals, there are typically seven standard sections within a journal article in addition to the tide page (see Exhibit 16.1).
Applied Reports
Unlike journal articles, applied reports are usually commissioned by a particular govefirment agency, corporation, or nonprofit organization. For this reason, the most imporant problem that applied researchers confront is the need to produce a final report that meets the fund- ing organization's expectations. This is called the bired-gun problem. Of course, the extent to which being a hired gun is a problem varies gready with the research orientation ofthe frrnding organization and with the nature ofthe research problem posed. The ideal situation is to have few constraints on the nature of the final report, but sometimes research reporB are suppressed or distorted because the researcher comes to conclusions that the firnding organization does not like.
Applied reports that are written in a less highly charged environment can face another problem-even when they are favorably received by the funding organization, their conclu- sions are often ignored. This problem can be more a matter of the organization not really knowing how to use research findings than a matter of not wanting to use them. And this is
CHAPTER 16. SUMMARIZING AND REPORTING RESEARCH 479
:s : .:.,:.:[it,e.iaiiUine.
i..'.,.,i,'.,',,R,6iv'i.,e.w..
".
..
#ililffi fii 1$ftH+6. .$,UUfi. .,$.ffiufin e s,a,S..
il6i6i:il ffi:8ii8iulffifi$lfi iHe,:i.diifi . ill,n
:l:':':1ffi : : i::t
r::':lilll:Re: ;elnel:n:0.:g:S
not merely a problem of the funding organization; many researchers are prepared only to
present their findings, without giving any thought to how research findings can be translated
into organizational policies or programs.
An Advisory Committee
An advisory comminee can help the applied researcher avoid the problems of incompatible
expectations for the final report and insufficient understanding of how to use the research
reiults, without adopting the more engaged strategy of Whyte's (1991) PAR or Guba and Lincoln's (1989) constructivist inquiry. An advisory commiftee should be formed before the
start of the project to represent the various organizational segments with stakes in the out- comes of the resea.ch. The researcher can use the committee as a source of intelligence about
how particular findings may be received and as a sounding board for ideas about how the
SECTION V . AFTER THE DATA IRE COLIE.TED480
Exhibit 16.1 General Sections of a lournal Article
organization or agency can use research findings. Perhaps most importandy an advisory com- mittee can help the researcher work out many problems in research design, implementation, and data collection. Because an advisory committee is meant to comprise all stakeholders, it is inevitable that conflicts will arise among advisory group members. In our experience, however, these conflicts almost invariably can be used to strateglze more effectively about the research design and the final product.
Advisory committees are particularly necessary for research investigating controversial issues. For example, after a study conducted n 1999 found that several death row inmates had been wrongly convicted of their crimes, the governor of Illinois placed a moratorium on all death sentences in the state. Other results of the study suggested that the death penalty was handed down unfairly; it found that proportionally more minority and poor offenders were sentenced to death than whites and those who could afford hired legal counsel. This caused a great deal ofmedia attention and led to calls for other states that practiced the death pen- alty to institute similar moratoriums. As a consequence of this attention, other states began to examine their implementation of the death penalty. Maryland is one such state. In 2001, the state legislature in Maryland commissioned Paternoster (Paternoster, Brame, Bacon, and Ditchfield 2004) at the University of Maryland to conduct a study of its practice of the death penalty. The primary goal of the study was to determine whether the administration of the death penalty in the state was affected by the race of the defendant or victim.
As you can imagine, when the study was released, it was controversial. To make sure all interests were represented, Paternoster et al. Q004) set up an advisory committee before undertaking the study. The advisory committee consisted of a gtoup of prosecutors and defense counsel who had experience in capital cases. They advised Paternoster on several critical issues, including the years that the study should cover, the sources where information could be found, and the particularly important variables related to sentencing outcomes. Not only did the advisory committee provide substantive input into the research, but by having a broad spectrum of the legal community on board, it also provided credibility for the study's findings.
DISPLAYING RESEARCH
You learned in Chapter 14 about some of the statistics that are useful in analyzing and report- ing data, but there are some additional methods of presenting statistical resul* that can improle research reports. Combined and compressed displays are used most often in applied research reports and government documents, but they can also help communicate findings more effectively in student papers and journal articles.
In a combined frequency display, the distributions for a set of conceptually similar variables with the same response categories are presented together, with common headings for the responses. For example, you could identify the variables in the lefrnost column and the value labels along the top. Exhibit 16.2 is a combined display reporting the frequency disributions in percentage form for responses to the question, "HolM worried are you that you or someone in your family will become a victim of terrorism?" for the years 2002 through 2013. From this table, you can infer several pieces ofinformation besides the basic distribu- tion of attitudes. Except for the high blip in early October wherc 24o/" of the population was very worried, for the most part, about 1 in 10 people have been very worried throughout the entire time period. llowever, during this time there has been a noticeable increase in the percentage of the population who are not worried at all.
Compressed frequency displays can also be used to present cross-tabular data and summary statistics more efficiendy, by eliminating unnecessary percentages (such as those corresponding to the second value of a dichotomous variable) and by reducing the need
Combined frequency display: ri l^!^1^ )-1.,-t 11 !,1i111, 1|t;-1 I ttl r';i1!,:ti \
i,.. ,^ ,. ol.. ,.. ," ir, ^
,.t', ,.,..iJ, , ,;.: ,, .., ...ItlilAJ t,!t liJF tt1\-Jl '.1\t'11\11114- L1,j t-..1 tJ L 1 i \r t t I I \r \ | I .J L t i t.;,-t L ; 1-j 1 i.J
{-.-^ --+-i-^.-^ tlrr /r \ r-t tzs | -rrrrr.;i\lrii,,lit;J\/j t,l. t.)1..t1. \.?r 1/\r1l\t1.ti,t r..rtij,?
r:tlilt i')l 1")l t.( i:1 i,i+ l1'.4t.ttl1i i lill.; 'Jijrt.iltlJ ':til tr-.,.lJl'.J..) t|{tt il'. L{ lrlL
-. i. -. .., .. .. "
,.. ... .\' irii* f *t lti,ilt.\* t' -ti*i liil ;*('\j 1.i 1 i | \J 1 \.r at lJ rJ 1 I,J L, 1, (.{ L\-. *
t.j 1 . \r.'1 .
{\ ii t ii ..i\
{'s 11 1",,; :, ri i r, f t r:. I i L: I 1 A i, d \tt1!llit\jt. l 1i\;{.t\.tlaJ\,tt.i l): rl tlil\,1{
{ nr ilt r, { firi 11 f. t\ u titt 1*i t_*,:j 1u,)r)\.J ti.)u;),
Compressed frequency
display: * t':ri:l ;: ll-::tl rli'r.,+r)r]?rtt tLLt,jlt.r J.tta_t_t ljt tj\)i-r:ti,) i\ r' t\t\ t-\ rrl., rrrr',1, r, + ti r. ;r el ti <t '-^t
t U 1.! .^! \) 1 \A j \; I I I i) U, 1_ I U ; I U lA t-L/-
;,lir i.', * nt ltt it r:li tri rr t ? i r i rr l! 1 | 1\: ; 1: I 1L.t t! Lj T I) I II l, I i 1 l-t | | I i 1-)
. . ,t^ ,- ,., -., ,- ^-*- - I 1l1 I 1Q.j'Ul! / I \; 1\Llf I 1 L)l |1' ;! \ (.1f, (r l i L r J t/ 1 r: 11 ) a t. l V .^t \ : t \r \J I i l.Lr,A U u,
*t if'.t;t :t tt 1?t t rl t, { r',r,i,l ;t rt +"'' 1' ''-',:3 " .^.^ .- ... ^1:... ^. -- Ll^ - ^ ^ ^ ^ .- ^1,\ttt /(-tt.ttlltti:111r,\ i!l ttt:\ t\,r.ll 1 1,.\,1lll,11 1l1i\'t I it Iitli .\T:l,t lltl 1
,,t ill i:',: *f a tll t:l t r: i.r;il't tli: - - -
l- I -
CHAPTER 16 . SUMMARIZING AND REPORTING RESEARCH 481
Sept. 21-22,2001
Oct. 5-6, 2001
June, 2005
July, 2005
July, 2007
September, 2008
December, 2009
January,2010
April, 2013
0% 107" 20% 30% 40% 50% 60% 70% 80% 90% 100% ffi Very Worried ffi Somewhat Worried ffi Not Too Worried ffi Not Worried at All
Source: gnivsrsity at Albany, Hindelang Criminal Justice Research Center, Sourcebook of Criminal Justice Statistics. Teble 2.29.2073. Available: http://www.albany.edu./sourcebooVpdf/A292013.pdf.
for repetitive labels. Exhibit 16.3 presents a compressed display used to highlight the per- centages of students in Grades 9 through 12 who reported using marijuana at least once in the past 30 days by select student characteristics and year. It took many cross-tabulations
SECTION V . AFTER THE DATA ARE COLLECTED
Oct. 1 1-1 4, 2001 Oct. 19-21 ,2001
Nov. 2-4,2001
Nov. 26-27,2001
March, 2002
April, 2OO2
May, 2002
September,2002
February, 2003
March, 2003
August, 2003
December, 2003
January,2004
February,2004
October,2004
482
Exhibit 16.2 Combined Frequenry Distribution of Responses to Question About Fear of Terrorism,2001-2013
{-, A l-4o U Lro Ar o LN
tg (d
o C|-)
14
o.onB9U ;-o5.9 'qJ oll-{ +) Iri E ri HRfi L.o
. OJ .-tb€vl L{']S .o 15trq,, o .F(d UH OtH,so,xu b.E urubEH iliUOE;E.q dF rl +rooNtr
EEo )-r-0D(UEE .g '3Ef;r+,5ut Pbo
t'rp& salofr(n+
483CHAPTER 16 o SUMMARIZING AND REPORTING RESEARCH
b0 .g (,tp EaP !.{ oso & Ot{ -c(6 F# NTng "\flUI (J O.ilE6(6.F L. tsl(rg AU.= gutsg -l r*
UU EE FAuts tr{ 5 :6 tD rr<
SEot'l,tra JJ-7(u uUt E3.. .gE s8 9ofi? p5 E.g cE,st TLg OE E'g, ti .d
UHs5 fle OH .d .-t]F
l{-{ Oog >,o GlLr =tn .h" E AFr EH EGotr(u(u Lad
s,=..1 .d- HL{x(6 UE nl \o F'I .= IAr* x rd
100
80
60
40
20
0
Source: Musu-Gillette, et al. z}l8.Indicators of Scho ol Crime and Safety: 20'J-7 ,Table 1,6,1..
484 SECTION V . AFTER THE DATA ARE COLLECTED
to create this display, but not all of the percentages are presented*only those percent- ages necessary to convey the bivariate distribution are presented. With this display, read- ers can easily compare the likelihood of different students smoking marijuana over time. Interestingly, despite the fact that several states have now legalized recreational marijuana use (although it typically does not apply to those under the age of 2l), the percentage of students using marijuana does not appear to have changed much over the past 20 years.
Combined and compressed statistical displays present a large amount of data in a rela- tively small space. To the experienced reader ofstatistical reports, such displays can convey much important information. They should be used with caution, howeveq because people who are not used to them may be baffled by the rows of numbers. Graphs can also provide an efficient tool for summarizing relationships among variables. A good example of the use of graphs is also reported inlndicators dSchool Crime and. Safety: 2017 (M,ast-Gillette et aI.2018). Exhibit 16.4 presens the number of student, staff, and other nonstudent school-associated violent deaths and the number of homicides and suicides of youth aged 5 to 19 at school from 1992 through 2015 (Musu-Gillette et al. 2018). As you can see, compressed displays in both table and graph form can be very illuminating!
Special Considerations for Reporting Qualitative or Mixed-Methods Research
The requirements for good research reports are similar in many respects for quantitative and qualitative research projects. Every research report should include good writing, a clear statement of the research question, an integrated literature review, and presenation of key
Number
48 48 91 48
11
3
1992- i99g- i994- i99s- 1996- 1 997- 1 998- 1 999- 2ooo- 2oo1- 2ooz- 2oog- 2oo4- 2005- 2006- 2007- 2oog- 2oo9- 2010- zoli- 2012- zolg- 2ou- 93 94 95 96 97 98 99 2000 01', 02' 03', 04' 05', 06', 07' 08', 09', '10', 'I1', 12' 13', 14' 15',
School year
Number of student, staff, and other nonstudent school-associated violent deaths2
Homicides of youth ages 5-18 at school
- Suicides of youth ages 5-18 at school
Exhibit 16.4 Number of Student, Staff, and Other Nonshrdent School-Associated Violent Deaths and the Number of Homicides and Suicides ofYouth Aged 5 to 19 At School ByYear
\ ,A 53
\ 37 34 " 96. _ _q0
findings with related discussion, conclusions, and limitations. The general sections of a quan- titative article @xhibit 16. 1) may also be used by some audrors of qualiative research reports. Iloweveq the differences between qualitative and quantitative research approaches mean that it is often desirable for research reports based on qualitative research to diverge in some rbspects from those reflecting quantitative research.
Reports based on qo"litrtirr" research should be enriched in each section with elements that reflect the more holistic and reflexive approach of qualitative projects. The inmoduc- tion should include background about the development of the researcher's interest in the topic, whereas the literature review should include some attention to the gpes of particular qualitative methods used in prior research. The methodology section should describe how the researcher gained access to the setting or individuals studied and the approach used to man- age relations with research participants. The presentation of findings in qualitative studies may be organized into sections reflecting different themes identified in interviews or obser- vational sessions. Quotes from participants or from observational notes should be selected to illustrate these themes, although qualitative research reports differ in the extent to which the researcher presents findings in summary form or uses direct quotes to identi$z key issues. The findings sections in a qualitative report may alternate between presentations of quotes or observations about the research participants, the tesearcher's interpretations of this material, and some commentary on how the researcher reacted in the setting, although some qualita- tive researchers will limit their discussion of their reactions to the discussion section.
Reports on mixed-methods projects should include subsections in the methods section that introduce each method and then distinguish findings from qualiative and quantitative analyses in the findings section. Some mixed-methods research reports may present analy- ses that use both qualitative and quantitative data in yet another subsection, but others may discuss implications of analyses of each gpe for the overall conclusions in the discussions and conclusions sections (Dahlberg, Wittink, and Gallo 2010). When findings based on each method are presented, it is important to explicidy consider the ways in which the specific methods influenced findings obtained with those methods and to discuss the implications of findings obtained using both methods for the overall study conclusions.
ETHICS, POLITICS,AND REPORTING RESEARCH
It is at the time of reporting research results that the researcher's ethical duty to be honest becomes paramount. Here are some guidelines:
. Prwide an honest atcuunting of haw the research was caried. oat and wbere the initial researclt deign had n be changed. Readers do not have to know about every change you made in your plans and each new idea you had, b-ut they should be informed about major changes in hypotheses or research design. If important hypotheses were not supported, acknowledge this rather than conveniendyforgettingto mention them @rown and Hedges 2009). If a different approach to collecting or analyzing the data could have led to different conclusions, this should be aclarowledged in the limitations section
@ergman 2008).
o Honestl! eaaluate the strmgths and weaknesses of your research design. Systemadc evaluations suggest that the stronger the research design from the standpoint of establishing internal (causal) validity, the weaker the empirical support that is likely to be found for hypothesized effects (compare Weisburd, Lum, and Petrosino 2001). Finding support for a hypothesis tested with a randomized experimental design is stronger evidence than support based on correlations amongvariables measured in a cross-sectional survey.
CHAPTER 16 O SUMMARIZING AND REPORTING RESEARCH 485
Refer to prir research and intayret yourfindings witbin tbe body of literaru.re renbingfrorn that prior research. Your resuls are Iikely to be only the latest research conducted to investigate a research question that others have studied. It borders on unethical practice to presentyour fiodi"Sr as if they are the only empirical informationwithwhich to answer your research question, yet many researchers commit this firndamental misake
@ergman 2008). For example, a systematic ev"aluation of ciation frequencyin articles reporting clinical trial results in medical journals found that on average, only 2lo/o of the available prior research was cited (for trials with at least three prior articles that could have been cited) @obinson and Goodman 201 1). The result of such omission is tlatreaders mayhave no ideawhetheryour own research suppors a larger bodyof evidence or differs from it-and so should be subject to even greater scrutiny.
Maintain a full record of the research projea so that questims can be artrwered if thel arise. M*ty deails will have to be omitted from all but the most comprehensive reports, but these omissions should not make it impossible to track down answers to specific questions about research procedures that may arise during data analysis or presentation. Tests of relationships that were conducted but not included in the report should be acl,rnowledged.
Aooid "lyingwitb statistits" or raing graphs n mislead.There is a more subde problem to be avoided, which is cherry-picking which result you will present. Although some studies are designed to test only one hlpothesis involving variables that are each measured in only one way, many studies collect data that can be used to test many hlpotheses, often with alternative measures. If many possible relationships have been examined with the data collected and only those found to yield a statistically significant result are reported, the odds of capitalizing on chance findings are multiplied.This is a major temptation in research practice and has the unfornrnate result of most published findings not being replicated or not sta:nding up to repeated tests over time (Lehrer 2010). Every statistical test presented can be adequately understood only in light ofthe entire body ofstatistical analyses that led to that particular result.
Acknowledge tbe sponsorc of the researcb. This is imporant pardy so that others can consider whether this sponsorship may have tempted you to bias your results in some way. Whether you conducted your research for a sponsor or together with members of an underserved community, grve research participants an opportunity to comment on your main findings before you release the findings to the public. Consider revising your report based on their suggestions or, ifyou disagree with their suggestions, include their comments in foomotes at relevant poins in your report or as an appendix to it @ledsoe and Hopson 2009).
Tbnnk staffwho ntade major contibutions.Thts is an ethical as well as a political necessity. Let's maintain our social relations!
Be v.r.re tbat the arder of aatborchip for coauthored reports is d.isatssed. in adaance ond reJlem agreed-on principles.Be sensitive to coauthors' needs and concefiN.
Ethical research reporting should not mean ineffective reporting. You need to tell a coher- ent story in the repoft and avoid losing track of the story in a thicket of minuscule details. You do not need to repoft every twist and tum in the conceptualizaton ofthe research problem or the conduct ofthe research. But be suspicious ofreports that dont seem to admit to the possibility of any room for improvement. Social science is an ongoing enterprise in which one research report makes its most valuable conribution by laying the groundwork for another, more sophisticated research project. Highlight important findings in the research report, but also use it to point out what are likely to be the most productive directions for future researchers.
SECTION V . AFTER THE DATA ARE COLLECTED486
Communicating With the Public
Even following appropriate guidelines such as these, howeveq will not prevent controversy and conflict over research on sensitive issues. Does tlds mean that ethical researchers should avoid political controversy by sidestepping media oudets for their work? Many social scien- tists argue that the media offer one of the best ways to communicate the practical application of sociological knowledge and that when we avoid these opportunities, "some of the best sociological insights never reach policy makers because sociologists seldom take advantage of usefirl mechanisms to get their ideas out" (Wilson 1998,+35).
The sociologistWilliamJulius Wilson (1998) urges the following principles for engagng the public through the media:
1. Focus on issues of national concern, issues that are high on the public agenda. 2. Develop creative and thoughtfrrl arguments that are clearlypresented and devoid of
technical language.
3. Present the big picture whereby the arguments are organized and presented so that the readers can see how the various parts are interrelated.
Illtimately, each researcher must make a decision about the most appropriate and impor- tant oudets for his or her work.
CHAPTER 16 o SUMMARIZING AND REPORTING RESEARCH 487
For Further Thought:
1,. The article finds that top-tier journals are as likely as lesser publications to retract articles. Beyond peer review, what measures should be taken to reduce this misconduct?
2. If research is funded by afederal agency, there is an expectation that the data will
be made publicly available if doing so does not compromise the confidentiality of research participants. Should all scientists be compelled to turn over their data for replication? Should journals put all papers through a plagiarism checker, as most universities now do with student papers?
Plagiarism:
Fr*sentinil frs *t1rj'* *w*t?t* irJras *i'wt:rd* af an*iher
ilor$t:i"r or p*rs*n.s frr a*arl ern i t rva I ualt*n,,ruith * ui p rr] il# r a*k**u:led U rn* nt,
PLAGIARISM
It may seem depressing to end a book on research methods with a section on plagiarism, but it would be irresponsible to avoid the topic. Of course, you may have a course syllabus deailing instructor or university policies about plagiarism and specilring the penalties for violating that policy, so we're not simply going to repeat that kind of warning. You probably realize that the practice of selling tenn papers is revoltingly widespread (our search of terru paperc on Google returned over 2 million websites onJune 3,2015), so we're not going to repeat that academic dishonesty is widespread. Instead, we will use this section to review the concept of plagiarism and to show how that problem connects to the larger issue of the integrity of social research. When you understand the dimensions of the problem and the way it affects research, you should be better able to detect plagiarism in other work and avoid it in your own.
You learned in Chapter 3 that maintaining professional integrity-honesty and open- ness in research procedures and results-is the foundation for ethical research practice. When it comes to research publications and reports, being honest and open means avoiding plagiarism-presenting as onet own the ideas or words of another person or persons for aca- demic evaluation without proper acknowledgment (Hard, Conway, and Moran 2006).
An increasing body ofresearch suggests that plagiarism is a growing problem on col- lege campuses. Stephens and his colleagues (Stephens,Young, and Calabrese 2007) found in a web-based survey of self-selected students at two universities that one-quarter acknowl- edged having plagiarized a few sentences (24.7"/"), while a much smaller fraction admitted plaglaizrnga complete paper (.3%) in courseworkwithin the pastyear. (Many others admit- ted to other forms of academic dishonesty, such as copymg homework.) Hard et al. (2006) conducted an anonymous survey in selected classes in one university, with almost all stu- dens participating, and found much higher plagiarism rates: 60.6Y" reported that they had copied "sentences, phrases, paragraphs, tables, figures or data direcdy or in slighdy modified form from a book, article, or other academic source without using quotation marks or giving proper acknowledgment to the original author or source", and 39-4o/" rcported that they had "copied information from Internet websites and submitted it as [their] work" (1069).
So, the plagiarism problem is not only about purchasing term papers-although that is really about as bad as it gets (Broskoske 2005); plagiarism is also about what you do with the information you obtain from a literature review or an inspection of research reports. And rest assured that this is not only about student papers; it is also about the work of established scholars and social researchers who publish reports that you want to rely on for accurate infor- mation. Several noted historians have been accused ofplagiarizing passages tha.t they used in popular bools; some have admitted to not checking the work of their research assistants, to not keeping track of their sources, or to being unable to retrieve the data they claimed they had analyzed. Whether the carxe is cutting corners to meet deadlines or consciously fudgrg fac6, the effect is to undermine the trustworthiness of social research.
Now that you are completing this course in research methods, it's time to think about how to do your part to reduce the prevalence of plagiarism. Of course, the first step is to mainain carefirl procedures for documenting the sources that you rely on for your own research and papers, but you should also think about how best to reduce temptations among bthers. After all, what people believe about what others do is a smong influence on their own behavior (Ilard et
^1.2006).Reviewing the definition of plagiarism and how it is enforced by your discipline's profes- sional association is an important first step. These definitions and procedures reflect a collec- tive effort to help social scientists maintain standards throughout the discipline. Awareness is the first step (American Sociological fusociation [ASA] 1999). In addition, your college or unirrersity also has rules that delineate its definition of and consequences for plagiarism.
Researchers have an obligation to be familiar with their institution's code of ethics, other applicable ethics codes, and these codes' application to sociologists' work. Lack of awareness
AFTER THE DATA ARE COLLECTED488 sECTloN v o
or misunderstanding of an ethical standard is not, in itself, a defense to a charge of unethical conduct.
ASAb (1999) code of ethics, which is used by the American Society of Criminology, includes an explicit prohibition of plagiarism:
14. Plagiarism
(a) In publications, presentations, teaching, practice, and service, sociologists explicitly identify, credit, and reference the author when they take data or material verbatim from another person's vritten work, whetler it is published, unpublished, or electronically available.
(b) In their publications, presentations, teaching, practice, and service, sociologists provide acknowledgment of and reference to the use of others' work, even ifthe work is not quoted verbatim or paraphrased, and they do not present others'work as their own whether it is published, unpublished, or electronically available. (16)
The next step toward combating the problem and temptation of plagiarism is to keep focused on the goal of social research methods: investigating the social world. If researchers are motivated by a desire to learn about social relations, to undersand how people understand society, and to discover why conflicts arise and how they can be prevented, they will be as con- cerned with the integrity of their research methods as are those, like yourself, who read and use the results oftheir research. Throughout this text, you have been learning how to use research processes and practices that yield valid findings and trustworthy conclusions. Failing to report honesdy and openly on the methods used or sources consulted derails progress toward that goal.
It worls the same as with cheating in school. When students are motivated only by the desire to ace their tests and receive better grades than others, they are more likely to plagia- rize and use other illicit means to achieve that goal. Students who seek first to improve their undersanding of the subject matter and to engage in the process of learning are less likely to plag1aize sources or cheat on exams (Kohn 2008). They are also building the foundation tor becoming successful social researchers who help others understand our world.
CONCLUSION
Good critical skills are essential when evaluating research reports, whether your own or those produced by others. There are always weak points in any research, even published research. It is an indication of strength, not weakness, to recognize areas where one's own research needs to be or could have been improved. And it is not merely a question of sharpening our knives and going for the jugular. You need to be able to weigh the strengths and weaknesses of par- ticular research results and to evaluate a study in terms ofits contribution to understanding its particular research question, not whether it gives a definitive answer for all time.
But this is not to say that anlthing goes. Much research lacks one or more of the three legs of validity-measurement validity, causal validity, and generalizability-and sometimes contributes more confusion than understanding about particular issues. Top journals generally maintain very high standards, pardy because they have good critics in the review process and distinguished editors who make the final acceptance decisions. But some daily newspapers do a poor job of screening, and research reporting standards in many popular magazines, TV shows, and boo[s are often abysmally poor. Keep your standards high and your view critical when reading research reports, but not so high or so critical that you turn away from stud- ies that make tangible contributions to the literature, even if they do not provide definitive
CHAPTER 16. SUMMARIZING AND REPORTING RESEARCH 489
answers. And dont be so intimidated by the need to mainain high sandards that you shrink from taking advantage ofopportunities to conduct research yourself.
The growth of social science methods from infanry to adolescence, perhaps to young adulthood, rants as a key intellectual accomplishment of the 20th century. Opinions about the causes and consequences of crime no longer need depend on the scattered impressions of individuals, and criminal justice policies can be shaped by systematic evidence of their effectiveness.
Of course, social research methods are no more usefrrl than the commitrnent of the researchers to their proper application. Research methods, like all knowledge, can be used poorly or well, for good purposes or bad, when appropriate or not. A claim that a belief is based on social science research in itself provides no extra credibility. As you have learned throughout this book, we must first learn which methods were used, how they were applied, and whether interpretations square with the evidence. To investigate the social world, we must keep in mind the lessons of research methods
).,,,Review, key: terms w,ith, eFl,a:shcards.
C onebiiled fr e $u enw... diS pla, Xr,., .,.....,. ..4 8,1,,.,
.C6,fi 65s6d'...frqquenC$.,1.diS.ptrbg...,r::.:::iri4:8X Participatory action research e'AR) q:7+' Plasiarism 488, c) : .. B ef$ie.... tlinin$,..,..,11....111477
Proposal writing should be a time for clariffrg the research problem, reviewing the literature, and thinking ahead about the report that will be required.
Relations with research subjects and consumers should be developed in a manner that achieves key research goals and prepares for an effective research report. The traditional scientific approach of minimizing the involvement of research s,rbjects and consum.rrln research decisions has been challenged bv proponents
of PAR and adherents of the constructivist paradigm.
Different types of reports typically pose different problems. Authors of student papers must be guided in part bv the expectations of their professor. Thesis writers have to meet the requirements of different committee members but can benefit greatly from the areas of expertise represented on a typical thesis commiffee.
Applied researchers are constrained by the expectations of the research sponsor; an advisory committee from the applied setting can help avoid problems. Journal articles
must pass a peer review by other social scientists and
often are much improved in the process.
Research reports should include an introductory statement of the research problem, a literarure review, a
SECTION V " AFTER THE DATA ARE COLLECTEDil,0,