Annotated Bibliography

04CH_Robinson_Justice.pdf

Home >Law homework help >Criminal homework help >Annotated Bibliography

Strategies for Reviewing the Research

Learning Objectives

After reading this chapter, you should be able to:

• Identify and locate resources to use in finding appropriate literature on social and criminal justice topics, including online sources and treeing.

• Evaluate both the breadth and depth of social and criminal justice research, and how these dimensions influence research and literature reviews.

• Evaluate the credibility of sources, using criteria such as relevance, primary versus secondary, authorship, expertise, currency, accuracy, and corroboration.

• Analyze journal articles in a systematic way to extract relevant and pertinent information.

iStockphoto/Thinkstock

CHAPTER 4Section 4.1 Locating Resources

Once a topic is identified and developed into a workable research question, the research begins. The ability to analyze and summarize information from a variety of sources,

which includes identifying common themes, is an incredibly valuable skill in college. It can also be viewed as a valuable skill by criminal justice employers and is sometimes nec- essary for advancement within the field. A “literature review” or “literature search” is the major component of the CRJ 422 Social and Criminal Justice Research Project and is con- ducted to identify the ideas that have already been made public or presented by scholars in the field. Although some of these ideas could be personal opinions or anecdotal reports, students should look for scholarly, peer-reviewed literature and learned opinions from experts in the field. The process of reviewing previous research allows students to build the foundation for answering the research question.

When students conduct a literature review, each aspect is related to specific components of the research question. The student presents the outcomes and results of past research in order to provide an answer for the research question, while concluding the literature review with recommendations for future research.

When outlining a literature review, the writing approach should focus on providing a pre- sentation of the facts in an impartial and objective manner, meaning the student should avoid providing personal opinions. Although the literature reviewed may persuade a stu- dent in his or her personal opinion, this opinion should not be expressed in this type of research paper. Students should remember that the evidence will answer the research question, not personal opinions.

A major component of this approach is the ability to incorporate evidence—specifically, research-based evidence—into the written text. Because of the unique nature of social and criminal justice research, a student might take a broad approach, which can include more than one of the fields, or a narrower approach, which would involve a specific focus on one field. Whatever approach the student chooses will ultimately affect the nature of the literature review.

4.1 Locating Resources

The first step in writing a literature review is to find appropriate literature that pertains to the research topic. Local public and university libraries, as well as a careful and

systematic Internet search (more on this in a later section), are all prospective options. In addition, the university offers a number of convenient resources for the online student, including an e-book database called ebrary, and the University Research Tutorial. In litera- ture reviews, journal articles are commonly utilized and summarized, but other types of resources can provide valuable academic contributions as well (e.g., books, monographs, recorded interviews, book chapters, websites, and so on). The criteria necessary to evalu- able the credibility of information (retrieved online or otherwise) are presented later in this chapter.

CHAPTER 4Section 4.1 Locating Resources

Tips & Tools: Making the Most of Databases—Advanced Features

Databases like ProQuest and others are called search engines and are indexed on a num- ber of criteria. This indexing is used to facilitate quicker retrieval when a student enters

the search terms in which he or she is interested. However, a literal search engine may miss relevant docu- ments if the wrong term is used or if the term is misspelled. Unless the student asks ProQuest to return hits that are variants of spellings, the database will find only literal, exact matches (i.e., the exact word). To provide more flexibility, two advanced features to consider using are the commands for truncation and wildcards, shown in Table 4.1.

Table 4.1: Truncation and wildcard characters

Character Explanation Example * The symbol * is used as a right-handed

truncation character only; it will find all forms of a word.

Searching for crim* will find “crime,” “criminal,” “criminology,” etc.

? The symbol ? is used to replace any single character, either inside the word or the end of the word. ? cannot be used to begin a word.

Searching for “wom?n” will find “woman” and “women.” Searching for “l?w” will find “law,” “low,” etc.

(continued)

The Online Library and Other Online Options Many institutions pay for institu- tional access to online databases that help search what is available and help locate the materials for direct access. These databases are housed in the online library (login via the Student Portal) and include electronic books, journals, and other types of articles. Social and criminal justice students will likely use EBSCO, JSTOR, Pro- Quest, Project MUSE, and ebrary (see the library’s Database List by Subject). However, students are not limited to these databases. In addition, local public libraries and/or other university libraries might have access to databases to which the university does not subscribe. But the online library does offer descriptions of each database, as well as tip sheets and guides for research and advanced search techniques, which may be helpful to students. Knowing the important keywords as well as reviewing all relevant databases are essential steps taken prior to conducting the actual research, source evaluation, and analysis of the information. For additional advanced techniques on searching databases, see Tips & Tools: Making the Most of Databases—Advanced Features.

iStockphoto/Thinkstock

The online library is a great tool for locating resources.

CHAPTER 4Section 4.1 Locating Resources

Tips & Tools: Making the Most of Databases—Advanced Features (continued)

The * symbol (asterisk) is used as a type of wildcard that will search for words that start with the same root letters but may end differently. For example, a database search on the

word legalization might not turn up related words like legal, legally, legalize, legalizing, or legalized unless these latter words were also mentioned in the journal article. But a search on legal* would return all pos- sible options for words starting with l-e-g-a-l. Using the ? (question mark) feature can be helpful as well, as in the example from ProQuest in the table above, using the term woman and women in one search.

Boolean operators are mathematical terms and descriptors that can be used in database searches to widen or narrow the scope of the search. The most common Boolean operators used for database searches are and, or, and not—but there are other advanced features as well. For example, the terms woman and women can be searched for simultaneously using woman or women, in addition to the wom?n feature. This means the database should retrieve any journal article that contains either term, as opposed to woman and women, which indicates that the journal article must contain both terms. The not function can be useful as well. For example, a search including not dissertation means disserta- tions should not be included in the list of database matches (hits). Such searches can also be combined on the same line, such as (woman or women) not dissertation. To find database matches with terms within the proximity of one another, see the advanced options in Table 4.2.

Table 4.2: Operators used in database searches

Operators Boolean, proximity, and adjacency operators are used to broaden and narrow your search.

AND Find all the words. When searching for keywords in “Citation and Document Text,” AND finds documents in which the words occur in the same paragraph (within approximately 1,000 characters) or the words appear in any citation field. Example: Internet AND education

AND NOT Find documents that have the first word, but not the second word. Example: Internet AND NOT html

OR Find any of the words. Example: Internet OR intranet W/# Find documents where these words are within some number of words apart (either

before or after). Use when searching for keywords within “Citation and Document Text” or “Document Text.” Example: computer W/3 careers

W/PARA Finds documents where these words are within the same paragraph (within approxi- mately 1,000 characters). Use when searching for keywords within “Document Text.” Example: Internet W/PARA education

W/DOC Find documents where all the words appear within the document text. Use W/DOC in place of AND when searching for keywords within “Citation and Document Text” or “Document Text” to retrieve more comprehensive results. Example: Internet W/ DOC education

NOT W/# Find documents where these words appear but are not within some number of words apart (either before or after). Use when searching for keywords within “Cita- tion and Document Text” or “Document Text.” Example: computer NOT W/2 careers

PRE/# Find documents where the first word appears some number of words before the second word. Use when searching for keywords within “Citation and Document Text” or “Document Text.” Example: world pre/3 web

(continued)

CHAPTER 4Section 4.1 Locating Resources

Tips & Tools: Making the Most of Databases—Advanced Features (continued)

One more useful tool that might prove helpful is ProQuest’s built-in thesaurus feature. This feature helps users identify other search terms that may be synonymous with the

initial search term or related to the term in some way. For example, a very broad search term, such as juvenile delinquency, would likely return hundreds of thousands of hits. But perhaps juvenile delin- quency isn’t quite the right term; the thesaurus feature will offer conceptually related ideas (and this feature is quite different from a literal search engine): broader terms, narrower terms, and then related terms. So if the true topic of interest is really juvenile female gangs involved in delinquency, clicking on the suggested Female Gang Members-Delinquency will make a search more precise. See Figure 4.1.

Figure 4.1: ProQuest’s Thesaurus feature

Access ProQuest’s Thesaurus feature by clicking on “Thesaurus” on the Advanced Search page or at the bottom of a search results page. Note that the “Suggested Topics” also provides additional search terms and related topics. As with other search engines, ProQuest includes dropdown menus of Boolean operators to help widen or narrow the scope of the search.

The screenshots are published with permission of ProQuest LLC. Further reproduction is prohibited without permission. www .proquest.com

The power and complexity of the available databases can be overwhelming to students who are first- time researchers. The key is to practice with these new tools, especially at the beginning of the research process. By becoming competent in database research in the beginning of the project, students will be better prepared to conduct the actual research and literature review. In addition, this competency will be beneficial to provide search skills that are useful well beyond the completion of this course.

CHAPTER 4Section 4.1 Locating Resources

In addition to the library resources available to students, another route for discovering research of interest involves using Google Scholar, which searches scholarly literature and abstracts available on the Web, and even on Google itself. For example, because of the unique nature of criminal justice and its four major fields of study, a library database may at times be too narrow and too focused, so resources that are more broad-based (such as Google Scholar) might identify research efforts not captured in other databases. There are also numerous governmental depositories of resources (e.g., the Educational Resource Infor- mation Clearinghouse, or ERIC) that may be helpful to students. A reference librarian who has been educated and trained to help others locate reference materials is a great resource to consult in order to determine whether government databases may be of value.

Knowledgeable People Faculty can also act as resources, or at least a starting point for students. Depending on the course, the instructor may have more expertise about a given topic, or at least be able to point students in a direction of additional sources. A reference librarian also has special talents that can be particularly useful. The university’s team of librarians is familiar with the available databases and can assist students with locating resources, generating search terms, and narrowing research topics. Librarians are available via e-mail, and by chat or phone.

Treeing Treeing is a technique by which a student examines the references section of an article to determine whether some of the sources cited might also be of value as part of his or her current research project. This technique is particularly effective when the student identi- fies a spot-on research article directly addressing the issue of interest; odds are, some of the references in that article may be references the student wishes to consult for his or her own research paper. Even if a particular research article is not ultimately cited in the research paper, some of the references from that article may be valuable for treeing. Thus, be sure to save PDFs and paper copies because those resources may be valuable in a dif- ferent context. For an example of treeing, see Tips & Tools: Example of Treeing.

Tips & Tools: Example of Treeing

A research question that evaluates the effectiveness of a juvenile restorative justice pro- gram could draw related resource material from social justice journals as well as criminal

justice journals. The article “Putting a human face on crimes: A qualitative study on restorative justice processes for youths” was published in the Child & Adolescent Social Work Journal (2011, October), 335–355. In the References section, 66 sources are presented. A review of the first 25 citations yields the following nine potential articles that could be relevant to the research question:

Applegate, B. K., Davis, R. K., & Cullen, F. T. (2009). Reconsidering child saving: The extent and correlates of public supports for excluding youths from the juvenile court. Crime & Delinquency, 55(1), 51–77.

Bazemore, G. & Green, D. L. (2007). “Yardsticks” for victim sensitive process: Principle-based standards for gauging the integrity of restorative justice process. Victims and Offenders, 2, 289–301.

(continued)

CHAPTER 4Section 4.2 Breadth of Field: Criminal Justice (Broad) and Topic-Specific Databases (Narrow)

4.2 Breadth of Field: Criminal Justice (Broad) and Topic-Specific Databases (Narrow)

One of the early decisions in writing a social and criminal

justice research paper is whether to take a broad approach or a nar- row approach. As mentioned in Chapter 3, criminal justice encom- passes four major fields of study (law enforcement, courts, correc- tions, and juvenile justice). Social justice encompasses three main principles (equality, solidarity, and human rights). A broad approach covers a breadth of topics so that a student may attempt to integrate multiple issues when examining phenomena and events of inter- est. This sort of broad viewpoint can include historical and cultural

iStockphoto/Thinkstock

Should you take the broad approach and explore the ocean, or the narrow approach and explore the shark?

Tips & Tools: Example of Treeing (continued)

Braithwaite, J. (1999). Restorative justice: Assessing optimistic and pessimistic accounts. Crime and Justice, 25, 1–127.

Braithwaite, J. (2002). Setting standards for restorative justice. British Journal of Criminology, 42, 563–577.

Butts, J. A. & Mears, D. P. (2001). Reviving juvenile justice in a get-tough era. Youth & Society, 33(2), 169–198.

Calhoun, A. & Pelech, W. (2010). Responding to young people responsible for harm: A comparative study of restorative and conventional approaches. Contemporary Justice Review, 13(3), 287–306.

Choi, J. J. & Severson, M. (2009). What! What kind of apology is this? The nature of apology in victim offender mediation. Children and Youth Services Review, 31, 813–820.

Daly, K. (2002). Restorative justice: The real story. Punishment & Society, 4(1), 55–79.

Hayes, H. & Daly, K. (2003). Youth justice conferencing and reoffending. Justice Quarterly, 20(4), 725–764.

Once these potential articles have been identified, the student should then search the database for the actual articles. If an article provides information worthy of inclusion in the research project, the student would synthesize that information and add the citation to the reference section of the project. As a result of the treeing process, if all nine articles are worthy of inclusion, the student has added nine resources to his or her research project, provided from the one initial article.

CHAPTER 4Section 4.2 Breadth of Field: Criminal Justice (Broad) and Topic-Specific Databases (Narrow)

issues within several or all of the fields, to examine the impact of events that influence our daily lives. For example, consider the topic “sex offenders.” A student might explore the context of how sex offenders are identified and labeled (legal), or the specific behaviors that sex offenders engage in (behavioral), or how the societal view of sex offenders has changed from the 20th to the 21st century (historical and cultural), or the role that the Inter- net has played in pedophilia (cultural and legal). To take a broad approach, the student might address all of these aspects on one literature review.

A narrow approach explores a subject in depth. In the earlier example about sex offenders, a student taking a narrow approach would perhaps explore only the legal perspective or only the cultural perspective but will explore it more deeply. This type of approach is common in criminal justice. Perhaps the student wants to understand a specific crime or examine a specific offender’s behavior. For example, a student may want to explore the early life of Ted Bundy in an attempt to identify whether events in his childhood had an effect on his later criminal behavior.

The type of approach would undoubtedly influence the types of journal articles a student might review in preparation of the research paper. There are journals such as The Journal of Crime and Justice and Crime and Delinquency, which publish a broad spectrum of articles and reflect the integrative work that criminologists often do. Numerous other journals focus on one of the specific fields or on specific topics or issues. For example, Police Stud- ies, Policing and Society, and the FBI Law Enforcement Bulletin all focus on law enforcement topics and issues. Criminal Law Quarterly, Criminal Law Review, and the Journal of Criminal Law all focus on legal and court issues, while Corrections Today focuses on current issues and trends in corrections at the local, state, and federal levels. In addition, specific topic journals include Criminal Justice Ethics, Child Abuse and Neglect, Journal of Drug Issues, Fed- eral Probation, and the Journal of Criminal Justice Education, which all focus on a specific topic or issue within criminal justice. For social justice topics, many journals are topic spe- cific, while others are more integrative in nature (i.e., George Mason University Civil Rights Law Journal, Journal of Human Rights, Journal of Sociology and Social Welfare, Race, Gender & Class, Social Justice, Social Justice Review, Social & Legal Studies, and Windsor Review of Legal & Social Issues).

Both broad and narrow approaches are effective in developing research papers that a stu- dent writes in social and criminal justice. However, depending on the topic and research question, the best approach might be a hybrid approach in which the student simultane- ously examines the big picture (broad) while also examining important specific details (narrow). Being able to develop the big picture (broad approach) while incorporating the important specific details (narrow approach) is an effective strategy, particularly when the research question involves a topic that can be evaluated on both a general and a specific perspective. For example, in a study of Ted Bundy, not only would his criminal behaviors be examined but the student would also examine his childhood history, as well as the social and cultural contexts in which his crimes were committed.

CHAPTER 4Section 4.3 Evaluating Sources

4.3 Evaluating Sources

However broad or narrow the research topic might be, the

student will still face the task of sifting through the numerous potential sources or “hits” on a database search. In evaluating resources, students should con- sider the source’s relevance to the topic at hand as well as its credibility, or trustworthiness. This is particularly true of Inter- net sources. Search engines such as Google Scholar and JSTOR can be useful in uncovering sources unavailable on library databases, such as government records and sources at college libraries. How- ever, because it is more difficult to determine an Internet source’s credibility, students should take extra care in evaluating these sources. The following sections recommend specific cri- teria for making the decision about the credibility of the source. In truth, sometimes it takes quite a bit of detective work to determine whether the information from a particular source is believable or not.

Relevance Imagine that a student is using library resources and search engines to research a topic of interest. A search result emerges with many good leads on the topic. How does the stu- dent decide which materials to retain and which to ignore? To be able to do this efficiently is a skill, and like any skill, it will take practice before it can be perfected. But how does a student make those decisions now?

Honestly, it is a bit of trial and error. For the research project in CRJ 422, it would be better to err on the side of retaining too much information (for possible inclusion in the research paper) than to not have enough information available. It is easier to discard a source that is not used in the final project than it is to go back and generate more searches and reviews. A strong outline and a clear research question can be the litmus test for how relevant a lead or source might be. That is, a student can compare the “gist” of a located resource to the research question and ask whether the resource under consideration could contribute to answering the question.

It may seem odd to include sources that contradict each other in answering the research question. However, it is important for all students to strive to be open-minded and criti- cal thinkers, looking for evidence on all sides of an issue. Do not ignore contradictory research just because it is contradictory; be sure to acknowledge alternative points of view. Critically thinking about social and criminal justice issues almost always means consider- ing multiple points of view.

Wu Tao/OJO Images/Getty Images

When conducting Internet research, it is important to verify sources and make sure that they are trustworthy.

CHAPTER 4Section 4.3 Evaluating Sources

Primary Versus Secondary Sources As much as possible, students will want to retrieve primary sources, which are original documents. For example, Doug Robinson conducted a research study on the effects of alcohol on criminal behavior and published an article in the Journal of Contemporary Criminal Justice. As Robinson was the researcher and he wrote the article, this is consid- ered a primary source.

It is also possible to have articles that review or summarize a primary source, written by another author. These are called secondary sources. For example, if Stan Mitchell were to review and critique Robinson’s article about alcohol and criminal behavior, and write and publish his own article in the Journal of Criminal Justice and Popular Culture, Mitchell’s article would be a secondary source. Although Mitchell’s article is written about the origi- nal research conducted by Robinson, it is Mitchell’s interpretation of the research and is thus a secondary source.

Essentially, authors cannot control how others interpret their work; they can only control what they write. Criminologists prefer primary sources whenever possible, as they pro- vide the opportunity to understand the researcher’s perspective in his or her own words, as opposed to someone else’s interpretation.

Authorship For any information to be considered credible (including a journal article), it is important that an author be listed (Ormondroyd, Engle, & Cosgrave, 2011). When an author (or authors) wants to make an evidence-based claim in criminal justice research, the credibil- ity of that claim is based in part on the author’s identity, which is also related to expertise (discussed in the next section). In general, scholars should be willing to attach their name to their own work. There are rare occasions when a criminologist might publish a work as “Anonymous,” but a student should question why. It could be that a topic is so contro- versial that an author fears some sort of academic retribution, such as the denial of tenure or a promotion. In some limited cases, it might make sense to publish anonymously, but those extenuating circumstances are rare. So when authors are unwilling to attach their names to an idea or claim, one might start to think, “Is this idea so flawed that no one was willing to put their name to it?”

Anonymous publishing happens frequently on webpages. A student might locate some great web-based information in the process of conducting research for the literature review, but there is no author listed on the page (or document) of interest. It may take some extended detective work to ascertain the author, but that process is important. If no one is willing to attach a name to the work, then this should raise a red flag and the mate- rial should not be used.

The question of authorship is why many educators become concerned when students cite Wikipedia entries in their assignments. Wikipedia contributors often do not identify themselves. Couple this anonymity with the unrestricted ability to change an entry (at least temporarily), and doubts about credibility will certainly emerge.

CHAPTER 4Section 4.3 Evaluating Sources

This is not to say that Wikipedia has no value. It can be a good source for generating ideas or seeing what others believe about a topic. In essence, the importance of Wikipe- dia hinges on how students will use the information. For example, if a student were to look for the crime rate in New York City and Wikipedia provided the wrong answer, that incorrect information may result in a point deduction for the student on a test or assign- ment, but getting the wrong information in this instance would not generate great harm. However, if that same student were employed by the New York Police Department and had gathered an erroneous crime rate for the city from Wikipedia, city and police policy decisions based on that erroneous information could lead to ineffective policies and polic- ing. Then the credibility of the information truly matters.

When authorship is tied to information, accountability is possible. Has anyone on Wikipe- dia ever been sued for providing incorrect information? Probably not, but the authoring of ideas and claiming that authorship are important aspects of credibility and accountability.

Expertise In addition to knowing the identity of an author, it is also important to be able to gauge the relative expertise of the person or organization making the claim. That is, does the person making the claim have the expertise, credentials, and know-how to make an appropriate and accurate claim? Every source has a point of view, whether a criminol- ogy professor at a university, a nonprofit agency working to reduce domestic violence, or a gov- ernment agency charged with fighting the war on drugs. No source is completely objective, so it is important to understand the author’s underlying motivations. Expertise is used to denote a special- ization in a scholarly discipline or topic and can be measured using the author’s track record in publishing, training in a particular area, etc.

In analyzing the expertise of the author and the material, students should be aware of two addi- tional factors. First, just because the author has graduate education (master’s degree, doctor- ate), this does not necessarily make him or her an expert. If an author makes a specific claim or statement regarding a point of research, students should also seek out others who have researched the topic and compare the findings. If the author

in question provides findings that are in sharp contrast to numerous others with similar results, the student should question the expert’s findings and research abilities. Although finding a new theoretical explanation for an issue is possible and highly sought after in social and criminal justice research, other authors should be testing the findings and either supporting or discrediting the new data.

Steven Senne/ASSOCIATED PRESS/AP Images

Harvard professor and African-American scholar Henry Lois Gates, Jr., has been at Harvard since 1991 and undoubtedly has gained considerable credibility in his area of expertise.

CHAPTER 4Section 4.3 Evaluating Sources

Second, just because an Internet source has “.edu” (a domain reserved for educational institutions) as part of the URL, this does not necessarily lend expertise to authors or to the information they provide. Ideally, the information retrieved from an educational entity would be accurate and credible, but students should not make that universal judgment. Ultimately, students need to fully and thoroughly explore the accuracy of information from multiple sources and utilize all the tools provided in this chapter to determine which sources to use and which not to use.

Currency Currency refers to the publication date of the journal article or other information. With a journal article, the year of publication should be clearly identifiable, as should the journal’s volume number and issue number. With web-based informa- tion, it is often much more difficult to determine its publication date, which is why “n.d.” is used in references at times, meaning “no date” was located or identified.

The date provides yet more evidence about the credibility and believability of information; undated information can still be useful, but the student needs to rely on other criteria (authorship, expertise, etc.) to make the case for credibility. However, currency doesn’t necessarily mean the article or material was published within the last year. For example, if a student is researching prohibition and its relationship to organized crime, gathering articles written in the late 1920s and 1930s would certainly be relevant, and cur- rency would not be an issue with this topic.

But if a student is examining the use of social networking sites by pedophiles to identify and locate potential victims, the currency of sources does matter. As social networking sites have become more plentiful and available in the 21st century, using a journal article from the 1990s about the emergence of social networking sites may not be current and could lead to erroneous research conclusions. Although currency may matter in some research and not in others, students should still seek out articles and sources for which a publication date is available, rather than those with no date.

Paul Sakuma/ASSOCIATED PRESS/AP Images

Currency is an essential facet of academic importance. For example, the impact of Facebook has significantly changed since its inception in 2004 and articles from 2007 may no longer be relevant.

CHAPTER 4Section 4.3 Evaluating Sources

Accuracy Judging the accuracy of academic sources is a complex task requiring some expertise on the student’s part to determine whether an author “got it right.” In some ways, this pro- cess is a bit analogous to looking up a word in the dictionary that one knows absolutely nothing about. Since no prior knowledge is available, one can either just trust the diction- ary definition or seek to acquire more knowledge about the topic. A student in social and criminal justice should rely on other indicators (at first) concerning the accuracy of a jour- nal article until acquiring enough expertise for independent judgments.

One of the key ideas underlying accuracy of research is the lack of bias, or prejudice, in the presentation of information (Harris, 2005). The more a student reads journal articles, the better he or she will become at identifying such bias; it may be slight at times and at other times flagrant. Useful information can be extracted from a biased source, but it is impor- tant to remember that the bias colors any ultimate interpretations. For different types of bias that can occur within the realm of research, see Table 4.3.

Table 4.3: Types of bias

Type of bias Description

Publication bias (Positive results bias) The tendency on the part of the investigators, reviewers, and editors to submit or accept manuscripts for publication based on the direction or strength of the study findings.

Language bias Languages of publication depend on the direction and strength of the study results.

Funding bias Bias in the design, outcome, and reporting of industry-sponsored research in order to show a favorable outcome.

Outcome reporting bias A study in which multiple outcomes were measured, but only those that are significant are reported—not those that were insignificant or unfavorable.

Database bias Biased indexing of published studies in literature databases. The literature search will be biased when it is based on a database in which the results of indexed studies are systematically different from those of nonindexed studies.

Study quality bias Studies of lower or higher quality are associated with positive or favorable results.

Gray literature bias The results reported in journal articles are systematically different from those presented in reports, working papers, dissertations, or conference abstracts.

From Cochrane Collaboration (2011).

Another consideration is the distinction between assertion and evidence (Galvan, 2006). An assertion is an author’s opinion about the meaning or interpretation of data; it may be the author’s own data reported in an empirical study or an assertion about the data of others. Evidence is the quantitative or qualitative information or data gathered using

CHAPTER 4Section 4.3 Evaluating Sources

appropriate research methods. Evidence is typically presented in the results section of an empirical journal article. Solid evidence should stand on its own, while the assertions made based on that evidence should be carefully evaluated. For example, based on the same research evidence regarding the benefits of medical marijuana, two students may develop very different assertions about the practical applications of the evidence (e.g., determining if the benefits of medical marijuana outweigh the costs or harms). So, when evaluating the accuracy of an academic source, students should strive to accurately dif- ferentiate between assertions and evidence.

Corroboration If research findings or results from a journal article seem too good to be true, they prob- ably are. Another old saying in research is that “if it hasn’t happened twice, it hasn’t happened.” Advances in research are based on the accumulation of corroborating, or sup- porting, evidence. When much evidence for a particular idea or proposition exists, we tentatively believe it until solid contradictory evidence is available.

In a critique of a resource, if the new findings help to confirm previous findings, then the journal article under review is helping to corroborate existing evidence. At times a newly published article may contradict existing research, and in that case it is the obligation of the author to explain why these new results do not corroborate existing research. For example, a new research approach may have been utilized, or a survey scale was refined, or some other methodological improvement was developed. In any case, it is incumbent upon the author to explain how the new evidence contributes to the current understand- ing on the topic (typically appearing in the discussion section of an empirical journal article). The more corroboration that exists for an idea, the more researchers can rely on that idea being correct.

For a slightly different conceptualization (and summary) of the preceding points that con- tribute to determining the value of information (especially online information), research librarians (e.g., University of Iowa, 2011; University of the Fraser Valley, 2009) use what is known as the CRAAP test, in which the characteristics of currency, relevance, authority, accuracy, and purpose are used to interpret the value of the information (see Table 4.4).

Table 4.4: CRAAP test

Currency: the timeli- ness of the source and the information

• When was the information posted? • When was it last revised? • Are links functional and up-to-date? • Is there evidence of newly added information or links? • Is the information still considered accurate? Has more recent

research challenged this information? Don’t exclude articles or information because of the publication date; instead think about the currency and relevance of the arguments presented.

(continued)

CHAPTER 4Section 4.4 Journal Articles and the Peer Review System

Relevance/coverage: the importance and scope of the information

• Does the information relate to your topic or answer your question? • Is the topic covered with sufficient depth and breadth? Is the infor-

mation comprehensive enough for your needs? Are the complexities of your topic adequately addressed?

• Could you find the same or better information in another source? • Is the information relevant to current scholarly discussions on the

topic? Do scholars refer to this source?

Authority: the source of the information

• Is the author/sponsor clearly identified? Is contact information easy to find?

• What are the author’s credentials? Is the author knowledgeable in his or her field (based on employment, publications, or sponsorship by reputable organizations)?

• Has the author published works in traditional formats? (Look up the author in Google Scholar.)

• Is the author affiliated with an organization? Does this organization appear to support or sponsor the page? (Google the authors and/or sponsoring organizations.)

• What does the sponsoring site (e.g., www.noaa.gov, www.uiowa .edu) and domain name (e.g., .com, .edu, .gov, .org, .net) reveal about the source of the information, if anything?

Accuracy: the reli- ability, truthfulness, and correctness of the content

• Where does the information come from? Can you verify any of the information in independent sources or from your own knowledge?

• Are the original sources of information listed? • What evidence is presented to support claims made? • Has the information been reviewed or refereed? • Does the language or tone seem objective and unbiased? • Is the information free of spelling, grammar, and typographical

errors?

Purpose: the reason for the website’s existence

• Is the purpose of the page stated? Is the purpose to: inform? teach? entertain? enlighten? sell? persuade? Are possible biases clearly stated?

• Is advertising content versus informational content easily distinguishable?

• Are editorials/opinion pieces clearly labeled?

4.4 Journal Articles and the Peer Review System

Journal articles are perhaps the most important communication mode for both social justice and criminal justice research. Journals have a more timely publication frequency,

can reach large numbers of people, have a rigorous acceptance and publication process, and are a well-established means of information distribution. It is important that social and criminal justice students become familiar with journal articles and their format, as journal articles will likely provide a significant amount of research material. For some tips, see Tips & Tools: How to Read a Journal Article.

Table 4.4: CRAAP test (continued)

www.noaa.gov

www.uiowa.edu

CHAPTER 4Section 4.4 Journal Articles and the Peer Review System

Tips & Tools: How to Read a Journal Article

Social psychologists Christian Jordan and Mark Zanna (1999) wrote a book chapter about how to read a journal article, and although their emphasis was on how to read journal

articles in social psychology, their advice is applicable to social and criminal justice as well.

When approaching a journal article for the first time, and often on subsequent occasions, most people try to digest it as they would any piece of prose. They start at the beginning and read word for word, until eventually, they arrive at the end, perhaps a little bewildered, but with a vague sense of relief. This is not an altogether terrible strategy; journal articles do have a logical structure that lends itself to this sort of reading. There are, however, more efficient approaches—approaches that enable you . . . to cut through peripheral details, avoid sophisticated statistics with which you may not be familiar, and focus on the central ideas in an article. Arming yourself with a little foreknowledge of what is contained in jour- nal articles, as well as some practical advice on how to read them, should help you read journal articles more efficiently. (p. 461)

Reading and extracting information from journal articles is an acquired skill, and like all skills, it takes time and practice to perfect it. In order to practice this skill development, Jordan and Zanna (1999) offered these tips for reading the major sections of research articles:

INTRODUCTION (Try to answer these questions when reading an introduction section.)

• What was the problem studied, and why was it studied? • How did this study go beyond previous studies of this same problem? • How did the researchers arrive at their hypotheses? • What questions do the researchers hope to answer by the study’s conclusion?

METHOD (Try to answer these questions when reading a methods section.)

• How were the hypotheses translated into testable questions? • How were the variables manipulated and/or measured? • Did the measures of the variables accurately represent the actual variables of interest?

RESULTS (Try to answer these questions when reading a results section.)

• Did the researchers provide evidence that the independent variables worked? • What were the major outcomes of the study? • Were the researchers’ hypotheses supported by the results of the study? • Did the researchers explain the findings that did occur (regardless of hypotheses)?

DISCUSSION (Try to answer these questions when reading a discussion section.)

• What conclusions can be drawn from the study? • What new insights are now available about this topic because of these research findings? • Did this study help fill the previous knowledge gap about the topic? • What are the practical implications of these results? • Did the results contradict previous findings, and if so, why? (continued)

CHAPTER 4Section 4.4 Journal Articles and the Peer Review System

Journals in social and criminal justice operate under a peer review system in which sev- eral professionals review article submissions before an acceptance decision is made. A criminologist who wants to professionally publish the results of her or his research starts by selecting a journal and sending the manuscript to the journal editor. The journal editor will send copies of the manuscript to criminal justice professionals, both academics and practitioners, for review. Thus the peer review process begins, as the journal editor asks the author’s peers in the field to review the manuscript and decide whether it is suitable for publication. The peers are also called referees, thus a “refereed journal” means the journal follows this peer review process.

An important note to students is that this process differs from submission of a manuscript to a popular magazine. Magazines pay people to write articles; authors of journal articles are not paid and may sometimes help defray the cost of journal publishing. Whereas mag- azine articles may be checked for accuracy, they do not undergo the same scrutiny, exami- nation, and review as journal articles. The majority of journal articles are well documented with supporting references that note when an idea has been adopted from another source. A magazine article is rarely as extensive in documenting the academic and scholarly work of the author. The distinction between journals and magazines is an important factor, especially when students are determining the credibility and reliability of the informa- tion. For some excellent guidelines for understanding the difference between a scholarly journal article and a popular magazine article, see Table 4.5.

Table 4.5: Differences between magazine and journal articles

Characteristic Popular magazine article Scholarly journal article

Length Shorter articles, broad overview of topics

Longer articles providing in-depth analysis of topics

Authorship Author is often a staff writer or journal- ist; sometimes name and credentials are not provided

Author (or authors) is usually an expert or specialist in the field, with name and credentials always provided

Language and audience

Written in a nontechnical language for anyone to understand

Written in the jargon of the field for scholarly readers (professors, research- ers, students) (continued)

Tips & Tools: How to Read a Journal Article (continued)

Journal articles are a key means of communication for researchers in social and criminal justice. Because criminal justice is such a vast area of study with the four major fields,

and social justice encompasses the main principles of equality, solidarity, and human rights, as research- ers, professors and students alike rely on journal articles to provide the most up-to-date information on all topics and issues. Since social and criminal justice research focuses mostly on a particular aspect of human behavior (i.e., criminal behavior), journal articles are invaluable for advancing knowledge of crimes and criminal behaviors, as they provide guidance and insight for future research.

CHAPTER 4Questions for Critical Thinking

Characteristic Popular magazine article Scholarly journal article

Format and structure

Articles do not necessarily follow a specific format or structure

Articles are usually very structured, with abstract, introduction, method, results, discussion, references

Special features

Illustrations with glossy or color photo- graphs, usually for advertising purposes

Illustrations that support the textual presentation, statistical tables, graphics

Editors Articles are not evaluated by experts but by editors on staff

Articles are reviewed and critically evaluated by experts in the field (called peer reviewers or referees)

Credits A bibliography is usually not provided, although names of reports or refer- ences may be mentioned in the text

A reference section is always pro- vided to document research sources thoroughly

University of Texas at San Antonio Library, 2007.

Chapter Summary

Research in social and criminal justice encompasses the various fields in criminal justice—law enforcement, courts, corrections, and juvenile justice—as well as the

major principles of equality, solidarity, and human rights of social justice. Within each of these areas there are specialties and subspecialties, demonstrating the potential depth of any one particular area. One challenge for students is to achieve a breadth (broad approach) of the topic, while also establishing a depth (narrow approach) of the topic. An effective literature review, which includes identifying past research on the topic that can provide insight for future research, is essential to answering the research question. Evaluation of online information, regardless of source, should involve the investigation of the currency, relevance, authority, accuracy, and purpose of the information available (the CRAAP test). The process of reviewing the literature can be enhanced when a research question is clearly defined, which can drive and inform the various search strategies that are employed. By gaining the skills and confidence to navigate online databases, includ- ing the use of advanced features, students in social and criminal justice will acquire valu- able skills that will serve them well beyond graduation.

Questions for Critical Thinking • In locating resources, what are some options for finding literature online? How

can the university’s team of librarians help in locating resources? What is the process for treeing?

• What is the difference between a broad approach and a narrow approach to a research topic? How does the approach influence the types of journals you would research for your project?

• Why is evaluating the credibility of a source important? Why are relevancy, currency, and accuracy important?

• Why are journal articles important for communicating in social and criminal justice research? What is a peer review, and why is it important for academic journal articles?

Table 4.5: Differences between magazine and journal articles (continued)

CHAPTER 4Key Terms

anonymity The absence of a connection between a specific participant and the data that he or she provides.

bias Prejudice or favor for one outcome versus another, generally unintentionally caused by the outcomes a researcher hopes to find.

Boolean operator Mathematical terms, phrases, and descriptors that can be used in database searches to widen or narrow the scope of the search.

corroborate A process of verification by ensuring that previous research supports claims that a researcher is attempting to make in a study.

CRAAP test A tool used to interpret the value of information using the character- istics of currency (timeliness), relevance (importance), authority (source), accuracy (reliability), and purpose (reason).

credibility The trustworthiness a researcher can place in a source based on where the information was obtained and how recent and relevant it is.

currency The practice of identifying when a journal article or other source was published.

expertise The experience, credentials, and know-how that people or organizations have in order to make their information credible.

peer review system A practice in the social sciences in which several profession- als review article submissions before an acceptance decision is made.

primary source The original docu- ment about the research, written by the researcher.

reference librarian An individual edu- cated and trained in helping others locate reference materials.

Web Links This website outlines tips for evaluating web content and determining an author’s expertise: http://library.albany.edu/usered/eval/evalweb/.

This website explains the criteria by which researchers evaluate print information: http://guides.library.jhu.edu/evaluatinginformation.

This website outlines ways to analyze the usefulness of a webpage and presents ques- tions to ask yourself when visiting webpages for the purposes of obtaining information: http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/Evaluate.html.

This website contains a detailed tutorial on how to use Boolean operators to maximize searches: http://www.internettutorials.net/boolean.asp.

This website provides videos, definitions, tips, and tricks for identifying scholarly journals versus popular periodicals: http://olinuris.library.cornell.edu/ref/research/skill20.html.

Key Terms

http://library.albany.edu/usered/eval/evalweb/

http://guides.library.jhu.edu/evaluatinginformation

http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/Evaluate.html

http://www.internettutorials.net/boolean.asp

http://olinuris.library.cornell.edu/ref/research/skill20.html

CHAPTER 4Key Terms

search engine Databases that index material in order to facilitate a user’s fast retrieval.

secondary source An article or source about original research that was not writ- ten by the individual person who com- pleted the research.

treeing A technique by which a student or researcher examines the references section of an article to determine whether some of the sources cited might also be of value as part of his or her current research project.