Discussion Board

ARTICLE EVALUATION GUIDE

1

SECTION I: INTRODUCTION The introduction is just that – an introduction to the research study. A good introduction will give you a complete picture of A) the topic being studied, B) the problem that the researchers wish to address through their study, C) the significance of the problem, D) previous research that has been conducted on the topic and/or problem, and E) the research question(s) and hypothesis(es). By the end of the introduction, you should understand why the problem is important to study and exactly what the researchers hope to do through their study. You should also feel confident that these researchers are capable of carrying out the study. A good introduction will hook the reader into wanting to find out more about the study and feeling like the study is worthwhile. For an evaluation of the introduction section, respond to at least six (6) of the following questions (include the questions in your evaluation to indicate which questions you’re answering):

1. In your own words, state the topic of the research study. Did the authors clearly introduce the topic? If not, what could they have done better?

2. In your own words, state the problem that the researchers say they will address. Did the authors clearly describe the problem? If not, what could they have done better?

3. In your own words, describe how the authors discussed the significance of the problem. Did the authors clearly describe the significance of the problem? If not, what could they have done better?

4. What are the important terms that the authors define? What other important terms do you think the authors should have defined?

5. Do the researchers have the knowledge and skills to carry out the proposed research? Whether yes or no, how do you know?

6. How is the background information on the problem presented? 7. Describe some of the major themes that the authors present in the literature review. Is the

review comprehensive? Why or why not? 8. Have the references been analyzed and critiqued, and the results of various studies

compared and contrasted? How did the authors organize their literature review? 9. Describe how the literature review educated you on the problem/topic. Describe what you

wish you knew more about regarding the problem/topic. 10. Describe how the authors justify the need for their study based on what they found in

their review of past literature. 11. How many different references do the authors include in their literature review? Do these

references come from different types of sources? 12. What are the research questions for the study? 13. What is/are the hypothesis/es? 14. What is the purpose of the study? Did the authors clearly describe the purpose? If not,

what could they have done better?

ARTICLE EVALUATION GUIDE

2

SECTION II: METHODOLOGY

The methodology section often gets a bad rap as one of the parts of an article that is “okay” to skip when you’re given an article to read for class. If you really want to be a smart consumer of research, though, the methodology section is an important piece to look at, as it gives you clues about whether or not the researchers were careful and did a good job of carrying out their study. The main purpose of a methodology section is to give the reader enough step-by-step information about how the study was carried out that the reader could then go out and replicate (repeat) the study themselves. Research is supposed to be transparent. If you conduct a study and say that you found such-and-such results, I should be able to conduct the same study and find similar results if your results are valid and reliable. A good methodology section will include detailed descriptions of A) the study participants, B) recruitment of study participants, C) the study design and procedures, and D) the measurement tools that were used to collect data. For an evaluation of the methodology section, respond to at least five (5) of the following questions (include the questions in your evaluation to indicate which questions you’re answering):

1. Describe how the researchers found participants for their study (sampling selection). Where did they go to recruit participants? How did they recruit participants?

2. Describe the inclusion and exclusion criteria for study participants. What characteristics were the researchers looking for in potential participants? What characteristics in a potential participant would lead the researchers to not include that individual in the study?

3. What are the limitations and/or biases either that you can pick out or that the authors stated in the methods they used to gather participants for their sample.

4. Describe the size and major characteristics of the population studied. Remember, the population is the group of individuals that the researchers want to study, but it is not always possible (because of time, money, size, etc.) to include every single individual from the population in the study itself. They have to gather a sample of individuals from the population that will allow them to generalize their study results to that larger population.

5. If you were a researcher who was looking to replicate (repeat) this study, what information do you need that is missing from the procedures described by the authors?

6. How are the design and procedures appropriate for examining the research question or testing the hypotheses of the study?

7. What do the authors say about the internal validity of their research design? 8. What instruments/measures did the researchers use to collect data? Are these

instruments/measures appropriate for the study? How so?

ARTICLE EVALUATION GUIDE

3

III. RESULTS/FINDINGS

Ah, the results section… The most feared of the article sections for most students. This is the other section of an article that most professors will tell you is okay to skip because you can read about the results in a less-confusing way in the discussion/conclusion section. A smart consumer of research knows better! Skipping the results and only reading the discussion/conclusion section is like letting an “expert” tell you that the earth is flat and believing them because they told you so. You’re missing out on the “evidence” that they’re using to support that claim. The results section is supposed to present the evidence that the authors will then use to make claims in the discussion/conclusion section. A good results section will describe A) the statistical tests that were used in the data analyses, B) the results of the data analyses, and C) the statistical significance level of the results. All of that information should be reported in clear, objective language and the results should be presented in an organized mix of narrative (words) and tables and figures. For an evaluation of the results/findings section, respond to at least four (4) of the following questions (include the questions in your evaluation to indicate which questions you’re answering):

1. What statistical tests did the researchers use for their data analyses? Were these tests appropriate to the type of data and study design? How so?

2. What descriptive statistics are presented? How are the descriptive statistics presented (narrative, table, figure, etc.)?

3. At what probability level (.01, .05, .1, etc.) were the results of the statistical tests evaluated in the data analysis? What rationale (reasoning) did the authors provide for choosing that probability level?

4. Which hypothesis/es was/were tested? 5. Describe, as best as you, some of the statistically significant results. Did the authors

clearly describe these results? 6. Is appropriate use made of tables and figures? How so? Are the data in each table and

figure described in the text? Are the tables and figures (if any) well organized and easy to understand?

7. Describe how the authors presented the results. Is the narrative discussion clear and concise? Did the authors discuss any conclusions or implications of the results? Did the authors seem to write in an objective (versus subjective) way?

ARTICLE EVALUATION GUIDE

4

IV. DISCUSSION AND CONCLUSION

The final section of an article, the discussion/conclusion is supposed to wrap up the findings of the study in tidy way and open up the floor for other studies on the same or similar problems. In the results section of the article, the authors were to objectively present the study results/findings. Now, they are to interpret what those results/findings mean in the context of the problem their study is addressing. You should see a clear link between what is said in the discussion/ conclusion, the information that was presented in the introduction (problem description, research question(s), hypothesis(es), etc.), and the results/findings. In this final section, the authors should also be transparent about limitations of the study and its findings. They should also present implications of their findings for this topic, population, problem, etc., and their recommendations for future research on the problem. A good discussion/conclusion provides: A) an objective description of the findings and conclusions of the study based on the results, B) implications of the findings and conclusions, C) limitations of the study and its findings, and D) recommendations for future research. For an evaluation of the discussion/conclusion section, respond to at least five (5) of the following questions (include the questions in your evaluation to indicate which questions you’re answering):

1. What major findings do the authors report in this section? Do they report these findings clearly, concisely, and objectively? How so?

2. How are the findings and conclusions justified by the data presented and analyzed? To answer this question, you will need to briefly read over the results section.

3. What do the authors say about how their findings relate to previous studies’ results and findings? Looking back at the literature review section, what could the authors have added about the previous studies’ relationship to their own study’s findings?

4. Describe what the authors say about whether the study addressed the problem. Do they restate the problem in this section?

5. Describe the limitations presented by the authors. If they don’t present any limitations, come up with 2-3 limitations that you see with the study. Remember, no study is perfect!!

6. What implications of the findings are discussed? These can be implications for theory, policy, and/or practice. If they don’t present any implications, come up with 2-3 implications that you see from the study.

7. What recommendations for future research are made? If they don’t present any recommendations, come up with 2-3 recommendations for future research based on the study.