Article Criticism

· Introduction and Review of Literature

· Does the researcher establish the importance of the problem area?

· What are the dependent and independent variables?

· Evaluating the Sampling of Generalizable and Non-Generalizable Studies

· Was random sampling used? Were the groups statistically the same? Hint: look at tables comparing demographics between groups – is there statistical similarity?

· If no random sampling, what type of sampling technique was used and how might that have introduced bias?

· How did the researchers recruit the sample participants? Could that be a source of bias?

· What excluding/including criteria were in place when choosing the participants?

· Has the author described relevant demographics of the sample?

· Is the overall size of the sample adequate?

· Has the researcher described the sample/population in sufficient detail?

· Who sponsored the study? Could that be a source of bias?

· Experimental Procedures

· Are the treatments/interventions described in sufficient detail?

· What type of study design was used, i.e. randomized controlled experimental study? Cross-sectional survey? Meta analysis?

· If the treatments were administered by people other than the researcher, were these people 
properly trained?

· What type of setting was used for the study?

· Has the researcher distinguished between random selection and random assignment?

· How have the researchers assured the validity of the design of their study? In other words, are they able to measure what they intended to measure?

· What steps have they taken to assure against major threats to validity?

Internal validity: History, maturation, testing, statistical regression, attrition, contamination

External validity: Selection bias, Hawthorne effect, placebo effect, pretest sensitization, etc.

· How have the researchers tried to minimize confounding variables?

· Have you identified any areas in the study that could bias the results? What are they?

Putting It All Together

· Have the researchers selected an important problem?

· Is the report cohesive?

· Does the report extend the boundaries of our knowledge on a topic?

· Are any major methodological flaws unavoidable or forgivable?

· Is the research likely to inspire additional research?