critique

RESEARCH STEP

QUESTIONS

Summary

Critique

Problem/

Justification

SUMMARY: What is the problem or need?

CRITIQUE: Is the problem statement logical? . . . convincing?

How might it have been improved?

#1--Purpose

SUMMARY: What is the author's purpose?

Does it relate to theory, practice, or both?

CRITIQUE: Does the purpose follow logically from the problem statement? Does the author convince you the study is worth doing? How?

Prior Research/

Literature Review

SUMMARY: What previous work has been done leading up to this study? If the authors present a theoretical base for their study, how do they connect it?

CRITIQUE: How comprehensive is the literature review section of the report? Justify your answer.

Does it include studies that you know to be relevant to the problem? If not, what might have been overlooked?

#2--Research Question(s)

SUMMARY: What is/are the research question(s)?

CRITIQUE: Is each research question clearly stated? If not, what might be improved?

What is the logical relationship to the purpose?

Approach

SUMMARY: Is the study descriptive, predictive, or explanatory? Or does it combine approaches? Explain.

CRITIQUE: What is the most convincing approach for this study? If that is not the approach actually used, explain what might have been better and why.

Hypothesis(es)

SUMMARY: Does the writer state one or more hypotheses? If so, summarize.

Identify each as directional, nondirectional, or null.

CRITIQUE: If the hypotheses are not explicitly stated, should they have been?

If stated, evaluate their relationship to the research questions.

#3--Variables

SUMMARY: List the variables.

When applicable, identify each as independent or predictor versus dependent or criterion.

Indicate whether each is measured as nominal, ordinal, or interval.

List any control variables.

CRITIQUE: Have any important variables been ignored? If so, what are they?

How might they have changed the findings?

Definition(s)

SUMMARY: Which terms are defined?

Are the definitions constitutive, operational, or exemplificatory (by example)?

CRITIQUE: Are all needed definitions provided? If not, what is missing?

Are definitions clear and appropriate to the study? If not, what should the author(s) have done differently?

#4—Research Design

SUMMARY: What is the category or type of research? . . . the design?

CRITIQUE: Is the research design clearly described? . . . feasible? . . . best choice for the study?

If not, what improvements can you suggest?

Validity Issues in Design

SUMMARY: What does the author say regarding issues of internal and external validity?

(Reminder: Design validity is NOT the same as instrument validity.)

CRITIQUE: Do the identified validity issues seriously weaken the study? What design validity issues has the author apparently overlooked? If they weaken the study, explain how.

How might any validity issues have been avoided or minimized?

Setting

SUMMARY: What is the setting—time and place?

CRITIQUE: To what extent does the setting limit—or facilitate—generalizing to other settings?

Can a reader determine whether or not his or her setting is similar to that of the study? If not, what information is missing?

#5—Population

and Sample

SUMMARY: Who or what is the target population (the group to which results would be generalized)?

Describe the sampling frame (if any) and sample.

Identify the sampling procedure. How many groups were formed or identified? On what basis?

CRITIQUE: Is the sample appropriate to the study? . . . adequate in size and characteristics?

If not, how might it have been improved?

What characteristics show that the sample is or is not representative of the intended population?

If there are problems, how might they have been better addressed or compensated for?

#6--Instrumentation

SUMMARY: Describe the instrument(s).

Were they pre-existing or specially created?

CRITIQUE: Map the instrumentation against the research questions.

Is the instrumentation appropriate for the study? Identify any problems.

Validity and reliability of Instrument(s)

SUMMARY: What type of instrument validity was sought and how was that established?

How was instrument reliability established?

CRITIQUE: What problems, if any, do you see with instrument reliability or validity?

How might they have been addressed?

Ethics

SUMMARY: Does the author discuss ethical issues? If so, how have they been addressed?

CRITIQUE: Do you see any ethical problems in the study? Describe.

Have any ethical issues been appropriately addressed? If so, how?

Are there any ethical issues that are not properly taken care of? What might have been done?

Data-Gathering

SUMMARY: What procedures were used to gather data and in what sequence?

CRITIQUE: Are the data-gathering procedures clearly described? . . . appropriate to the study? . . .correctly implemented? …replicable?

Was there adequate participation? Explain your answers.

#7—Analysis

SUMMARY: What procedures were followed?

What statistics were used?

CRITIQUE: Are the procedures and statistics suitable to the research questions and data? . . . appropriately executed? . . .effectively reported?

On what basis can the reader place confidence in the analysis? …lack confidence?

#8—Conclusions/

Findings

SUMMARY: What conclusions and findings does the author report?

CRITIQUE: Are the conclusions and findings clearly stated?

Do they answer the research question(s)?

Are they justified from the evidence? Does the author indicate the direction of the findings?

Support your answers.

Implications and Recommendations

SUMMARY: What are the implications and recommendations for theory, further research, and practice?

CRITIQUE: Are the implications and recommendations clearly stated? . . . justified from the findings? . . . important? . . . related to broader issues?

Your Evaluation

Overall, what do you think of the study? How well was it carried out? How important and useful are the findings? Other comments? Are you persuaded to follow the author's recommendations? Why or why not?