article critique
© 2000 by Lillian Biermann Wehmeyer & Beverly Hardcastle Stanford
CRITIQUING A QUALITATIVE RESEARCH STUDY
Adapted from Gall, Borg, & Gall and Beverly Hardcastle Stanford, Ph.D.
As researchers, we cannot avoid orienting our studies with our beliefs about what knowledge is. As a consequence, all research is biased. What is important is that the bias or perspective be recognized and incorporated into our evaluations. We need also to be aware of how our theoretical biases are influencing how we evaluate others.
Gall, Borg & Gall appear to subscribe to the positivist orientation toward research. As a consequence, some of their evaluation questions (pp. 609-612) are biased in that direction. While there are positivist qualitative researchers, they are in the minority. Most who engage in qualitative research are interpretive or critical theorists. I am a researcher who subscribes to the latter two perspectives.—Dr. Stanford
Gathering resources on a topic is challenging enough, yet it is only the beginning. One must ask whether the research was well designed and properly executed. Then one must consider whether the reported conclusions are supported by the study. The form on the following pages has been designed for that purpose.
Suggested procedure:
(1) Read the research article. Then complete the NUMBERED items in the SUMMARY column. This gives an overview of the entire study.
Note: If the study is complex, involving several research questions, you may find it easier to complete a separate form for each question, or for each group of closely related questions.
(2) Next, complete the rest of the SUMMARY column. Keep in mind that not all boxes will be applicable to all studies.
(3) Now fill in the CRITIQUE column. Do not just answer "yes" or "no," but explain and justify your opinion.
(4) Next, review the critique column; then fill in the EVALUATION box at the end of the form.
(5) Finally, write a brief SUMMARY, as described.
QUALITATIVE CRITIQUE FORM
Your Names _. _______
List the article in APA format
|
RESEARCH STEP |
QUESTIONS |
SUMMARY |
CRITIQUE |
|
Researcher(s) |
SUMMARY: Are there multiple researchers, or only one? What information do the researchers present regarding their background and potential biases? CRITIQUE: How might the researchers' background influence their interpretations of the data? Does that seem to be the case? |
|
|
|
#1--Purpose/ Research Question(s) |
SUMMARY: What is the purpose or research question(s)? CRITIQUE: Are the researchers trying to prove something they already believe? How do you know? Do the researchers demonstrate undue positive or negative bias in describing the subject of the study? If so, In what way? |
|
|
|
Literature Review/ Theoretical Framework |
SUMMARY: What previous work had been done leading up to this study? Do the authors present a theoretical orientation for their study? If so, how do they connect it to the study? CRITIQUE: How comprehensive is the literature review section of the report? Justify your answer. Does it include studies that you know to be relevant to the problem? If not, what has been overlooked? |
|
|
|
Approach |
SUMMARY: Is the study exploratory or confirmatory? How do you know? CRITIQUE: Is the approach appropriate to the purpose or research question(s)? Why or why not? If not, what approach would you recommend and why? |
|
|
|
#2--Methodology |
SUMMARY: Which of the types of qualitative research is being used (grounded theory or inquiry, historical, case study, etc.)? CRITIQUE: To what extent is the methodology appropriate to the purpose or research question(s)? Explain. If inappropriate, what might have been better? |
|
|
|
#3--Participants |
SUMMARY: Who are being studied? How were they selected? CRITIQUE: Did the selection procedure result in a case or cases that were particularly interesting? What would you expect the researcher to learn from them about the phenomena of interest? Why do you think so? What improvements would you suggest? |
|
|
|
Context |
SUMMARY: What is the context or setting of the study? CRITIQUE: What information enables the reader to transfer the findings to other settings? Is it adequate? Can a reader determine whether or not his or her context is similar to that of the study? If not, what information is missing? |
|
|
|
#4—Data Sources |
SUMMARY: What sources were used to gather data (interviews, questionnaires, artifacts, documents, etc.)? CRITIQUE: Were the data sources appropriate? What might have been more useful? Evaluate the adequacy of the instruments in relation to the research questions. |
|
|
|
Trustworthiness of Data Sources |
SUMMARY: What evidence is presented to demonstrate the trustworthiness of each data source? Were the data sources triangulated, that is, with multiple sources of data for the same phenomenon? If so, how? CRITIQUE: Are you convinced that the data collected match the data needed to respond to the purpose or research question(s)? …that the data were honest and accurate? To what extent are you convinced—or do you doubt—the trustworthiness of the data? Explain. |
|
|
|
Ethics |
SUMMARY: Does the author discuss ethical issues? If so, how have they been addressed? CRITIQUE: Do you see any ethical problems in the study? Have ethical issues been appropriately addressed? If so, how? Are any ethical issues not properly taken care of? What might have been done to address these concerns? |
|
|
|
Data Collection |
SUMMARY: What procedures were used to gather data and in what sequence? What was the intensity of data collection? For instance, over what period of time were observations made? …documents collected? etc. CRITIQUE: How do the authors demonstrate that their data collection procedures were unbiased and complete? …of sufficient intensity, e.g., continuous rather than fragmented? What additional information about procedures might be needed so that others could replicate the study? |
|
|
|
#5--Analysis |
SUMMARY: How were the data analyzed? What evidence is there of triangulation (for example, multiple persons working independently as they analyze data and then comparing results, multiple theories, multiple methods, or perspectives of multiple disciplines)? CRITIQUE: How clearly is the process of analysis described? Why do you say so? Did the report include a "thick" description (or, in a short article, quotations or other evidence of rich data)? To what extent is the analysis trustworthy? Why or why not? |
|
|
|
Themes/Patterns |
SUMMARY: Did any themes, patterns, or constructs emerge from the data? If so, what? CRITIQUE: Consider the categories or patterns and the themes or constructs that emerged? How well do they relate to the data? How plausible are they? Are you convinced? Why or why not? |
|
|
|
#6--Findings |
SUMMARY: What did the researchers learn? …or what insights did they gain? CRITIQUE: Consider the findings, hypotheses, or further questions in logical relationship to the data and analysis. Evaluate the match. Were alternative explanations considered? …needed? If so, what? |
|
|
|
Conclusions: Implications and Recommendations |
SUMMARY: What do the authors indicate as the significance of the study? What implications for practice are suggested? What hypotheses or questions are recommended for further research? CRITIQUE: How logically were the conclusions derived from the findings? Were statements of generalizability, if made, appropriately qualified? If the study was theory-based, do the conclusions relate to the theory? Is a reader motivated to take action? Explain your response. |
|
|
|
Reference List and Appendixes |
SUMMARY: Do the authors provide a list of references? …in APA style? Did they choose to provide a resource list for further study? Appendixes? CRITIQUE: If the format is not APA, is it at least consistent? Are all cited references provided? Did the authors look for an appropriate range of information sources as to date and format (books, journals, government documents, etc.)? Provide all necessary detail? |
|
|
|
EVALUATION Overall, what do you think of the study? How well was it carried out? How important and useful are the findings? Other comments? Are you persuaded to follow the recommendations? Why or why not?
|
Write a SUMMARY, probably one to two paragraphs in length, to summarize the article as you might if the study appeared in your literature review for an article, grant application, or dissertation. Be sure to tell the reader who did what, why, where and when, and with whom. Mention the theoretical base (if any), and point out notable strengths and problems (if any).