Critical Appraisal

MSN5300 Faculty: Dr. K. Richards

of a Critical Appraisal for Quantitative Research

a Critical Appraisal for • Authors • Title • Abstract • Problem Statement • Literature Review • Conceptual Framework • Purpose/Aims/Objectives/

Research Question, Hypothesis • Protection of Human Subjects • Definitions (Conceptual and

Operational)

• Methods: • Sample • Study Design • Study Procedures (Data Collection) • Instruments

• Data Analysis and Results • Discussion/Conclusions/Recommendations

• Limitations • Implications for Practice

• References

of a Critical Appraisal for Quantitative Research

Authors: (Are the researchers qualified to conduct this research?) - qualifications, job title

Title: (Does the title fit the research design?) - describes research - specific - succinct

Abstract: (Does the abstract succinctly and adequately summarize the research?) - structured in sections: Ex., introduction, objective(s), methods, results, and conclusions - generally 150-250 words

Problem Statement: (What is the rationale for the study?) • should include:

• independent and dependent variables • population of interest • key study concepts

• may be in form of either research question or purpose statement • should fill a gap in knowledge/theory or identify a specific single, relevant

nursing issue • ‘problems’ with problem statements:

• too broadly stated or overly generalized • research questions that can’t be answered by proposed methods

Literature Review: (What is the state of the science on the research topic?)

• primary purpose: to develop the research question • systematic, critical review of relevant literature that:

• highlights weaknesses in previous studies • identifies previously studied concepts • relates the current research project to historical research • identifies knowledge deficit • states importance of further study

• Theoretical frameworks • narrower in scope • can be tested directly

• Conceptual frameworks • express assumptions • can’t be tested directly • explains relationships among variables

• allows better understanding of relationships between major concepts • more fully explicates relationships between variables

Frameworks: (What is the relationship among the concepts?)

Purpose/Aims/Objectives/Research Question/Hypothesis: (Is there congruency between research purpose and research question?)

• demonstrates a link b/w research problem and how study will be undertaken • should be congruent with data in lit review

Protection of Human Subjects: (Did the study receive IRB approval?) • primary purpose: protection of human subjects • statement of approval; statement of informed consent • describes:

• overview of the study • potential safety risks • conflicts of interest (if any) • informed consent

Definitions:

Conceptual Definitions: (Are concepts well-defined?) • to ensure concepts are understood • levels of measurement of variables (NOIR)

Operational Definitions: (How will concepts be measured?) • to ensure measurement of concepts is understood

Methods:

Methods - Sample: (Does the sample represent the population of interest?) • description of subjects; recruitment strategies; inclusion/exclusion criteria • random sampling produces strongest levels of evidence • should reflect population (= representativeness) • should include sampling error • power analysis to determine sample size

Methods - Design: (Is the study design appropriate to answer the research question?)

• enough detail to replicate • ranked according to level of evidence • Experimental is most commonly used and most rigorous (“gold standard”)

• researcher has most control: • selection of study subjects • introduction of the independent variable • random assignment to treatment and control groups

Methods – Design (…con’t…): (Is the study design appropriate to answer the research question?)

• Quasi-experimental • lacks one of the following:

• control group • intervention • random assignment

• Non-Experimental • lacks control of independent variable(s)

Methods – Procedures/Data Collection: (How were data collected?) • procedures for data collection (how, where, who, time frame) • training for data collectors

Methods – Instruments: (How were data measured?) • match b/w instrument and conceptual definition • reliability, validity

Data Analysis: (Were data analyzed using appropriate statistics?) • description of how data were handled • statistical tests used and appropriateness of fit r/t assumptions and levels of

measurement of variables • both descriptive and inferential statistics should be used

• descriptive measures: • central tendency (mean, median, mode) • dispersion (range, variance, standard deviation)

• inferential statistics: • parametric • non-parametric analog

• p value; α level; interpretation of statistical significance

Results: (Do study findings answer the study question?)

• statements of findings in data critical to answering study question • should state clearly whether the data analysis supports (accept), or fails to

support (reject), the research hypothesis • tables and/or graphs • consistency with description of statistical tests in Methods section

Discussion/Conclusions/Recommendations: (What are the researcher’s conclusions from the study?) • answers research question • uses statistical results to draw conclusions regarding research question • review of current knowledge; explains how current study findings add to body of

knowledge • clinical significance • indicates what research questions should be answered next

Study Limitations: (Are study weaknesses addressed?) • should be an assessment of factors r/t design, sample, sampling, and analysis re.

external validity (generalizability)

Implications for Practice: (What is the clinical interpretation of study findings?) • evidence for clinical practice • should be suggested with associated caution r/t study limitations

References: (Are references comprehensive and current?) • should conclude w/ accurate list of all sources used • < 5 years

Rigor (Is the study valid and reliable?):

2 types of Validity: 1. internal validity: Does the study use research methods and measures

that allow legitimate inferences to be made from the results? 2. external validity: Given the methodology used, is it reasonable to

generalize study results to other populations and/or settings?

Reliability: Is the study reproducible?

References  Coughlan, M., Cronin, P. & Ryan, F. (2007). Step-by-step guide to critiquing

research Part 1: Quantitative research. British Journal of Nursing, 16(11), 658-663.

 Fineout-Overholt, E., Melnyk, B.M., Stillwell, S.B., & Williamson, K.M. (2010). Critical Appraisal of the Evidence Part I: An introduction to gathering, evaluating, and recording the evidence. American Journal of Nursing, 110(7), 47-52.

Gray, J.R., Grove, S.K., & Sutherland, S. (2017). Burns and Grove's The practice of nursing research: Appraisal, synthesis, and generation of evidence (8th Ed.). Elsevier: St. Louis, MI.

 Hudson-Barr D. (2005). From research idea to research study: The how. Journal of the Specialty of Pediatric Nursing, 10(3), 147-150.