Systematic Review Study

The Problem

· Did the report clearly state the research problem and/or research questions? Is the scope of the project appropriate?

· Is the topic of the review important for nursing?

· Were concepts, variables, or phenomena adequately defined?

· Was the integration approach adequately described, and was the approach appropriate?

Search Strategy

· Did the report clearly describe criteria for selecting primary studies, and are those criteria reasonable?

· Were the bibliographic databases used by the reviewers identified, and are they appropriate and comprehensive? Were key words identified, and are they exhaustive?

· Did the reviewers use adequate supplementary efforts to identify relevant studies?

· Was a PRISMA-type flowchart included to summarize the search strategy and results?

The Sample

· Were inclusion and exclusion criteria clearly articulated, and were they defensible?

· Did the search strategy yield a strong and comprehensive sample of studies? Were strengths and limitations of the sample identified?

· If an original report was lacking key information, did reviewers attempt to contact the original researchers for additional information—or did the study have to be excluded?

· If studies were excluded for reasons other than insufficient information, did the reviewers provide a rationale for the decision?

Quality Appraisal

· Did the reviewers appraise the quality of the primary studies? Did they use a defensible and well-defined set of criteria, or a respected quality appraisal scale?

· Did two or more people do the appraisals, and was interrater agreement reported?

· Was the appraisal information used in a well-defined and defensible manner in the selection of studies, or in the analysis of results?

Data Extraction

· Was adequate information extracted about methodologic and administrative aspects of the study? Was adequate information about sample characteristics extracted?

· Was sufficient information extracted about study findings?

· Were steps taken to enhance the integrity of the dataset (e.g., were two or more people used to extract and record information for analysis)?

Data Analysis—General

· Did the reviewers explain their method of pooling and integrating the data?

· Was the analysis of data thorough and credible?

· Were tables, figures, and text used effectively to summarize findings?

Data Analysis—Quantitative

If a meta-analysis was not performed, was there adequate justification for using a narrative integration method? If a meta-analysis was performed, was this justifiable?

For meta-analyses, were appropriate procedures followed for computing effect size estimates for all relevant outcomes?

Was heterogeneity of effects adequately dealt with? Was the decision to use a random effects model or a fixed effects model sound? Were appropriate subgroup analyses undertaken—or was the absence of subgroup analyses justified?

Was the issue of publication bias adequately addressed?

Data Analysis—Qualitative

In a metasynthesis, did the reviewers describe the techniques they used to compare the findings of each study, and did they explain their method of interpreting their data?

If a metasummary was undertaken, did the abstracted findings seem appropriate and convincing? Were appropriate methods used to compute effect sizes? Was information presented effectively?

In a metasynthesis, did the synthesis achieve a fuller understanding of the phenomenon to advance knowledge? Do the interpretations seem well grounded? Was there a sufficient amount of data included to support the interpretations?

Conclusions

Did the reviewers draw reasonable conclusions about the quality, quantity, and consistency of evidence relating to the research question?

Were limitations of the review/synthesis noted?

Were implications for nursing practice and further research clearly stated?