Critical Appraisal Checklist and Worksheet

Critical Appraisal Worksheet for Group Project 1, Part b

Elements of Appraisal

Discussion

Study Design

Was the study a qualitative or quantitative design? Explain.

Practice Problem

State the practice problem/issue that is the focus of the study.

How does this practice problem/issue affect your nursing practice?

Study Purpose

State the purpose of the study.

Discuss whether this study was feasible to conduct in terms of money commitment; the researchers' expertise; availability of subjects, facilities, and equipment; and ethical considerations.

Review of Literature

Was the literature review organized to show the progressive development of evidence from previous research?

Was a clear, concise summary presented of the current empirical and theoretical knowledge in the area of the study?

Did the literature review summary identify what was known and not known about the research problem? Did it logically direct the research purpose?

Theoretical Framework

Is a conceptual model or theoretical framework used? If so, is it presented with clarity? Does it adequately explain the phenomenon of concern?

Is the framework linked to the research purpose? If not, would another framework fit more logically with the study?

If a proposition or relationship from a theory is to be tested, is the proposition clearly identified and linked to the study hypothesis?

Research Question(s) and Hpothesis(es)

What is the research question? Is it clearly stated?

Does the research question match the purpose of the study?

What is the research hypothesis? Is it clearly stated?

Does the hypothesis match the purpose of the study?

Formulate a null hypothesis for this study.

Were the objectives, questions, or hypotheses logically linked to the concepts and relationships (propositions) in the framework? Explain.

Variables

List all research variables with corresponding levels of measurement on the NOIR scale.

Do variables represent the concepts identified in the framework?

How is each study variable defined (both conceptually and operationally)?

Are conceptual definitions of variables consistent with operational definitions?

Study Design

What was the design was used in the study? Was it the most appropriate design to answer the study question? Explain.

Was the design logically linked to the sampling method and statistical analyses?

Did the design provide a means to examine all objectives, questions, or hypotheses?

Was there an intervention (treatment) in the study? If so:

· was the intervention clearly described?

· was the treatment appropriate for examining the study purpose and hypotheses?

· did the study framework explain the links between the treatment (independent variable) and the proposed outcomes (dependent variables)?

· were subjects randomly assigned to the treatment group, or were the treatment and comparison groups dependent?

· were the treatment and comparison group assignments appropriate for the purpose of the study?

· was a protocol developed to promote consistent implementation of the treatment to ensure intervention fidelity?

· did the researcher monitor implementation of the treatment to ensure consistency? How might this impact findings?

Study Sample

Were any subjects excluded from the study because of age, socioeconomic status, or ethnicity? If so, explain. Was sound rationale provided?

Did the sample include an understudied or vulnerable population, such as young, elderly, pregnant, or minority subjects?

What sampling criteria (inclusion and exclusion) were used? Were sampling criteria appropriate for the type of study conducted? Describe.

Were the rights of human subjects protected? Explain.

What was the refusal rate for the study? If greater than 20%:

· how might this have affected the representativeness of the sample?

· did the researchers provide rationale for the refusals?

Was the attrition rate for the study provided? If so:

- what was it?

- did the researchers provide a rationale for the attrition of study participants?

- how did attrition influence the final sample as well as study results and findings?

How large is the sample?

How was sample size determined?

Research Instruments

Did the measurement methods selected for the study adequately measure the study variables?

What instruments or tools were used to collect data?

How did you determine if the instruments are valid? Reliable?

Respond to the following questions, which are relevant to the measurement approaches used in the study:

1. Scales and questionnaires

(a) Were the instruments clearly described?

(b) Were techniques for completion and scoring of the instruments provided?

(c) Were validity and reliability of the instruments described?

(d) Did the researcher reexamine the validity and reliability of instruments for the present sample?

(e) If an instrument was developed for the study, was the instrument development process described?

2. Observation

(a) Were phenomenon that were to be observed clearly identified and defined?

(b) Was interrater reliability described?

(c) Were techniques for recording observations described?

3. Interviews

(a) Did interview questions address concerns expressed in the research problem?

(b) Were interview questions relevant for the research purpose and objectives, questions, or hypotheses?

(c) Did the design of the questions tend to bias subjects' responses?

(d) Did the sequence of questions tend to bias subjects' responses?

4. Physiological measures

(a) Were physiological measures clearly described? If appropriate, are brand names (ex., Hewlett-Packard) of instruments identified?

(b) Were accuracy, precision, and error of physiological instruments discussed?

(c) Were physiological measures appropriate for the research purpose and objectives, questions, or hypotheses?

(d) Were methods for recording data from physiological measures clearly described? Was recording of data consistent?

Data Collection

State the data collection procedures.

How often were data collected and for how long?

Was the data collection process conducted in a consistent manner?

Did the data collected address the research objectives, questions, or hypotheses?

Did any adverse events occur during data collection? If adverse events did occur, how were they managed?

Data Analysis

Were data analysis procedures clearly described?

What statistical tests were used to analyze data?

Were statistical tests suitable to the types of data collected/levels of measurement? Explain.

What assumptions in the data must be met for the type of statistical tests used? How do you know if these assumptions were met?

Were data analysis procedures clearly described? Did the researcher address any problems with missing data and if so, how was this problem managed?

Were data analysis techniques consistent with the study purpose research objectives, questions, and/or hypotheses? Explain.

Were data logically organized/presented in tables, graphs and/or charts? Describe.

What was the alpha for each statistical test?

Describe how statistical significance was demonstrated (or not) for each variable.

Study Results and Interpretation

Discuss study results. What were the findings?

What was the statistical interpretation of study results? Was this interpretation correct?

What was the statistical interpretation of study results? Was this interpretation correct?

Were findings discussed in relation to each objective, question, or hypothesis? In other words, was the research question/hypothesis answered? Describe.

Were the findings linked to the study framework?

Were study findings an accurate reflection of reality and valid for use in clinical practice?

Did the conclusions fit the results from the data analyses? Were the conclusions based on statistically significant and clinically important results? Explain.

Were study limitations described? Discuss. Did the study have weaknesses not identified by the researcher?

Did the researcher generalize the findings appropriately? Who will benefit from results of the study? Discuss.

Were there any unexpected findings? Discuss.

Recommendations for Future Research

Discuss study recommendations.

Is need for further research identified?

Implications for Nursing Practice

Do study findings have clinical significance? If so, discuss implications of the study for nursing practice.

What changes could you make in your practice based on the results of this study?

Study Rigor

Discuss elements of rigor based on study design. In your opinion, did the researchers address these aspects adequately in the published report? Why or why not?

How does rigor impact study results?