Critical Appraisal Checklist and Worksheet

principles and practice

The Research Critique General Criteria for Evaluating a Research Report CHERYL TATANO BECK, CNM, DNSc

General criteria for evaluating a research report are addressed. This outline of criteria can be used as a guide for nurses in critiquing research studies. A sample research report is summarized followed by a critique of the study. Readers have an opportunity to practice critiquing by doing their own analyses before reading the critique presented in the article.

A research critique is a critical appraisal of a research study that has been systematically reviewed based on some known criteria. The focus is on judging the inves- tigation’s scientific merit and worth so that a decision can be made as to whether or not the re- sults of the study should be incor- porated into clinical practice. An appraisal of t h e research’s strengths and limitations is made. The critic should not concentrate onIy on the flaws in the investiga- tion.

Given the constraints of con- ducting clinical research, the cri- tic needs to assess what the inves- tigator attempted to d o and to judge the strategies chosen. Iden- tification of t h e strengths of a study is necessary for generating scientific knowledge and imple- menting the findings into clinical practice. To critique a research study systematically, each section of the research report must be an- alyzed. Critiquing involves identi- fying answers to questions for

Accepted: March 1989

each component of the research report.

One of t h e purposes of a re- search critique is to provide feedback to help researchers strengthen not only the studies being critiqued but also future re- search studies being planned on the same topics. A second purpose of a research critique is to aid re- search consumers in making deci- sions on how to use the study find- ings. For example, is the study in- teresting but its research design not sound enough to warrant ap- plication of the findings to clinical practice?

This second purpose of a re- search critique is of vital impor- tance to all nurses, not just nurse researchers. One of the roles of all professional nurses is to be knowl- edgeable consumers of nursing research. Professional nurses must develop t h e habit of criti- cally reading nursing research journals and research studies published in their specialty jour- nals. Only through applying valid research findings in practice can the nursing profession advance.

In this article excerpts from a published research report are first summarized. This exploratory study is a replication of earlier re- search on women’s responses to fetal monitoring during labor. A criteria outline for the evaluation of research reports is presented (Table 1) as a guide to critique re- search.’-3 These suggested ques- tions are not a complete or ex- haustive list. Readers are given the opportunity to practice apply- ing the critiquing criteria to this nursing research on women’s re- sponses to fetal monitoring. For each section of the critique, the reader should do his or her own analysis before reading the study critique presented in the final seg- ment of this article.

SAMPLE RESEARCH REPORT

R e s e a r c h Problem

Title: Patient Acceptance o f Fetal Monitoring as a Helpful Tool4

In 1972, a study was conducted to investigate women’s cognitive

18 1 9 1 January/February 1990 JOGNN

Table 1. General Criteria For Evaluating a Research Report

Step 1. Research Problem 1. Is the problem clearly and concisely stated? 2. Is the problem adequately narrowed down into a researchable problem? 3. Is the problem significant to nursing? 4. Is the relationship of the identified problem to previous research clear?

1. Is the literature review logically organized? 2. Does the review provide a critique of the relevant studies? 3. Are the gaps in knowledge about the research problem identified? 4. Are important relevant references omitted?

1. Is the theoretical framework easily linked with the problem, or does it seem forced? 2. If a conceptual framework is used, are the concepts adequately defined and are the

Step 2. Literature Review

Step 3. Theoretical or Conceptual Framework

relationships among these concepts clearly identified? Step 4. Research Variables

1. Are the independent and dependent variables operationally defined? 2. Are any extraneous or intervening variables identified?

1. Is a predicted relationship between two or more variables included in each hypothesis? 2. Are the hypotheses clear, testable, and specific? 3. Do the hypotheses logically flow from the theoretical or conceptual framework?

1. Is the sample size adequate? 2. Is the sample representative of the defined population? 3. Is the method for selection of the sample appropriate? 4. Are the sample criteria for inclusion into the study identified? 5. Is there any sampling bias in the chosen method?

1. Is the research design adequately described? 2. Is the design appropriate for the research problem? 3. Does the research design control for threats in internal and external validity of the study? 4. Are the data collection instruments described adequately? 5. Are the reliability and validity of the measurement tools adequate?

1. Are the data collection methods appropriate for study? 2. Are the data collection instruments described adequately? 3. Are the reliability and validity of the measurement tools adequate?

1. Is the results section clearly and logically organized? 2. Is the type of analysis appropriate for the level of measurement for each variable? 3. Are the tables and figures clear and understandable? 4. Is the statistical test the correct one for answering the research question?

1. Are the interpretations based on the data obtained? 2. Does the investigator clearly distinguish between actual findings and interpretations? 3. Are the findings discussed in relation to previous research and to the conceptual/

theoretical framework? 4. Are unwarranted generalizations made beyond the study sample? 5. Are the limitations of the results identified? 6. Are implications of the results for clinical nursing practice discussed? 7. Are recommendations for future research identified? 8. Are the conclusions iustified?

Step 5. Hypotheses

Step 6. Sampling

Step 7. Research Design

Step 8. Data Collection Methods

Step 9. Data Analysis

Step 10. Interpretation and Discussion of the Findings

and emotional reactions t o fetal sidered routine during labor. After monitoring in labor.5 Thirty-one of t h e five-year period, t h e re- 50 women in labor had negative s e a r c h e r replicated t h e original initial responses t o fetal monitor- study to determine to what extent ing. Five y e a r s after t h e initial women’s responses to fetal mon- study, fetal monitoring was con- i t o r i n g h a d c h a n g e d . O n c e

One of the roles of all professional nurses is to be knowledgeable consumers I of nursing research.

women’s reactions to fetal moni- toring are known, nurses can tailor nursing interventions specifically t o these responses.

Literature Review

The initial study was conducted in an 800-bed university teaching hospital in New England.5 Fifty women on a postpartum unit were interviewed sometime during their first three days after deliv- ery. The researcher intended t o gain information that would de- s c r i b e women’s cognitive a n d emotional responses t o fetal mon- itoring during labor. The reactions of the 50 women to fetal monitor- ing were categorized into initial responses a n d later subsequent responses after t h e women had been attached to the monitors for periods of time. Thirty-one women had negative initial responses, and 1 9 women had neutral initial responses. None of the 50 women in t h e initial study had a positive initial response. Six mothers had negative subsequent responses, 13 had neutral subsequent re- sponses, and 3 1 had positive sub- sequent responses.

In a 1976 study, 7 1 women com- pleted questionnaires within one month before t h e i r d e l i v e r i e s 6 T h e questionnaires concerned knowledge of fetal monitoring and attitudes. Thirty-one of t h e 7 1 women were monitored during labor a n d interviewed during postpartum hospitalization. In comparing predelivery and post- delivery attitudes, the researchers found t h a t 2 6 women exhibited positive attitudes postdelivery, while only 12 women had reported

19:l JanuaryIFebruary 1990 JOCNN 19

positive predelivery attitudes. In a 1977 study, 10 women who had no prior fetal loss a n d n o c u r r e n t labor difficulties a n d 25 women who had experienced fetal l o s s and/or current labor difficulties were interviewed during the post- partum p e r i ~ d . ~ Among t h e 25 women who had experienced complications, 14 indicated posi- tive attitudes, 10 indicated nega- tive attitudes, and 1 indicated a neutral attitude. Of t h e 14 with positive attitudes, 9 had suffered prior fetal loss. In the 10 mothers who had not experienced compli- cations, 2 indicated positive atti- tudes and 8 indicated negative at- titudes.

Research Questions

1) During labor, what a r e women’s cognitive a n d emo- tional responses to fetal moni- toring when they are initially attached t o t h e monitor and, subsequently, after they have been on the monitor for a pe- riod of time?

2) After a five-year period, have women’s cognitive a n d emo- tional responses to fetal moni- toring during labor changed?

3) Do any significant associations exist between any of the demo- graphic variables studied and women’s initial a n d s u b s e - quent responses to fetal moni- toring?

Definition of Terms

1) Initial response-a woman’s recollection of her immediate reaction to the thought of fetal monitoring a n d t o t h e moni- toring equipment when first told that s h e a n d h e r fetus would be attached to the fetal monitoring machine.

2 ) S u b s e q u e n t r e s p o n s e - a woman’s total reaction to fetal monitoring as the woman re- calls her entire fetal monitor-

A research critique is a critical appraisal of a study that has been systematically reviewed based on some known I criteria. ing experience while attached to a monitor.

3) Positive reaction-a favorable, helpful, a n d desirable re- sponse to fetal monitoring.

4) Negative reaction-an unfavor- able, upsetting, and distressing response t o fetal monitoring.

5 ) Neutral reaction-a response that cannot be categorized as being favorable o r unfavorable toward fetal monitoring.

Sampling

The convenience sample con- sisted of 50 postpartum women who had delivered a t a 350-bed, university-affiliated community hospital in Baltimore, Maryland. Respondents’ ages ranged from 14 t o 37 years. Thirty mothers were white, and 20 were black. Thirty- two women were married, while 18 w e r e single. Twenty-seven mothers were clinic patients, and 23 were private patients. T h e length of time women were on the fetal monitor ranged from 30 min- utes to 16 hours, with a mean time of 4 hours and 42 minutes.

Research Design and Data Collection Procedures

This study is a replication of the original research d o n e in 1972. Fifty women w e r e interviewed sometime during their first three postpartum days. As in the 1972 study, t h e s e 50 women’s re- sponses were divided into initial and subsequent responses. Each patient’s initial a n d subsequent responses to fetal monitoring were categorized further into positive,

negative, o r neutral reactions. Re- liability of t h e categories was checked by having another faculty member independently categorize the 50 mothers’ initial and subse- quent responses. The percentage agreement with t h e researcher’s categorization was 95%.

Data Analysis

The most striking difference the investigator noticed in comparing the results of her two studies was in women’s initial responses t o fetal monitoring. In 1972, not one woman interviewed had a positive initial response, while 31 women had negative initial responses. Five years later, 11 women had positive initial responses t o fetal monitoring and only 11 had nega- tive initial responses. In regard to s u b s e q u e n t r e s p o n s e s t o fetal monitoring, 37 mothers had posi- tive reactions, 9 mothers had neu- tral reactions, and only 4 mothers had negative reactions. When comparing t h e subsequent re- sponses identified in this current study with those of t h e original study, 6 more women had positive responses, 2 fewer had negative responses, and 4 fewer had neu- tral responses in 1977.

Chi-square tests were done to test for associations between any of t h e demographic variables studied and women’s initial and s u b s e q u e n t r e s p o n s e s t o fetal monitoring. Two of these associa- tions were statistically significant a t more than the -01 level. A chi- square t e s t of 12.31 (df 2) indi- cated that a significant association between age and initial responses existed. Older mothers initially reacted more positively to fetal monitoring than younger mothers. The second significant association found was between marital status and initial responses (x’ = 9.97, df 2). Married women initially re- sponded more positively to fetal monitoring than single women.

20 19:l January/February 1990 JOGNN

Discussion

The data collected in this study support t h e conclusion that cur- rently initial fetal monitoring is a less upsetting experience for women in labor than such moni- toring was in 1972. The most fre- quent reason given by the women for their positive reactions was that they were familiar with t h e monitor before coming into t h e hospital in labor. Nurses who come in contact with pregnant women, either through work in ob- stetrical clinics, private offices, o r prenatal education classes, should discuss fetal monitoring with their patients and show pictures of the monitor t o h e l p familiarize pa- tients with the equipment.

One of the purposes of a research critique is to help decide whether or not the results of a study should be incorporated into clinical I practice. More than half of the mothers in

the study had neutral initial reac- tions to fetal monitoring. These 28 women appeared t o begin their initial experiences with fetal mon- itoring in an open, nonjudgmental manner. This finding suggests that, in labor, a woman’s initial ex- perience with fetal monitoring may be a crucial period affecting how s h e may react t o t h e re- mainder of her monitoring experi- ence. Consequently, labor nurses must u s e a positive a p p r o a c h when introducing fetal monitoring to women.

Two significant relationships identified in this study have im- portant nursing implications. When caring for young and single women who are about t o experi- ence fetal monitoring, labor and delivery n u r s e s need t o make greater efforts t o give t h e s e women support.

Study Limitations

The findings of this study can- not be generalized t o cover t h e entire maternity population be- c a u s e t h e sample was not ran- domly selected.

CRITIQUE OF THE SAMPLE RESEARCH REPORT

Research Problem

The research problem is clearly stated. The investigator sought to compare women’s responses t o fetal monitoring in 1972, when fetal monitoring was still new, and five years later in 1977, when fetal monitoring was considered a rou- tine p a r t of labor. Identifying w h e t h e r o r not women’s emo- tional and cognitive responses to fetal monitoring had changed is relevant a n d useful t o nursing. With t h i s information, labor nurses will be able to adapt their nursing intervention techniques.

Literature Review

The references are pertinent to t h e topic u n d e r investigation. When this study was published, the references were current. One weakness in this section, however, is the lack of any critique of the research studies cited. The inves- tigator only summarized the find- ings but did not critique the stud- ies’ designs for any limitations.

Theoretical Framework

The researcher did not identify a theoretical o r conceptual frame- work on which the study had been based.

Research Variables

Independent o r dependent vari- ables were not identified, which was appropriate since t h e study was exploratory. Theoretical defi- nitions were specifically provided for the variables of initial, subse-

To critique a research study systematically, each section of the research

1 report must be analyzed.

quent, positive, negative, and neu- tral responses.

Research Questions

Because the study was descrip- tive, research questions instead of hypotheses were appropriate. The t h r e e research questions were clear and specific.

Sampling

Because t h e research sample was a convenience sample, t h e findings have limited generaliza- bility. N o criteria for sample eligi- bility were identified. In addition, t h e r e s e a r c h e r did not discuss how the subjects’ rights had been protected.

Research Designs and Data Collection Procedures

All of the subjects were inter- viewed within three days after de- liveries, which helped prevent problems of recall ability. The in- vestigator did not, however, pro- vide readers with a description of the interview schedule, the num- ber of researchers who did the in- terviewing, or the validity of the interview schedule. The re- searcher addressed the reliability of t h e coding categories, which was determined a s 95%. Test-re- t e s t reliability of t h e interview schedule was not done.

Data Analysis

The statistical tests chosen to analyze the data were appropriate for the measurement level of the variables a n d for t h e research questions. For example, chi- square analysis was used to deter- mine whether or not any signifi-

19:l January/February 1990 JOCNN 21

Professional nurses must develop the habit of critically reading nursing research studies.

cant associations existed between t h e demographic variables a n d t h e women’s responses t o fetal monitoring. The probability level t o determine significance was identified.

D i s c u s s i o n of the F i n d i n g s

The study findings were com- pared with t h e original research done in 1972. Differences between t h e results of t h e s e two studies were identified. The researcher did not discuss t h e findings in re- lation t o t h e theoretical frame- work because no framework had been used. One limitation of the study-the limited generalizabil- ity of t h e findings-was a d -

dressed. Other limitations, such a s the lack of validity testing of the interview schedule, were not identified. Result implications for clinical nursing practice were dis- cussed, but recommendations for future research were not made.

CONCLUSION

General criteria for evaluating a research report were discussed. These criteria should help facili- tate nurses in becoming knowl- edgeable c o n s u m e r s of nursing research.

REFERENCES

1. Polit, D., a n d B. Hungler. 1987. Nursing Research: Principles and Methods. Philadelphia: J. B. Lip- pincott Company.

2. Wilson, H. 1987. Introducing Re- search in Nursing. Menlo Park, Cal- ifornia: Addison-Wesley Publish- ing Company.

3. LoBiondo-Wood, C., and J. Haber. 1986. Nursing Research: Critical Ap- praisal and Utilization. St. Louis: C. V. Mosby Company.

4. Beck, C. 1980. Patient acceptance of fetal monitoring a s a helpful tool. JOGN Nurs. 9(6):350-53.

5. Beck, C. 1972. Patients’ Cognitive and Emotional Responses to Fetal Monitoring. Masters Thesis, Yale University.

6. Dulock, H., and M. Herron. 1976. Women’s responses t o fetal moni- toring. JOGN Nurs. 5:68-70.

7. Starkman, M. 1977. Fetal monitor- ing: Psychologic c o n s e q u e n c e s a n d management recornrnenda- tions. Obstet Gynecol. 50:500.

Address for correspondence: Cheryl Tatano Beck, Florida Atlantic Univer- sity, Division of Nursing, 500 NW 20th Street, Boca Raton, FL 33431.

Cheryl Tatano Beck is an associate profes- sor at Florida Atlantic University in Boca Raton, Florida. Dr. Beck is a member of NAACOG, the American Nurses’ Associa- tion, and Sigma Theta Tau.

DO YOU WANT TO BE AN AUTHOR? If you’re thinking about writing an article for JOGNN, write for author guidelines.

- Papers must have no more than 25 references. - Papers must be no longer than 15 pages (not including references, tables, and

- U.S. and Canadian statistics should be used, e.g., “The incidence of neonatal

To obtain a complete set of guidelines, write to Editor, JOGNN, 409 12th St.,

Submission requirement include

illustrations).

hyperbilirubinemia is -O/O in the U.S. and -O/O in Canada.”

S.W., Washington, DC 20024-2191; or call 1-202-863-2441.

22 19:l JanuaryfFebruary 1990 JO(;NN