PICOT Statement Paper
SP E C IA L FEATURE r By Denise M . Bowen, RDH, MS, and Jane L. Forrest, RDH, EdD
Translating Research for Evidence-Based Practice
lthough most dental hygienists received education regarding evi dence-based decision making while in
school, many report their preparation was less than adequate for them to confidently search the literature for good quality evidence and to appraise it for its validity and relevance.1 The first phase involves understanding the evi dence-based decision making (EBDM) process, including asking a good clinical question, and efficiently searching databases to find credible
evidence to answer that question.2,3 An online continuing education course regarding those steps is available for dental hygienists wanting a review.4 The second phase involves critical ly appraising the evidence to determine its credibility and assessing the practicality of the information. The final phase involves applying the evidence to one’s clinical decision and pa tient care. The focus of this article is on Phase 2, evaluating research so you can translate it into your practice.
Figure 1. Hierarchy of Research Designs and Levels of Scientific Evidence
se co n d a ry p reapp ra tsed o r f ilte re d s tud ies
p rim a ry , e xp e rim e n ta l s tud ies p ro sp e c tive , tes ts tre a tm e n t
p rim a ry , n o n -e x p e rim e n ta l/o b s e rv a tio n a l s tud ies prospective: have been exposed to a risk; observe fo r c
p rim a ry , n o n -e x p e rim e n ta l/o b s e rv a tio n a l s tud ies re tro sp e c tive : su b je c ts have o u tc o m e o f In te res t; I
no design n a rra tive review s, e x p e rt op in io n s , e d ito r ia ls
does not involve humans
Reprinted from Forrest JL, M iller SA. EBDM in Action: Developing competence in EB practice, ©2016. Used by permission of the author.
10 January 2017
Questions to Guide the Process of Critically Appraising Research
First, consider where the article is published. For scientific evidence, the source needs to be peer-reviewed (or “refereed”). Peer-reviewed means the manuscripts have been reviewed by experts who have evaluated the study design and the strength of the conclu sions.5 Typically, there are at least two reviewers. If the study information is reported clearly and accurately, the manuscript can be accepted without additional work. When the informa tion is incomplete or confusing, the reviewers’ critiques are sent to the au thors for consideration and implemen tation. The manuscript is revised to address the issues raised, to strengthen and improve its quality before publi cation. After revision, the manuscript may be accepted without further review, or sent back to the reviewers for another review. In this case, there is no guarantee that the manuscript will be accepted. When a manuscript is extensively flawed or not appropriate for a particular publication, it will be rejected upon initial review.
When evaluating research articles, determine if the source is peer-reviewed, or if it is a practice-based publication that publishes expert opinions, product overviews and continuing education. Both types of publications have value to dental hygiene practitioners, but the most credible scientific information is generally found in peer-reviewed journals. While considering the type of publication, also review the references cited to see if they represent higher levels of evidence (research studies) or if they represent personal interviews, expert or layman opinions, commercial websites, magazine articles and online materials from less credible popular sources. High-quality scientific articles cite other high-quality evidence like clinical practice guidelines; systematic reviews; meta-analysis; randomized con trolled trials (RCTs); controlled clinical trials; or governmental reports, laws and regulations.
Forrest and Miller have succinct ly outlined questions from evi dence-based sources that health care providers can use to evaluate, or crit ically appraise, scientific literature.6'8 These questions provide the basis for the critical evaluation process discussed in this article. A preliminary assess ment helps to determine what the article actually is about.7 For example,
• Why was the study conducted, and what question did the author examine? Is there a clear statement of purpose, a specific PICO question, or a list of precise objectives stated at the beginning of the article? A PICO question identifies the patient problem or population (P), intervention (I), comparison (C) and outcome(s) ( O ) .
• W hat type of research was conducted? Where does this type of research fall on the Hierarchy of Research Designs and Levels of Evidence (Figure 1)?
- In general, secondary research is the highest level of research because it follows a distinct, systematic process to synthesize primary research studies that address the same question. Secondary research includes systematic reviews and meta-analyses, representing a body of knowledge versus an individual study (Figure 2). It provides the basis for clinical practice guidelines (CPG) (Box 1).
• Was the appropriate research design used based on the purpose of the study? For example, an RCT with experimental and control groups is the most appropriate design for answering questions related to a therapy or preventive procedure/technique, whereas a controlled clinical trial would be used for answering questions related to a diagnostic test/ instrument as there would be no control group. Both types
A SURVEY OF DENTAL HYGIENISTS' PREFERENCES, ATTITUDES 01* PRACTICES WOULD NOT ANSWER QUESTIONS ABOUT THE EFFECTIVENESS OF CLINICAL PROTOCOLS.
of research use experimental research designs.6 A survey of dental hygienists’ preferences, attitudes or practices would not answer questions about the effectiveness of clinical protocols or be able to assess the outcomes of an intervention on a patient or community.
Assuming this initial screening using the preliminary questions indicates good quality thus far, the process continues by asking three specific questions.8
1. Is the study valid? 2. W hat are the results? 3. Do the results apply to my
patient?
1. Is the study valid? In other words, was the question
or objective clearly focused, and did the investigators use the correct study design to answer the question? If the answer is no, it will be difficult to have confidence in the findings, so why bother continuing to read the study?
To begin assessing validity, identify whether the authors directly address the stated purpose and/or the research questions or hypothesis. Then consider whether rigorous methods were used. Methods to assure objectivity and rig or, depending on the design, include following strict pre-established criteria, selecting and recruiting subjects
A c c e s s 11
without bias, including subjects that represent the attributes to be stud ied, randomly assigning participants or treatments to groups, blinding participants and/or examiners to group or intervention assignment, including a control or comparison group or using a standard reference, and using research instruments with documented established validity and reliability (ver sus the authors’ simply creating one they thought would assess the desired outcomes).
The study also should be conducted with a large enough sample to test the stated hypothesis or answer the research questions. Well-designed small studies are acceptable; the results just need to be interpreted carefully. While small studies can provide results quickly, they do not normally yield definitive results. Thus, the authors should not make strong conclusions about a risk factor or trial intervention, regardless of the results. Instead, small
studies should be used to design larger studies or confirm preliminary results. If study’s aim is to provide valid and reliable evidence on a risk factor or new intervention, the study should be large enough to do so.
Less rigorous methods negatively impact the validity of a study’s results. For example, studying oral hygiene with one group of dental hygiene students to determine effectiveness of a preventive technique or approach has always seemed odd, as it would be assumed they might value oral hygiene or oral health differently than the aver age patient. Measuring plaque, gingival bleeding and inflammation but not probing depth and clinical attachment level (CAL) would not be valid in a study of a periodontitis intervention but would be valid outcomes to assess in a study of gingivitis. In educational research, determining if an educational program improved knowledge from pretest to posttest with no comparison
or control group may not be valid because knowledge would be expected to increase following education.
2. What are the results?
After determining if the study is valid, review the results. Are the findings statistically and/or clinically significant? Clinical significance can be interpreted as importance.11 Sta tistical significance estimates the like lihood or probability that the results truly are related to the variable being studied versus due to chance. Some results can be statistically significant without being clinically significant. For example, when comparing two treatments for periodontitis, a statis tical difference in favor o f treatment X might be found. If treatment X resulted in a gain of CAL of 0.25 mm and treatment Y resulted in a CAL
Prim ary vs. Secondary Research
Primary Research
Individual Research Studies
All answering the sam e question
Secondary Research Reviews of Already Conducted Research
1 t----------------------------- ----------------------------- 1 Systematic Review & Meta-Analysis
Statistical S y n th e s iz e d . A nalysis of
R e s u lt s S ynthesized Results
B ody o f E v idence
T J
Reprinted from Forrest JL, Miller SA. EBDM in Action: Developing competence in EB practice, ©2016. Used by permission o f the author.
12 January 2017
Table I. Checklists and Standards for Evaluating Research Literature
Resource Website Type of Tool(s) Provided
Critical Appraisal Skills Programme (CASP)
http://www.casp- uk.net/casp-tools- checklists
Critical appraisal checklists for:
1. systematic reviews 2. randomized controlled trial 3. diagnostic 4. economic evaluation 5. qualitative 6. case control 7. cohort study 8. clinical prediction rules
Consolidated Stan dards of Reporting Trials (CONSORT)
http://www.con- sort-statement.org/
Statement regarding minimum recommendations for RCTs and a checklist and flow diagram for criti cal appraisal of RCTs
Preferred Reporting Items for Systematic Reviews and Me ta-Analyses (PRISMA)
http ̂ /p ris ma-statement.org/
K •• - I #
Statement regarding minimum recommendations for systematic reviews and meta-analyses and a checklist and flow diagram for crit ical appraisal of systematic reviews and meta-analyses
Standards for Re porting Studies of Diagnostic Accuracy (STARD)
http://www.equa- tor-network.org/ wp-content/up- loads/2015/03/ STARD-2015-check- list.pdf
A checklist for evaluating studies of diagnostic tests and devices
Strengthening the reporting of observa tional studies in epi demiology (STROBE)
http ://w w w . strobe-statement. org/?id=avail- able-checklists
Checklists (combined and individu al) for evaluating
1. Cohort studies 2. Case study 3. Cross-sectional studies 4. Conference abstracts
Wiley Critical apprais al checklists
http ://onlinelibra ry. wiley.com/doi/ 10.1002/ 9780470987407. app2/pdf
Checklists for various study designs
1. Experimental (randomized and non-randomized controlled trial) designs. 2. Quasi-experimental (interrupt ed time series) designs 3. Attribution studies 4. Questionnaire surveys 5. Qualitative studies 6. Mixed-methodology case stud ies 7. Real world and action research
Access 13
Box 1. Clinical Practice Guidelines (CPGs)
• Based on the h ighest levels o f evidence to provide guiding principles fo r specific s itua tions o r conditions.
• Examples re lated to dental hygiene practice include p it-and fissure sealants9 and non- surgical tre a tm e n t o f chronic pe riodon titis .10
gain of 0.10 mm, the dental hygienist has to determine whether the differ ence in gain in CAL between the two approaches (0.15 mm) is clinically significant. Is it important enough to change protocols, purchase a device or revise patient self-care practices? Also, consider how many additional visits were required to obtain the 0.15 mm gain and at what cost? Remember, statistical significance does not deter mine clinical significance. Conversely, findings can be clinically significant without being statistically significant, especially when they make a tangible difference to the patient.12
3. "Will the results help in caring for my patients?"
Once the study’s validity is estab lished, and the results are determined, consider this final question in the critical appraisal process.6 Compare your patient(s) or communities of in terest to those studied in terms of age, education or socioeconomic level, and disease parameters and assess whether they are similar or different. In other words, would they have qualified to be in the study? If they are different, the results would not apply. Consider how your experiences and your patient’s preferences will influence the decisions about incorporating these results into that patient’s care. Also, be careful how you apply results of population-based research to an individual patient. At times, research examines the char acteristics of a group of patients or a selected community to estimate population-based risks. Results from these studies can indicate incidence of a given disease or risk factors. Findings can be used to generate hypotheses for studies to assess individual’s risks. However, conclusions from such popu lation-based research cannot be applied to individual patients.5
After you have evaluated the strength of the evidence and deter mined its relevance, decide whether it should be incorporated into your treatment plan. If so, take action by actually planning its implementation during patient care when similar circumstances arise. Share the findings with others in your practice setting who might have patients with similar conditions. This EBDM process not only is likely to improve treatment outcomes, but also has the potential to make your work more interesting. H
Denise Bowen, RDH, M S, professor emeritus in dental hygiene a t Ida ho State University, has served as a consultant to dental industry, as well as government, universities and private organizations. She has presented at meetings o f the A D H A , Canadian Dental Hygienists’ Association, Amer ican Association o f Dental Schools, American Public Health Association, North American Conference on Dental Hygiene Research, and International Symposiums on Dental Hygiene in Can ada, Russia, China and Costa Rica. She
has contributed to several dental hygiene texts and is widely known through her numerous published articles in dental hygiene and her dynamic continuing education programs related to nonsurgi- calperiodontal therapy, preventive oral care, research and education.
Jane L. Forrest, RDH, EdD is professor o f Clinical Dentistry; section chair, Behavioral Science, Ostrow School o f Dentistry, University o f Southern Cali fornia; and director, National Center fo r Dental Hygiene Research & Practice. She is an internationally recognized aitthor and presenter on evidence-based decision making (EBDM). Projects have included preparing the A D H A white paper on E BD M in dental hygiene, serving on the planning committee and participating in the American Dental Association's Champions Conference on EBD. Her
publications include book chapters on E BD M fo r Clinical Periodontology and fo r Dental Hygiene: Theory and Practice. She is the lead co-author o f two textbooks, the most recent being Evidence-Based Decision Making: Devel oping Competence in EB Practice.
References
1. Osborne SD, Henley GL, Josey-Baker YI, Freyer CE. Information-seeking practices of dental hygienists. J Dent Educ. 2014; 78(12): 1615-22.
2. Forrest JL, Miller SA. Translating evidence based decision making into practice: EBDM concepts and finding the evidence. J Evid Base Dent Pract. 2009; 9(2): 59-72.
3. Miller SA, Forrest JL. Translating evidence based decision making into practice: appraising and applying the evidence. J Evid Base Dent Pract. 2009; 9(4): 164-82.
4. Forresr JL. Evidence-based decision making: introduction and formulating good clinical questions. Available at: www.dentalcare.com/en- us/professional-education/ce-courses/ce311
5. Hollister C. Reading dental literature: just rhe basics. Nashville Area Dental Support Center United South and Eastern Tribes. Inc. n.d. Available at: www.nappr.org/files/dental- resource-gu ide/Cl i n ical%20Resources/ Reading%20Dental%20L.iterature-Basics.pdl. Accessed October 24,2016.
6. Forrest JL, Miller SA. EBDM in action:
developing competence in eb practice. ebdLibrary: Colbert, Wash., 2016.
7. Greenhalgh T. Getting your bearings (deciding what the paper is about). BMJ 1997; 315(7102): 243-6.
8. Critical Appraisal Skills Programme (GASP). Appraising the evidence. Available at: www. casp-uk.net. Accessed October 30, 2016.
9. Wright JT, Crall JJ, Fontana M, et al. Evidence-based clinical practice guideline for the use o f pit-and-fissure sealants. J Am Dent Assoc. 2016; 147(8): 672 -82 .e l2.
10. Smiley CJ, Tracy SL, Evidence-based clinical practice guideline on the nonsurgical treatment o f chronic periodontitis by means o f scaling and root planing with or without adjuncts. J Am D ent Assoc. 2015; l46(7):525-35.
11. University o f Ortowa. Statistical Significance and Clinical Importance. Available at https://www.med.uottawa.ca/sim/data/ StatisticaLsign i ficance_importance_e.htm. Accessed Nov. 5, 2016.
12. Hujoel P. Levels o f clinical significance, j Evid Base D ent Pract 2004; 4:32-6.
14 January 2017
Copyright of Access is the property of American Dental Hygienists Association and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.