Searching for evidence - Criminal Procedures

www.elsevier.com/cuog

Searching for evidence to inform clinical practice

Khalid S. Khan*, Aravinthan Coomarasamy

Education Resource Centre, Birmingham Women’s Healthcare NHS Trust, Birmingham B15 2TG, UK

Summary Literature search and retrieval are essential steps in evidence-based practice. However, locating the relevant research to answer a specific clinical question can be daunting due to the volume and scattering of published literature. For clinicians, this necessitates the acquisition of basic skills to make literature searching effective and efficient. The first step is to take a hierarchic approach to searching: begin by looking for well-developed guidelines and evidence summaries; these may be found on the websites of various professional bodies. If guidelines and evidence summaries do not exist, searches for well-conducted systematic reviews are likely to give more precise and accurate answers than single studies. Such reviews may be found within databases such as the Cochrane Library or the Reproductive Health Library. If systematic reviews do not exist, a carefully planned search, with appropriate input from a clinical librarian, will need to be carried out in primary bibliographic databases such as MEDLINE and EMBASE. If all else fails, the final port of call will be contacting experts, Internet discussion forums or manufacturers. Once evidence has been obtained, following assessment of its quality, importance and relevance, it may be adopted in practice. & 2003 Published by Elsevier Ltd.

Introduction

Identification and retrieval of evidence is a key step in evidence-based medicine (EBM), that encourages the explicit and judicious use of current best evidence in healthcare decision-making. EBM re- quires a high level of skill in systematically finding, appraising and applying contemporaneous research findings. There is evidence that the weakest link in these three steps of EBM is the finding of appro- priate literature. Our primary objective is to describe literature searching for practicing EBM, focusing on the clinical reader rather than librar- ians and information scientists.

Clinicians have limited time, so they are unable to examine all of the available evidence. There are over 17 000 medical journals, so it is inconceivable that clinicians would be able to undertake this task,

even if they wanted to, given their minimal training in literature searching. Therefore, their approach has to be pragmatic. Even this requires training in its own right, and this paper will outline one such approach for clinician’s consideration. Critical appraisal of literature for its quality (validity) is an integral part of the practice of EBM. This paper does not provide details of objective criteria for quality assessment, but a relevant reference is cited (Greenhalgh).

The general outline of searching literature for informing practice is shown in Fig. 1. In the current day and age, EBM requires more than just individual studies. Many systematic reviewers and guideline developers prepare evidence summaries and re- commendations for practice, that have now come to occupy a pivotal role in clinician’s professional lives. Pragmatically, this should be the starting point of a search. There should be a systematic and structured approach to reduce the risk of missing relevant citations higher up in the flow chart (Fig. 1). This is crucial because if up-to-date and

ARTICLE IN PRESS

KEYWORDS

Literature search;

Evidence-based Medicine

(MESH);

Information Science

(MESH);

Review Literature (MESH);

Guidelines (MESH)

*Corresponding author. Tel.: þ 44-121-66272695; fax: þ 44- 121-4141576. E-mail address: [email protected] (K.S. Khan).

0268-0890/$ - see front matter & 2003 Published by Elsevier Ltd. doi:10.1016/j.curobgyn.2003.12.006

Current Obstetrics & Gynaecology (2004) 14, 142–146

quality guidelines and reviews can be identified, the pain of selecting search terms, exploding them, finding word variants, combining them with AND/ OR/NOT (appropriately), sifting through the resul- tant citations and abstracts, selecting them for appraisal of quality and findings (using full papers), and considering them for application in practice can all be avoided. The validity of any recommen- dations that clinicians make following a search of bibliographic databases of primary studies lower down in the flow chart (Fig. 1) will be related directly to the comprehensiveness of the literature search. Without input from a clinical librarian, there would be little confidence that important studies have not been missed.

Searching for clinical practice guidelines

Guidelines provide statements to assist clinicians and patients in making decisions about specific clinical situations. They often, but not always, use evidence from systematic reviews. When devel- oped formally and systematically, they can be expected to be more rigorous than guidance preached by experts. For every expert, there is usually an equal and opposite expert, so there cannot really be much faith in their sermons, particularly when well-developed guidelines exist. Such guidelines would be expected to employ an explicit methodology, a well-conducted systematic literature review (see below), a multidisciplinary working group, graded recommendations with direct links to the evidence reviewed, quality- control input by an independent advisory board or by independent peer review, and other such features to avoid bias. It is important to be aware that not all guidelines are developed this way.

Guidelines are often not published in peer- reviewed journals, and are usually not indexed in either MEDLINE or other bibliographic databases. The Internet provides the most useful way to access guidelines (Table 1), but one-click searches for these do not usually exist. There will be a need to develop a personal strategy for searching guidelines depending on the clinician’s subspecialty. Once developed, with frequent use, the search strategy will become second nature. If a guideline, identi- fied by the search, deals with the clinical problem that the clinician faces and it is methodologically rigorous, the relevant recommendations could be adapted and adopted in practice. If the guideline

ARTICLE IN PRESS

Search for Guidelines and evidence summaries (e.g. websites of professional bodies, NeLH)

Guidelines exist

Appraise, adapt and adopt guideline Search for systematic reviews

(e.g. Cochrane Database (CDSR, DARE), RHL, PubMed, Clinical Evidence, Effective Health Care Bulletin)

Search for primary literature in bibliographic databases (eg PubMed, Embase)

Consider asking a clinical librarian to search or attempt an Internet search or contact manufactures/experts

No Guidelines

Reviews exist

No Reviews

Primary studies exist

Appraise and adopt evidence

No papers

Figure 1 A suggested flow chart for identifying evidence to inform practice.

Table 1 A selected list of Internet sourcesn of guidelines.

CMA InfobaseFwww.cma.ca/cpgs/ Guidelines and Guidelines in PracticeFwww.eguidelines.co.uk Guidelines in Obstetrics and GynaecologyFarchive.uwcm.ac.uk/uwcm/lb/pep/maternal/index.html matweb.hcuge.ch/matweb/endo/cours 4e MREG/Obstetrics guidelines.htm Health services/technology assessment text (HSTAT)Ftext.nlm.nih.gov/ National Coordinating Centre for Health Technology AssessmentFwww.hta.nhsweb.nhs.uk/ National Electronic Library for HealthFwww.nelh.nhs.uk/ National Institute for Clinical Excellence (NICE) Fwww.nice.org.uk/ National Guidelines ClearinghouseFwww.guideline.gov/index.asp OMNIFomni.ac.uk (use advanced search and specify Practice Guidelines in Resource Type) Royal College of Obstetricians and GynaecologistsFwww.rcog.org.uk ScHARRFLock’s Guide to the evidenceFwww.shef.ac.uk/uni/academic/R-Z/scharr/ir/scebm.html SIGN GuidelinesFwww.show.scot.nhs.uk/sign/home.htm Turning Research Into Practice (TRIP)Fwww.tripdatabase.com/ Clinical EvidenceFwww.clinicalevidence.org Effective Health Care BulletinsFwww.york.ac.uk/inst/crd/ehcb.htm

nMany of these provide evidence summaries with or without recommendations for practice. Web addresses correct at the time of preparing manuscript.

Searching for evidence to inform clinical practice 143

was not up-to-date, it may be necessary to search for reviews and primary studies that have been published since the guideline was completed.

Some sources may provide evidence summaries following a relatively systematic search to answer specific clinical questions. These include secondary publications (some may be searchable on the electronic databases), such as Best Evidence (which includes ACP Journal Club and Evidence Based Medicine), Clinical Evidence, Bandolier, Obstetrics and Gynecology Survey, Evidence-based Obstetrics and Gynecology and others. Products containing evidence summaries have mushroomed in recent years. They are very popular with clinicians as they scan a number of journals to summarize the results of reviews and primary evidence with explanatory commentaries. Indeed, many of the websites listed in Table 1 include a mixture of evidence summaries and guidelines. The clinician would have to be aware of this distinction and should not have blind faith in these. Just as with guidelines, the evidence summaries should be examined with a critical eye before adoption in practice.

Searching for systematic reviews

Systematic reviews are research articles that summarize the evidence on a clearly formulated question using systematic and explicit methods to identify, select and appraise relevant primary studies, and to extract, collate and report their findings. By following this process, they represent a quantum leap in review methodology, and are likely to lead to less biased assessment of tests and treatments in comparison with traditional expert reviews where the experts can review the litera- ture selectively to support almost any personal opinion. If no relevant guidelines can be found, the next step would be to search for systematic reviews. A well-conducted systematic review in- volves five stepsFpre-defined questions, extensive literature search, study quality assessment, synth- esis of findings and their interpretation. It may or may not use meta-analysis, a statistical technique for combining (pooling) the results of a number of studies addressing the same question to produce a summary result. If a systematic review employs this technique, it might be called a meta-analysis and indexed under this term in bibliographic databases. However, it is important to be aware that not all meta-analyses are based on systematic reviews, and meta-analyses without a systematic approach to finding the evidence may be biased by selective inclusion of studies.

The key source for systematic reviews (of effectiveness of interventions) is the Cochrane Library (www.update-software.com/cochrane/), and this should be searched first. The Cochrane Library contains the following databases of sys- tematic reviews: Cochrane Database of Systematic Reviews (CDSR), Database of Reviews of Effective- ness (DARE), and Health Technology Assessment Database (HTA) (Table 2). There are more systema- tic reviews around than one might think. For example, in the third issue of the 2003 Cochrane Library alone there were 1754 complete reviews and 1304 protocols of reviews in CDSR, 4123 abstracts of quality assessed reviews in DARE, and 3178 abstracts of completed technology assess- ments in the HTA database. Cochrane Reviews relevant to reproductive health can also be found on the WHO Reproductive Health Library; this is available, free of charge, to health workers in developing countries.

If no relevant reviews can be located in the above libraries, the next step would be to search alternative sources where citations to published systematic reviews are included. Bibliographic databases such as MEDLINE and EMBASE may be searched for systematic reviews. MEDLINE, avail- able freely via PubMed (www.ncbi.nlm.nih.gov/ PubMed), contains bibliographic records (with and without abstracts) of biomedical literature from 1966 onwards. PubMed’s Clinical Queries feature allows one-click searches for systematic reviews. At the time of writing, there were 71 125 citations included in the PubMed Systematic Reviews subset search strategy. For other databases, filtered searches (see below), well beyond the remit of busy clinicians, will be required to identify systematic reviews efficiently.

Searching for primary studies

Much advice is given on searching bibliographic databases, but none of it is of any particular help to busy clinicians. The advice is often geared towards teaching Boolian logic (AND/OR/NOT), indexing trees, exploding, limiting and other features peculiar to bibliographic databases. Most of this information is not clinically intuitive. What is needed is a link between what the clinician needs to know and how this can be captured quickly with a degree of certainty. Searching thus needs to be adapted to the clinical process (Fig. 2).

If we accept the clinical process and the thesis that a clinician’s mantra is to employ current best knowledge about aetiology, diagnosis, prognosis

ARTICLE IN PRESS

144 K.S. Khan, A. Coomarasamy

and therapy, why should literature searches not be tailored to meet the requirements of the clinical process? PubMed’s Clinical Queries feature (www.ncbi.nlm.nih.gov/entrez/query/static/ clinical.html) does just that. It provides one-click searches for citations concerning the various clinical knowledge needs. The manner in which it does this behind the screen is worth considering, mainly to convince readers about the worth of this utility. For each clinical concept (aetiology, diag- nosis, prognosis, therapy), there are particular study designs, and these have subject indexing terms in databases. Therefore, search term combi- nations (also known as search filters) can be developed to capture studies of particular designs and related clinical concepts. This way, filters can be used to perform quick searches to support day- to-day evidence-based practice. A high proportion of citations retrieved by filtered searches will be relevant to the clinical concept used, but there is a risk that some relevant citations that are not indexed in a way that the filter can pick up may be missed. This is a major drawback for guideline developers and systematic reviewers, but not so for pragmatic clinicians. For therapy questions, in particular, where indexing of design-related terms is reasonable, filtered searches can identify rele- vant citations quickly.

If the initial searches do not reveal much, one may broaden the search strategy. This will generally result in the retrieval of more

articlesFmany irrelevantFbut with a greater chance of containing a relevant article. This can be achieved in PubMed’s Clinical Queries web page by selecting the radio-tab for sensitivity rather than specificity. Do not confuse the librarians’ use of these terms with sensitivity and specificity of diagnostic tests. Sensitivity of a search describes the ability of a search to identify all relevant studies on a given topic, increasing our ability to capture a large proportion of the relevant studies, although this would have to be at the cost of having to go through a very large number of citations. If this reveals too many citations to go through, use

ARTICLE IN PRESS

Table 2 A selected list of sources of systematic reviews.

The Cochrane Database of Systematic Reviews (CDSR)n

Contains the full text of regularly updated systematic reviews of healthcare interventions carried out by the Cochrane Collaboration, plus protocols for reviews currently in preparation. Database of Abstracts of Reviews of Effectiveness (DARE)n

Critical appraisals of systematic reviews found in sources other than CDSR. These reviews are identified by regular searching of bibliographic databases, hand searching of key major medical journals, and by scanning grey literature (available free at www.york.ac.uk/inst/crd). Health Technology Assessment (HTA) Databasen

Abstracts of completed technology assessments and ongoing projects being conducted by members of the International Network of Agencies for Health Technology Assessment (INAHTA) and other healthcare technology agencies. Most of these include systematic reviews (available free at www.york.ac.uk/inst/crd). The WHO Reproductive Health Library (RHL) Cochrane Reviews relevant to reproductive health. This is available free of charge to health workers in developing countries by e-mailing [email protected] PubMedFMEDLINE This provides the option to search for systematic reviews in the Clinical Queries feature (available free at www.ncbi.nlm.nih.gov/entrez/query/static/clinical.html). CINAHL, EMBASE and other databases These may be searched for reviews by adapting one of the search filters (a combination of text words, indexing terms and subject headings that captures relevant articles) available from the Centre for Reviews and Dissemination search strategies available at www.york.ac.uk/inst/crd/search.htm

nAvailable in the Cochrane Library. Web addresses correct at the time of preparing manuscript.

Patient presentation knowledge about diagnosis

Testing •History

•Examination •Investigations

Diagnosis knowledge about prognosis

Therapy •Changes prognosis

knowledge about therapeutic effectiveness Clinical outcome

Etiology knowledge about causation

Research evidence sought from literature searches

Clinical Process and knowledge requirements

Etiologic Research

Diagnostic Research

Prognostic Research

Therapy Research

Figure 2 The relationship between clinical process and literature searches for evidence.

Searching for evidence to inform clinical practice 145

the specificity option. Specificity of a search describes the ability of a search to exclude irrelevant studies on a given topic, increasing our ability to quickly identify relevant studies by sifting through a relatively small number of citations, although we risk missing out on some other relevant studies. If this reveals too few or no relevant citations, use the sensitivity option.

Other sources of evidence

If you come up empty handed in your search for evidence-based guidelines, appraised evidence summaries, systematic reviews and primary studies in the usual databases, you have a number of choices. At this stage, one should involve an experienced clinical librarian to broaden the search strategy and search a larger number of databases. If the input from an expert librarian yields no fruits, where will one turn next? There are at least three options: first, locate a specialist in the field of interest and ask for the evidence (so that you can make up your mind about the quality of evidence yourself). Second, you can identify relevant Internet discussion forums (by searching in www.google.com, for example) and put a query to the members of the discussion forum. Third, you may approach the manufacturers of the diagnostic test kit, treatment or any other technology you wish to appraise for supporting evidence. However, be well aware that it is in the interest of manufacturers to promote evidence favouring their product, and it would generally be naive to expect them to provide a well-balanced critique of all evidence. What about traditional textbooks? Well, one can burn them as soon as one has passed examinations, as they are likely to be out of date and they may contain erroneous messages. The exception is if the textbook is revised frequently and the statements are explicitly supported by evidence. Sadly, such texts are very rare.

Retrieving articles

Many guidelines can be downloaded freely from the Internet. The problem arises when reviews and primary articles published in journals need

to be obtained. Some of these may also be freely available on the Internet (see www. freemedicaljournals.com/), or may be obtained through membership of a professional body. How- ever, when seeking information to inform practice at short notice, e.g. on a ward round, the clinician would need a link with their local medical library, that may, in turn, have a working arrangement with regional or national library networks.

Further reading

Coomarasamy A, Latthe P, Papaioannou S, Publicover M, Gee H, Khan KS. Critical appraisal in clinical practice: sometimes irrelevant, occasionally invalid. J R Soc Med 2001;94:573–7.

Coomarasamy A, Ola B, Gee H, Khan KS. Quality of brief guidelines produced by professional bodies: a study of the ‘green-top’ guidelines by the UK Royal College of Obstetricians and Gynaecologists. J Obstet Gynaecol 2003;23:479–83.

Deshpande N, Publicover M, Gee H, Khan KS. Incorporating the views of obstetric clinicians in implementing evidence- supported labour and delivery suite ward rounds: a case study. Health Info Libr J 2003;20:86–94.

Greenhalgh T. How to read a paperFthe basics of evidence based medicine. London: BMJ Publishing Group; 2000.

Kavanagh J, Khan KS. Searching for evidence. Clinical govern- ance advice no. 3. London: The Royal College of Obstetricians and Gynaecologists Press; 2001.

Khan KS, Awonuga AO, Dwarakanath LS, Taylor R. Assessments in evidence-based medicine workshops: loose connection be- tween perception of knowledge and its objective assessment. Med Teach 2001;23:92–4.

Khan KS, Kunz R, Kleijnen J, Antes G. Systematic reviews to support evidence-based medicine: how to review and apply findings of systematic reviews. London: Royal Society of Medicine; 2003.

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med 2003;96:118–21.

Rosenberg W, Donald A. Evidence based medicine: an approach to clinical problem-solving [see comments]. BMJ 1995;310:1122–6.

Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine: how to practice and teach EBM. Edinburgh: Churchill Livingstone; 2000.

Khan KS, ter Riet G, Glanville J, Sowden AJ, Kleijnen J. Undertaking systematic reviews of research on effective- ness. CRD’s guidance for carrying out or commissioning reviews. CRD Report Number 4, 2nd ed. York: NHS Centre for Reviews and Dissemination, University of York; 2001.

Wyatt JC. Number of medical journalsFWyatt’s apology (Following on from the article ‘ Wyatt JC. Knowledge for the clinician 5. Reading journals and monitoring the published work. J R Soc Med 2000;93:423–30). lis-medical@ mailbase.ac.uk 2000.

ARTICLE IN PRESS

146 K.S. Khan, A. Coomarasamy