CRH

Best Practice & Research Clinical Rheumatology 27 (2013) 295–306

Contents lists available at SciVerse ScienceDirect

Best Practice & Research Clinical Rheumatology

journal homepage: www.elsevierheal th.com/berh

10

How to perform a systematic search

Else Marie Bartels, PhD, DSc, Research Librarian DB *

The Parker Institute, Department of Rheumatology, Copenhagen University Hospital Frederiksberg and Bispebjerg, Ndr. Fasanvej 57, 2000 Frederiksberg, Denmark

Keywords: Bibliographic databases Evidence-based medicine Information literacy Information services Internet Literature

* Tel.: þ45 38164168; fax: þ45 38164159. E-mail address: else.marie.bartels@regionh.dk.

1521-6942/$ – see front matter � 2013 Elsevier Lt http://dx.doi.org/10.1016/j.berh.2013.02.001

All medical practice and research must be evidence-based, as far as this is possible. With medical knowledge constantly growing, it has become necessary to possess a high level of information literacy to stay competent and professional. Furthermore, as patients can now search information on the Internet, clinicians must be able to respond to this type of information in a professional way, when needed. Here, the development of viable systematic search stra- tegies for journal articles, books, book chapters and other sources, selection of appropriate databases, search tools and selection methods are described and illustrated with examples from rheu- matology. The up-keep of skills over time, and the acquisition of localised information sources, are discussed.

� 2013 Elsevier Ltd. All rights reserved.

Introduction

Medical information, mainly in the form of scientific papers but also as books and other types of resources (mostly as Internet sites), is growing at a remarkable rate. One result of this is that a high level of information literacy is required by all who wish to keep up-to-date in their field. Another part of information literacy is to be able to trace the information patients have found on the Internet, and to assess this in a professional way, in order to keep a good patient–doctor relationship, where the patient has confidence in the doctor’s knowledge and skills.

Although general-purpose searching via a search engine (e.g., using Google [1]) or a search-engine- type search inMedline via PubMed [2] (see later) may cover your information need to some degree, it is important to be disciplined and focussed and to know the available information sources if you wish to practise your daily work in an evidence-based way [3,4]. It is only when you wish to find “something about a subject” that you might try your luck with a general-purpose search; and in this case you still

d. All rights reserved.

E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306296

have to make sure that you have found valid information, at least in the form of a review article from a peer-reviewed journal or a textbook chapter of an acceptable standard and level. For efficient up-keep of the needed level of professional knowledge at any given time, you need to be able to carry out a proper systematic search and make a correct choice of information sources.

There are four steps towards reaching a level of information literacy that will make keeping up with medical literature manageable: (1) learn to define your questions in a meaningful way; (2) get to grips with the ins and outs of literature searching; (3) make a time schedule for necessary searches; and (4) update yourself in new information sources at least once a year.

Literature search

Scientific papers in peer-reviewed journals

Most of the new medical literature appears as papers in peer-reviewed journals. To keep up with this part of the information flow, you have to follow steps 1–8, below:

1. Define your problem. 2. Create a search strategy. 3. Select the right bibliographic databases. 4. Search. 5. Select suitable references from those that have been retrieved. 6. Assess whether the search was satisfactory. 7. Redesign the search strategy and/or choose other databases/search tools, where needed. 8. Repeat steps 2–6, if necessary.

Define your problem

A successful search is based on looking for the key issues, but how do you ensure that you do exactly that? As an illustration, suppose that you wish to be updated on the effects of biologics on rheumatoid arthritis (RA). This is not exactly awell-defined problem in search terms. There are three important key issues: effects, biologics and RA.

Starting with RA, this might very well be as fully defined as it should be, but ask yourself whether a further specification is needed. Is it a particular patient group in terms of age, gender, genetics or similar? It could be that the group concerned is ‘young women with RA during pregnancy’ or another specific group of RA patients.

Effects must also be specified – effects measured in what manner and compared to what? An example of the question that has to be addressed could be: What is the effect of treatment with bi- ologics compared to non-steroidal anti-inflammatory drugs (NSAIDs) treatment or to steroids? Furthermore, is it the effect of a specific type of biologics compared with a specific type of NSAIDs? The other part of the question is which effect am I looking at? Is it DAS28, pain reduction, joint destruction, function or quality of life? There are many types of outcome measures, and usually you will have a fair idea of the important ones for a particular patient group or for a specific treatment.

The last step towards a clear definition of the problem in question is to define biologics. Howbroad a definition is allowed and which biologics are the most important ones to include. Is it really a com- parison between biologics in general with another defined type of treatment, or is it a specified biologic treatment, you have in mind?

The definition of the problem about which you want to find information is the base on which the whole procedure is built, and more experienced practitioners and researchers will have a great advantage here, being able to write out the problem of interest quickly. Often, it will be necessary to break down the problem into sub-questions to create clear search strategies, which will lead to better results. You must also decide if your question asks for an epidemiological approach, where you are looking at effects of the past in awhole population and therefore cannot ask for randomised controlled trials (RCTs), or if you aremainly interested in looking for designed studies in controlled and, if possible, randomised studies. In all this, you also have to think clearly, and make the best use of the material

E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306 297

available. Nearly all high-powered RCTs started as pilot studies. Many large epidemiological studies began as more humble studies of smaller groups, which provided the ideas for the full-scale studies. It is important to understand what type of studies you are looking for to get a valid answer to your question [5].

Create a search strategy

Having delineated a well-defined question, it is possible to create a search strategy based on a search table. Instead of rushing into a search by typing in the first words that come to mind, it is worthwhile working out a search table. If the question is effect of exercise on physical function in juvenile idiopathic arthritis, a search table could look like Fig. 1.

Juvenile idiopathic arthritis has several names, and it is wise to check with the MeSH database in PubMed [2] or with keywords in EMBASE (Exerpta medica) [6] (see below) to get further ideas concerning the various names used for the same condition by different authors.

For the juvenile idiopathic arthritis patients exercise will be limited, and it might be carried out under the supervision of a physiotherapist. Keeping this in mind whenworking out the search table, a list of suggestions covering exercise, again using some known keywords from the medical databases, is given, and more possibilities may be added. For the last term, physical function, a set of known outcome measures are given. There are more of these, and the choices here are scales that specifically are used for children, because this is a child-specific disease. In each column, each representing a key issue, at least one of the given terms has to be found in a reference to include this in the retrieved references. The terms in each column are therefore combined with ‘OR’ when searching. This will give rise to three sets of results, one from each column.

As all of the threemain issues in this example (other problems could havemoremain issues) have to be included in the total search, the results of the three searches, one for each column, have to be combined with ‘AND’, when searched. The end search will be (juvenile idiopathic arthritis OR juvenile rheumatoid arthritis OR juvenile chronic arthritis OR juvenile onset Still’s disease) AND (exercise* OR physical therapy OR jogging OR swimming OR pool therapy OR dancing) AND (physical function OR exercise test OR CHAQ OR JASI OR JAFAS OR joint range of motion).

The above is a simple example. The search can usefully be extended much further, and the given list here is a mixture of ‘free text words’, words appearing anywhere in a reference in the searched database, and defined keywords (MeSH words), which might be specific for the particular database. It is necessary to search both terms that are given keywords in the database, if a keyword covering the term exists, and other words covering the term in question. Although it is important to search the keywords/MeSH words when available, they should in general be searched as both keywords and as

Juvenile idiopathic arthritis Exercise* Physical function

Juvenile rheumatoid arthritis Physical therapy Exercise test

Juvenile chronic arthritis Jogging CHAQ

Juvenile-onset Still’s disease Running JASI

SAFAJgnimmiwS

Aquatic therapy Joint range motion

gnicnaD

gnilcyC

AND

OR

Fig. 1. A possible search table for the question ‘Effect of exercise on physical function in juvenile idiopathic arthritis’. The table may be developed further, especially in the Exercise and in the Physical function column.

E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306298

free text words. This will be the case if all search words are searched as ‘free text words’. The reason for this is that a keyword may only have been in use for some of the years covered by a database and not always been used to index a relevant reference during these years. In the example, it should be noted that exercise has an * after the word. This causes the search to include any term that starts with ‘ex- ercise’. This is called truncation of or adding a wild card to the search term.

Truncation is also used to account for different types of English language spelling, for instance *edema for edema or oedema (American and British English spellings).

The broader you search in terms of keeping each term less defined, the more ‘noise’ (useless/un- wanted references) will come out of the search. As an example, you could search osteoarthritis without limiting your search to a joint such as the knee. You will get several studies on other joints than the knee. However, you will also catch some studies of knee osteoarthritis that did not appear in your narrower search. You will therefore achieve a higher sensitivity (a better coverage of the literature) with the broader search, but the cost is the high number of retrieved references you have to assess for inclusion, where several do not give an answer to your request.

The alternative is a very specific, narrow definition of the problem in question in your designed search strategy. This will give a high specificity (more or less all of the retrieved references will be relevant), but the coverage of the problem will most likely not be optimal. Depending on your infor- mation needs, you should aim at a search strategy that will give ‘enough information’ for your purpose. For updating, a more specific searchmay be preferred, whereas youmay need several broader searches for research purposes.

Select the right bibliographic databases

When the question is clearly defined and a search strategy created, it is time to choose the right database(s) or other search tools. For medical literature, there is a good range of bibliographic databases.

Bibliographic databases cover only a certain chosen number of journals, and the selection varies from database to database. This is why you have to consider which databases are the ones to search for your particular subject area. Furthermore, it is important to remember that although databases may seem alike, they are not. Two databases may have the same structure but their keywords and names for the various fields (‘tags’ such as author, address and abstract) are very often different. The way one database searches may also differ from all the others. When designing a search strategy, it is important to look for possible keywords that define your search terms andmake sure the meaning of these words really is the same as your understanding of the words. You have to understand that a term used in your local clinical or laboratory setting is not necessarily used for the same notion in a database and that a term used in a database might have a different meaning than the one you normally would expect. Further, the terminology and spelling will vary between countries and languages (e.g., between American English and British English). You will, as mentioned, take this into account by using trun- cation/wild cards.

In rheumatology there will be a need to search a set of bibliographic databases, depending on the area of interest. For a quick update, you will get by with a search in Medline [2] and/or EMBASE [6], but if you want to get a complete update of your field, you will probably need to search more than these two databases.

Table 1 shows a selection of bibliographic databases of interest, with a short explanation and a suggestion of what to search where. It is worth knowing that new research areas first will appear as meeting abstracts, and these are mainly found in Web of Science [7], Biosis Previews [8] and EMBASE [6]. For special areas, a set of smaller databases is also shown in Table 1.

EMBASE [6] and – to a certain degree –Medline [2] cover some journals written in languages other than English, but if you wish to search articles in other languages you may find that the country speaking the language provides a bibliographic database inwhich you can search in the given language.

Bibliographic databases come in various wrappings, depending on who delivers them to the user. The delivery firm is called the ‘database host’, and one host can give access to a wide variety of da- tabases. What the host provides is the design of the search page. PubMed is really the host for several databases, apart from Medline. Other examples of hosts are STN, OVID and EBSCO. A database will

Table 1 List of databases of interest for rheumatologists.

Name [Ref.] Description Seen from the rheumatologist’s point of view, good for searching:

Medline/PubMed [2] Medicine, human biology, general physiology, cell biology. Medline is provided from most database hosts; the PubMed version is the hosting service from US National Library of Medicine, which is the creator of the database Medline.

All clinical and physiological questions. Good coverage of accreditation and management.

EMBASE [6] Medicine, human biology, general physiology, cell biology. Abstracts from some larger medical conferences.

All clinical and physiological questions. Very good coverage of musculoskeletal diseases and their treatment. Strong in the area of Pharmacology. Broader coverage of European journals than Medline.

Psycinfo [9] Psychology. Includes books and book chapters, as well as journal articles. References include reference lists.

Human psychology such as patient– doctor relationships and cognitive therapy. Useful references from reference lists of found references.

Cochrane Library [10] The database consists of Cochrane Database of Systematic Reviews (CDSR), Database of Abstracts of Reviews of Effectiveness (DARE) and Cochrane Controlled Trials Register (CCTR). A small limited database of high quality.

Systematic reviews with meta-analyses, and RCTs, concerning clinical treatment approaches as a base for evidence-based treatment.

Web of Science [7] Science, technology, health sciences, sociology, and humanities. Abstracts from some larger medical conferences.

Coverage of a wide variety of subjects. Not so strong on a specific search of a subject area, but excellent for catching references from interdisciplinary areas, as well as conference abstracts.

Biosis Previews [8] Microbiology, genetics, cell biology, general physiology and biochemistry, behaviour, botany, ecology. Some conference abstracts.

Cell biology, genetics, and other basic bio-medical areas. Includes a good selection of conference abstracts.

Chemical Abstracts [11] Chemistry and biochemistry. Drugs and drug treatment. Search at very high level due to use of CAS numbers which will relate to any name given to a particular drug.

Toxnet [12] NLMs special entrance to free databases concerning toxicology.

LactMed for drug effects during breast feeding. TOXLINE for adverse effects of drugs.

PEDro [13] PEDro covers physiotherapy and includes references to systematic reviews, RCTs, and clinical practice guidelines. Small, limited, high-quality database. Very basic search system.

Evidence-based physiotherapy. Only very basic searches can be performed

CINAHL [14] Nursing and allied health research database. References include reference lists.

Useful when searching for nursing and physiotherapy information. Reference lists of found references may give some useful guidelines etc.

AMED [15] Allied and complementary medicine. Physiotherapy, palliative care, occupational therapy

MANTIS� [16] Manual, alternative, and natural therapy index system (earlier chirolars)

Manual medicine, chiropractic, osteopathy. Generally strong on rehabilitation.

Clinical Trials.gov [17] A registry and results database of publicly and privately supported clinical studies of human participants conducted around the world.

Clinical trials which have happened or are on the way.

PROSPERO (International Prospective Register of Systematic Reviews) [18]

Prospective register of systematic reviews. Place for registering protocols for systematic reviews and meta-analyses.

(continued on next page)

E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306 299

Table 1 (continued )

Name [Ref.] Description Seen from the rheumatologist’s point of view, good for searching:

CrossRef [19] CrossRef’s classifies itself as the citation linking backbone for all scholarly information in electronic form. CrossRef links via CrossRef Digital Object Identifiers (CrossRef DOI) a reference to the electronic full-text reference, such that the DOI-number is unique for the particular reference.

Searching references via DOI number, or finding DOI-number for a reference.

Derwent Innovation Index [20]

Patents. Useful if one wishes to patent a treatment and need to find what is already patented in the area.

Journal Citation Reports [21]

Gives impact at the journal and category levels, as well as presenting the relationship between citing and cited journals.

Useful for finding impact factors (remember these change every year), as well as cited and citing half life.

E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306300

therefore look different on different hosts, but – despite this – the database behind will be exactly the same, the search system will be the same and you should not search the same database twice by searching the same database via two hosts. For example, it is only necessary to search Medline either via PubMed or via OVID, despite the different appearances of this database in the two hosting systems.

Another problem can occur when searching several databases provided from the same host in one search. This will not give the best and most professional results because the databases behind the common search face are different in structure, and the full benefit of your selection of keywords, publication types, etc. will not be obtained. It is valuable to search each database separately. This will also allow you to download chosen references from your searches for import to reference handling systems (Reference Manager [22], Endnote [23], Procite [24] and others), where knowledge of both database and host is demanded for successful import into your own reference databases.

Search

There is a continuous development in the ways search systems search databases. Artificial intelligence is part of many search systems, but its quality varies. Occasionally you will get

great benefits from the artificial intelligence and get a better search, but at other times you will find some search results that seem very far from your intended search. It all depends on the way you approach your database in the form it is made available to you. In Medline searched via PubMed [2], which uses artificial intelligence, you can search by introducing the whole search string (the end search). You can see how the database has been searched by looking at ‘search details’ (the box at the right-hand side of the screen), which explains why your search gives the references retrieved. Instead of relying on the artificial-intelligence approach, you could choose to search each term alone. When you have carried out the single-term searches, you can then go to ‘Advanced’where from ‘History’ you combine the terms with AND or OR as appropriate, where each searched termwill be represented with #1 or #2, etc. Your further search will look something like this: (#1 OR #3) AND (#4 OR #5).

In the PubMed version of Medline, it is easier to learn to handle the total search string created from your search strategy as soon as you feel confidentwith thewhole search technique and have designed a good search strategy for updating. The advantage in searching one term at a time to start with is, on the other hand, that you are able to understand if your choice of search terms is satisfactory. If you have only two hits on one of your search terms, and you know the area is well described, you know you have to find another word for your term.

Apart from the operators AND and OR, you can use NOT, although this operator should be used carefully and only in situations inwhich you are completely certain about what to exclude. For instance, you can use NOT when you have carried out two different searches with different search strategies and you want to exclude the references in the second set of results that have already appeared in the first set. If your first set of results come from search #15 and your second set of results come from search #26, you will get the results only appearing in #26 and not in #15 by searching: #26 NOT #15.

E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306 301

The consequences of using the Boolean operators AND, OR or NOT is shown Fig. 2. The main medical databases such as Medline and EMBASE have keywords organised as a tree

structure, withmain headings and subheadings. In PubMed [2], you can find these in theMeSHDatabase (choose MeSH in the search field next to the PubMed logo). If you search the term ‘osteoarthritis’ you will get an explanation of the use of this term in Medline. By going one step deeper, you will get sub- headings and the tree structure (where this term appears in the Medline hierarchy of keywords):

All MeSH categories Disease category Musculoskeletal diseases

Joint diseases Arthritis Osteoarthritis Osteoarthritis, hip Osteoarthritis, knee Osteoarthritis, spine

All MeSH categories Disease category Musculoskeletal diseases

Rheumatic diseases Osteoarthritis Osteoarthritis, hip Osteoarthritis, knee Osteoarthritis, spine

You can see that osteoarthritis appears in two parts of the tree: rheumatic diseases and joint diseases.

InMedline youwill search the term in all parts of the tree, and youwill also search all narrower terms belonging to the particular MeSH. If you search for ‘osteoarthritis’, you will automatically search ref- erences with osteoarthritis; osteoarthritis, hip; osteoarthritis, knee; and osteoarthritis, spine. However, if you wish to limit your search to, for instance, diagnosis, you may in the PubMedMeSH database do so by choosing the subheading Diagnosis and ‘send to box with AND’, search and see the results.

In Medline searches via other hosts than PubMed, and in EMBASE, there might be a choice of ‘explode search’ or not to do so. This will achieve exactlywhat it describes: ‘explode search’will include all references with the search term or with any of the subheadings; by not exploding, you will limit yourself to the chosen term.

Select suitable references from the retrieved ones

Depending on howmuch you choose to limit your search, you are likely to find several (and perhaps many) references which are not relevant for your purpose. As mentioned above, you have to find the

Fig. 2. The Boolean operators.

E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306302

right balance between sensitivity and specificity in the given situation. Around 200 or less is a reasonable number to skim through by looking at titles and abstracts, where these are available. Often, the title will help you decide whether you read further or not. Many databases have the possibility of looking at ‘related articles’ and, if you go just one layer into the ‘related articles’ search for the chosen references, you will get a useful supplement and a more complete coverage of the literature with your search.

The last step is to go through the reference lists of your chosen references to determine whether there are other references you have missed, either because they have not been included in the data- bases searched or because your search has not been able to catch them. You need to be aware that the electronic versions of the bibliographic databases start at different times; some go back a long way whereas others start around 1980. Reference lists are good sources for covering important, older literature, as well as covering important book chapters and conference proceedings.

Assess whether the search was acceptable

Whenyour initial set of relevant references has been selected, it is time to checkwhether something is missing. A person who is new to a particular field will have a problem in assessing the expected number of references, because it may not be easy to see whether one should expect five studies or 1000. If the subject is a ‘hot topic’, a lot of studies ought to appear in that area. However, if the subject is brand new, there might not be more than – say – three conference abstracts. If you already know about one or two important studies, these ought to appear as part of your retrieved references. If they do not, your search strategies are not good enough, or you have not chosen the right combination of databases, or the references in question are not included in any of the available databases and have to be found elsewhere.

Redesign the search strategy and/or choose other databases/search tools

If your coverage of the literature was found to be unsatisfactory, you have to go back to step 1 and start again. You may get some help by looking at the references you knowwere missing to see whether you can get a clue about which additional search terms could usefully be added. In addition, look at Table 1 to see whether a search in another database would be the answer. You will then have to repeat steps 2–6. Otherwise, try other search tools.

Keeping up with the peer-reviewed journal articles

When you are satisfied with your search result, save your search strategy for use at later up-dates. Every time you update your knowledge in a field, you have to consider whether it is necessary to improve your search strategies. This is because your field develops all the time and you have to follow the various ways diseases and treatments change names over time to be able to include the new terms in your future searches. You will also find that keywords develop over time. As an example, fibro- myalgia as a MeSH word appeared in 1989. If you want to search a term as a keyword earlier than the keyword appears in a database, you can try to search the keyword word above in the hierarchy/tree.

Most rheumatologists will have access to a variety of bibliographic databases via their workplace or research library. The licence fees for many of these databases are high, and you cannot expect to have access to all databases shown in Table 1. For anybody who has no access to databases paid by their study or workplace, the US National Library of Medicine (NLM) provides free access to Medline [2] and some other databases via PubMed, and PEDro [13] is also free.

A major part of the medical profession will have access to the full-text versions of the journals through their hospital or research libraries. For those who do not have access to all licenced scientific journals, it is important to be aware of that more and more journals are becoming ‘open access’. This means that it is free to access the papers published and that the author or the author’s employer have paid a fee to make their paper freely available. Today many grant-giving bodies demand that the results of the research paid by their grants must be published as an open-access paper. Several of the old journals will therefore now provide an open-access option, if the authors or the grant-giving body

E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306 303

will pay for this. This has suddenly given a much wider free access to electronic journals. A large selection of open-access papers in health science can be searched in PubMed Central, which is a part of PubMed. To get to PubMed Central, choose PMC in the box for choosing database at the top of the front page next to the PubMed logo. When you open PubMed, this box shows ‘PubMed’, which is the entrance to Medline via PubMed. There is a choice of several databases you can search via the search host PubMed.

Some open-access journals, which may be too new to fulfil the entrance criteria for bibliographic databases such as Medline or EMBASE, will be acceptable if they at least are searchable via CrossRef [19] and have a digital object identifier (DOI) number. This will guarantee peer review and some scientific standard. All electronic scientific papers should have a DOI registration number, which is presented at the front page of the paper.

Other search tools

Bibliographic databases are the ‘safe’ places to search because the references included in these databases are from peer-reviewed journals or, in cases where books or book chapters are included, from recognised scientific publishers. However, it should be recognised that there are other limits to this ‘safe strategy’, because a useful part of the available information is published in journals that are not included in the described databases and these journals may also be peer reviewed and of perfectly acceptable quality. The major bibliographic databases concentrate on covering publications from publishers in Europe and the United States, and to some degree Japan, Australia and New Zealand, but valuable information could still be missed by searching these major bibliographic databases. The other issue is that many patients, and their relatives, nowget their information about illnesses and treatment from the Internet. It is therefore also important to keep up with what the layman’s sources of knowledge are communicating in your field in order to be ahead of your patients and to supplement your more scientific sources of medical information, before you are asked questions about a certain condition or therapy or drug treatment.

Search engines

For scientific search via Internet search engines, it is important to understand the difference in search techniques in these engines from search in bibliographic databases.When using a search engine such as Google [1] or Ixquick [25], you have to think of the main words and put them in the order of importance in the search field. Different search engines search in different ways, but most place a high importance to words appearing in headings and in the first paragraph of the text. The word placed as number one in a search will be counted as more important than the second or the third. Although it is possible to perform an ‘advanced search’, this does not make a great difference to the result and you cannot combine searches in the way described for bibliographic databases.

For scientific questions, it is recommended to search either Google Scholar [26] or Scirus [27], which aim to provide high-quality answers. Another choice could be a multi-search engine such as Ixquick [25] (this type of tool harvests results from searches in several search engines and presents them in one list) due to its large coverage.

Through search engines you will find many homepages on rheumatology subjects, including everything from stories of individual patients or from a next of kin to the information pages of patient organisations, homepages of learnt societies and lectures from university courses. You can also find online encyclopaedias such as Wikipedia [28]. Wikipedia is not peer reviewed but because it is free on the Internet and anybody can make corrections in the written articles, you can often find solid and up- to-date information. However, it is also possible for the information to be incomplete or even some- times incorrect, and you therefore should consider yourself responsible for evaluating any information that you take from Wikipedia or similar sources.

There are some important sites on the Internet if you need health statistics. The main ones are the World Health Organization’s (WHO) extended homepage [29] and free pages of useful statistics from various countries’ National Board of Health. These sources can be considered reliable and need no evaluation.

E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306304

Another important issue, well covered by the Internet, is bioethics. If you need up-to-date advice on bioethics, you can find the legal documents or interpretations of these under NLM’s Bioethics Infor- mation Resources [30] or under Council of Bioethics Europe Division [31].

Evaluation of Internet resources where peer-review is not applied

It is necessary to evaluate any found useful resource before the information provided can be taken into account. The evaluation is simple and builds on common sense:

1. Does the page cover the information you are looking for at an acceptable level? 2. What is the URL (Uniform Resource Locator) address? 3. Is the information given sufficiently complete, correct and precise? 4. Who is the creator of this page? 5. Has the page an acceptable structure? 6. If there are links, what quality are these?

If you find that a page gives you the desired information, you can check whether the address in- dicates commercial interests or a private person, or whether it takes you to a well-known university, hospital site or a similar acceptable institution. Although the medical industry has commercial in- terests, research laboratories may post acceptable information on the Internet. You just have to use your judgement about the firms’ main marketing objectives when assessing the information given.

If a page gives any information you know is incorrect, you should not use any of the information given there. You should also require some recognisable expertise of the creator of a page that you will accept as an information source. Today, it is fairly easy to create a well-structured homepage, so you may ask yourself if you can trust a pagewhere it is very difficult to find the information. Assessing links is usually easy at sites of interest for rheumatologists. There will most often be only a couple of ref- erences to peer-reviewed journals or to health-care institutions, and therewill be very few others. Only commercial pages will have advertising and similar andmust be looked uponwith some apprehension. Useful sites are homepages from patients’ organisations who, in many cases, will have professional staff providing the information on their homepages.

Books and book chapters

During your professional life, it is important to keep up with the development of the subjects in your areas, and this is often most easily done at any given time by reading the newest textbooks in the field. Another source of useful books is the many doctoral theses. To follow the books published in your field over time, you can search any medical research library’s catalogue. You may not find all published literature of interest, but you will, if you search at yearly intervals, be made aware when it is time to bring your knowledge up-to-date. If there are a couple of theses and two new textbooks published during the year, it is time to read up on the subject.

Open access has also reached the book market. You may therefore find several new subject-specific books in monograph series such as for instance InTech Press [32] or Future Medicine [33], where all books are edited by specialists and all are free to access. Older handbooks may also appear on the Internet for free. With these you must be aware that the freely available version often may be an older edition, or for book chapters, perhaps a pre-proof version with yet-to-be-corrected mistakes.

Searching for a systematic review

The most advanced type of literature search is the search necessary to be able to write a systematic review. A systematic review has to qualify for being ‘systematic’ and not being an editorial or similar, whichmeans that it has to cover ‘all’ relevant published studies in the area. The reviewhas to be based on a protocol, and this protocol has to be published via sites such as PROSPERO [34] prior to starting the search. The Cochrane Handbook [35] will guide through how such a review – and a possible planned

E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306 305

meta-analysis–has to be structured, and the (PRISMA) Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement [36] must also be considered, but this is outside the scope of this paper.

The main message when preparing a search for a systematic review is:

- Define the aim and objectives of the systematic review clearly, you are not just writing an essay. - Work hard on the search strategy, it has to be good and comprehensive, as you are bound to follow

it after publishing your protocol. - Choose all bibliographic databases you can imagine. - Search meeting abstracts, as well as databases covering protocols for studies such as Clinical

Trials.gov [17]. - Scrutinise reference lists of chosen studies and major reviews.

Summary

Keeping up with the literature is necessary in all medical practice and research to provide up-to- date diagnosis and treatment. Possessing the latest knowledge is essential if you are to find the best way of achieving the highest level of competence in your field. Defining a subject for the purpose of carrying out a literature search helps to clarify how everybody sees the problem, both broadly and in detail, and this is useful in itself. All soundly based research and innovation projects start with an information search, supplemented with further searches when new aspects appear.

The virtual library, with its many electronic search tools, may look as if it changes all the time and may deter a busy professional from carrying out thorough searches. As a general rule, there should be no worries in terms of searching. The basic design remains the same. You just have to find where everything is located.Whatever the appearance, the aim seen from the user’s point of view is the same: “Find me the relevant information in my subject over a defined period of time.” No search tool is so complicated that a personwith basic information-literacy skills and an education in the health sciences cannot work out how to use it. However, courses in information literacy are available and important, if you wish to be highly competent in this area, especially if you intend to carry out a systematic review.

To keep up with the literature, you must build searching into your work routines, and it is good practice to search at least every 6 months to keep up with your field.

Practice points

- Search the literature on a regular basis, at least every 6 months. - Define your subject using a search table. - Choose relevant databases. - Create a search strategy adjusted to the database(s) you need to search. - Search each database separately. - Search the Internet for supplementary information. - Remember to evaluate all resources acquired via an Internet search.

Conflict of interest statement

The author had no conflicts of interest concerning this work.

Acknowledgement

This work was supported by the OAK Foundation.

References

[1] Google. http://www.google.com [accessed 17.01.2013].

E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306306

*[2] PubMed. http://www.ncbi.nlm.nih.gov/pubmed [accessed 17.01.2013]. *[3] Grant MJ. How does your searching grow? A survey of search preferences and the use of optimal search strategies in the

identification of qualitative research. Health Information & Libraries Journal 2004;21:21–32. *[4] Glasziou P, Vandenbroucke J, Chalmers I. Assessing the quality of research. British Medical Journal 2004;328:39–41. [5] Hacksaw A. A concise guide to clinical trials. Oxford, UK: Wiley-Blackwell; 2009. *[6] EMBASE. http://www.embase.com [accessed 17.01.2013]. [7] Web of Science. http://thomsonreuters.com/products_services/science/science_products/a-z/web_of_science/ [accessed

17.01.2013]. [8] BiosisPreviews. http://thomsonreuters.com/products_services/science/science_products/a-z/biosis_previews/ [accessed

17.01.2013]. [9] Psycinfo. http://www.apa.org/pubs/databases/psycinfo/index.aspx [accessed 17.01.2013].

[10] The Cochrane Library. http://www.cochranelibrary.com/view/o/index.html [accessed 17.01.2013]. [11] Chemical Abstracts. http://cas.org [accessed 17.01.2013]. [12] TOXNET. http://toxnet.nlm.nih.gov/ [accessed 19.07.2008]. [13] PEDro. http://www.pedro.org.au [accessed 17.01.2013]. [14] CINAHL. http://www.ebscohost.com/academic/cinahl-plus-with-full-text/ [accessed 17.01.2013]. [15] AMED. http://www.library.nhs.uk/help/resource/amed [accessed 17.01.2013]. [16] MANTIS�. http://healthindex.com [accessed 17.01.2013]. *[17] Clinical Trials.gov. http://www.clinicaltrials.gov/ [accessed 17.01.2013]. *[18] Prospero. http://www.crd.york.ac.uk/PROSPERO/ [accessed 17.01.2013]. [19] CrossRef. http://www.crossref.org [accessed 17.01.2013]. [20] Derwent Innovations Index. http://thomsonreuters.com/products_services/legal/legal_products/a-z/derwent_

innovations_index/ [accessed 17.01.2013]. [21] Journal Citation Reports. http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_

reports/ [accessed 17.01.2013]. [22] Reference Manager. http://www.refman.com/ [accessed 17.01.2013]. [23] Endnote. http://www.endnote.com/ [accessed 17.01.2013]. [24] Procite. http://www.procite.com/ [accessed 17.01.2013]. [25] Ixquick. http://www.ixquick.com/ [accessed 17.01.2013]. [26] Google Scholar. http://scholar.google.com [accessed 17.01.2013]. *[27] Scirus. http://www.scirus.com/ [accessed 17.01.2013]. [28] Wikipedia. http://www.wikipedia.org/ [accessed 17.01.2013]. *[29] WHO. http://www.who.int/en/ [accessed 17.01.2013]. *[30] NLM’s Bioethics Information Resources. http://www.nlm.nih.gov/bsd/bioethics.html [accessed 17.01.2013]. *[31] Council of Bioethics Europe Division. http://hub.coe.int/what-we-do/health/bioethics [accessed 17.01.2013]. [32] InTech Press. http://www.intechopen.com/ [accessed 17.01.2013]. [33] Future Science Group. http://www.future-science-group.com/ [accessed 17.01.2013]. [34] PROSPERO. http://www.crd.york.ac.uk/PROSPERO/ [accessed 17.01.2013]. [35] The Cochrane Handbook for Systematic Reviews. http://www.cochrane.org/training/cochrane-handbook [accessed 17.01.

2013]. [36] Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting sys-

tematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Journal of Clinical Epidemiology 2009;62(10):e1–34.