Discussion Thread: AI Research Tools and Proper Use AI Research Tools
Serenity3203
Chapter7forHtdylr.pptx
PowerPoint 7 Systematic literature reviews
Chapter 7 from
How to do your literature review
This is PowerPoint 7 for Chapter 7 …
We will look at
What is a systematic literature review?
Criticisms of systematic literature review
How a systematic review works
Differences between systematic and narrative reviews
All of this is discussed in much more detail in Chapter 7 of How to Do Your Literature Review
Before we look at systematic literature review, note the following …
Systematic literature review is a specialised process which is time-consuming and is resource-intensive. It can’t easily be done by students on their own.
However:
It is useful to know about the process because you will come across systematic reviews (or reviews which claim to be systematic) in your own searches, and it is useful to understand their methods and claims.
There are elements of the process of systematic literature review that can usefully be borrowed for a student review (such as the specification of search parameters).
What is a systematic literature review?
The systematic literature review has features that make it rather different from a traditional or narrative review. In the main, these concern:
The specification at the outset of the review of a ‘protocol’, which lays out its plans and must be followed exactly.
A clear specification of the search strategy, specifying the keywords and databases to be used.
Screening of the studies discovered in (b) by using eligibility criteria (which must have been specified in the protocol) about relevance and quality.
The origins of systematic review
Systematic literature review’s claim is that it provides a systematic and reliable way to assess the quality and scale of evidence on a topic.
It owes its origins mainly to what is called ‘evidence-based medicine’.
Archie Cochrane, the originator of ‘evidence-based medicine’ argued that evidence from a specific kind of experiment should be the benchmark for quality for inclusion in a review.
That specific kind of experiment is the randomised controlled trial (RCT), wherein experimental conditions are carefully controlled and sources of bias eliminated.
Evidence from RCTs, Cochrane argued, should be systematically collated and summarised.
Criticisms
Selection bias. In their choice of question and selection of search terms, researchers may be able to skew their search toward the kind of finding they expect or hope to make, therein invalidating claims about the elimination of ‘subjectivity’.
Reviews may be vulnerable to publication bias, where studies with positive results are more likely to be published.
Lack of heterogeneity. Studies with a particular methodology may be favoured, thereby distorting analysis of a topic.
Implied or explicit downgrading or marginalisation of alternative approaches to review.
Time-consuming and resource intensive.
How does a systematic literature review work?
Plan … and put plan into a ‘protocol’, which can be scrutinized subsequently to see if researchers have followed the plan and haven’t (for instance) just cherrypicked parts of their findings or altered their approach part-way through the review to discover more interesting and publishable findings.
Search … databases using keywords and identify (usually thousands of) studies.
Screen … the studies in (2) by applying eligibility criteria (which must have been specified in the protocol) about relevance and quality.
Extract … the pre-planned data from the studies. The systematic review extracts just the key information needed to reduce a body of studies to a single account that summarises the key results.
Analyse … After searching and screening, the hope is that all or most of the relevant studies have been located and these can then be analysed to reach the overall judgement on whether the approach under investigation works. At the analysis stage, while there are several options, typically one conducts either …
a ‘narrative synthesis’, where the researcher describes and summarises each of the studies in prose, or
a ‘meta-analysis’ where the (numerical) effect size from each individual study is collected in a database and an average effect is found.
Report … The results of the analysis and the overall process of the review are reported.
Flowchart of the kinds of decisions and processes followed in systematic review
Database search with keywords
Merge
Sort for duplicates
Discard duplicates
Preselect based on titles and abstracts
Sort for duplicates
Discard irrelevant
Full text assessment and evaluation
Sort for duplicates
Discard unsatisfactory quality
Final set of papers for review
Additional search from expert advice and snowballing
Sort for duplicates
Discard irrelevant
Reviewed additional
Database search with keywords
Differences between systematic and narrative reviews
| Process | Traditional or narrative review | Systematic literature review |
| Formulate questions | Set prima facie question. | Question clearly defined from the outset and laid out in a protocol. There is no deviation from this. |
| Revise questions? | If necessary, yes, after pre-reading with key references and consideration of initial questions following brainstorming/mindmap/storyboard. | No. The assumption is that you can’t rethink and revise the question depending on what you’re finding. |
| Begin search | Eclectic use of sources – colleagues, senior researchers, personal contacts, newspapers, databases, AI tools. | Clear definition of databases, search terms and other criteria such as dates. |
| Slim down search discoveries and select literature for synthesis | Mainly based on relevance and structure of review (chronological, mixed, etc). May also use specific inclusion and exclusion criteria but open, for example, to commentary, and not just empirical work. | Using strictly defined inclusion and exclusion criteria, most importantly about relevance and nature/quality of methodology in found sources. |
| Analyse and synthesise | Emphasis on interpretation. | Emphasis on ‘data extraction’. |
| Conclude and report | | |
Summary
The aim of systematic literature review is to enable thorough search and methodical synthesis of relevant literature.
It involves a detailed, reproducible process.
It is best suited to questions involving interventions and outcomes.
It must include explicit planning, searching databases using explicit and reproducible terms, screening the findings from the search for quality and relevance, extracting relevant information, analysing that information and reporting on the analysis.
Criticism of systematic literature review centres on its claims to eliminate bias, its difficulty in incorporating studies using methods other than those favoured in evidence-based medicine, and its time-consuming and resource-intensive nature.
Literature reviews of any kind can learn from systematic literature review the importance of a comprehensive, explicitly defined search for relevant literature, in place of reliance on possibly idiosyncratic knowledge of a field.
Activity
Make sure you have access to one or more of these academic databases: PubMed, Scopus, Google Scholar. Google Scholar and PubMed should be openly available – you can search for them from Google. Scopus should be available through your university or college.
Select a research topic: choose a topic in your own area of interest and set yourself a question or thesis statement for your review.
Define keywords that target your interest.
Define inclusion and exclusion criteria: these might include publication date, study design, language, geographic location.
Search for relevant literature: Using one or any combination of your search tools – Google Scholar, PubMed or Scopus – search using your keywords. If possible, also use Boolean operators (see p. ???), and other techniques such as embedding a term in quotation marks.
You’ll emerge from your search with a great many ‘finds’ – perhaps thousands. In a systematic review, the researchers would now spend a great deal of time going through each of these to determine which satisfied their expectations on relevance and quality. You don’t have time to do this. So select 10 or so of your ‘finds’ that look interesting and write a digest of them in 100-200 words.
