Critique of two research articles

Michelle_Michy

Lecture8Criticalappraisalquant.pdf

Home >Education homework help >Critique of two research articles

Critical appraisal of quantitative research.

Neil O’Connell 2019 PH2600 Research Methods

Learning outcomes

By the end of the lecture students should:

 Understand the basic principles of critical appraisal of quantitative papers

https://www.pedro.org.au/english/downloads/pedro-statistics/

https://www.pedro.org.au/english/dow nloads/pedro-statistics/

Zoldan et al. Arch Phys Med Rehab 2018; 99: 1: 129-136

Critical Appraisal Skills

Is the method of research appropriate for the question?

Is the research of good quality?

Are the results relevant to your group/individual?

Is the research generalisable or sample specific?

What is the size of the beneficial/negative effect?

Are these results reliable - can I trust

them?

Do these results apply to my practice?

What do they mean for my practice?

APPROPRIATE DESIGN?

APPROPRIATE ANALYSIS?

RISKS OF BIAS?FAIR INTERPRETATION?

CLEARLY PRESENTED?

Internal validity

RELEVANT POPULATION?

RELEVANT INTERVENTION?

EFFECT SIZE MEANINGFUL?

SHOULD IT CHANGE MY PRACTICE?

External validity

Know thyself: You’re biased when you read the evidence too!

 Death Penalty evidence experiment

 Republican/ Democrat supporter experiment

 Irrationality and bias is the norm - it is a constant struggle to outwit the inner zombie!

Things you might (will) do

 Cherry pick only the evidence that supports your world view

 Ignore evidence that doesn’t

 Selectively criticise research you don’t like (Rescue Bias)

 Ignore research and wallow in comfortable “truths”!

What are my biases?

Am I being fair?

Am I prepared to accept any outcome?

Title /Abstract

Is the title clear? Does it reflect the content?

Is the abstract clear - does it fairly summarise the study and results?

Spinning a better story

 “spin” in more than one section in 40% of negative RCT reports (Boutron et al. JAMA. 2010;303(20):2058-2064).

 Rheumatology RCTs - 23% judged as having misleading conclusions in the abstract.

 The strongest predictor of misleading conclusions was a negative trial! (Mathieu Joint Bone Spine 79 (2012) 262–267)

The question

Do the authors give a fair and balanced summary of the preceding literature?

Is the question clear and well justified?

Is the choice of design appropriate for that question?

The study design

Why was this design chosen?

Is this the best design to answer this question? Is this the best design to answer this question?

What are the broad strengths and weaknesses of this design?

My question is… Look for this….

Does this intervention work? RCT, Systematic review/ meta-analysis of RCTs

Diagnosis/ Screening tests….

• Is it accurate?

• Does it improve outcomes?

Cross-sectional studies where subjects get the test & a gold standard reference.

RCTs

What is the prognosis/ natural history of a condition?

Longitudinal cohort study

Is this risk factor important? Cohort study Case-control study Cross sectional study (v exploratory)

Describe this population and the relationships within it.

Cohort study Cross-sectional study

Sampling and recruitment

What sampling strategy did they use? Probabilistic? Convenience? Is it reasonable?

What were the inclusion exclusion criteria? (reasonable? Too tight? Too loose?)

Is the final sample representative of the relevant broader population? If not how do they differ?

Is the sample size adequate?

The study process

Are the methods rigorous? Are potential biases well controlled? (details depend on the design)

Can you think of any variables that might confound the results?

Did many participants drop out? What do we know of the missing data?

Is the analysis fair and appropriate given the type of data?

THE INTERVENTION(S)

THEORY ?CLEAR

?PLAUSIBLE

CONTENT

?specific details education Exercises Modalities Clinicians Tailored

DELIVERY

SETTING DOSING FIDELITY

ADHERENCE

The results and interpretation

Are the results clearly and fairly presented (can you see the necessary data?)

Is any important information not reported?

Is the authors’ interpretation a fair/ measured reflection of the evidence?

Where do these results sit in the wider evidence base?

How big is the effect? Useful/ relevant outcomes?

Appropriate/ achievable intervention? Similar Patient Group?

Do I even have enough

information?

Conclusions

 Assessing the quality and relevance of evidence requires a broad range of skills

 It is a challenge of measured judgement

 This is the challenge set by the assignment

 You can start practising now.