Answer five questions in requirement . Each question requires more than 300 words， a total of 1,500 words
ccc777
TOPIC9_Aggregate_SLIDES.pptxEvidence-based Management MGMT 7250
TOPIC 9: Aggregating evidence
1
Formal methods to aggregate & visual tools to map what we know
Truth, proof, & evidence
The outcome of an evidence-based process is an estimate of a probability given the available evidence, not the ‘truth’.
The probability of a claim, assumption, or hypothesis being true is always conditional on the available evidence.
Evidence only exists in the context of a claim or an assumption.
Data only become evidence when they stand in a ‘testing’ relationship with a claim or an hypothesis.
Evidence is always evidence for (or against) something.
Formal methods to aggregate
Qualitative data
Content analysis
Thematic analysis
Quantitative data
Descriptive statistics.
Inferential statistics for hypothesis testing.
Formal methods to aggregate scientific findings
Meta-analysis
Systematic reviews
Expedient systematic reviews:
Rapid evidence assessments
CATs
Conceptual maps
Source: CIPD 2017
Variants of snake-oil charts – Example 1
SNAKE OIL?
Bayesian Thinking
Probability of hypothesis being true depends on the trustworthiness of evidence
Bayes’s Theorem:
Aggregating across sources
It can be the case that when faced with a problem, experts say one thing, and the scientific evidence says something different. What do we do when evidence says different things?
In the first instance, it is always better to lean more towards the most trustworthy evidence which is likely to be scientific evidence. Another option is to generate local evidence and pilot test or run an experiment to test an intervention in a local setting. Yet another option, particularly when there are time constraints, is to use Bayes Theorem to reconcile conflicting evidence.
11
Informative priors & alternative hypotheses
The following table helps us think about how to quantify the trustworthiness of evidence (P(E|Htrue); informative prior), and the likelihood of an alternative explanation for the effect found (P(E|Hfalse); alternative hypothesis).
16
Aggregating evidence involves weighing and pulling together different sources of evidence to address a particular question. In addressing a question relating to a problem or practical issue, it is important to gravitate towards the strongest or most trustworthy evidence – the best available evidence.
Developing a conceptual map or framework can be helpful for synthesis as it can illustrate causal paths, mediating and moderating factors, critical antecedents and outcomes relevant to a problem and question.
Source: Chartered Institute of Personnel and Development Rapid Evidence Assessment on performance appraisal.
Aggregating evidence from different sources
17
17
Advantates of organisational data
Non-reactive measures
Allow analysis of actual patterns
Can analyse change: if data exist for many years
Can compare different organisational contexts and situations
Increased objectivity and transparency
Reduced research costs
Conflicting evidence
What to do when evidence tells us different things
It can be the case that when faced with a problem, experts say one thing, and the scientific evidence says something different. What do we do when evidence says different things?
In the first instance, it is always better to lean more towards the most trustworthy evidence which is likely to be scientific evidence. Another option is to generate local evidence and pilot test or run an experiment to test an intervention in a local setting. Yet another option, particularly when there are time constraints, is to use Bayes Theorem to reconcile conflicting evidence.
Activity – Worksheet
Claim made by your Human Resources director: If you substantially increase job satisfaction, productivity will increase by at least 10% (professional experience and judgment). Indeed, most senior managers in your organisation believe the claim is plausible.
How trustworthy is the judgement of the executive managers that the claim is plausible ?
The probability of a hypothesis (claim, assumption) being true given the evidence depends both on the likelihood (trustworthiness) of the evidence being found and prior knowledge.
20
