Your Lecturer

Flexible, analytic approach for evaluating cost effectiveness of intervention decisions as long as effects do not occur over long time course that would require:

· Discounting of outcomes and costs

· Complicated tree structures to capture multiple health-state changes over time

10 of 51

· Developed to overcome the limitations of decision trees

· Core component: Exhaustive list of mutually exclusive "states" that a participant can occupy over time

· E.g., "healthy," "sick," or "dead"

· Each associated with a health-related quality of life (HRQOL) and potentially a cost value

· Used to evaluate quality-adjusted life years (QALYs) and cost effectiveness of different decisions

Published cost-effectiveness model comparing combination antiretroviral therapy to monotherapy for the treatment of HIV

· Used to explain structure and function of STMs

· Lists four different states

· State A: CD4 cell count >200 and <500

· State B: CD4 cell count <200

· State C: AIDS

· State D: Dead

· Explicitly model passage of time over a series of discrete time periods

· Have lengths that are chosen depending on disease and intervention under study

· Range: Short time periods (e.g., minutes in hospital ER) to years for chronic disease

· Key consideration: Whether a participant would be likely to experience more than one event before the end of the cycle

· HIV treatment case study: One-year cycle length

36 of 51

Define likelihood that an individual will move from one state to another during each cycle:

· Mutually exclusive states: Probabilities within each row must sum to 1

· Zero probability: Impossible to transition from one state to another

· Absorbing states: No possibility of leaving state (e.g., death)

39 of 51

44 of 51

· Markov models: Class of STMs with defining characteristic known as Markov property (alternatively, Markovian property or Markovian assumption)

· Assume that transition probabilities between states are independent of individual's history (i.e., "memory-less")

· E.g., probability of dying from AIDS not dependent on speed of transition from other two HIV states or length of time in AIDS state

· Markov chains: Models that also assume that transition probabilities remain constant over time

51 of 51

· Costs are generally implemented for each cycle that an individual is in a given state.

· Can also be assigned based on transitions to states (e.g., cost for transitioning to "dead" state)

· In the HIV treatment case study, cost are assigned for each year in a given state and equal for both treatment arms.

· Only cost differential: Cost of different treatments

· Only outcome measured: Life expectancy

· Each year spent in nondead state counted equally

· Can be changed to assign different health states different HRQOL in order to estimate QALYs

· Evaluates expected outcome for average (vs. single) person

· Runs a group of identical individuals through state-transition model (STM)

· Exact size of cohort irrelevant to average per-person results

· In HIV treatment case study: 1,000 individuals run through model, all starting in State A during Cycle 0 and running for 20 cycles

13 of 56

1. Calculate probability of participant or proportion of group of patients being in a given state in a given cycle.

· Performed in spreadsheet or in specialized software (e.g., TreeAge)

· Called "Markov trace"

2. Estimate expected costs and outcomes from each cycle, weighted by proportion of cohort in each state.

· When life expectancy is used: Multiply those alive in each state by one.

· When quality-adjusted life years (QALYs) are used: Multiply values by health-related quality of life (HRQOL) for each state.

14 of 56

24 of 56

40 of 56

Average per-person life years in Cycle 5:

(195 + 186 + 364) ÷ 1,000

745 ÷ 1,000 = 0.745

43 of 56

· Annual cost by state (British pounds)

· State A: £2,756

· State B: £3,052

· State C: £9,007

· Additional annual cost of treatment with monotherapy: £2,278

45 of 56

Average per-person cost in Cycle 1:

(721 × (£2,756 + 2,278) + 202 × (£3,052 + 2,278) + 67 × (£9,007 + 2,278)) ÷ 1,000 = £5,462

48 of 56

Discounting in Cycle 5:

£6,077 × 1 ÷ (1 + 0.06)5 = £4,541

56 of 56

· Membership in each state is counted either at beginning or end of each cycle.

· Yet transition into states occurs throughout cycle (i.e., at midpoint).

· Biased estimates result from ignoring difference between calculation and smooth transition.

· Addition of half of cycle at beginning of analysis corrects bias (i.e., half-cycle correction).

· Can be implemented using software solutions

· Debated in terms of its efficacy

· Easy to implement and favored by reviewers

8 of 35

Despite "memory-less" property of Markov models, time dependency can still be modeled.

· Allow transition probabilities to change depending on time cohort in model.

· View time as age-related changes in transition probabilities if cohort is all the same age.

· Increase of background mortality risk as cohort ages and faces competing risk of death.

11 of 35

· Added by creating a special health state only for a single cycle

· Can incorporate short-term changes in risks related to patient history without violating the "memory-less" property

13 of 35

24 of 35

Special category of temporary states

· Arranged with each state leading to next (i.e., like a tunnel)

· Creates series of substates

· Allows costs, health-related quality of life (HRQOL), and other transition probabilities to vary based on time spent in given state

· E.g., for models of cancer prognosis

· Called semi-Markov processes

25 of 35

35 of 35

· Become overly complicated if many different temporary states added to create memory

· Called "state explosion"

· Estimate only expected value of average participant vs. factoring in heterogeneity

· Could create separate, parallel cohorts for each type of person but would result in same state explosion

· Known as first-order Monte Carlo simulation

· Retain time cycles, transition probabilities, health states from Markov cohort models

· Follow individuals, not cohorts

· Evaluate costs and effects for simulated individuals, estimate average effects

· Allow transition probabilities, other aspects to vary based on individual characteristics, history

12 of 22

· Require substantial data

· Insufficient data assumptions may add to complexity, detract from meaning

· Increase computational burden of running models

· Limitation declining due to increased computer power and cloud access

20 of 22

· Previous models assume individual outcomes do not influence others.

· This is untrue for infectious, chronic, and social-norm-component diseases.

· New infections depend on pool of infected individuals.

· Proportions change in epidemics and should be included in the model.

· This is relevant to evaluating vaccination, screening, and transmission models.

· International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and Society for Medical Decision Making (SMDM) collaborated on 2010 Good Research Practices in Modeling Task Force

· Goal: Create consensus-based guidelines for modeling studies in healthcare

· Focus: Use of state-transition models

13 of 70

· Risk reduction

· Events occurring prior to disease onset

· Cohort based on individuals free of the disease and complications

18 of 70

· Two types

· One-time: Newborns or genetic screening

· Repeated (interval): HPV or cervical cancer

· Evaluating strategies with respect to type and sequence of testing, diagnostic modes, screening interval, age at which screening begins and ends

23 of 70

· Finding optimal diagnostic strategies

· Focusing on differences in test performance, sequence of different testing strategies, cutoff scores for positivity

28 of 70

· Evaluating different treatment options

· Characterizing disease natural history, expected prognosis in the absence of treatment, and characterization of treatment effects and cost

36 of 70

· If possible, cohort simulations should be chosen due to:

· Transparency

· Efficiency

· Ease of debugging

· Ability to conduct value-of-information analyses

38 of 70

· If valid representation of decision problems would lead to an unmanageable number of health states, then individual-level models are recommended.

· Do not sacrifice validity for simplicity.

44 of 70

Best Practice:

· Defined by demographic and clinical characteristics that affect transition probabilities, state values

· Outputs from a single cohort analysis allow comparison of alternative strategies for cohort.

· If optimal strategies vary by characteristics of subgroups, they should be modeled in parallel, not same cohort.

51 of 70

Best Practice:

Specification: Biological or theoretical understanding of disease or condition

· Identify states that reflect the disease-health process in the absence of intervention.

· Capture benefit or harm of intervention, natural disease history.

· Combine decision tree and state-transition models when short-term states precede longer-term transition process.

57 of 70

· Transition probabilities and intervention effects should come from the most representative data sources.

· Natural history transition probabilities use population-based epidemiological studies.

· Control arms of intervention studies are less generalizable.

63 of 70

Group should be large enough to generate stable estimates.

· Compare variance across model runs with same number of individuals to expected differences between strategies.

· Between-model-run variance due to random differences in sampled microsimulation populations should be smaller than smallest expected difference between strategies.

68 of 70

· 30 different guidelines

Childhood Obesity Intervention Cost Effectiveness Study

· Goal: Identify and prioritize cost-effective strategies for prevention.

· One in three adults, one in five children affected

· Higher healthcare costs and increased morbidity

· How to choose which policy or program to implement?

· Cost-effectiveness analysis (CEA) of options

· CHOICES provides menu of options for policy makers.

16 of 48

· Multidisciplinary approaches to public health:

· Regulatory: Tax on sugary drinks

· School policy: Increase in physical education

· Clinical approach: Gastric banding and weight loss surgeries

· Modeling offers a unified framework for evaluation and comparison.

· Structured, transparent results sensitive to uncertainty and different assumptions

· How are interventions being compared?

25 of 48

· Compare intervention scenario with what would have happened without the intervention

· Example: Increasing prevalence of obesity complicates results

· Not a comparison with current rates, a comparison with what rates would have been without an intervention

· CHOICES microsimulation (Monte Carlo simulation at the individual level)

· Counterfactual comparison done person by person

· Individual's life simulated

· Intervention applied and differences noted

27 of 48

38 of 48

44 of 48

47 of 48

· Compare costs and effects between scenarios.

· Measure cost effectiveness.

· Determine which interventions offer value for money.

48 of 48

· Cross-sectional population: A collection of virtual individuals as they exist at one point in time

· Should incorporate all relevant domains

· Linked data systems in Netherlands and Scandinavia, fragmented in United States

· US data available but not in one place

· Statistical matching: The "borrowing" of information from different data sets and synthesizing it to create a virtual person in multiple dimensions

8 of 38

18 of 38

23 of 38

34 of 38

37 of 38

· Important to have the right starting place

· Make counterfactual comparison for each intervention

· Compare cost effectiveness of each intervention to the most realistic baseline

38 of 38

· CHOICES model 2015–2025

· Burn-in period to 2015

· Run model for 10 years

· Long enough to measure longer-term outcomes

· Not so long that many large assumptions are made

· Attribute: Height and weight change over time

9 of 66

12 of 66

· Use a statistical matching approach to pair segments with individuals with similar height and weight segments.

· Use a minimization technique to find similar segments and stitch them together over time.

· Model out how height and weight change for a virtual individual as they age.

13 of 66

21 of 66

· Observed data sets 10–15 years old

· Weight trajectories need adjustment to reflect current trends

· Calibration: Adjust parameters of model to realign output

· Fit regression models to mean BMI and obesity prevalence in NHANES 1999-2012

· Calibrate weight trajectories so aggregate obesity and mean BMI population trends match regression-based models

· Reasonable population-level trends

· Heterogeneity in height and weight growth maintained

22 of 66

28 of 66

· Look at joint distribution of smoking and obesity at the population level.

· When matching to Behavioral Risk Factor Surveillance System (BFRSS) 2011, pull in joint distribution of smoking and obesity at the individual level.

· Simulate smoking trajectories based on National Health Interview Surveys (NHIS) 1965–2009.

30 of 66

31 of 66

35 of 66

40 of 66

· The counterfactual scenario assumes obesity costs would be lower after an intervention.

· However, non-obesity-related costs might go up as individuals live long enough to have other health-related expenditures.

· Competing risks should stay in the model to avoid overestimating effect of an intervention.

44 of 66

· National Health and Nutrition Examination Survey (NHANES) data on calories consumed from different food groups

· No longitudinal (repeated) measures

· Make assumptions on how these change over time

53 of 66

· Use the model to calculate percentile of consumption, ranked from low to high consumption.

· Assume behavior tracks over time.

· Put a bandwidth around the percentile, and the next year sample a new percentile within it.

· Use this percentile to look at what the implied consumption is at the new age.

· Repeat process as people in the model age.

· Maintains distribution of consumption observed in the population

· Represents heterogeneity at the individual level

· Lets behaviors track over time in a realistic way

56 of 66

· 2020 population will be different from the 2010 population

· More people, more different kinds of people

59 of 66

64 of 66

· Repeat the process 50 times to capture uncertainty in the matching process.

· Create 50 populations from cross-sectional forward, independently and together.

· Explore the range of possible scenarios.

· Get a realistic comparator with which to evaluate potential interventions.

65 of 66

· Repeat the process 50 times to capture uncertainty in the matching process.

· Create 50 populations from cross-sectional forward, independently and together.

· Explore the range of possible scenarios.

· Get a realistic comparator with which to evaluate potential interventions.

· Findings more robust to changes in underlying parameters

66 of 66

6 of 44

1. Reach

2. Cost

3. Effect

11 of 44

· Who is exposed to the intervention? Who is the target?

· Example: States or counties; children in public schools; families on the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) or the Supplemental Nutrition Assistance Program (SNAP)

· Define interventions at different levels, e.g., geographic and individual characteristics from different domains.

· Create tree of recruitment probabilities to select individuals into the intervention.

19 of 44

· Societal perspective

· Account for opportunity cost of resources

· Geographic vs. individual

· Geographic: E.g., passing policy at state or county level (cost is the same regardless of number of people in intervention)

· Individual: E.g., training a teacher, sending a mailing, buying equipment

· Start-up vs. recurrent

· Doesn't change cost effectiveness but useful for budgeting and schedules

21 of 44

· Capital is amortized over useful life.

· Costs are standardized to 2014 USD using Consumer Price Index.

26 of 44

· Modeled through individual weight trajectories

· Literature may report effect using different metrics

· BMI

· Behavior, e.g., purchasing patterns, diet, physical activity

33 of 44

· Weight changes nonlinearly and can take a long time.

· Adults

· One year for half of the effect

· Up to three years for full effect

· Children

· 12–18 months

34 of 44

41 of 44

· There is uncertainty about reach, cost, and effect.

· Account for this by specifying distributions about uncertain parameters.

· Example: Information from a clinical trial using confidence intervals about effect size on weight and/or BMI

· Result: A best guess with a range of uncertainty

· Model: Sample from normal distribution using different values to explore the effect on results

· Costs: Typically skew right, sample from gamma or log-normal distribution

44 of 44

· Specify interventions as realistically as possible.

· Incorporate relevant sources of uncertainty.

· Inform policy makers of best-guess results of intervention implications.

· First-order uncertainty (stochastic)

· Some probabilities are known but not what happens to individuals in the model

· Eliminated when large enough sample size is simulated (1 million)

· Second-order uncertainty (parameter or systematic)

· Uncertainty about model parameters

· Model run repeatedly with different parameters

· Millions of individuals, thousands of repetitions, sampling at random

· Computationally intensive

18 of 44

· Decide how parameters relate to each other.

· Specify if parameters are independent or linked within intervention.

· Example: Point estimate for effect is different according to gender.

· Sampling the high end of one population and low end of another cancels out the difference.

· Link parameters to avoid underestimating variance.

· Convey appropriate uncertainty; don't imply realistically precise results.

25 of 44

· Distribution of outcomes needs to be distilled into a simple message.

· Take expected value (mean) as point estimate.

· Rank each outcome and take 2.5 and 97.5 percentiles to represent uncertainty intervals.

· Interpretation: 95% of the time the outcome will fall within this interval.

· It may be more complicated to report point estimates and uncertainty for ratio measures.

· Example: ICER = Cost/QALY

31 of 44

Point Estimate:

1. Expected value of ratios calculated within each iteration

· Calculate ratio in each iteration, order them, report 95% uncertainty interval based on the distribution.

2. Take a ratio of expected value across all iterations

· If results were plotted in 2D on cost-effectiveness plane, with a line drawn to center of mass, the slope of the line would be the ICER.

38 of 44

· Overview of analytic framework

· Importance of counterfactual comparisons

· Statistical matching and synthesizing data to create cross-sectional populations

· Other data sets and assumptions about population changes over time (longitudinal)

· Running models and distilling results into digestible indicators

· Conveying uncertainty

44 of 44

· You should:

· Understand the CHOICES model and how microsimulation modeling is used in economic evaluation

· Appreciate the challenges and assumptions

· Understand the right questions to ask of models

· See the value of a consistent framework that allows us explore scenarios in many domains

· Be able to provide guidance as to what are likely to be good value investments in health