Epidemiology quiz

Validity of Diagnostic and Screening Tests

Jennifer Deal, PhD Johns Hopkins University

The material in this video is subject to the copyright of the owners of the material and is being provided for educational purposes under rules of fair use for registered students in this course only. No additional copies of the copyrighted work may be made or distributed.

Defining Sensitivity and Specificity in the Context of Screening Tests

3

Recall…

Three types of prevention

Type of prevention

Definition Examples

Primary Preventing the initial development of a disease

Immunization, reducing exposure to a risk factor

Secondary Early detection of existing disease to reduce severity and complications

Screening for cancer

Tertiary Reducing the impact of the disease

Rehabilitation for stroke

4

How Can We Distinguish between Persons Who Have a Disease and Persons Who Don’t?

► Screening: The presumptive identification of unrecognized disease or defect by the application of tests, examinations, or other procedures which can be applied rapidly. Screening tests sort out apparently well persons who probably have a disease from those who probably do not. A screening test is not meant to be diagnostic. Persons with positive or suspicious findings must be referred to their physicians for diagnosis and necessary treatment…

Source: Porta, M. A dictionary of epidemiology (6th ed.).

5

How Do We Evaluate a Screening or Diagnostic Test?

► The characteristics of a screening test must include accuracy, estimates of yield, precision, reproducibility, sensitivity, and validity

6

Evaluating a Screening or Diagnostic Test: Reliability

► The characteristics of a screening test must include accuracy, estimates of yield, precision, reproducibility,* sensitivity, and validity

*Precision and reproducibility ← reliability

7

Definition of Validity

► Validity: the ability of a test to distinguish between individuals who have a disease and individuals who do not have the disease

8

Components of Validity

► Validity: the ability of a test to distinguish between individuals who have a disease and individuals who do not have the disease

► The components of validity 1. Sensitivity 2. Specificity

9

Definition of Sensitivity

► Validity: the ability of a test to distinguish between individuals who have a disease and individuals who do not have the disease

► The components of validity 1. Sensitivity: ability of a test to correctly identify individuals who have the disease 2. Specificity

10

Definition of Specificity

► Validity: the ability of a test to distinguish between individuals who have a disease and individuals who do not have the disease

► The components of validity 1. Sensitivity: ability of a test to correctly identify individuals who have the disease 2. Specificity: ability of a test to correctly identify individuals who do not have the

disease

11

Concept of Sensitivity and Specificity—1

► Assume a population of 1,000 individuals, of whom…

100 have the disease 900 do not have the disease

► A screening test is used to identify individuals with and without the disease

12

Concept of Sensitivity and Specificity—2

► Assume a population of 1,000 individuals, of whom…

900 do not have the disease

13

Concept of Sensitivity and Specificity—3

► Assume a population of 1,000 individuals, of whom…

14

Concept of Sensitivity and Specificity—4

15

Concept of Sensitivity and Specificity—5

16

Concept of Sensitivity and Specificity—6

17

Concept of Sensitivity and Specificity—7

18

Sensitivity: The Ability of the Test to Correctly Identify Individuals Who Have the Disease—1

► Sensitivity: the ability of the test to correctly identify individuals who have the disease

19

Sensitivity: The Ability of the Test to Correctly Identify Individuals Who Have the Disease—2

► Sensitivity: the ability of the test to correctly identify individuals who have the disease

► 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 80

20

Sensitivity: The Ability of the Test to Correctly Identify Individuals Who Have the Disease—3

► Sensitivity: the ability of the test to correctly identify individuals who have the disease

► 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 80 𝟏𝟏𝟏𝟏𝟏𝟏

21

Sensitivity: The Ability of the Test to Correctly Identify Individuals Who Have the Disease—4

► Sensitivity: the ability of the test to correctly identify individuals who have the disease

► 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 80 𝟏𝟏𝟏𝟏𝟏𝟏

= 0.80 = 80%

22

Specificity: The Ability of the Test to Correctly Identify Individuals Who Do Not Have the Disease—1

► Specificity: the ability of the test to correctly identify individuals who do not have the disease

23

Specificity: The Ability of the Test to Correctly Identify Individuals Who Do Not Have the Disease—2

► Specificity: the ability of the test to correctly identify individuals who do not have the disease

► 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 800

24

Specificity: The Ability of the Test to Correctly Identify Individuals Who Do Not Have the Disease—3

► Specificity: the ability of the test to correctly identify individuals who do not have the disease

► 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 800 𝟗𝟗𝟏𝟏𝟏𝟏

25

Specificity: The Ability of the Test to Correctly Identify Individuals Who Do Not Have the Disease—4

► Specificity: the ability of the test to correctly identify individuals who do not have the disease

► 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 800 𝟗𝟗𝟏𝟏𝟏𝟏

= 0.89 = 89%

26

Formulas for Calculating Sensitivity and Specificity

► 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑇𝑇𝑇𝑇 𝑇𝑇𝑇𝑇+𝐹𝐹𝐹𝐹

► 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑇𝑇𝐹𝐹 𝑇𝑇𝐹𝐹+𝐹𝐹𝑇𝑇

The material in this video is subject to the copyright of the owners of the material and is being provided for educational purposes under rules of fair use for registered students in this course only. No additional copies of the copyrighted work may be made or distributed.

Effect of Choosing Different Cutpoints to Define a Positive Test Results for Continuous Measures

2

Title: Purified Protein Derivative (PPD) Skin Test for Tuberculosis Infection

Source: Berrien County, Michigan, government website.

3

Distribution of Tuberculin Reactions

Source: Adapted from Edwards, L. B., Palmer, C.E., & Magnus, K. (1953). BCG vaccination: Studies by the WHO Tuberculosis Research Office, Copenhagen. WHO Monograph No. 12. WHO: Geneva.

4

Distribution of Systolic Blood Pressure (mmHg), Multiple Risk Factor Intervention Trial (MR FIT)

Data source: Stamler, J., Stamler, R., & Neaton, J. D. (1993). Blood pressure, systolic and diastolic, and cardiovascular risks: US population data. Arch Intern Med, 153(5), 598–615.

5

Distribution of Blood Sugar Levels in Hospital Patients with and without Diabetes

Source: Gordis. (5th ed.).

6

Example 1: Blood Sugar Cutpoint Set to ≥80 mg/dL

Source: Gordis. (5th ed.).

7

Example 1: Blood Sugar Cutpoint Set to ≥80 mg/dL, Negative Test Results

Source: Gordis. (5th ed.).

8

Example 1: Blood Sugar Cutpoint Set to ≥80 mg/dL, Positive Test Results

Source: Gordis. (5th ed.).

9

Example 1: Blood Sugar Cutpoint Set to ≥80 mg/dL, Validity

► 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 28 28+0

= 100%

► 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 39 39+181

= 18%

Sensitivity = TP / TP + FN; Specificity = TN / TN + FP

10

Example 2: Blood Sugar Cutpoint set to ≥200 mg/dL

Source: Gordis. (5th ed.).

11

Example 2: Blood Sugar Cutpoint Set to ≥200 mg/dL, Validity

► 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 11 11+17

= 39%

► 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 212 212+0

= 100%

Source: Gordis. (5th ed.).

12

Comparing Cutpoints—1

► Cutpoint of ≥80 mg/dL ► Sensitivity = 100% (no FN) ► Specificity = low (lots of FP)

► Cutpoint of ≥200 mg/dL ► Sensitivity = low (lots of FN) ► Specificity = 100% (no FP)

13

Comparing Cutpoints—2

► Cutpoint of ≥80 mg/dL ► Sensitivity = 100% (no FN) ► Specificity = low (lots of FP)

► Cutpoint of ≥200 mg/dL ► Sensitivity = low (lots of FN) ► Specificity = 100% (no FP)

► Which cutpoint is better?

14

Comparing Cutpoints—3

► Cutpoint of ≥80 mg/dL ► Sensitivity = 100% (no FN) ► Specificity = low (lots of FP)

► Cutpoint of ≥200 mg/dL ► Sensitivity = low (lots of FN) ► Specificity = 100% (no FP)

► Which cutpoint is better? ► It depends on the importance of FP and FN

15

Comparing Cutpoints—4

► Cutpoint of ≥80 mg/dL ► Sensitivity = 100% (no FN) ► Specificity = low (lots of FP)

► Consequences of FP ► Emotional cost ► Financial cost to re-test

► Cutpoint of ≥200 mg/dL ► Sensitivity = low (lots of FN) ► Specificity = 100% (no FP)

► Consequences of FN ► Missed opportunity to treat

► Which cutpoint is better? ► It depends on the importance of FP and FN

16

Effect of Choosing Different Cutpoints for a Positive Test Result for Diabetes

Source: Gordis. (5th ed.).

17

High Cutpoint

Source: Gordis. (5th ed.).

18

Low Cutpoint

Source: Gordis. (5th ed.).

19

But in the Real World, We Don’t Know Who Has Diabetes and Who Does Not

Source: Gordis. (5th ed.).

20

Persons with and without Diabetes Are Mixed Together in a Population

Source: Gordis. (5th ed.).

21

The Choice of Cutpoint Has Real Consequences in the Real World

Source: Gordis. (5th ed.).

22

Diagnosing Diabetes— Criteria for the Diagnosis of Diabetes

► FPG ≥ 126 mg/dL (7.0 mmol/L). Fasting is defined as no caloric intake for at least 8 h.* or

► 2-hPG ≥ 200 mg/dL (11.1 mmol/L) during OGTT. The test should be performed as described by the WHO, using a glucose load containing the equivalent of a 75-g anhydrous glucose dissolved in water.* or

► A1C ≥ 6.5% (48 mmol/mol). The test should be performed in a laboratory using a method that is NGSP certified and standardized to the DCCT assay.* or

► In a patient with classic symptoms of hyperglycemia or hyperglycemic crisis, a random plasma glucose ≥ 200 mg/dL (11.1 mmol/L)

*In the absence of unequivocal hyperglycemia, results should be confirmed by repeat testing Source: American Diabetes Association. (2018). Diabetes Care, 41(Suppl 1), S13–S27. http://dx.doi.org/10.2337/dc18-S002

23

Screening for Diabetes

Population Recommendation Grade

Adults aged 40 to 70 years who are overweight or obese

The USPSTF recommends screening for abnormal blood glucose as part of cardiovascular risk assessment in adults aged 40 to 70 years who are overweight or obese. Clinicians should offer or refer patients with abnormal blood glucose to intensive behavioral counseling interventions to promote a healthful diet and physical activity.

B This recommendation applies to adults aged 40 to 70 years who are seen in primary care settings and do not have obvious symptoms of diabetes. Persons who have a family history of diabetes, have a history of gestational diabetes or polycystic ovarian syndrome, or are members of certain racial/ethnic groups (that is, African Americans, American Indians or Alaskan Natives, Asian Americans, Hispanics or Latinos, or Native Hawaiians or Pacific Islanders) may be at increased risk for diabetes at a younger age or at a lower body mass index. Clinicians should consider screening earlier in persons with 1 or more of these characteristics.

Source: US Preventive Services Task Force website. Accessed December 10, 2018, at https://www.uspreventiveservicestaskforce.org/Page/Document/RecommendationStatementFinal/screening-for-abnormal-blood-glucose-and-type-2-diabetes

The material in this video is subject to the copyright of the owners of the material and is being provided for educational purposes under rules of fair use for registered students in this course only. No additional copies of the copyrighted work may be made or distributed.

Use of Multiple Tests

2

Thought Experiment—1

► You are a public health worker in a clinic in the field

► You have two screening tests for a disease ► First test: Sensitivity = 70%, Specificity = 80% ► Second test: Sensitivity = 90%, Specificity = 90%

► It is much more important to minimize false positives than to catch all cases of a disease

3

Thought Experiment—2

► Minimize false positives (maximize specificity)

4

Thought Experiment—3

► Minimize false positives (maximize specificity)

► Consequences of FP ► Emotional cost ► Financial cost to re-test ► More invasive test

5

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—1

► Test 1

6

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—2

► Test 1

7

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—3

► Test 1

► 10,000 × 0.05 = 500

► With a 5% prevalence, 500 of the 10,000 people have disease

8

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—4

► Test 1

► 500 people have disease

► How many people do not have the disease?

9

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—5

► Test 1

► 500 people have disease

► How many people do not have the disease

► 10,000 − 500 = 𝟗𝟗, 𝟓𝟓𝟓𝟓𝟓𝟓

10

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—6

► Test 1 ► Sensitivity = 70% ► Specificity = 80%

11

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—7

► Test 1 ► Sensitivity = 70% ► Specificity = 80%

12

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—8

► Test 1 ► Sensitivity = 70% ► Specificity = 80%

13

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—9

► Test 1 ► Sensitivity = 70% ► Specificity = 80%

14

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—10

► Test 1 ► Sensitivity = 70% ► Specificity = 80%

15

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—11

► Test 1 ► Sensitivity = 70% ► Specificity = 80%

16

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—12

► Test 1 ► Sensitivity = 70% ► Specificity = 80%

► Now let’s take only the people who tested positive on Test 1 and give them a second test

► This is called sequential testing

17

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—13

► Test 1 ► Sensitivity = 70% ► Specificity = 80%

► Test 2

18

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—14

► Test 1 ► Sensitivity = 70% ► Specificity = 80%

► Test 2 ► Sensitivity = 90% ► Specificity = 90%

19

A Possible Solution (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—15

► Test 1 ► Sensitivity = 70% ► Specificity = 80%

► Test 2 ► Sensitivity = 90% ► Specificity = 90%

20

Sequential Testing: Net Sensitivity—1

► Net sensitivity

► Proportion of those with the disease who test positive on both Test 1 and Test 2

► 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑃𝑃𝑃𝑃𝑇𝑇𝑃𝑃𝑇𝑇𝑃𝑃𝑃𝑃𝑇𝑇 𝑃𝑃𝑜𝑜 𝑩𝑩𝑩𝑩𝑩𝑩𝑩𝑩 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

𝑇𝑇𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇 𝑤𝑤𝑃𝑃𝑇𝑇𝑤 𝐷𝐷𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

21

Sequential Testing: Net Sensitivity—2

► Test 1 ► Test 2

22

Sequential Testing: Net Sensitivity—3

► Net sensitivity

► Proportion of those with the disease who test positive on both Test 1 and Test 2

► 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑃𝑃𝑃𝑃𝑇𝑇𝑃𝑃𝑇𝑇𝑃𝑃𝑃𝑃𝑇𝑇 𝑃𝑃𝑜𝑜 𝑩𝑩𝑩𝑩𝑩𝑩𝑩𝑩 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

𝑇𝑇𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇 𝑤𝑤𝑃𝑃𝑇𝑇𝑤 𝐷𝐷𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

► 315 500

= 0.63 = 63%

23

Sequential Testing: Net Sensitivity—4

► Net sensitivity

► Proportion of those with the disease who test positive on both Test 1 and Test 2

► 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑃𝑃𝑃𝑃𝑇𝑇𝑃𝑃𝑇𝑇𝑃𝑃𝑃𝑃𝑇𝑇 𝑃𝑃𝑜𝑜 𝑩𝑩𝑩𝑩𝑩𝑩𝑩𝑩 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

𝑇𝑇𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇 𝑤𝑤𝑃𝑃𝑇𝑇𝑤 𝐷𝐷𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

► 315 500

= 0.63 = 63%

● This number is lower than the sensitivity of either Test 1 or Test 2. Net sensitivity is decreased.

24

Sequential Testing: Net Specificity—1

► Net specificity

► Proportion of those without the disease who test negative on either Test 1 or Test 2

► 𝑁𝑁𝑇𝑇𝑁𝑁𝑇𝑇𝑇𝑇𝑃𝑃𝑃𝑃𝑇𝑇 𝑃𝑃𝑜𝑜 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 1 + 𝑁𝑁𝑇𝑇𝑁𝑁𝑇𝑇𝑇𝑇𝑃𝑃𝑃𝑃𝑇𝑇 𝑃𝑃𝑜𝑜 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 2

𝑇𝑇𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇 𝑤𝑤𝑃𝑃𝑇𝑇𝑤𝑃𝑃𝑤𝑤𝑇𝑇 𝐷𝐷𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

25

Sequential Testing: Net Specificity—2

► Test 1 ► Test 2

26

Sequential Testing: Net Specificity—3

► Test 1 ► Test 2

27

Sequential Testing: Net Specificity—4

► Net specificity

► Proportion of those without the disease who test negative on either Test 1 or Test 2

► 𝑁𝑁𝑇𝑇𝑁𝑁𝑇𝑇𝑇𝑇𝑃𝑃𝑃𝑃𝑇𝑇 𝑃𝑃𝑜𝑜 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 1 + 𝑁𝑁𝑇𝑇𝑁𝑁𝑇𝑇𝑇𝑇𝑃𝑃𝑃𝑃𝑇𝑇 𝑃𝑃𝑜𝑜 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 2

𝑇𝑇𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇 𝑤𝑤𝑃𝑃𝑇𝑇𝑤𝑃𝑃𝑤𝑤𝑇𝑇 𝐷𝐷𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

► 7,600+1,710

9,500 = 98%

► Net specificity is increased

28

Thought Experiment

► You are a public health worker in a clinic in the field

► You have two screening tests for a disease ► First test: Sensitivity = 70%, Specificity

= 80% ► Second test: Sensitivity = 90%,

Specificity = 90%

► It is much more important to minimize false positives (maximize specificity) than to catch all cases of a disease

► Consequences of FP ► Emotional cost ► Financial cost to re-test ► More invasive test

29

Thought Experiment

► You are a public health worker in a clinic in the field

► You have two screening tests for a disease ► First test: Sensitivity = 70%, Specificity

= 80% ► Second test: Sensitivity = 90%,

Specificity = 90%

► It is much more important to minimize false positives than to catch all cases of a disease (maximize sensitivity)

► Consequences of FN ► Missed opportunity to treat

30

A Possible Solution: Simultaneous Testing (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—1

► Test 1 ► Sensitivity = 70% ► Specificity = 80%

► Test 2 ► Sensitivity = 90% ► Specificity = 90%

31

A Possible Solution: Simultaneous Testing (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—2

► Test 1 ► Sensitivity = 70% ► Specificity = 80%

► Test 2 ► Sensitivity = 90% ► Specificity = 90%

32

Simultaneous Testing: Net Sensitivity—1

► Net sensitivity

► Proportion of those with the disease who test positive on either Test 1 and Test 2

► 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑃𝑃𝑃𝑃𝑇𝑇𝑃𝑃𝑇𝑇𝑃𝑃𝑃𝑃𝑇𝑇 𝑃𝑃𝑜𝑜 𝑬𝑬𝑬𝑬𝑩𝑩𝑩𝑩𝑬𝑬𝑬𝑬 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

𝑇𝑇𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇 𝑤𝑤𝑃𝑃𝑇𝑇𝑤 𝐷𝐷𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

33

Simultaneous Testing: Net Sensitivity—2

► Net sensitivity

► Proportion of those with the disease who test positive on either Test 1 and Test 2

► 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑃𝑃𝑃𝑃𝑇𝑇𝑃𝑃𝑇𝑇𝑃𝑃𝑃𝑃𝑇𝑇 𝑃𝑃𝑜𝑜 𝑬𝑬𝑬𝑬𝑩𝑩𝑩𝑩𝑬𝑬𝑬𝑬 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

𝑇𝑇𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇 𝑤𝑤𝑃𝑃𝑇𝑇𝑤 𝐷𝐷𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 → Don’t double count!

34

Example: Simultaneous Testing (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—1

► Test 1 ► Test 2

35

Simultaneous Testing: Net Sensitivity—3

36

Simultaneous Testing: Net Sensitivity—4

► 350 test positive on Test 1

► Of the 350, how many would test positive on Test 2?

37

Simultaneous Testing: Net Sensitivity—5

► 350 test positive on Test 1

► Of the 350, how many would test positive on Test 2?

► Multiply 350 by the sensitivity of Test 2

► 350 × 0.90 = 𝟑𝟑𝟑𝟑𝟓𝟓

► 315 people correctly tested positive on both Test 1 and Test 2

38

Simultaneous Testing: Net Sensitivity—6

► Net sensitivity

► Proportion of those with the disease who test positive on either Test 1 and Test 2

► 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑃𝑃𝑃𝑃𝑇𝑇𝑃𝑃𝑇𝑇𝑃𝑃𝑃𝑃𝑇𝑇 𝑃𝑃𝑜𝑜 𝑬𝑬𝑬𝑬𝑩𝑩𝑩𝑩𝑬𝑬𝑬𝑬 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

𝑇𝑇𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇 𝑤𝑤𝑃𝑃𝑇𝑇𝑤 𝐷𝐷𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 → Don’t double count!

► 350+450−315

500 = 97%

► Net sensitivity is increased

39

Simultaneous Testing: Net Specificity—1

► Net specificity

► Proportion of those without the disease who test negative on both Test 1 and Test 2

► 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑁𝑁𝑇𝑇𝑁𝑁𝑇𝑇𝑇𝑇𝑃𝑃𝑃𝑃𝑇𝑇 𝑃𝑃𝑜𝑜 𝑩𝑩𝑩𝑩𝑩𝑩𝑩𝑩 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

𝑇𝑇𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇 𝑤𝑤𝑃𝑃𝑇𝑇𝑤𝑃𝑃𝑤𝑤𝑇𝑇 𝐷𝐷𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 → Only interested in the overlap!

40

Example: Simultaneous Testing (Assume a Population of 10,000 People with a Disease Prevalence = 5%)—2

► Test 1 ► Sensitivity = 70% ► Specificity = 80%

► Test 2 ► Sensitivity = 90% ► Specificity = 90%

41

Simultaneous Testing: Net Specificity—2

► 7,600 test negative on Test 1

► Of the 7,600, how many would test negative on Test 2?

► Multiply 7,600 by the specificity of Test 2

► 7,600 × 0.90 = 𝟔𝟔, 𝟖𝟖𝟖𝟖𝟓𝟓

► 6,840 people correctly tested negative on both Test 1 and Test 2

42

Simultaneous Testing: Net Specificity—3

► Net specificity

► Proportion of those without the disease who test negative on both Test 1 and Test 2

► 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑁𝑁𝑇𝑇𝑁𝑁𝑇𝑇𝑇𝑇𝑃𝑃𝑃𝑃𝑇𝑇 𝑃𝑃𝑜𝑜 𝑩𝑩𝑩𝑩𝑩𝑩𝑩𝑩 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇

𝑇𝑇𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇 𝑤𝑤𝑃𝑃𝑇𝑇𝑤𝑃𝑃𝑤𝑤𝑇𝑇 𝐷𝐷𝑃𝑃𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 → Only interested in the overlap!

► 6,840 9,500

= 72%

► Net specificity is decreased

43

Comparing Sequential and Simultaneous Testing

► Sequential screening ► Two tests administered

● Everyone is given Test 1 ● A subset are given Test 2

► Simultaneous testing ► Two tests administered

● Everyone is given both tests

Sensitivity or specificity Sequential Simultaneous

Net sensitivity ↓ ↑

Net specificity ↑ ↓

44

When to Choose Sequential vs. Simultaneous Testing—1

► Practical considerations ► Resources are limited ► Minimizes burden on patients

45

When to Choose Sequential vs. Simultaneous Testing—2

► Goal of testing ► It is more costly to…

● Incorrectly label someone as positive when s/he does not have the disease (FP), or…

● Incorrectly label someone as negative when s/he does have the disease (FN)

46

When to Choose Sequential vs. Simultaneous Testing—3

► Goal of testing ► It is more costly to…

● Incorrectly label someone as positive when s/he does not have the disease (FP)  To minimize FP, maximize specificity → sequential

testing, or… ● Incorrectly label someone as negative when s/he does

have the disease (FN)

47

When to Choose Sequential vs. Simultaneous Testing—4

► Goal of testing ► It is more costly to…

● Incorrectly label someone as positive when s/he does not have the disease (FP)  To minimize FP, maximize specificity → sequential

testing, or… ● Incorrectly label someone as negative when s/he does

have the disease (FN)  To minimize FN, maximize sensitivity →

simultaneous testing

The material in this video is subject to the copyright of the owners of the material and is being provided for educational purposes under rules of fair use for registered students in this course only. No additional copies of the copyrighted work may be made or distributed.

Predictive Value

2

Validity

► How good is the test at identifying people with the disease and people without the disease? ► Validity ► Fixed characteristics of a test (do not change)

3

Validity vs. Predictive Value

► What proportion of individuals who test positive (or negative) actually have (or do not have) the disease? ► Predictive value ► Depends on disease prevalence and validity of the test

4

Types of Predictive Value

► Predictive value = the probability of disease given the results of the test 1. Positive predictive value (PPV) 2. Negative predictive value (NPV)

5

Defining PPV and NPV

► Predictive value = the probability of disease given the results of the test 1. Positive predictive value (PPV) = the probability that a person with a positive test

does have the disease 2. Negative predictive value (NPV) = the probability that a person with a negative test

does not have the disease

6

Concept of Predictive Value—1

7

Concept of Predictive Value—2

8

Concept of Predictive Value—2

9

Probability That a Person with a Positive Test Does Have the Disease— 1

► PPV (positive predictive value) = the probability that a person with a positive test does have the disease

► 𝑃𝑃𝑃𝑃𝑃𝑃 = 80 180

= 0.44 = 44%

10

Probability That a Person with a Positive Test Does Have the Disease— 2

► PPV (positive predictive value) = the probability that a person with a positive test does have the disease

► 𝑃𝑃𝑃𝑃𝑃𝑃 = 80 180

= 0.44 = 44%

11

Probability That a Person with a Negative Test Does Not Have the Disease—1

► NPV (negative predictive value) = the probability that a person with a negative test does not have the disease

► 𝑁𝑁𝑃𝑃𝑃𝑃 = 800 820

= 0.98 = 98%

12

Probability That a Person with a Negative Test Does Not Have the Disease—2

► NPV (negative predictive value) = the probability that a person with a negative test does not have the disease

► 𝑁𝑁𝑃𝑃𝑃𝑃 = 800 820

= 0.98 = 98%

13

What Influences the Positive Predictive Value (PPV) of a Test?

► The PPV primarily depends on the…

► Prevalence of the disease in the population tested

► Specificity of the test

14

Example: Prevalence and PPV (Assume Sensitivity = 99% and Specificity = 95%)—1

► Prevalence = 1%

15

Example: Prevalence and PPV (Assume Sensitivity = 99% and Specificity = 95%)—2

► Prevalence = 1%

16

Example: Prevalence and PPV (Assume Sensitivity = 99% and Specificity = 95%)—3

► Prevalence = 1%

17

Example: Prevalence and PPV (Assume Sensitivity = 99% and Specificity = 95%)—4

► Prevalence = 1% ► Prevalence = 5%

18

Distributions of Amniotic Fluid α-Fetoprotein (AFP) Levels in Pregnant Women Who Have Children with and without Spina Bifida—1

Source: Gordis. (5th ed.).

19

Distributions of Amniotic Fluid α-Fetoprotein (AFP) Levels in Pregnant Women Who Have Children with and without Spina Bifida—2

Source: Gordis. (5th ed.).

20

Example: Specificity and PPV (Assume Prevalence = 20% and Sensitivity = 50%)—1

► Specificity = 50%

21

Example: Specificity and PPV (Assume Prevalence = 20% and Sensitivity = 50%)—2

► Specificity = 50%

22

Example: Specificity and PPV (Assume Prevalence = 20% and Sensitivity = 50%)—3

► Specificity = 50% ► Specificity = 90%

23

Example: Specificity and PPV (Assume Prevalence = 20% and Sensitivity = 50%)—4

► Specificity = 50% ► Specificity = 90%

24

Example: Specificity and PPV (Assume Prevalence = 20% and Sensitivity = 50%)—5

► Specificity = 50% ► Specificity = 90%

25

Example: Specificity and PPV (Assume Prevalence = 20% and Sensitivity = 50%)—6

► Specificity = 50% ► Specificity = 90%

26

Lessons Learned

► Sensitivity and specificity are measures of validity and are fixed characteristics of a test

► Cutpoints to define a positive test affect the number of false positives and false negatives that result from the test ► Important to maximize sensitivity or specificity?

► In sequential testing, net sensitivity is decreased, but net specificity is increased. We see the opposite pattern with simultaneous testing: net sensitivity increases and net specificity decreases.

► Predictive value is important to patients. Positive predictive value increases with increasing disease prevalence and with increasing specificity.