probability
overmorrow888
P2-TypeITypeII.pdf
LSP 121
P2 - Type I and Type II Errors
Test results
• medical tests
• diagnostic tests
• presence/absence of a condition
2
Test Results
• Positive – condition is present – Not a value judgment
• Negative – condition is not present – Not a value judgment
3
Take action….
• based on test results
• concerns about taking action – side effects – cost – many others
4
No test is 100% accurate
Our Concerns
• If test results say condition exists, (test result is positive)
• does it actually exist ?
• If test results say condition does not exist, (test result is negative)
• does it actually exist ?
6
How likely is it that (given the test results)
a condition exists ?
…that we need to take action
probability
7
Four Possible Outcomes (test results vs. reality)
• True Positive – test results are correct – condition exists
• True Negative – test results are correct – condition does not exist
• False Positive – test results are not correct – condition does not exist
• False Negative – test results are not correct – condition exists
8
Test Results vs. Reality
Reality �
Test Results
Has Condition Does Not Have Condition
Positive Test True Positive
False Positive Type I Error
Negative Test False Negative Type II Error
True Negative
9
Technique for Determining Probability
of each of the four possible outcomes
10
Three important factors stated in each problem
• Test accuracy
• Occurrence in the real world
• Number of test participants
11
Test Accuracy
• What percent of the time does a test correctly identify the condition ?
• Example: A disease detection screening is 85% accurate
12
Occurrence in the real world
• How frequently does the condition occur in the population ?
• This could be stated is a variety of ways: • percentage, • proportion, • x of y
• Example: 80% of homes have termites .80 homes 8 of 10 homes
13
Number of test participants
• Number of tests run
• Example: – 10,000 people were tested for high blood
sugar
14
Example
• We conduct a study in which diagnostic tests are given to 10,000 people who have symptoms of Condition D.
• Assume that 1% of people who have symptoms of Condition D actually have it.
• The test for Condition D is 85% accurate
15
How many people we test for Condition D
are in each category ?
• True positive • False positive (Type I Error) • True negative • False negative (Type II Error)
16
Step 0. Draw basic 4 x 4 matrix and label cells
Has Condition D
Does not have Condition D
Total
Positive Test
True Positive Type I Error
Negative Test
Type II
Error
True Negative
Total
17
Number of test participants
• We conduct a study in which diagnostic tests are given to 10,000 people who have symptoms of Condition D.
18
Step 1. Post total participants
Has Condition D
Does not have Condition D
Total
Positive Test
True Positive Type I Error
Negative Test
Type II Error True Negative
Total 10,000
19
Occurrence in the Population
• 1% of people who have symptoms of Condition D actually have it.
20
Step 2. calculate # in population who do and do not have the condition, based on
occurrence in the population Has Condition D
Does not have Condition D
Total
Positive Test
True Positive Type I Error
Negative Test
Type II Error True Negative
Total 100
99% of 10,000
10,000
1% in the population have Condition D
99% do not 21
1% of 10,000
9,900
Test Accuracy
• The test for Condition D is 85% accurate
Step 3. Calculate inner cell values based on test accuracy
Has Condition D
Does not have Condition D
Total
Positive Test
True Positive
(85% of 100) 85
Type I Error
(15% of 9900)
1485
Negative Test
Type II Error
(15% of 100)
15
True Negative
(85% of 9900) 8415
Total 100 9,900 10,000
85% accurate
85% accurate
23
Do not round totals in the interior cells of the matrix
• if the calculation of the values in the four quadrants result in a number that is not a whole number (e.g. 15 x .10 = 1.5)
• do not round those values
24
Complete the matrix
calculate total positive and total negative
results
Step 4. Calculate Total Positive and Negative Test Results
Has Condition D Does not have Condition D
Total
Positive Test
True Positive
(85% of 100) 85
Type I Error
(15% of 9900)
1485
(85+1485)
1570
Negative Test
Type II Error
(15% of 100)
15
True Negative
(85% of 9900) 8415
(15+8415)
8430
Total Tests 100 9,900 10,000
26
What is the probability that a participant actually has the
condition (so we need to take action) ?
true positives/total positives and
false negatives/total negatives
27
Probability that a Positive Result is correct (true positive)
• Overall, the diagnostic test gives • positive results to 1570 people (85+1485)
• 85 people who actually have the condition (correct) and to • 1485 people who do not have the condition (incorrect)
• 85 of these are true positives,
• So the probability of a positive result actually being correct is 85/1570
28
Step 5. Calculate Probability that a Positive Result is correct
True Positives/Total Positives
85 / 1570 = .054 or 5.4%
Has Condition Does not have Condition
Total
Positive Test
True Positive
(85% of 100) 85
Type I Error
(15% of 9900)
1485 1570
29
Probability that a Negative result is incorrect (false negative)
• Suppose you are a doctor seeing a patient with symptoms of Condition D.
• The diagnostic test comes back "not Condition D".
• Based on the matrix values, what is the chance that the patient really has Condition D ?
30
Probability that a Negative result is incorrect (false negative)
• Overall, the diagnostic test gives – negative results to 8430 (15+8415) people
• 15 people who do have the condition (incorrect) • 8415 people who actually do not have the condition (correct)
• 15 of these are false negatives,
• So the probability of a negative result actually being incorrect is 15/8430
31
Step 6. Calculate the Probability that a Negative result is incorrect
False Negatives/ Total Negatives
15 / 8430 � .0018 or .18%
Has Condition Does not have Condition
Total
Negative Test
Type II Error
(15% of 100)
15
True Negative
(85% of 9900) 8415 8430
32
What’s the big deal ?
• False positives (Type I error) lead to un- needed efforts to “cure” the problem
• False negatives (Type II error) give a false sense of security and no effort to “cure” the problem
33
