Research Article Review Masters Level Course

Letters

RESEARCH LETTER

LESS IS MORE

Comparison of Observed Harms and Expected Mortality Benefit for Persons in the Veterans Health Affairs Lung Cancer Screening Demonstration Project The Veterans Health Affairs (VHA) lung cancer screening (LCS) demonstration project identified a much higher false- positive rate following initial low-dose computed tomo- graphic screening than did the National Lung Screening Trial

(58. 2% vs 26.3%).1 , 2 Most false-positive results (nod- ules not confirmed to be lung

cancer [LC] after follow-up) resulted in repeated imaging, but 2.0% of people screened also required nonbeneficial down- stream diagnostic evaluation to determine these nodules were not cancer.2 We sought to put these findings into context by examining how this high false-positive rate influences the harm-to-benefit ratio for higher- vs lower-risk patients.

Methods | From March 31, 2015, through June 30, 2015, 2106 patients were screened across 8 academic VAs. Screening processes and population-average outcomes for this project have been reported.2 In trials, LCS’s 20% relative risk reduc- tion (RRR) in LC mortality did not vary by baseline LC risk,3 so we estimated each patient’s absolute risk reduction (ARR) by multiplying the 20% RRR by their baseline LC mortality risk (ARR = Baseline Risk × RRR). We estimated annual baseline L C mortality risk using the B ach risk model.4 Unlike other models, the Bach model’s inputs are obtainable in VHA’s Corporate Data Warehouse. In addition, a recent analysis indicates it is one of the best performing models.5

Next, we separated patients into risk quintiles and as- sessed for each: number of LC cases observed; screening ef- fectiveness (number needed to screen [NNS] per LC death pre- vented); and screening efficiency (number of false-positive results and downstream diagnostic procedures [eg, ad- vanced imaging, bronchoscopies, biopsies] per LC death pre- vented). Following VHA policy and as part of the VA Quality Enhancement Research Initiative, this evaluation was not con- sidered to be research and was declared to be nonresearch qual- ity improvement activities by the VHA National Center for Health Promotion and Disease Prevention, and the Ann Ar- bor Veterans Affairs Medical Center institutional review board. As a quality improvement activity, patient consent was not re- quired. Patient data were deidentified in analyses.

Results | Patients in higher quintiles of LC risk had signifi- cantly more lung cancers diagnosed during the project, sup- porting the Bach model's ability to risk stratify in this popu- lation (Figure, A: 4.8 LCs per 1000 in quintile 1 vs 29.7 per 1000 in quintile 5). Initial screens were least effective for veterans in quintile 1 (lowest LC risk) (NNS of 6903) and most effec tive for veterans in quintile 5 (NNS of 687 ) (Figure). Rates of false-positive results and downstream evaluations did not differ significantly across risk quintiles (P = .52 and P = .15 for trend, respectively). That is, the over- all 56.2% rate of false-positive results requiring tracking remained relatively stable across risk quintiles (95% CI, 53.1%-62.6% in quintile 1 vs 51.9%-61.5% in quintile 5), as did the overall 2.0% rate of false-positive results requiring downstream diagnostic evaluations (95% CI, 0.3%-2.6% in quintile 1 vs 1.7%-5.2%). This relationship of increasing absolute benefit and relatively stable harms enhances the favorable harm vs benefit balance for higher-risk vs lower- risk persons. The initial sc reen was least effic ient for

Figure. Observed Rate of Lung Cancer Diagnosis and Predicted Effectiveness With Initial Low-Dose Computed Tomography Screening

0

50

40

Lu ng

C an

ce rs

p er

1 00

0, N

o.

20

30

10

Baseline Lung Cancer Risk Quintile Q1 Q2 Q3

P = .004

Q5Q4

Observed rate of lung cancer diagnosesA

0

8000

7000

6000

N N

S to

P re

ve nt

1 L

C D

ea th

5000

1000

2000

3000

4000

Baseline Risk Quintile Q1 Q2 Q3

P < .001

Q5Q4

Screening effectivenessB

A, Observed rate of lung cancer diagnoses (per 1000 persons screened once). B, Screening effectiveness: number needed to screen (NNS) to prevent 1 lung cancer death. Error bars indicate 95% CIs.

Editorial

jamainternalmedicine.com (Reprinted) JAMA Internal Medicine Published online January 22, 2018 E1

© 2018 American Medical Association. All rights reserved.

Downloaded From: by donna hammaker on 02/17/2018

patients in quintile 1 (2749 false-positive results and 68 non- beneficial diagnostic procedures per LC death prevented) and most efficient for those in quintile 5 (eg, 363 false- positive results and 22 nonbeneficial diagnostic procedures per death prevented) (Table).

Discussion | The high rate of false-positive results identified in the VHA’s LCS demonstration project has caused concern about whether LCS should be implemented in this population. We reexamined these data and found that the high false-positive rate results in a more concerning harm-to-benefit ratio for those eligible persons at lower LC risk, but a much better harm-to- benefit ratio for high-risk patients (Table). We found that even given these very high false-positive rates, the overall balance of pros and cons among patients at high LC risk still surpasses those of most established cancer screening programs.

These results should be interpreted with several caveats in mind. The high rate of false-positive results found in the VA demonstration project may represent a substantial overesti- mate of future rates for 2 reasons: (1) initial screens likely have more false-positive results than recurrent screening, and (2) newer nodule management guidelines are showing great prom- ise in lowering false-positive rates.6 Reducing the rate of false- positive findings would improve the harm-to-benefit balance for all quintiles. However, our analysis did not include all po- tential harms of LCS, such as overdiagnosis and psychologi- cal effects from false-positive results. In addition, effective- ness studies are still needed to confirm the extent to which the mortality benefit observed in the National Lung Screening Trial, a 20.0% reduction in lung cancer and a 6.7% reduction in all- cause mortality,1 applies in actual practice.

These real-world findings reinforce the need to risk- stratify patients for LCS and provide support for personal- ized, risk-based harm-benefit estimates for all eligible per- sons during LCS decision-making.

Tanner J. Caverly, MD, MPH

Angela Fagerlin, PhD Renda Soylemez Wiener, MD, MPH Christopher G. Slatore, MD, MS Nichole T. Tanner, MD, MSCR Shira Yun, MD Rodney Hayward, MD

Author Affiliations: VA Center for Clinical Management Research, Ann Arbor, Michigan (Caverly, Yun, Hayward); University of Michigan Medical School, Ann Arbor (Caverly, Yun, Hayward); Institute for Health Policy Innovation, University of Michigan, Ann Arbor (Caverly, Hayward); VA Salt Lake City Center for Informatics Decision Enhancement and Surveillance (IDEAS), Salt Lake City, Utah (Fagerlin); University of Utah School of Medicine, Salt Lake City (Fagerlin); Center for Healthcare Organization and Implementation Research, Edith Nourse Rogers Memorial Veterans Affairs Hospital, Bedford, Massachusetts (Wiener); Boston University School of Medicine, Boston, Massachusetts (Wiener); VA Portland Health Care System Center to Improve Veteran Involvement in Care, Portland, Oregon (Slatore); Oregon Health & Science University School of Medicine, Portland (Slatore); Health Equity and Rural Outreach Innovation Center (HEROIC), Ralph H. Johnson Veterans Affairs Hospital, Charleston, South Carolina (Tanner); Medical University of South Carolina, Medicine, Charleston (Tanner).

Corresponding Author: Tanner J. Caverly, MD, MPH, VA Center for Clinical Management Research and University of Michigan Medical School, 2800 Plymouth Rd, Building 16, Room 321, Ann Arbor, MI 48109 (tcaverly@med .umich.edu).

Accepted for Publication: November 27, 2017.

Published Online: January 22, 2018. doi:10.1001/jamainternmed.2017.8170

Author Contributions: Dr Caverly had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Caverly, Fagerlin, Slatore, Yun, Hayward. Acquisition, analysis, or interpretation of data: Caverly, Wiener, Tanner, Yun, Hayward. Drafting of the manuscript: Caverly. Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: Caverly, Hayward. Obtained funding: Caverly. Administrative, technical, or material support: Caverly, Yun. Study supervision: Caverly, Fagerlin.

Conflict of Interest Disclosures: None reported.

Funding/Support: Funding for this study was provided by the US Department of Veterans Affairs (VA) Quality Enhancement Research Initiative. Dr Caverly is

Table. Outcomes of Initial Low-Dose Computed Tomography Screening According to Risk Quintile

Quintile of Risk (1-y Cumulative Risk LC)a

No. (%) Predicteda LC Deaths Efficiency Calculation (No. of Harms per LC Death Prevented)

Participantsb

Observed (During VHA Demonstration Project)

Total, No.c Prevented, No.c Observed LC Casesb,c

FPs Requiring Tracking

FPs Requiring Diagnostic Evaluation

FPs per LC Death Preventedc

Nonbeneficial Diagnostic Evaluations per LC Death Preventedc

All quintiles 2084 (100) 31 (100) 1175 (100) 42 (100) 7.97 1.59 737 26

Quintile 1 420 (20.2) 2 (6.5) 243 (20.7) 6 (14.3) 0.45 0.09 2749 68

Quintile 2 459 (22.0) 2 (6.5) 249 (21.2) 5 (11.9) 1.07 0.22 1152 23

Quintile 3 379 (18.2) 5 (16.1) 205 (17.4) 8 (19.0) 1.29 0.26 793 31

Quintile 4 422 (20.3) 10 (32.3) 249 (21.2) 9 (21.4) 2.0 0.40 622 22

Quintile 5 404 (19.4) 12 (38.7) 229 (19.5) 14 (33.3) 3.16 0.63 363 22

Abbreviations: FP, false-positive screening result; LC, lung cancer; VHA, Veterans Health Administration. a Based on lung cancer risk prediction model of Bach et al,4 which uses the

following inputs to calculate an individual’s 1-year cumulative risk of LC: sex, age, smoking status (current/former smoker), years since quitting if former smoker, mean number cigarettes per day while smoking, and asbestos exposure. Asbestos exposure was not available for participants and was not considered in these calculations. The Bach model has been shown to have

excellent predictiveness without this variable.3,5 For example, the Bach model (in the absence of asbestos exposure information) showed satisfactory calibration and excellent discriminative ability in a recent external validation study (areas under curves of 0.68 to 0.8 for predicting LC death).5

b Twenty-two of the 2106 participants had incomplete smoking history and were excluded from this analysis.

c P < .05 by linear test of trend for continuous outcomes.

Letters

E2 JAMA Internal Medicine Published online January 22, 2018 (Reprinted) jamainternalmedicine.com

© 2018 American Medical Association. All rights reserved.

Downloaded From: by donna hammaker on 02/17/2018

coinvestigator on a research grant from Genentech’s Corporate Giving Scientific Project Support Program that is unrelated to this study and unrelated to any Genentech or Roche products. No other disclosures are reported.

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: All authors were employees of the VA at the time this work was conducted. The views expressed in this article are those of the authors and do not necessarily represent the views of the VA or the US Government.

Meeting Presentation: An earlier version of this work was an oral presentation at the 2017 Veterans Affairs Health Services Research & Development (HSR&D)/ Quality Improvement Enhancement Initiative (QUERI) National Conference; July 18-20, 2017; Arlington, Virginia.

1. Aberle DR, Adams AM, Berg CD, et al; National Lung Screening Trial Research Team. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365(5):395-409.

2. Kinsinger LS, Anderson C, Kim J, et al. Implementation of lung cancer screening in the Veterans Health Administration. JAMA Intern Med. 2017;177(3): 399-406.

3. Kovalchik SA, Tammemagi M, Berg CD, et al. Targeting of low-dose CT screening according to the risk of lung-cancer death. N Engl J Med. 2013;369(3): 245-254.

4. Bach PB, Elkin EB, Pastorino U, et al. Benchmarking lung cancer mortality rates in current and former smokers. Chest. 2004;126(6):1742-1749.

5. Ten Haaf K, Jeon J, Tammemägi MC, et al. Risk prediction models for selection of lung cancer screening candidates: a retrospective validation study. PLoS Med. 2017;14(4):e1002277.

6. Pinsky PF, Gierada DS, Black W, et al. Performance of Lung-RADS in the National Lung Screening Trial: a retrospective assessment. Ann Intern Med. 2015;162(7):485-491.

Letters

jamainternalmedicine.com (Reprinted) JAMA Internal Medicine Published online January 22, 2018 E3

© 2018 American Medical Association. All rights reserved.

Downloaded From: by donna hammaker on 02/17/2018