SUMMARIZE OF 3 ARTICLES

helpneded

nejmsr065990.pdf

Home >Nursing homework help >SUMMARIZE OF 3 ARTICLES

n engl j med 357;2 www.nejm.org july 12, 2007 181

s p e c i a l r e p o r t

T h e n e w e n g l a n d j o u r n a l o f m e d i c i n e

Quality of Primary Care in England with the Introduction of Pay for Performance

Stephen Campbell, Ph.D., David Reeves, Ph.D., Evangelos Kontopantelis, Ph.D., Elizabeth Middleton, M.Sc., Bonnie Sibbald, Ph.D., and Martin Roland, D.M.

In 2004, the United Kingdom committed £1.8 bil- lion ($3.2 billion) to a new pay-for-performance contract for family practitioners.1 During the first year, the levels of achievement exceeded those an- ticipated by the government, with an average of 83.4% of the available incentive payments claimed.2 However, the quality of care in English family practices had already begun to improve in response to a wide range of initiatives,3-6 including nation- al standards for the treatment of major chronic diseases and a national system of inspection (Ta- ble 1). Family practitioners already had some ex- perience with financial incentives from the lim- ited use of incentive programs that were initiated in 1990.7,8 It is therefore unclear whether the high levels of quality attained after the pay-for-perfor- mance contract was introduced in 2004 ref lect improvements that were already under way or whether existing trends toward improvement were accelerated. The effect of the incentive program must be understood in the context of the com- prehensive quality-improvement strategy within which the contract was introduced.

This report presents data from a longitudinal cohort study that measured the quality of care in a representative sample of primary care practices in England at two time points (1998 and 2003)

before the pay-for-performance program was in- troduced and at one time point (2005) after its introduction. A validated set of criteria was used to assess quality in the management of three chronic conditions: asthma, coronary heart dis- ease, and type 2 diabetes. Because some clinical indicators — the measures of the quality of clini- cal care — were not rewarded with financial pay- ments in the 2004 pay-for-performance program, the study design also permitted a comparison of trends in the quality of care for indicators for which financial incentives were provided and for those for which they were not provided in the management of these three conditions.

M e t h o d s

In 1998, we measured the quality of care in a strat- ified, random sample of 60 primary care practices in six geographic areas of England. These practices were nationally representative in terms of size, whether the practice was approved for residency training, and the sociodemographic characteris- tics of their populations.9 We followed up 42 of these practices in 2003 and 2005. The reduction in the number of practices was due partly to attri- tion and partly to the retirement of solo physi-

Table 1. Examples of Key Initiatives in the Broad National Quality-Improvement Strategy.

National standards for the treatment of major chronic diseases, such as the National Service Frameworks for coronary heart disease (1999) and diabetes (2003)

Contractual requirement for practitioners to undertake a clinical audit (initially a requirement in the 1990 contract)

Financial incentives for cervical cytologic testing and immunization (early 1990s)

Widespread use of audit and feedback by the Primary Care Trusts

Release of comparative data for quality of care to practitioners (common) and the public (rare) by the Primary Care Trusts

Annual appraisal of all primary care physicians (by the Primary Care Trusts and including discussion of some audit data)

National system of inspection and monitoring of performance (by the Healthcare Commission)

The New England Journal of Medicine Downloaded from nejm.org on June 11, 2016. For personal use only. No other uses without permission.

T h e n e w e n g l a n d j o u r n a l o f m e d i c i n e

n engl j med 357;2 www.nejm.org july 12, 2007182

cians and the closing of other practices. The 42 practices for which data were available for the longitudinal analysis were still nationally repre- sentative in terms of socioeconomic status, but solo practitioners were underrepresented.4 How- ever, these 42 practices have values close to the national averages for socioeconomic status, pop- ulation density, and type of housing of the pa- tient population, and their performance was also typical of English family practices during the first year of the pay-for-performance program.2 The analysis was restricted to the 42 practices for which data were available for all three time points (1998, 2003, and 2005). The research protocol was approved by the ethics committee of the multi- center Manchester National Health Service.

Data Collection

Trained research staff extracted the data to assess the quality of clinical care for the categories of coronary heart disease (15 clinical indicators), asthma (12 clinical indicators), and type 2 dia- betes (21 clinical indicators). Data were collected from both computerized and handwritten medi-

cal records with the use of evidence-based review criteria10,11 developed with the RAND–UCLA ap- propriateness method.12 Patients with these three conditions were randomly selected from lists of those receiving the relevant drugs (see the Sup- plementary Appendix, available with the full text of this article at www.nejm.org) according to re- peat prescriptions within the previous 6 months, and separate samples of patients treated in 1998, 2003, and 2005 were selected. In 1998, for two practices, there were no eligible patients who had coronary heart disease because of the young age of the patient population, so for that time point, the results for this condition are based on only 40 practices. Data were collected for up to 20 pa- tients for each of the three conditions in each practice in 1998 (some small practices did not have 20 patients for each of the conditions) and for up to 12 patients for each condition in each practice in 2003 and 2005. Data were collected for a total of 2300 patients in 1998, for 1495 pa- tients in 2003, and for 1482 patients in 2005. These data are presented as a pooled analysis across practices.

Although the study did not include conditions that were not rewarded with financial incentives in the pay-for-performance program, there were clinical indicators for coronary heart disease, asthma, and type 2 diabetes for which financial incentives were not provided in 2004. We com- pared 30 indicators for which financial incentives were provided with 17 indicators for which finan- cial incentives were not provided. In this analysis, we excluded three clinical indicators for which this distinction was unclear — that is, it was not clear whether there were financial incentives pro- vided for the indicator at all three time points.

Statistical Analysis

An overall score for the quality of care was com- puted for each patient included for 1998, 2003, and 2005. For each patient with asthma, coronary heart disease, or type 2 diabetes, the score was computed as a ratio: the number of clinical indi- cators for which appropriate care was provided, divided by the number of indicators relevant to that patient. Expressed as a percentage, this score represents the percentage of “necessary care” 10 provided to each patient, within a range from 0 to 100. We adopted this measure for consistency with our previous investigation of this sample.4 Scores for the quality of care at the practice level were

M ea

n P

ra ct

ic e-

Q u

al it

y S

co re

( %

)

0 1998 1999 2000 2001 2002 2003 2004 2005

CHD

Asthma

Diabetes

Year

AUTHOR:

FIGURE:

JOB:

4-C H/T

RETAKE

SIZE

ICM

CASE

EMail Line H/T Combo

Revised

AUTHOR, PLEASE NOTE: Figure has been redrawn and type has been reset.

Please check carefully.

REG F

Enon

1st 2nd 3rd

Campbell

1 of 1

07-12-07

ARTIST: ts

35702 ISSUE:

22p3

Figure 1. Mean Scores for Clinical Quality at the Practice Level for Coronary Heart Disease, Asthma, and Type 2 Diabetes, 1998 to 2005.

The quality of care for coronary heart disease (CHD), asthma, and type 2 diabetes was improving between 1998 and 2003, before the introduction of pay for performance. The rate of improvement in quality of care increased significantly for diabetes and asthma between 2003 and 2005, after the in- troduction of pay for performance; the rate for coronary heart disease, which was increasing most rapidly before pay for performance, continued at the same rate after pay for performance was introduced.

The New England Journal of Medicine Downloaded from nejm.org on June 11, 2016. For personal use only. No other uses without permission.

special report

n engl j med 357;2 www.nejm.org july 12, 2007 183

computed as the simple average of the scores for individual patients within each practice.

We also performed analyses for individual clinical indicators. Data were available for at least five patients in a practice for each indicator ana- lyzed and for geographic areas where the number of practices meeting this criterion was at least 10. In 1998, data were collected for indicators re- quiring an activity to be undertaken annually on the basis of data recorded during the previous 14 months, whereas in 2003 and 2005, data were collected for indicators requiring an activity to be undertaken annually on the basis of data record- ed during the previous 15 months, in line with the pay-for-performance contract. For consis- tency, we generated 15-month versions of the clini- cal indicators for 1998 from our original data for use in the present analysis.

We compared scores for observed quality in 2005 with the scores predicted on the basis of the trend between 1998 and 2003. Practice scores for individual indicators, computed as a percent- age of patients receiving appropriate care as the indicator, were subject to a ceiling effect of 100%. In calculating the expected scores for 2005, it was inappropriate to use a simple linear model, be- cause such a model would fail to account for ceiling effects (i.e., some predicted scores would have exceeded 100% if the previous linear trend had been extrapolated from 2003 to 2005). Probit and logit models are most commonly used to model binary data. We adopted the logit model a priori for the current analysis, calculating ex- pected values for 2005 on the basis of the logit curve that the scores from 1998 to 2003 followed and extrapolating this curve to 2005. After per-

forming the analysis, we computed probit predic- tions for comparative purposes and found these all to be within 1 percentage point of their logit equivalent. For quality-of-care scores between 20 and 80%, the logit and probit curves are essen- tially linear.

For each practice, we therefore computed a predicted score for 2005 using a logit projection from 1998 and 2003. We computed predicted val- ues for overall scores and for individual clinical indicators. The predicted scores were then com- pared with the actual scores for the practices in 2005. However, because of floor and ceiling ef- fects, the differences between actual and predict- ed scores are not equivalent across the scale: the difference between an observed score of 54 and a predicted score of 50 does not have the same import as the difference between a score of 99 and a score of 95. To adjust for this difference, ob- served and predicted scores were converted into their logit equivalents before the analysis. Under the transformation, a proportion, P, is trans- formed into a log odds, as Logit(P) = ln[P ÷ (1 − P)]. Since Logit(P) cannot be computed where the value of P is 0 or 1, we used the empirical logit, Logit(P) = ln[(P + 0.5 ÷ n) ÷ (1 − P + 0.5 ÷ n)], in these cases, where n is the number of observations over which P is calculated.13 The logit transformation maps the scale of 0 to 100% to a scale of less than infinity to infinity.13 The transformation therefore “stretches” the scores at the extremes, which increases the effect on the results of the analysis of practices for which scores are close to the floor or the ceiling.

The transformed observed and predicted scores for 2005 were then compared by means of a

Table 2. Changes in Mean Scores at the Practice Level for Quality of Care for Coronary Heart Disease, Type 2 Diabetes, and Asthma, 1998 to 2005.*

Variable Coronary Heart

Disease Diabetes Asthma

Mean score for 1998 — % 58.6 61.6 60.2

Mean score for 2003 — % 76.2 70.4 70.3

Mean score for 2005 — % 85.0 81.4 84.3

Mean predicted score for 2005 (logit model) — % 80.7 73.2 72.3

Mean difference between transformed observed score and predicted score for 2005 — % (95% CI)

0.22 (−0.02 to 0.45) 0.68 (0.27 to 1.1) 0.44 (0.27 to 0.62)

P value 0.07 0.002 <0.001

* CI denotes confidence interval. P values are for the comparison between transformed observed and predicted scores for 2005.

The New England Journal of Medicine Downloaded from nejm.org on June 11, 2016. For personal use only. No other uses without permission.

T h e n e w e n g l a n d j o u r n a l o f m e d i c i n e

n engl j med 357;2 www.nejm.org july 12, 2007184

Ta b

le 3

. S co

re s

fo r

Q u

al it

y o

f C

ar e

at t

h e

P ra

ct ic

e L

ev el

in 1

9 9

8 , 2

0 0

3, a

n d

2 0

0 5.

C o

n d

it io

n C

lin ic

al I

n d

ic at

o r

M ea

n S

co re

†

M ea

n D

if fe

re n

b et

w ee

n T

ra n

sf o

rm ed

b se

rv ed

S co

re a

n d

re d

ic te

d S

co re

i n

2 0

0 5

5% C

I) ‡

P V

al u

19 98

20 03

20 05

pe rc

en t

C o

ro n

ar y

h ea

d is

ea se

F re

q u

en cy

o r

p at

te rn

o f

an gi

n a

at ta

ck s

re co

rd ed

d u

ri n

g th

e p

re vi

o u

s 15

m o

43 .4

60 .3

70 .1

0. 22

( −0

.4 2

to 0

.8 6)

0. 49

B lo

o d

p re

ss u

re r

ec o

rd ed

d u

ri n

g th

e p

re vi

o u

s 15

m o

87 .0

95 .3

98 .8

0. 26

( 0.

02 t

o 0

.5 1)

0. 04

E xe

rc is

e ca

p ac

it y

re co

rd ed

d u

ri n

g th

e p

re vi

o u

s 15

m o

36 .5

66 .3

72 .8

0. 21

( −0

.5 0

to 0

.9 1)

0. 55

C h

o le

st er

o l l

ev el

r ec

o rd

ed d

u ri

n g

th e

p re

vi o

u s

5 yr

63 .5

89 .7

98 .7

0. 54

( 0.

26 t

o 0

.8 2)

< 0.

00 1

D ie

ta ry

a d

vi ce

r ec

o rd

ed d

u ri

n g

th e

p re

vi o

u s

5 yr

58 .8

75 .7

87 .4

0. 64

( −0

.0 6

to 1

.3 4)

0. 07

S m

o ki

n g

st at

u s

re co

rd ed

d u

ri n

g th

e p

re vi

o u

s 5

yr 85

.1 86

.6 98

.3 0.

87 (

0. 47

t o

1 .2

7) <

0. 00

R ef

er ra

l t o

s p

ec ia

lis t

fo r

ex er

ci se

s tr

es s

te st

in g

o r

as se

ss m

en t

(E C

G )

ev er

co rd

ed 76

.2 88

.2 89

.2 −0

.0 1

(− 0.

40 t

o 0

.3 8)

0. 95

P re

sc ri

p ti

o n

o r

ad vi

ce t

o t

ak e

as p

ir in

r ec

o rd

ed u

n le

ss r

ec o

rd o

f co

n tr

ai n

d i-

ca ti

o n

o r

in to

le ra

n ce

76 .2

80 .7

92 .5

0. 71

( 0.

27 t

o 1

.1 5)

0. 00

B lo

o d

p re

ss u

re c

o n

tr o

lle d

t o

≤ 15

0/ 90

m m

H g

47 .9

72 .2

81 .5

0. 15

( −0

.2 3

to 0

.5 2)

0. 44

S er

u m

c h

o le

st er

o l c

o n

tr o

lle d

t o

1 90

µ g/

d l

16 .9

60 .7

72 .9

0. 10

( −0

.4 1

to 0

.6 1)

0. 69

C o

n d

it io

n al

ct o

S m

o ki

n g

ad vi

ce t

o s

m o

ke rs

r ec

o rd

ed d

u ri

n g

th e

p re

vi o

u s

5 yr

66 .5

76 .8

99 .1

N A

W ei

gh t

ad vi

ce f

o r

o ve

rw ei

gh t

p at

ie n

ts r

ec o

rd ed

d u

ri n

g th

e p

re vi

o u

s 5

yr 63

.7 78

.8 82

.5 0.

28 (

−0 .7

2 to

1 .2

9) 0.

A ct

io n

t ak

en o

n b

lo o

d p

re ss

u re

if s

ys to

lic p

re ss

u re

> 16

0 m

m H

o r

if >

14 0

m m

H g

an d

t o

ta l s

er u

m c

h o

le st

er o

l > 19

0 m

g/ d

l 34

.3 44

.4 58

.3 N

A N

A ct

io n

t ak

en if

c h

o le

st er

o l >

19 0

m g/

d l i

n p

at ie

n t

ag e

< 70

y r

53 .8

73 .6

79 .2

N A

B et

a- b

lo ck

er p

re sc

ri b

ed a

s m

ai n

te n

an ce

t h

er ap

y if

s o

le t

h er

ap y

47 .4

58 .1

79 .5

N A

A st

h m

N o

rm al

o r

p re

d ic

te d

p ea

k ex

p ir

at o

ry f

lo w

o r

re co

rd o

f d

if fi

cu lt

y u

si n

g m

et er

co rd

ed d

u ri

n g

th e

p re

vi o

u s

5 yr

76 .1

85 .7

91 .6

0. 28

( −0

.0 7

to 0

.6 4)

0. 12

D ai

ly , n

o ct

u rn

al , o

r ac

ti vi

ty -l

im it

in g

sy m

p to

m s

re co

rd ed

d u

ri n

g th

e p

re vi

- o

u s

15 m

o 43

.7 58

.1 76

.5 0.

91 (

0. 31

t o

1 .5

2) 0.

00 4

S m

o ki

n g

st at

u s

re co

rd ed

d u

ri n

g th

e p

re vi

o u

s 5

yr 82

.6 84

.3 96

.0 0.

59 (

0. 16

t o

1 .0

1) 0.

00 8

In h

al er

t ec

h n

iq u

e re

co rd

ed d

u ri

n g

th e

p re

vi o

u s

5 yr

53 .1

58 .7

76 .0

0. 71

( 0.

13 t

o 1

.2 8)

0. 02

The New England Journal of Medicine Downloaded from nejm.org on June 11, 2016. For personal use only. No other uses without permission.

special report

n engl j med 357;2 www.nejm.org july 12, 2007 185

C o

n d

it io

n al

ct o

S m

o ki

n g

ad vi

ce t

o s

m o

ke rs

r ec

o rd

ed d

u ri

n g

th e

p re

vi o

u s

5 yr

64 .2

86 .6

96 .6

N A

S el

f- m

an ag

em en

t p

la n

f o

r p

at ie

n ts

r ec

ei vi

n g

h ig

h -d

o se

c o

rt ic

o st

er o

id s

o r

th o

se w

h o

h ad

in p

at ie

n t

tr ea

tm en

t fo

r as

th m

a re

co rd

ed d

u ri

n g

th e

p re

- vi

o u

s 5

22 .5

56 .0

77 .8

N A

S h

o rt

-a ct

in g

b ro

n ch

o d

ila to

rs p

re sc

ri b

ed f

o r

u se

b ef

o re

e xe

rc is

e fo

r p

at ie

n ts

it h

e xe

rc is

e- in

d u

ce d

b ro

n ch

o sp

as m

r ec

o rd

ed d

u ri

n g

th e

p re

vi o

u s

5 yr

40 .1

74 .2

90 .9

N A

P ea

k ex

p ir

at o

ry f

lo w

d u

ri n

g a

co n

su lt

at io

n f

o r

an e

xa ce

rb at

io n

r ec

o rd

ed f

o r

m o

st r

ec en

t ep

is o

d e

10 0

N A

O ra

l c o

rt ic

o st

er o

id s

p re

sc ri

b ed

if p

ea k

ex p

ir at

o ry

f lo

w <

60 %

o f

n o

rm al

co rd

ed f

o r

m o

st r

ec en

t ep

is o

d e

61 .7

73 .6

93 .6

N A

A ct

io n

t ak

en if

p at

ie n

ts h

ad n

o ct

u rn

al s

ym p

to m

s re

co rd

ed d

u ri

n g

th e

p re

vi -

o u

s 5

yr 30

.6 80

.8 10

0 N

A N

A ct

io n

t ak

en if

p at

ie n

ts h

ad a

ct iv

it y-

lim it

in g

sy m

p to

m s

re co

rd ed

d u

ri n

g th

e p

re vi

o u

s 5

yr 50

.0 0

0 N

A N

R ef

er ra

l t o

r es

p ir

at o

ry p

h ys

ic ia

n if

o ra

l s te

ro id

s w

er e

u se

d in

m ai

n te

n an

tr ea

tm en

t, r

ec o

rd ed

d u

ri n

g th

e p

re vi

o u

s 5

yr 76

.1 85

.7 91

.6 N

A N

T yp

e 2

d ia

b et

G ly

ca te

d h

em o

gl o

b in

r ec

o rd

ed d

u ri

n g

th e

p re

vi o

u s

15 m

o 87

.1 93

.1 99

.4 0.

54 (

0. 29

t o

0 .8

0) <

0. 00

V is

u al

e xa

m in

at io

n o

f fe

et r

ec o

rd ed

d u

ri n

g th

e p

re vi

o u

s 15

m o

57 .4

69 .6

88 .0

0. 99

( 0.

45 t

o 1

.5 3)

0. 00

P er

ip h

er al

p u

ls es

o r

vi b

ra ti

o n

s en

se r

ec o

rd ed

d u

ri n

g th

e p

re vi

o u

s 15

m o

60 .0

62 .9

90 .2

1. 58

( 0.

98 t

o 2

.1 7)

< 0.

00 1

S er

u m

c re

at in

in e

re co

rd ed

d u

ri n

g th

e p

re vi

o u

s 15

m o

79 .6

89 .9

96 .4

0. 33

( −0

.0 8

to 0

.7 3)

0. 11

U ri

n e

p ro

te in

u ri

a re

co rd

ed d

u ri

n g

th e

p re

vi o

u s

15 m

o 66

.3 74

.8 82

.8 0.

43 (

−0 .1

2 to

0 .9

8) 0.

E xa

m in

at io

n o

f fu

n d

i o r

vi su

al a

cu it

y re

co rd

ed d

u ri

n g

th e

p re

vi o

u s

15 m

o 69

.4 72

.2 82

.7 0.

58 (

0. 10

t o

1 .0

6) 0.

W ei

gh t

re co

rd ed

d u

ri n

g th

e p

re vi

o u

s 15

m o

80 .2

86 .5

97 .2

0. 60

( 0.

20 t

o 1

.0 1)

0. 00

B lo

o d

p re

ss u

re r

ec o

rd ed

d u

ri n

g th

e p

re vi

o u

s 15

m o

92 .6

95 .8

99 .0

0. 15

( −0

.1 0

to 0

.4 0)

0. 22

S m

o ki

n g

st at

u s

re co

rd ed

d u

ri n

g th

e p

re vi

o u

s 5

yr 86

.5 88

.5 98

.4 0.

58 (

0. 13

t o

1 .0

3) 0.

S er

u m

c h

o le

st er

o l r

ec o

rd ed

d u

ri n

g th

e p

re vi

o u

s 5

yr 75

.1 97

.6 99

.4 0.

05 (

−0 .0

8 to

0 .1

8) 0.

B lo

o d

p re

ss u

re c

o n

tr o

lle d

t o

≤ 14

0/ 85

m m

H g

(r ec

o rd

ed in

p re

vi o

u s

15 m

o )

21 .8

35 .4

49 .0

0. 49

( 0.

04 t

o 0

.9 4)

0. 03

T o

ta l s

er u

m c

h o

le st

er o

l c o

n tr

o lle

d t

o ≤

19 0

m g/

d l (

re co

rd ed

in p

re vi

o u

s 5

yr )

21 .8

52 .0

72 .5

0. 42

( 0.

03 t

o 0

.8 1)

0. 03

G ly

ca te

d h

em o

gl o

b in

c o

n tr

o lle

d t

o ≤

7. 4%

( re

co rd

ed in

p re

vi o

u s

15 m

o )

37 .8

39 .8

50 .6

0. 40

( 0

to 0

.8 1)

0. 05

The New England Journal of Medicine Downloaded from nejm.org on June 11, 2016. For personal use only. No other uses without permission.

T h e n e w e n g l a n d j o u r n a l o f m e d i c i n e

n engl j med 357;2 www.nejm.org july 12, 2007186

C o

n d

it io

n al

ct o

D o

cu m

en ta

ti o

n o

f p

at ie

n t

ed u

ca ti

o n

if d

ia b

et es

d ia

gn o

se d

< 5

yr r

ec o

rd ed

u ri

n g

th e

p re

vi o

u s

5 yr

84 .8

87 .7

10 0

0. 29

( −0

.1 4

to 0

.7 3)

0. 17

A d

vi ce

g iv

en t

o s

m o

ke rs

r ec

o rd

ed d

u ri

n g

th e

p re

vi o

u s

5 yr

70 .0

78 .3

95 .2

N A

R ef

er ra

l t o

a s

p ec

ia lis

t if

c re

at in

in e

> 2.

26 µ

g/ d

l r ec

o rd

ed d

u ri

n g

th e

p re

vi -

o u

s 5

yr 10

0 10

0 N

A N

In p

at ie

n ts

a ge

< 80

y r,

t re

at m

en t

o ff

er ed

if a

ve ra

ge o

f th

e th

re e

p re

vi o

u s

b lo

o d

-p re

ss u

re r

ea d

in gs

w as

d ia

st o

lic >

10 0

o r

sy st

o lic

> 15

0 an

d d

ia -

st o

lic >

90 m

m H

53 .6

90 .5

66 .7

N A

In p

at ie

n ts

t re

at ed

f o

r h

yp er

te n

si o

n , i

f p

ro te

in u

ri a

p re

se n

t, p

at ie

n ts

s h

o u

re ce

iv e

A C

E in

h ib

it o

r 68

.8 10

0 75

.0 N

A N

In p

at ie

n ts

s ta

rt in

g A

C E

in h

ib it

o r,

m ea

su re

m en

t o

f cr

ea ti

n in

e an

d p

o ta

ss i-

u m

w it

h in

1 m

o b

ef o

re t

re at

m en

t st

ar te

d r

ec o

rd ed

d u

ri n

g th

e p

re vi

o u

s 5

38 .1

29 .9

38 .8

N A

In p

at ie

n ts

r ec

ei vi

n g

su lf

o n

yl u

re a,

h yp

o gl

yc em

ia s

ym p

to m

s re

co rd

ed d

u r-

in g

th e

p re

vi o

u s

15 m

o 18

.2 8.

1 7.

8 −0

.1 2

(− 0.

57 t

o 0

.3 3)

0. 60

In p

at ie

n ts

≤ 70

y r

o f

ag e,

if p

re vi

o u

s gl

yc at

ed h

em o

gl o

b in

m ea

su re

d w

> 9%

, p at

ie n

ts o

ff er

ed a

t h

er ap

eu ti

c in

te rv

en ti

o n

t o

im p

ro ve

g ly

ce m

co n

tr o

81 .3

71 .9

66 .0

N A

* C

o n

d it

io n

al i

n d

ic at

o rs

a re

i n

d ic

at o

rs t

h at

d id

n o

t ap

p ly

t o

a ll

p at

ie n

ts .

T o

c o

n ve

rt v

al u

es f

o r

ch o

le st

er o

l to

m ill

im o

le s

p er

l it

er ,

m u

lt ip

ly b

y 0.

02 6.

T o

c o

n ve

rt v

al u

es f

o r

cr ea

ti n

in e

m ic

ro m

o le

s p

er l

it er

, m

u lt

ip ly

b y

88 .4

. C

I d

en o

te s

co n

fi d

en ce

i n

te rv

al ,

E C

G e

le ct

ro ca

rd io

gr ap

h y,

N A

n o

t an

al yz

ed (

an al

ys is

w as

p er

fo rm

ed o

n ly

w h

en t

h er

e w

er e

at l

ea st

1 0

p ra

ct ic

w it

h a

m in

im u

m o

f fi

ve p

at ie

n ts

e ac

h t

h at

p ro

vi d

ed d

at a

fo r

an i

n d

ic at

o r

at e

ac h

o f

th e

th re

e ti

m e

p o

in ts

), a

n d

A C

E a

n gi

o te

n si

n -c

o n

ve rt

in g

en zy

m e.

† T

h e

re su

lt s

ar e

b as

ed o

n a

ll p

ra ct

ic es

f o

r w

h ic

h d

at a

w er

e av

ai la

b le

. ‡

T h

e re

su lt

s ar

e b

as ed

o n

p ra

ct ic

es w

it h

a t

le as

t fi

ve p

at ie

n ts

w h

o h

ad t

h e

co n

d it

io n

a t

ea ch

o f

th e

th re

e ti

m e

p o

in ts

. S

ta ti

st ic

al t

es ts

w er

e n

o t

p er

fo rm

ed f

o r

ge o

gr ap

h ic

a re

as i

n w

h ic

fe w

er t

h an

1 0

p ra

ct ic

es m

et t

h is

c ri

te ri

o n

. §

R es

u lt

s b

ec am

e si

gn if

ic an

t (P

= 0

.0 04

) u

si n

g th

e b

o o

ts tr

ap m

et h

o d

The New England Journal of Medicine Downloaded from nejm.org on June 11, 2016. For personal use only. No other uses without permission.

special report

n engl j med 357;2 www.nejm.org july 12, 2007 187

matched-pairs two-tailed t-test. In view of the relatively small number of practices included in the analysis and the distributional assumptions made by this test, a bootstrap procedure based on 1000 bootstrap samples was used to confirm the significance of the tests.

Although the logit model is appropriate to an analysis of individual binary indicators, the over- all scores for quality used here may not conform to the logit curve as the scores approach the ceiling. Therefore we repeated the analysis, using a linear model applied to the untransformed scores. The linear model makes no adjustment for ceiling effects, except that we did not allow predictions greater than 100%. Hence the results of the sensitivity analysis are likely to be conser- vative.

For comparisons of clinical indicators for which financial incentives were provided with those for which they were not provided, we derived observed and predicted scores for clinical quality at the practice level separately for the groups of indi- cators in the management of each condition for which financial incentives were provided and for those indicators for which they were not provid- ed. The logit transformation was then applied to these scores. A matched-pairs t-test was used to compare the difference between the observed scores for indicators for which financial incen- tives were provided and those for which financial incentives were not provided with the predicted difference. The significance of the test was con- firmed with the use of the bootstrap procedure.

R e s u lt s

The quality of care in the categories of coronary heart disease, asthma, and type 2 diabetes im- proved between 2003 and 2005, continuing the earlier trend (Fig. 1). However, the increase in the rate of improvement between 2003 and 2005 was significant for asthma (P<0.001) and diabetes (P = 0.002) (Table 2). Scores for coronary heart disease also increased, but the change in the rate of improvement was not significant (P = 0.07). Sim- ilarly, the sensitivity analysis performed with the use of the more conservative linear model showed significant increases in the rate of improvement for asthma and diabetes, as compared with the rates for coronary heart disease.

On the basis of the conservative linear model, the annual rate of increase in the quality of care

between 2003 and 2005 for diabetes was faster than the annual rate between 1998 and 2003 in 34 practices and was slower in 8 practices (sig- nificantly so for 13 and 0 practices, respectively; P<0.05). For asthma, the annual rate of improve- ment in the quality of care was unchanged be- tween 1998 and 2003 in 1 practice, faster in 28 practices, and slower in 13 practices (significant- ly so for 10 and 2 practices, respectively; P<0.05). For coronary heart disease, although the annual rate of improvement between 1998 and 2003 was faster in 23 practices, it was slower in 17 practic- es (significantly so for 7 and 5 practices, respec- tively; P<0.05).

Observed scores for clinical quality for 1998, 2003, and 2005 and the predicted score for 2005 for each indicator in each condition are shown in Table 3. The table includes examples of changes in individual clinical indicators that are likely to be particularly important for improving patient outcomes, such as control of cholesterol and blood pressure in the management of coronary heart disease.14

We then compared changes in the quality of care between clinical indicators for which finan- cial incentives were provided in 2004 and those for which financial incentives were not provided. The quality of performance for indicators with incentives in all three conditions was substan- tially higher at all three time points than for those without incentives. However, in all condi- tions, the rate of improvement between 2003 and 2005 for clinical indicators for which financial incentives were provided, as compared with those for which they were not, did not differ significant- ly from the rate predicted on the basis of the trend between 1998 and 2003 (Table 4).

D i s c u s s i o n

Although the quality of care in the categories of asthma, coronary heart disease, and type 2 dia- betes was improving before the introduction of the 2004 contract, our results suggest that the introduction of pay for performance was associ- ated with a modest acceleration in improvement for two of these three conditions: diabetes and asthma. In most of the 42 practices for which data were available, the annual improvement for both was accelerated. The results are based on care reported in the medical records but not nec- essarily on care provided, and it is a common

The New England Journal of Medicine Downloaded from nejm.org on June 11, 2016. For personal use only. No other uses without permission.

T h e n e w e n g l a n d j o u r n a l o f m e d i c i n e

n engl j med 357;2 www.nejm.org july 12, 2007188

criticism of pay-for-performance programs that their main effect is to promote better recording of care rather than better care. However, the pan- els used to develop the indicators judged that to provide good care, it was necessary both to pro- vide the care and to record the processes and in- termediate outcomes assessed in terms of the in- dicators we used.10

Because some details of the new contract were publicized before the contract was introduced in 2004, it is possible that some practices were pre- paring for the incentives during our second round of data collection in 2003. If this is true, we might have overestimated improvements in qual- ity made before pay for performance was intro- duced,4 so that the results reported here could represent a conservative estimate of the actual im- provement resulting from pay for performance.

The key question is whether the increased rate of improvement in quality of care after the new contract was introduced can be attributed to pay for performance or to other factors. The pay-for- performance program was the only major national policy implemented in primary care in England in 2004 that targeted the types of care processes evaluated in this study. However, since practices were observed at only two time points before the introduction of pay for performance, we were un- able to determine whether the rate of improvement had already accelerated as a result of earlier but still ongoing initiatives. No control group could be recruited, because financial incentives were applied simultaneously across the whole of the United Kingdom. A final concern is that the pa- tients included in the study were selected on the basis of the presence or absence of treatment with relevant drugs, and those who were untreated or

who did not comply with treatment were excluded from the analysis. This selection bias could have resulted in overestimation of the quality of care at all three time points, although the trends in quality should have been unaffected.

The study focuses on three chronic conditions — asthma, coronary heart disease, and type 2 diabetes — for which financial incentives were provided under the pay-for-performance program and which had also been subject to considerable quality-improvement activity in the United King- dom as part of a national quality-improvement strategy. The finding of a significant increase in the rate of improvement for asthma and diabe- tes but not for coronary heart disease may re- flect the fact that in 2003 scores for quality for coronary heart disease were already higher than those for the other two conditions. Coronary heart disease had been a particular target of ear- lier quality-improvement initiatives, with 98% of the Primary Care Trusts reporting coronary heart disease initiatives in 2001 and 2002.15

The finding of no significant difference in the rate of improvement between clinical indica- tors for which financial incentives were provid- ed and those for which they were not provided suggests that the pay-for-performance program may not necessarily have been responsible for the acceleration in improvement that we found be- tween 2003 and 2005. However, the study was not designed or powered for this analysis, and the broad confidence limits for many of the clinical indicators shown in Table 3 reflect the uncer- tainty associated with the small sample available for the analysis. In addition, there may have been a “halo effect,” as a result of which some indi- cators for which financial incentives were not provided may have been indirectly rewarded. For example, the clinical indicator “control of total serum cholesterol in coronary heart disease to 190 mg per deciliter (5 mmol per liter) or less,” which in the 2004 contract became an indicator for which a financial incentive was provided, is likely to have influenced performance on “evi- dence of action being taken if cholesterol was raised,” a clinical indicator for which a financial incentive was not specifically provided. Improve- ments may therefore have spilled over onto oth- er aspects of care that were not subject to per- formance monitoring. This effect has previously been noted in the Department of Veterans Af- fairs quality-improvement programs.16 The study

Table 4. Mean Difference in Improvement for Indicators with and without Incentives.*

Category Mean Difference

(95% CI) P Value

Coronary heart disease

0.53 (−0.01 to 1.08) 0.054

Asthma 0.03 (−0.45 to 0.51) 0.904

Type 2 diabetes 0.08 (−0.32 to 0.49) 0.682

* The mean difference is the amount by which the ob- served difference between transformed overall scores for clinical indicators for which financial incentives were or were not provided exceeded the predicted dif- ference. CI denotes confidence interval.

The New England Journal of Medicine Downloaded from nejm.org on June 11, 2016. For personal use only. No other uses without permission.

special report

n engl j med 357;2 www.nejm.org july 12, 2007 189

was not able to assess what is perhaps a more important question, namely, what the effect of financial incentives was on care for conditions for which no financial incentives were provided at all.

The introduction of the pay-for-performance program has been associated with a general trend in the National Health Service away from placing implicit trust in health care profession- als and toward more active monitoring of their performance than before the program was in- troduced.17 Financial incentives are most likely to be an effective means of influencing profes- sional behavior when performance targets and rewards are aligned to the values of the staff be- ing rewarded.18,19 Professional motivation alone may not be sufficient to improve the quality of care, especially when physicians have to make financial investments in their practices — for example, by employing more staff to achieve gains in quality. Sustained improvement in qual- ity of care, which involves a range of health care providers (e.g., physicians, nurses, and adminis- trative staff), requires a combination of other factors, including clear goals, good teamwork, and effective leadership.20

Owing to the inherent limitations of our study design and data, it was not possible to de- termine whether improvements in quality of care resulted only from the pay-for-performance pro- gram; our findings are consistent with previous work, which suggested that financial incentives can change professional behavior21,22 and that patients receive higher-quality care in geograph- ic areas where performance measures and mon- itoring have been established.16 However, there are also potential, unintended consequences of such schemes.1,23 These include the possible neglect of geographic areas where financial incentives for improvements in care are not provided and of “myopia” (the pursuit of short-term targets at the expense of legitimate long-term objectives) or “misrepresentation” (deliberate manipulation of data so that reported behavior differs from actual behavior).24 In addition, external incentives may crowd out motivation — the desire to do a task well for its own sake.25,26 In the United Kingdom, family practitioners have predicted that among the adverse consequences of financial incentives may be a reduction in the continuity of care, fragmentation of care as a result of specialization within practices, and neglect of conditions for

which financial incentives are not provided.27 Despite these concerns, overall job satisfaction among family physicians was higher in 2004 than in 2001.28 Moreover, a recent report from the United States suggests that targeted quality- improvement programs have not resulted in a deterioration in the quality of care in untargeted disease areas.29 Our results generally support the view of the Institute of Medicine that pay- for-performance programs can make a useful contribution to improving quality,30 particularly when such programs are part of a comprehensive quality-improvement program.31

The size of the gains in quality in relation to the costs of pay for performance remains a po- litical issue in the United Kingdom,32 and the government now accepts that it paid more than it had expected to pay for the improvements in performance.33 The proportion of practice income taken as profit by general practitioners appears to have increased after the new contract was in- troduced, suggesting that gains in quality could have been achieved at a lower cost. For the years 2006 through 2007, the pay-for-performance framework has been amended to introduce high- er payment thresholds, new targets, and new dis- ease areas34 without increasing physicians’ max- imum available income from incentive payments. Physicians in the United Kingdom may now need to work harder or employ more staff to earn the same rewards that they had received before 2006.

Supported by the U.K. Department of Health. The views pre- sented here are those of the authors and not necessarily of the U.K. Department of Health.

Dr. Roland reports serving as an academic advisor to the gov- ernment and the British Medical Association negotiating teams during the development of the United Kingdom pay-for-perfor- mance scheme during 2001 and 2002. No other potential con- flict of interest relevant to this article was reported.

From the National Primary Care Research and Development Centre, University of Manchester, Manchester, United King- dom. Address reprint requests to Dr. Campbell at the National Primary Care Research and Development Centre, University of Manchester, Oxford Rd., Manchester M13 9PL, United Kingdom.

Roland M. Linking physician pay to quality of care: a major experiment in the United Kingdom. N Engl J Med 2004;351: 1448-54.

Doran T, Fullwood C, Gravelle H, et al. Pay-for-performance programs in family practices in the United Kingdom. N Engl J Med 2006;355:375-84.

Campbell S, Steiner A, Robison J, Webb D, Raven A, Roland M. Is the quality of care in general medical practice improving? Results of a longitudinal observational study. Br J Gen Pract 2003;53:298-304.

Campbell SM, Roland MO, Middleton E, Reeves D. Improve- ments in the quality of clinical care in English general practice

The New England Journal of Medicine Downloaded from nejm.org on June 11, 2016. For personal use only. No other uses without permission.

special report

n engl j med 357;2 www.nejm.org july 12, 2007190

1998-2003: longitudinal observational study. BMJ 2005;331: 1121-3.

Campbell S, Wilkin D, Roland M. Primary care groups: im- proving quality of care through clinical governance. BMJ 2001; 322:1580-2.

Campbell SM, Roland MO. The experience of the United Kingdom. In: Garcia-Pena C, Munoz O, Duran L, Vazquez F, eds. Family medicine at the dawn of the 21st century. Mexico City: CAMS, 2005:391-416.

Baker D, Middleton E. Cervical screening and health in- equality in England in the 1990s. J Epidemiol Community Health 2003;57:417-23.

Middleton E, Baker D. Comparison of social distribution of immunisation with measles, mumps, and rubella vaccine, En- gland, 1991-2001. BMJ 2003;326:854.

Campbell SM, Hann M, Hacker J, et al. Identifying predic- tors of high quality care in English general practice: observa- tional study. BMJ 2001;323:784-7.

Campbell SM, Roland MO, Shekelle PG, Cantrill JA, Buetow SA, Cragg DK. Development of review criteria for assessing the quality of management of stable angina, adult asthma and non- insulin dependent diabetes mellitus in general practice. Qual Health Care 1999;8:6-15.

Campbell SM, Hann M, Hacker J, Durie A, Thapar A, Roland MO. Quality assessment for three common conditions in pri- mary care: validity and reliability of review criteria developed by expert panels for angina, asthma and type 2 diabetes. Qual Saf Health Care 2002;11:125-30.

Brook RH, Chassin MR, Fink A, Solomon DH, Kosecoff J, Park RE. A method for the detailed assessment of the appropri- ateness of medical technologies. Int J Technol Assess Health Care 1986;2:53-63.

Collett D. Modelling binary data. London: Chapman and Hall, 1991.

McElduff P, Lyratzopoulos G, Edwards R, Heller RF, Shekelle P, Roland M. Will changes in primary care improve health out- comes? Modelling the impact of financial incentives introduced to improve quality of care in the UK. Qual Saf Health Care 2004;13:191-7.

Campbell S, Wilkin D. Clinical governance. In: Wilkin D, Coleman A, Dowling B, Smith K, eds. The National Tracker Sur- vey of primary care groups and trusts 2001/2002: taking respon- sibility? Manchester, United Kingdom: University of Manchester, National Primary Care Research and Development Centre, Uni- versity of Manchester, 2002. (Accessed June 21, 2007, at http:// www.npcrdc.ac.uk/Publications/TRACKER_REPORT_2002.pdf.)

Asch SM, McGlynn EA, Hogan MM, et al. Comparison of quality of care for patients in the Veterans Health Administra- tion and patients in a national sample. Ann Intern Med 2004; 141:938-45.

Checkland K, Marshall M, Harrison S. Re-thinking account- ability: trust versus confidence in medical practice. Qual Saf Health Care 2004;13:130-5.

Marshall M, Smith P. Rewarding results: using financial in-

10.

11.

12.

13.

14.

15.

16.

17.

18.

centives to improve quality. Qual Saf Health Care 2003;12:3 97-8.

Spooner A, Chapple A, Roland M. What makes British gen- eral practitioners take part in a quality improvement scheme? J Health Serv Res Policy 2001;6:145-50.

Campbell S, Steiner A, Robison J, et al. Do Personal Medical Services contracts improve quality of care? A multi-method eval- uation. J Health Serv Res Policy 2005;10:31-9.

Gosden T, Forland F, Kristiansen IS, et al. Impact of pay- ment system on behaviour of primary care physicians: a system- atic review. J Health Serv Res Policy 2001;6:44-55.

Epstein AM, Lee TH, Hamel MB. Paying physicians for high- quality care. N Engl J Med 2004;350:406-10.

McGlynn EA. Intended and unintended consequences: what should we really worry about? Med Care 2007;45:3-5.

Smith P. On the unintended consequences of publishing per- formance data in the public sector. Int J Publ Admin 1995;18:277- 310.

Deci EL, Koestner R, Ryan RM. A meta-analytic review of experiments examining the effects of extrinsic rewards on in- trinsic motivation. Psychol Bull 1999;125:627-68.

Gagné M, Deci EL. Self-determination theory and work mo- tivation. J Organ Behav 2005;26:331-62.

Roland MO, Campbell SM, Bailey N, Whalley D, Sibbald B. Financial incentives to improve the quality of primary care in the UK: predicting the consequences of change. Prim Health Care Res Dev 2006;7:18-26.

Whalley D, Bojke C, Gravelle H, Sibbald B. GP job satisfac- tion in view of contract reform: a national survey. Br J Gen Pract 2006;56:87-92.

Ganz DA, Wenger NS, Roth CP, et al. The effect of a quality improvement initiative on the quality of other aspects of health care: the law of unintended consequences? Med Care 2007;45: 8-18.

Fisher ES, Davis K. Pay for performance — recommendations of the Institute of Medicine. N Engl J Med 2006 (Web only). (Avail- able at http://content.nejm.org/cgi/content/full/NEJMp068216/ DC1.)

Committee on Redesigning Health Insurance Performance Measures, Payment, and Performance Improvement Programs. Rewarding provider performance: aligning incentives in Medi- care. Washington, DC: National Academies Press, 2007. (Ac- cessed June 21, 2007, at http://www.nap.edu/catalog/11723.html.)

Galvin R. Pay-for-performance: too much of a good thing? A conversation with Martin Roland. Health Aff (Millwood) 2006;25:w412-w419.

GP pay raise ‘was mistake.’ BBC News Health. (Accessed June 21, 2007, at http://news.bbc.co.uk/player/nol/newsid_6280000/ newsid_6280000/6280077.stm?bw=bb&mp=rm.)

British Medical Association. Quality and outcomes frame- work guidance: summary of indicators — clinical domain. (Ac- cessed June 21, 2007, at http://www.bma.org.uk/ap.nsf/Content/ qof06~summclinical#SecondaryPreventionofCoronary.) Copyright © 2007 Massachusetts Medical Society.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

34.

The New England Journal of Medicine Downloaded from nejm.org on June 11, 2016. For personal use only. No other uses without permission.