Casestudy 2

Is Access Sufficient? An Examination of the Effects of the MedShare Program to Expand Access to Prescription Drugs for Indigent Populations

Thomas Shaw University of South Alabama, Mobile Mark Carrozza Institute for Policy Research, Cincinnati, Ohio

We conduct an evaluation of MedShare, a program designed to enhance access to prescription drugs for indigent patients in the Greater Cincinnati area. The program expands access to drugs by providing subsidies to reduce the costs paid by patients for their prescriptions. The assumption is that by expanding access to prescription drugs, participant health outcomes as measured by qual- ity of life improve. Although the program appears outwardly successful, we found little difference between program participants and comparison groups. We feel that these findings point to a major flaw with existing health policy: access alone is not sufficient to improve health outcomes. Too often programs are created and, provided they show outwards signs of success (e.g., enrollment and utilization), are assumed to be improving the health of the community. Our findings indicate that one must look beyond just expanding access to ensure that programs are indeed achieving their overall objectives.

Keywords: health; prescriptions; evaluation; policy; access

Introduction: Access and Health Care Outcomes

One of the trends in health care throughout the 1990s and into this century has been a focus on expanding access to help alleviate disparities,

526

Evaluation Review Volume 32 Number 6

December 2008 526-546 © 2008 Sage Publications

10.1177/0193841X08315884 http://erx.sagepub.com

hosted at http://online.sagepub.com

Authors’ Note: Please address correspondence to Thomas Shaw, Department of Political Science and Criminal Justice, 221 Humanities Building, University of South Alabama, Mobile, AL, tshaw@usouthal.edu

Shaw, Carrozza / MedShare and Prescription Drugs 527

particularly among the poor and racial minorities (Anderson, 1995; Ensor & Cooper, 2004; Institute of Medicine, 2001, 2002). Access both in terms of proximity or availability of services and financial ability to pay for services is a critical component in reducing disparities (Institute of Medicine, 2001, 2002). However, beyond just expanding access there are still a number of aspects that play into the continuation of disparities. It is not simply a case of “build it and they will come.” One aspect in this regard is ethnic and cul- tural characteristics that mitigate against accessing traditional health care services (Freiman, 1998). Thus, even if access is expanded, certain groups may choose not to take advantage of services because of mistrust, apprehen- sion, or other reasons. Even more significant though are programs that are created and widely used but which may fail to produce desired outcomes.

In this article, we elaborate on a program evaluation conducted for the Health Foundation of Greater Cincinnati (HFGC) of their MedShare pre- scription drug subsidy program for indigent populations within Greater Cincinnati. The MedShare program had already been shown to provide a cost-savings to participants; however, HFGC wanted to further examine whether MedShare had an impact on the quality of life of participants. MedShare permitted individuals with limited economic resources greater access to prescriptions drugs because of reduced costs. In other words, prescriptions were more readily available to the indigent population. The program was so successful that within a few years enrollments had doubled and then tripled. However, to our surprise, our initial results indicated that the program had little or no impact on the quality of life of MedShare parti- cipants. We found that economic necessity and low health comprehension contributed to reduced compliance. However, accounting for compliance, health improvements were modest at best.

We feel that the results of our analysis point to a more general problem in the domain of health care. Many programs are created (e.g., Medicare Part D) to expand access to medical resources. Unfortunately, just provid- ing or enhancing access does not necessarily translate into improved health care outcomes.

Searching for a Control Group: Research and Study Design

Because we were evaluating a program that had already been implemented, we used a post-test experimental comparison group evaluation research design (Bingham & Felbinger, 2002; Spector, 1981). However, because of the recruit- ing success of the program, an easily accessible comparison group was diffi- cult to identify.

528 Evaluation Review

To overcome this problem, two strategies were employed. The first strategy concentrated on what we termed the “internal” comparison group. This internal comparison group approach postulated that any positive effects of access to pharmaceutical drugs related to quality of life (particu- larly for specific disease groupings) would take time to be fully realized by MedShare participants. Therefore, newer MedShare enrollees could be compared to existing/older MedShare participants to determine if program- matic access to pharmaceutical drugs had an effect on participant quality of life. We felt that those that had entered the program within the previous 3 months would constitute new members and were less likely to have expe- rienced a significant increase in their quality of life as a result of access to drug therapies.

The second strategy focused on creating an external comparison group from existing data that had been collected in the 2002 Community Health Status Survey (CHSS) also conducted in the Greater Cincinnati area. Using propensity scores, we matched sample members from the CHSS survey to those in the MedShare participant survey along 13 characteristics present in both surveys. The matches using the propensity scores resulted in a subset sample from the CHSS survey that mirrored the MedShare sample on rele- vant characteristics thereby allowing for comparison between the two groups. We then examined quality of life using SF12 physical and mental scores using a comparison of means test between the CHSS and MedShare subsamples created from the propensity score matches.

Hypotheses

Our primary hypotheses were that MedShare would have a positive impact on the mental and physical quality of life of participants. That is, those indi- viduals participating in MedShare would possess better (higher) quality of life scores, where quality of life is measured using the SF12 physical and mental scores, than those individuals not in MedShare. This expectation was driven by the idea that the application of drug therapies will have a positive impact on individual’s lives when they experience health problems.

H1: MedShare participants should possess significantly higher quality of life scores as measured by the SF12 physical component, than those not partici- pating in MedShare.

H2: MedShare participants should possess significantly higher quality of life scores as measured by the SF12 mental component than those not participat- ing in MedShare.

Shaw, Carrozza / MedShare and Prescription Drugs 529

In addition to these two main hypotheses, we also had three more hypothe- ses related to specific disease categories. The specific disease categories under investigation were asthma, diabetes, and hypertension. We expected differential benefits from drug therapies related to each of these three diag- noses. It is important to note that our expectations are not necessarily rooted in a medical understanding of how these drug therapies impact the disease or the health of the patient but rather derive from a patient’s self perception of the benefit of the drug therapy. From this standpoint, there may well be a medication that improves a condition that the patient is generally oblivious to; however, because the patient does not see any direct effect from taking the medication, the patient is unlikely to perceive any change in quality of life by virtue of taking the medication. We feel that for a patient to report an increase in quality of life because of a drug therapy, there needs to be a direct symp- tom(s) that can be linked to the diagnosis and commonly observable improve- ments in the condition related to the drug therapy.

Given the acute and episodic nature of asthma and its treatment, we felt that access to drug therapies would provide the most direct relief of symp- toms for this condition. Consequently, we felt that access to drug therapies would contribute to the highest quality of life scores among asthmatics. Although not as episodic as asthma, there are a number of distinct symptoms related to diabetes such as fatigue, thirst, and consistent voiding that access to drug therapies should help to alleviate. Although maybe not as noticeable as relief from an asthmatic attack, the alleviation of these symptoms should provide for an observable improvement in quality of life. Thus, we do not expect to see as much of an improvement in quality of life among diabetic participants. Finally, we expect to see little or no effect on quality of life among hypertensive participants. For many, hypertension remains a covert diagnosis. To be sure, the patient is likely well aware that they have been diagnosed with this condition; however, many will possess few if any day- to-day or episodic symptoms. As such, drug therapy treatment of this disease will likely produce few tangible improvements discernable to the patient. Consequently, we expect to see either the least amount of patient observable improvement or no improvement in quality of life scores among hyperten- sive patients, particularly when compared to asthmatic and diabetic patients.

H3: Participants diagnosed with asthma will have higher quality of life scores, as measured by SF12 physical and mental scores, than either diabetic or hypertensive patients.

H4: Participants diagnosed with diabetes will have lower quality of life scores, as measured by SF12 physical and mental scores, than asthma patients but higher quality of life scores than hypertensive patients.

530 Evaluation Review

H5: Participants diagnosed with hypertension will have lower quality of life scores, as measured by SF12 physical and mental scores, than either asthma or diabetic patients.

Finally, in addition to the question of drug therapies and their effects on quality of life, we were also interested in the effects of self-efficacy and social support networks on individual perceptions of quality of life. Both of these characteristics have been shown to have an impact on quality of life (Robert, 1999; Schlenk et al, 1997); therefore, we included scales of each in our questionnaire. We expected to find effects similar to previous studies showing that increased self-efficacy and social support lead to increases in perception of quality of life. Thus, we have two final hypotheses.

H6: Participants scoring higher on the self-efficacy scale would have higher quality of life measures than those scoring lower on the self-efficacy scale.

H7: Participants scoring higher on the social support scale would have higher quality of life measures than those scoring lower on the social support scale.

Finally, our models included controls for gender, age, and race. These three factors contribute to health disparities and could account for differen- tial outcomes if not included in the models.

Collecting the Data

To measure the impact of the MedShare Program on quality of life, it was necessary to conduct a survey of participants. In addition to informa- tion on quality of life, HFGC wanted to be able to examine the data by spe- cific disease groupings including asthma, diabetes, and hypertension. Even though participants were indigent, HFGC indicated that telephone informa- tion for recipients was generally reliable. A telephone survey was therefore deemed to be the best method for data collection.1

The beginning of the data collection coincided with the implementation of the Health Information Portability and Accountability Act privacy rule. The University of Cincinnati Institutional Review Board issued a waiver to allow for the collection of protected health information in the case of our survey. The data were collected by telephone survey between October 10 and November 6, 2003. A randomized listed sample was used to select respondents with additional quotas used to ensure adequate numbers of dis- ease specific respondents. A total of 928 MedShare participants responded to the survey for a margin of error of +/-3.2%.

Shaw, Carrozza / MedShare and Prescription Drugs 531

Measuring Quality of Life, the Dependent Variable

Although there are a number of different indicators for quality of life (Bowling, 1997), we chose to focus on the SF12 physical and mental scores. The reason for selecting the SF12 was threefold:

1. The use of the SF12 would permit us to create the external comparison group for the analysis using propensity score matching. Any other measure of quality of life would rule out this group for comparison because they had not been asked extensive questions regarding quality of life.

2. The SF12 has been shown to be an effective measure that can be adminis- tered by telephone, our preferred method of data collection.

3. The SF12 is short and because of resource constraints, we needed to keep survey administration costs to a minimum.

Independent Variables

Year of Entry into MedShare Program

This information was essential for the internal comparison. We wanted to separate out the most recent enrollees from all others. Initially, we thought we would include information regarding each annual cohort to see if there were any additional time effects beyond those associated with the most recent enrollees. However, the analysis revealed that the combination of the various dichotomous variables for year of entry contributed to abnormal variance inflation factor scores in the overall models as well as the disease specific subgroup models. Consequently, because the most recent enrollees were the focus of the analysis, we limited the model to just a dichotomous variable for entering the program in 2003.2 The constant therefore contains information on anyone entering the MedShare program prior to 2003. The expectation is that participants entering the program in 2003 will have sta- tistically significant negative coefficients, i.e., they will have lower quality of life scores than those participants who have been in the program for a year or more. This expectation reflects the premise that access to these medica- tions enhances the quality of life of program participants.

Diagnosed with Specific Condition

We asked each participant about six possible conditions: asthma, dia- betes, hypertension, back pain, cholesterol, and depression. In each case, we asked if a doctor had ever told them they had the particular condition. We

532 Evaluation Review

then used these conditions as a set of dichotomous variables in the analysis with “no condition” being the excluded dummy because it was possible that a participant may not have been diagnosed with any of the conditions. It is this “no condition” that is represented in the constant. Our expectation was that each condition would contribute to deteriorating health with some dif- ferentiation among the categories. In particular, we conducted additional analysis of three of the disease groupings: asthma, diabetes, and hyperten- sion. In each case, we ran separate models examining the effects of the inde- pendent variables exclusively on the quality of life scores for individuals with these three diagnoses. As indicated earlier, we expected asthma to have the most pronounced effect, followed by diabetes, and then hypertension.

Social Support and Personal Agency

We incorporated scales of both social support and personal agency into the questionnaire to be able to take account of environmental and nonmed- ical factors that may impact estimations of quality of life (Robert, 1999). In both cases we expected these scales to have a positive impact on the qual- ity of life scores of participants.

Controls: Gender, Age, and Race

Both gender and race are shown to have differential impacts on health outcomes because of unequal social, economic, health, and geographic con- ditions (Shi & Singh, 2004). Consequently, it was important to include these variables as controls. Our expectation is that both women and minori- ties will exhibit slightly worse quality of life scores than males and nonmi- norities. Increased age is also clearly associated with deteriorating health outcomes and declining quality of life (Shi & Singh, 2004). Again, we felt it was important to include age to fully differentiate the impact of our other variables in accounting for the quality of life of MedShare participants.

Analyzing the Data

The Internal Control Group—Comparing Recent Enrollees to Existing Enrollees

The Overall Model

The overall model incorporated the entire applicable MedShare sample group as well as all of the independent variables. We ran each model for the

two dependent variables, physical quality of life as measured by the SF12 physical score and mental quality of life as measured by the SF12 mental score. Table 1 presents the findings from each of these models. Both the physical and mental models were statistically significant at the 99.9% con- fidence interval and showed moderate relationships in terms of the adjusted R2 estimates. The independent variables explained approximately 38% of the variation in the SF12 physical model and approximately 43% of the variation in the SF12 mental model.

The constant in each of these models reflects the score of a young, white, male starting the MedShare program prior to 2003 with no diagno- sis of any of the six disease categories. In both cases, the baseline scores reflect that MedShare participants are less well off than the average individ- ual with scores below the 50% SF12 baseline. Young, white, male MedShare participants with none of the specified diseases possess a statistically sig- nificant baseline SF12 physical score of 43.3 and a baseline SF12 mental score of 33.7. Consequently, MedShare participants start out below average in terms of both physical and mental quality of life.

Time of entry into the program had no statistically significant effect on either the physical or mental quality of life scores. Our expectation was that those individuals joining MedShare in 2003 would have a strong negative coefficient indicating that they were considerably less well off in terms of quality of life relative to existing MedShare participants. Ultimately, start- ing the MedShare program in 2003 did not attain statistical significance in either model. Consequently, relying on our internal control group, the null hypotheses were confirmed as we found no evidence to support either H1 or H2 that MedShare participants experienced an improved quality of life, either physical or mental, relative to non-MedShare participants.

Impact of Specific Diseases

Having been diagnosed with any of the listed illnesses resulted in a sta- tistically significant relationship and a decline in a participants physical quality of life score. Chronic back pain contributed to the greatest decline in physical health (-9.3). Asthma, diabetes, and hypertension all contributed to more than a 3-point decline in the physical score while depression and cholesterol reduced the physical score by more than 2 points. The various disease categories had considerably less impact on a participant’s mental quality of life. The only disease category that mattered here was whether one had been diagnosed with depression which resulted in a 9 point reduc- tion in mental quality of life.

Shaw, Carrozza / MedShare and Prescription Drugs 533

534 Evaluation Review

Table 1 Regression Results Internal MedShare Comparison

SF12 Physical Scorea SF12 Mental Scoreb

B (se) Sig B (se) Sig

Constant 43.34 (2.064) .000*** 33.705 (1.907) .000*** Started MedShare 2003 −0.895 (.760) .240 −0.899 (.703) .184 Respondent told has asthma −3.624 (.894) .000*** −1.156 (.826) .145 Respondent told has diabetes −3.113 (.880) .000*** 0.561 (.813) .473 Respondent told has hypertension −3.557 (.833) .000*** -0.830 (.770) .263 Respondent told has back pain −9.252 (.909) .000*** −0.403 (.840) .618 Respondent told has cholesterol −2.162 (.870) .013* -0.130 (.804) .867 Respondent told has depression −2.396 (.811) .003** −9.081 (.749) .000*** Gender 0.272 (.817) .739 0.506 (.754) .485 Age −0.071 (.030) .017* 0.045 (.028) .090 Race 0.971 (.786) .217 1.577 (.727) .025* Social support 0.128 (.047) .007** 0.120 (.043) .004** Personal agency 0.269 (.049) .000*** 0.432 (.045) .000*** Adj. R2 = .384 .000*** .432 .000*** N = 785 779

a. Diagnostics revealed: (a) two outliers; however, they did not affect the estimates and as they were real data were kept in the analysis; (b) the variance inflanction factors indicated that mul- ticollinearity was unlikely; and (c) both the residual vs. fitted plot and the Breusch-Pagan/ Cook-Weisberg test for heteroskedasticity (.0022) revealed the presence of heteroskedasticity in the model. Attempts to correct the heteroskedasticity through transformations was unsuc- cessful; however, as heteroskedasticity has a tendency to underestimate statistical signifi- cance, we are erring on the side of not identifying an existing relationship rather than identifying a relationship that does not really exist (Type 1 error), e.g., a more conservative estimate (Hamilton, 1992). b. Diagnostics revealed: (a) six outliers—we opted to remove these outliers from the analy- sis because the model without the outliers boosted the overall adj R2 and affected the significance of the race variable; (b) the variance inflation factors indicated that multi- collinearity was unlikely; and (c) both the residual vs fitted plot and the Breusch- Pagan/Cook-Weisberg test for heteroskedasticity (.0000 ) revealed the presence of heteroskedasticity in the model. Attempts to correct the heteroskedasticity through trans- formations was unsuccessful; however, as heteroskedasticity has a tendency to underesti- mate statistical significance, we are erring on the side of not identifying an existing relationship rather than identifying a relationship that does not really exist (Type 1 error), e.g., a more conservative estimate (Hamilton 1992). * sig at p < .05 level, ** sig at p < .01 level, *** sig at p < .001 level.

We expected those with asthma to see the biggest reduction in quality of life followed by those with diabetes and then those with hypertension. The results show that for the physical quality of life model hypertension had an almost similar impact to asthma, and diabetes had the least impact of the three. For the mental model, none of the three had an impact on quality of life. Consequently, our hypotheses regarding the individual disease cate- gories were not confirmed.

Social Support and Self-Efficacy

The scores for social support and personal agency attained statistical sig- nificance in the predicted direction for both models allowing us to reject the null hypotheses and lending support to our predictions for these variables. Social support had a mild positive effect in both models such that as a MedShare participant experiences greater feelings of community and social bonding within their neighborhood, both their physical and mental quality of life tend to improve. Personal agency had a statistically significant effect on physical quality of life and a slightly stronger effect on mental quality of life. Thus, as MedShare participants’ feelings regarding self-worth and self-determination increase, their quality of life tends to improve.

Controls

Finally, age attained significance in the physical quality of life model, and race attained significance in the mental quality of life model. As one would expect, aging resulted in a declining physical quality of life score. Interestingly, the race variable in the mental model indicates that blacks possess slightly higher mental quality of life scores than whites. Gender did not possess statistically significant effects.

Disease Specific Models: Asthma

Table 2 presents the results for MedShare participants diagnosed with asthma. The results indicate the following:

• Both models were statistically significant with the physical score explaining 22% of the participant’s physical quality of life and 39% of the participant’s mental quality of life.

• Time of entry did not have any affect on quality of life in either model—conse- quently our primary hypotheses (H1 and H2) to be tested using the internal control group were rejected in favor of the null hypotheses.

Shaw, Carrozza / MedShare and Prescription Drugs 535

536 Evaluation Review

• In the physical model only age and personal agency had statistically signifi- cant effects. In both cases, they conformed to expectations.

• In the mental model, both social support and personal agency attained statistical significance. For both, mental quality of life improves as these scores improve.

Disease Specific Models: Diabetes

Table 3 presents the results for MedShare participants diagnosed with diabetes. The results indicate the following:

• Both models were statistically significant with the physical score explaining just 16% of the participant’s physical quality of life and 35% of the partici- pant’s mental quality of life.

• Again, time of entry into the MedShare program did not have any effect on quality of life.

• In the physical model only age and personal agency had statistically signifi- cant effects, and both conformed to expectations.

• In the mental model, age, race, social support, and personal agency attained statistical significance. The age coefficient shows a somewhat counterintuitive

Table 2 Regression Results Internal MedShare Comparison,

Asthma Participantsa

SF12 Physical Score SF12 Mental Score

B (se) Sig B (se) Sig

Constant 35.810 (4.770) .000*** 16.717 (4.358) .000*** Started MedShare 2003 −1.243 (1.876) .509 −1.874 (1.714) .276 Gender 0.728 (2.228) .744 3.864 (2.036) .060 Age −0.264 (.066) .000*** .006 (.060) .921 Race .794 (1.920) .680 2.563 (1.755) .146 Social support 0.123 (.113) .278 0.226 (.103) .031* Personal agency 0.489 (.112) .000*** 0.848 (.103) .000*** Adj. R2 = .217 .000*** .390 .000*** N = 158 158

a. Diagnostics revealed: (a) no outliers; (b) the variance inflation factors indicated that multi- collinearity was unlikely; and (c) the Breusch-Pagan/Cook-Weisberg tests for heteroskedastic- ity (.6188 and .2041 for the pcs12 and mcs12 models respectively) were nonsignificant indicating that heteroskedasticity is not present in either model; this was confirmed through examination of the residual versus fitted plots. * sig at p < .05 level, ** sig at p < .01 level, *** sig at p < .001 level.

Shaw, Carrozza / MedShare and Prescription Drugs 537

result in that as age increases, the model indicates a slight improvement in mental quality of life. The race coefficient shows that blacks tend to experience a better mental quality of life than whites. Again both personal agency and social support show positive effects such that as each of these scores increases, mental quality of life improves.

Disease Specific Models: Hypertension

Table 4 presents the results for MedShare participants diagnosed with hypertension. The results indicate the following:

Table 3 Regression Results Internal MedShare

Comparison, Diabetes Participants

SF12 Physical Scorea SF12 Mental Scoreb

B (se) Sig B (se) Sig

Constant 32.684 (4.832) .000*** 19.509 (3.725) .000*** Started MedShare 2003 .291 (2.063) .888 1.570 (1.590) .325 Gender −1.296 (1.943) .505 −2.842 (1.498) .059 Age −0.191 (.073) .009** 0.167 (.056) .003** Race 1.786 (1.830) .330 4.749 (1.416) .001** Social support 0.187 (.103) .071 0.202 (.080) .012* Personal agency 0.500 (.104) .000*** 0.616 (.080) .000*** Adj. R2 = .164 .000*** .350 .000*** N = 209 208

a. Diagnostics revealed: (a) no outliers; (b) the variance inflation factors indicated that multi- collinearity was unlikely; and (c) the Breusch-Pagan/Cook-Weisberg test for heteroskedastic- ity (.4383) was non-significant indicating that heteroskedasticity is not present in the model; this was confirmed in the residual versus fitted plot. b. Diagnostics revealed: (a) one outlier—removal of the outlier did not have a major impact on the individual variables but did affect the overall explanatory power of the model—given the amount a single case affected the overall explanatory power of the model, the analysis was conducted without the outlier even though it was an actual case; (b) the variance inflation fac- tors indicated that multicollinearity was unlikely; and (c) the Breusch-Pagan/Cook-Weisberg test for heteroskedasticity (.0923) was nonsignificant indicating that heteroskedasticity is not present in the model; the residual versus fitted plot was examined but was inconclusive, there- fore we are relying on the results of the Breusch-Pagan/Cook-Weisberg test as confirmation of the lack of heteroskedasticity. * sig at p < .05 level, ** sig at p < .01 level, *** sig at p < .001 level.

538 Evaluation Review

• Both models were statistically significant with the physical score explaining just 17% of the participant’s physical quality of life and 31% of the partici- pant’s mental quality of life.

• Time of entry into the MedShare program again had no affect on mental quality of life; however, for physical quality of life there was a statistically significant impact in the predicted direction (b = −2.861, p >|t| = .030). Unfortunately, this confirmation of our hypothesis occurs in only one model; consequently, we cannot conclude that time of entry has a real impact because it fails to attain significance in any of the other models.

• In the physical model, time of entry, age, race, social support, and personal agency had statistically significant effects. Time of entry has the anticipated negative impact. Age had the anticipated effect while the race coefficient indicated slightly improved physical quality of life for whites. Physical qual- ity of life improved as social support and personal agency improved.

• In the mental model, age, social support, and personal agency attained statis- tical significance. Age again shows a counterintuitive result such that as age increases, there is a slight improvement in mental quality of life. Both per- sonal agency and social support exhibit positive effects on a participant’s mental quality of life.

Table 4 Regression Results Internal MedShare Comparison,

Hypertensive Participantsa

SF12 Physical Score SF12 Mental Score

B (se) Sig B (se) Sig

Constant 31.051 (3.331) .000*** 19.233 (2.951) .000*** Started MedShare 2003 −2.861 (1.316) .030* −0.542 (1.166) .642 Gender −1.643 (1.305) .209 −2.098 (1.156) .070 Age −0.113 (.049) .021* 0.144 (.043) .001*** Race 3.083 (1.232) .013* 1.917 (1.092) .080 Social support 0.204 (.074) .006** 0.162 (.065) .013* Personal agency 0.450 (.073) .000*** 0.709 (.065) .000*** Adj. R2 = .167 .000*** .312 .000*** N = 390 390

a. Diagnostics revealed: (a) no outliers; (b) the variance inflation factors indicated that multi- collinearity was unlikely; and (c) the Breusch-Pagan/Cook-Weisberg tests for heteroskedastic- ity (.6538 and .0815 for the pcs12 and mcs12 models respectively) were nonsignificant indicating that heteroskedasticity is not present in either model; this was confirmed through examination of the residual versus fitted plots. * sig at p < .05 level, ** sig at p < .01 level, *** sig at p < .001 level.

Shaw, Carrozza / MedShare and Prescription Drugs 539

We now summarize the effects found in examining the internal control group using the overall model as well as the various disease specific models. We found only one significant effect for time of entry into the program across all eight models. Thus, generally those who entered the program most recently had quality of life scores that were essentially the same as those who had been in the program for varying but longer lengths of time. This finding disconfirms our hypotheses (H1 and H2) that exposure to the program and prescription drugs leads to an improvement in one’s physical or mental quality of life.

In terms of the disease specific hypotheses, the overall model showed that our expectations regarding asthma, diabetes, and hypertension were not supported. There were no effects for these variables in the overall mental model. In the overall physical model, these variables did have an effect but not in the anticipated ways. Thus, the particular pattern of expectations identified in hypotheses H3, H4, and H5 were not supported.

Both social support and personal agency performed as predicted across almost all models. Only in the disease specific asthma and diabetes physi- cal quality of life models did the social support score fail to attain statisti- cal significance. Otherwise both attained statistical significance in the predicted direction. Thus, two of our ancillary hypotheses (H6 and H7) were supported.

The External Control Group—Propensity Score Matching and Comparison With the Community Health Status Survey

The analysis using propensity score matching focused on identifying a group of individuals comparable to MedShare participants on a number of relevant characteristics (see Table 5). We used logistic regression to calcu- late the propensity scores and thereby facilitate matching on variables across different samples (D’Agostino, 1998; Parsons, 2000). Thus, using the propensity scores, we were able to identify a comparable subsample from the 2002 CHSS conducted in Greater Cincinnati.

Prior to creating and matching the propensity scores, we tested to see if there were differences between the MedShare sample and the overall CHSS sample. Table 5 shows the results of these tests both before and after the propensity score matching. Prior to the analysis, we expected to find statis- tically significant differences because the two samples should have been dissimilar.

540 Evaluation Review

The data were then combined and the propensity scores obtained from the probability estimates using logistic regression. At this point, the data were again separated into the two independent samples and matched according to the propensity scores. The matching procedures resulted in a much lower n size (259) for each group. However, after matching was com- plete, retesting the differences in the two samples along the relevant char- acteristics resulted in only one statistically significant difference. Thus, the matching resulted in a CHSS subsample similar to the MedShare subsam- ple across the identified characteristics.

Having obtained two separate but comparable samples, we then calculated the SF12 mental and physical scores for both groups. A t-test for independent samples was used to determine if there was a statistically significant differ- ence between the means of the two groups for each of these scores. We antic- ipated a positive statistically significant difference to demonstrate that MedShare participants had a different and improved outcome in terms of quality of life relative to this comparable group of non-MedShare partici- pants. However, we again found no statistically significant differences in either physical or mental quality of life between these two groups (see Table 6). We therefore still cannot attribute any impact from the MedShare program on a participant’s quality of life.

Table 5 Analysis Using Propensity Score Matching

Prematching Significance Postmatching Significance

Number of children 0.1466 0.5397 Number of adults < .0001 0.3505 Marital status < .0001 0.5881 Employment status < .0001 0.8540 Education < .0001 0.9471 Religion < .0001 0.8274 Race < .0001 0.4490 Chronic back pain < .0001 0.3932 High cholesterol < .0001 0.7743 Asthma < .0001 0.4018 Diabetes < .0001 0.0497* Depression < .0001 0.5611 Hypertension < .0001 0.7898 MedShare n size 770 259 CHSS n size 1645 259

* sig at p < .05 level.

Shaw, Carrozza / MedShare and Prescription Drugs 541

Initial Results—No Programmatic Effect

After comparing both the internal and external control groups to MedShare participants, we were left with the conclusion that MedShare participation, that is, access to dramatically discounted prescriptions, does not improve the quality of life of participants. This finding is not particu- larly pleasant for at least two reasons. First, although MedShare has been shown to provide a financial savings to participants, there is an expectation that over time enhanced access to drug therapies should lead to better health outcomes as represented in measures such as quality of life. Second, if MedShare does not improve participant quality of life, it could threaten the existence of the program by convincing donors to withdraw their financial support for the program.

In light of our findings, a rational policy perspective would advocate the dismantling of the program to reallocate funds to more effective purposes. However, the general trend in the use of prescription drugs for treating ill- nesses gave us pause in accepting the conclusion that there was no connec- tion between MedShare participation and quality of life. We considered three possible explanations for this lack of connection.

First, it is possible that we were not using the best quality of life mea- sure. Unfortunately, resource constraints coupled with the unique aspects of identifying appropriate control groups drove the decision to use the SF12 as the measure of participant quality of life.

Second, it is possible that the overall detrimental environment in which MedShare participants find themselves masks any positive effects that MedShare participation provides. That is, it may not be possible for any measure of quality of life to be sensitive enough to register mild improve- ments because of prescription drug access in the face of poverty and gener- ally poor living conditions.

Table 6 T-Test for Matched Samples from Propensity Scores

Dependent Variable Study Mean t Sig

PCS12 MedShare 44.62 -1.42 .156 CHSS 45.98

MCS12 MedShare 49.02 0.88 .380 CHSS 48.16

N = 259

542 Evaluation Review

Furthermore, although MedShare participation provides a savings over the normal cost of prescription drugs, there is still a cost associated with acquiring drugs for a MedShare participant. Although the cost may be greatly reduced, e.g., paying $10 instead of $40, there is still an out-of- pocket expense. According to a rational actor model of behavior, an indi- vidual would not turn down such a cost savings given the potential importance of the medications. Unfortunately, even at a reduced cost, the relative benefit versus cost for these prescriptions is likely perceived differ- ently by someone living in poverty and for whom $10 or $15 dollars can mean the difference in meals or other goods and services that may be more immediately attractive than an abstract improvement in health, particularly if one’s health conditions are not acute.

Third, compliance could be a factor. Are MedShare participants taking their medications as instructed by physicians? Because we asked about compliance in the disease specific questions, we turned to an analysis of these disease subgroups to try to identify the impact of compliance.

Could Compliance be a Factor?

For those patients with asthma, diabetes, and hypertension, the follow- ing questions were asked:

Q1. During the last year, while taking your prescription medication for (asthma/diabetes/hypertension), did you always take your (asthma/diabetes/ hypertension) medications according to your doctor’s instructions?

Q2. During the last year, did you not receive prescription medications for (asthma/diabetes/hypertension) because you needed the money to buy food, clothing, or pay for housing?

We found that many MedShare participants were not compliant either by virtue of not having taken their medications per instructions or because they needed the money for other necessities (see Table 7).

Having identified a pattern of noncompliance, we anticipated that physi- cian noncompliance and financial noncompliance did not overlap com- pletely. That is, noncompliance overall would be larger than for either physician or financial noncompliance separately. The results of drawing out these distinctions are displayed in Table 8.

Thus, once overlap among the groups is accounted for, noncompliance for participants among all three disease specific groupings is more than 40%. To separate out and test for the effects of compliance and noncompliance on the

Shaw, Carrozza / MedShare and Prescription Drugs 543

Table 7 Compliance Among MedShare Participants

No, did not No, did not Yes, always always take go without Yes, did go took meds meds per meds for without meds

Patient per physician physician financial for financial diagnosed instructions instructions reasons in reasons in with: in past year in past year past year past year

Compliant Noncompliant Compliant Noncompliant Asthma 75.1 24.9 55.6 44.4

(130) (46) (99) (79) Diabetes 83.0 17.0 59.7 40.3

(186) (38) (132) (89) Hypertension 84.8 15.2 62.5 37.5

(368) (66) (275) (165)

Table 8 Compliance Among the Three Disease Groupings

Noncompliant, Noncompliant, did not did go

always take without Noncompliant, meds per meds for did not follow

Patient physician financial instructions diagnosed instructions reasons and financial Overall with: Compliant in past year in past year reasons Noncompliance

Asthma 43.9 10.4 31.2 14.5 56.1 (76) (18) (54) (25) (97)

Diabetes 52.3 7.2 31.4 9.1 47.7 (115) (16) (69) (20) (105)

Hypertension 56.1 6.0 28.6 9.2 43.9 (243) (26) (124) (40) (190)

models, we created a dichotomous variable. Compliant participants were coded 1 and noncompliant participants were coded 0. We then re-analyzed the disease specific regression models incorporating the compliance variable. Essentially, the primary results were relatively unchanged in the models such that Table 9 provides a summary of just the compliance variable from each model.

544 Evaluation Review

Table 9 shows that in four of the six models, compliant participants expe- rienced statistically significant improvements in their physical and mental quality of life. These improvements amounted to a 4-point increase in mental quality of life for asthmatics; a 3-point increase in mental quality of life for diabetics; an almost 4-point increase in physical quality of life and just over a 3-point increase in mental quality of life for hypertensive participants. It is also possible that there is some measurement error in terms of compliance. Cognitive recall over a 12-month period is such that compliance may demon- strate an even stronger effect if the error could be reduced. Ultimately though, we must be cautious of our interpretation of these findings. They are related only to the disease subgroups; they primarily affect the mental rather than the physical quality of life; and they are nominal gains at best.

At this point, we feel that the findings regarding MedShare are mixed. The evidence presented here points to little or no programmatic effect on health outcomes via the MedShare program. Conversely, MedShare has been very successful in at least attracting clients and thereby expanding access to health services albeit with little impact on health outcomes. There may however be a slight positive programmatic effect if compliance is accounted for; however, this conclusion requires additional study and the preliminary evidence from this study does not hold out hope for strong effects. The results do however point to a significant theoretical finding: access is a necessary but not a suffi- cient step in promoting healthy outcomes.

Table 9 Summary Effects of Compliance Variable in

Disease Specific Regression Models

Physical SF12 Mental SF12

B (se) Sig B (se) Sig

Asthma model: always takes asthma meds 0.772 (1.904) .686 4.083 (1.766) .022*

Diabetes model: always takes diabetes meds −0.612 (1.837) .740 3.143 (1.472) .034*

Hypertension model: always takes hypertension meds 3.689 (1.232) .003** 3.201 (1.083) .003**

* sig at p < .05 level, ** sig at p < .01 level, *** sig at p < .001 level.

Shaw, Carrozza / MedShare and Prescription Drugs 545

Discussion

As it stands, the current study tends to emphasize that there is little con- nection between MedShare and improved quality of life. We stand by the utilization of the SF12 as the quality of life measure for the study because of the program and resource constraints involved; however, different measures may provide different insights. It deserves to be noted again that the SF12 was necessary to allow us to compare the MedShare participants to similar individuals surveyed in the CHSS (i.e., our external comparison). The use of a different measure would have severely hampered our ability to make com- parisons. Also, compliance seems to play into the findings; however, it is unclear to what extent compliance is an issue. Where we were able to exam- ine compliance, it tends to show up as a statistically significant issue, but the strength of its impact in the current study seems minimal at best. Thus, with- out further examination, it is difficult to suggest just how important compli- ance is to MedShare outcomes. However, the study does point to important theoretical issues related to access and the evaluation of health policies.

We feel that health policymakers tend to suffer from an “if you build it, they will come” syndrome. According to this idea, policymakers tend to inherently associate increased access with improved health care outcomes. The logic underlying this syndrome is sound because access remains a critically impor- tant link in the health care policy chain. However, it is not the last link. Yet, because in many ways, it is relatively straightforward and discernable (con- structing new facilities and programs, providing subsidies for patients), we overemphasize our policy efforts around access. Similar to other policy domains though, measuring outcomes and determining impact are much more difficult and less politically sexy; however, it is at this postaccess stage that we need to concentrate more effort than we previously have. Our current study highlights how easy it is to see outward signs of programmatic success and fall into the assumption that a program is therefore having the anticipated impact. If there had been no desire to quantify this perceived success to attempt to acquire additional funding, this study would likely never have been conducted and policymakers would continue to assume a nexus between MedShare and improved community health outcomes. These findings however illustrate that (a) it is important not to fall into the “if you build it, they will come” syndrome and ignore postaccess issues and (b) it is vital to devote resources and efforts to program evaluation to gauge the broader impact of any given policy.

Notes

1. Although in-person surveys would have been the optimal solution there were a number of mitigating factors including cost, coverage, availability of clinic personnel, and comparability

546 Evaluation Review

with the Community Health Status Survey that caused us to focus our attention on a tele- phone survey.

2. We considered whether to use an ordinal time scale (e.g., 1, 2 . . . 5) for year of entry or a set of dichotomous variables. After testing for curvilinearity, we opted to utilize dichotomous variables to account for year of entry into the program (Hardy, 1993).

References

Anderson, R. (1995). Revisiting the behavioral model and access to care: Does it matter? Journal of Health and Social Behavior, 36, 1–10.

Bingham, R. D., & Felbinger, C. L. (2002). Evaluation in practice (2nd ed.). New York: Seven Bridges Press.

Bowling, A. (1997). Measuring health, a review of quality of life measurement scales. Philadelphia: Open University Press.

D’Agostino, R. B. (1998). Tutorial in biostatistics, propensity score methods for bias reduc- tion in the comparison of a treatment to a non-randomized control group. Statistics in Medicine, 17, 2265–2281.

Ensor, T., & Cooper, S. (2004). Overcoming barriers to health service access: Influencing the demand side. Health Policy and Planning, 19, 69–79.

Freiman, M. P. (1998). The demand for health care among racial/ethnic subpopulations. Health Services Research, 34 (1, Part II), 215–227.

Hamilton, L. C. (1992). Regression with graphics. Belmont, CA: Doxbury Press. Hardy, M. (1993). Regression with dummy variables. Thousand Oaks, CA: Sage. Institute of Medicine. (2001). Coverage matters, insurance and health care. Washington, DC:

National Academies Press. Institute of Medicine. (2002). Unequal treatment: Confronting racial and ethnic disparities in

health. Washington, DC: National Academies Press. Parsons, L. S. (2000). Using SAS software to perform a case-control match on propensity score

in an observational study. Poster 225–25 presented at the SAS Users Group International Conference in Indianapolis, IN.

Robert, S. (1999). Socioeconomic position and health: The independent contribution of com- munity socioeconomic context. Annual Review of Sociology, 25, 489–516.

Schlenk, E. A., Erlen, J. A., Dunbar-Jacob, J., McDowell, J., Engberg, S., Sereika, S. M., et al. (1997). Health-related quality of life in chronic disorders: A comparison across studies using the MOS SF-36. Quality of Life Research, 7, 57–65.

Shi, L., & Singh, D. A. (2004). Delivering health care in America, a systems approach. Sudbury, MA: Jones and Bartlett Publishers.

Spector, P. (1981). Research designs. Thousand Oaks, CA: Sage.

Thomas Shaw is an assistant professor at the University of South Alabama where he teaches courses in political science and public administration with a focus on policy analysis and evaluation.

Mark Carrozza is a senior research associate at the Institute for Policy Research and direc- tor of the Southwest Ohio Regional Data Center, Institute for Policy Research, Cincinnati.