Summer end 1

1

Article Review

The purpose of this evaluation was to examine the efficacy of a Rural Infant Care

Program (RICP) designed to reduce infant mortality rates in rural communities from nine states

(Gortmaker, Clark, Graven, Sobol, & Geronimus, 1987). The RICP is based on the notion that

high infant mortality rates are due to deficits in the perinatal system (e.g., system deficits--See

Table 1). Accordingly, the RICP proposes that by bringing together key personnel (e.g., local

providers, medical school personnel, state health department) improvements in the perinatal

system (e.g., training providers, increasing referral rates, regionalizing tertiary centers) can be

made which would lead to lower levels of infant mortality in rural communities (See Table 1).

The RICP provided funding to 10 medical schools with programs designed to improve

the delivery of health services to mothers and infants in rural areas. These programs aimed to

improve access to perinatal care, improve the transportation of sick neonates, upgraded

professional skills in rural hospitals and increase referrals of high-risk pregnancies to tertiary

centers (See Inputs/Obj. in Table 1). These ten sites were selected because they had infant

mortality rates above their state’s level for 1977, had a minimum of 1000 births per year and

were located in states with IPO projects.

A time series design was employed to examine whether the RICP reduced infant

mortality rates above those expected in the absence of the program (See figure 1). This design

has the advantage of controlling for both maturation and history effects. It allows researchers to

determine if the changes in infant mortality rates can be attributed to the RICP intervention.

However, a time series design can not control for instrumentation effects--the use of the

same/different instrument over various time periods. In the present study infant mortality rates,

natality data and vital statistics from various sources (e.g., National Center for Health Statistics,

2

State Health Departments, and published/unpublished State vital statistics) were utilized to

examine changes in mortality rates both pre and post-intervention. The reliance on multi-source

data can increase the potential for instrumentation effects (e.g., reliability/validity of the DV

measures) and thus limit the strength of the results of the study. Infant mortality rates from

various sources (e.g., National Center for Health Statistics, State Health Departments, and

published/unpublished State vital statistics) were compared to rule out instrumentation effects.

The results of these analyses showed that there were no significant differences in infant mortality

rates. Thus, the design employed in the present study (partially) controlled for instrumentation

effects as well as maturation and history effects. In addition, this design has the advantage of

controlling for regression and (partially controlling) selection effects as well. These design

characteristics are noteworthy since they allow the evaluators to rule out a number of alternative

explanations for the results including (1) that the program effects were due spillover effects from

the IPO projects, (2) that the program effects were due to some special event co-occurring with

the intervention (e.g., history), (3) that the program effects were due to some change in the

participants (e.g., maturation), and (4) that the program effects were due to the composition of

the participants (e.g.,selection; regression).

The use of a time series design is particularly appropriate for evaluating the RICP

intervention. This design take advantage of the fact that multiple data points can be retrieved for

consecutive time periods both before and after the initiation of the RICP (See figure 1). In fact,

three separate comparisons tests were conducted to determine whether the RICP was effective.

These included a comparisons between RICP and non-RICP areas (e.g., non-RICP areas

included counties not targeted to receive RICP funding with lower infant mortality rates), a

comparison between RICP areas and eligible RICP states not funded, and a comparison between

3

RICP areas to matched rural areas with IPO funding. Time-series regression models were fit to

examine whether the RICP was effective. Success of the RICP was determined by a change in

infant mortality rates beginning in 1979.

In general, the RICP was successful in reducing infant mortality rates in nine out of ten

sites. Time series regression models revealed that declines in neonatal mortality were

attributable to the RICP. Furthermore, there was a sharp drop in neonatal mortality beginning in

1979 in the RICP areas. By 1982-1984 the neonatal rates in the RICP areas were similar to those

found in non-RICP areas. In addition, there were no significant differences in postneonatal

activity associated with the RICP and there were no significant changes observed in the non-

RICP comparison areas. Thus, the authors conclude that “The RICP demonstrated the value of

local initiative in addressing these problems and showed that effective cooperation can be

achieved among local physicians and nurses, hospital administrators, local health departments,

state health departments, and tertiary hospitals” (Gortmaker et al., 1987, p. 114).

Though encouraging, the result of the present study have several limitations that pertaing

to the external validity of the results. First, as the authors note, the ten sites included in the study

were chosen because they were well organized. That is, the sites already had an existing network

which facilitated the implementation of the RICP intervention. This is particularly important

since it raises questions about the extent to which the RICP may be equally successful in rural

communities without such established networks (e.g., generalizability). The authors note that

despite this limitation infant mortality rates were still reduced. However, the magnitude of this

effect remains an open question. It may be that a selection factor may account for the program

effects.

4

A second limitation, also noted by the authors, concerns the lack of random assignment

of geographic areas to treatment and control groups. While there are ethical issues involved with

this decision (e.g., withholding of treatment to control group participants), it is important to

remember that a selection factor may be (partly) responsible for the results of the study. The

lack of random assignment compounds the potential for a selection bias already noted. For

example, it may be that patients who are the recipients of RICP benefits (e.g., pregnant women)

may be different from those attending other rural hospitals. Though, the authors tried to deal

with this issue by comparing Non-RICP areas with lower mortality rates (e.g., non-RICP

comparison areas), similar mortality rates (e.g., IPO-76 programs) and a matched rural area (e.g.,

IPO-78 programs), it remains unsolved. This issue becomes even more problematic when we

consider the fact that though there were overall reductions in mortality rates in the nine sites,

only “three of the reductions were statistically significant” (Gortmaker et al., 1987, p. 106). This

may suggest that the reductions in infant mortality may be due to some selection X treatment

effect. That is, the combination of a specific site and the treatment used at that site.

Thirdly, because the nature of the intervention required prior planning and coordination

on the part of network participants it is possible that this activity alone may account for the

obtained results. This is clearly an alternative hypothesis which cannot be ruled out by the study.

In fact, the authors note that program meetings in the targe areas “were mostly informational [at

first]. . . Important contacts, however, were made at this stage. . .local physicians. . . often

[met] for the first time [with] doctors from tertiary centers” (Gortmaker et al., 1987, p. 97).

Thus, the extent to which these early meetings may have influenced the results of the study are

unknown.

5

Finally, a larger question remains unanswered--are the expenses associated with the

implementation of the RICP justified by the results? A cost-benefits analysis (Shortell &

Richardson, 1978) would shed some light on this issue. If the cost to benefit ratio was such that

the benefits outweighed the cost then clearly the program could be deemed effective. On the

other hand, if the cost outweighed the benefits, then it would certainly call into question the

desirability of replicating such efforts.

Clearly, these issues raise some concerns about the implementation of this program in

other hospitals. In the best case scenario, this program could be implemented in hospitals with

existing networks of providers that are willing to participate in the RICP intervention. In the

worst case, replication of this project in a random fashion would not be desirable. Given that

only three sites reported statistically significant reductions in infant mortality rates, careful

considerations must be given to the fit between the RICP intervention and the characteristics of

the setting in which it will be implemented.

References

Gortmaker, S.L., Clark, C.J.G., Graven, S.N., Sobol, A.M., & Geronimus, A. (1987). Reducing

infant mortality in rural America: Evaluation of the rural infant care program. Health

Services Research, 22(1), 91-116.

Shortell, S. M., & Richardson, W. C. (1978). Health program evaluation. Saint Louis, MO. The

C. V. Mosby Company.

6

Table 1. Process Model of Evaluation.

Preexisting Conditions Program Components Intervening Events Impact/Consequences

High infant mortality

Individual Differences

-Low SES

-underinsurance

-isolation

-low Ed. levels

-inadequate housing

System Deficits

-Poor communication

among providers

-Low referrals rates to

teriary centers

-Limited Knowledge,

Skills, Abilities

(KSAs) among

providers

-No regionalization of

tertiary centers

-Limited success with

births of LBW infants

Improve delivery

services to mothers &

infants in target areas

Inputs/Obj.

-Increase access to

perinatal centers

-Improve

transportation of sick

neonates

-Upgrade KSAs of

providers

-Increase referral of

high-risk pregnancies

Resources

-Special funding

-Medical School

-Local providers

-State health Dept.

-Adm. Support

-Travel Costs

Activities

-Conduct program

meetings

-Identify problems

-Conduct needs

assessment

-Provide training

-Upgrade facilities

-Transport sick

neonates

-Expand Well-child

clinics

-Develop High-risk

OBGYN clinics

External

-IPO projects

-Announcement of

RICP

Internal

-Organization of

provider network

-Regionalization of

services

-Lack of interest in

RICP

-Reduce infant

mortality

-Increase referrals

-Greater cooperation

/communication

among providers

-Increase KSAs of

providers

7

Figure 1. Evaluation Design.

O1965 O1966 O1967 O1968 O1969 O1970 O1971 O1972 O1973 O1974 O1975 O1976 O1977 O1978 X1979 O1979 O1980 O1981 O1982 O1983

O1984 O1985