Program Implementation & Evaluation

TheChallengesforPolicyResearch.pdf

Home >Political Science homework help >Program Implementation & Evaluation

s we approach the 21st century, the public seems increasingly disen-

chanted with the record of govern- ment, and less and less inclined to believe in the value of empirical analysis as a guide to action. Evidence of the loss of confidence in the public sector’s ability to operate effectively and efficiently is found in opinion polls, falling rates of electoral participation, and the rising influence of “anti-govern- ment” politicians. In such an environment, it is useful to reflect on the historical role that applied social science has played in the public sector and the role it might play in the future.

The Past Influence of Social Policy Research

Social scientific research can dramatically shape public policy, both directly and indirect- ly. Directly, research can prove pivotal in situ- ations where policymakers have reached con- sensus about goals but have inadequate infor- mation about alternative means of reaching them. Perhaps the clearest illustrations have been the impacts of demonstration research on guiding choices among different types of housing subsidies, personnel retraining pro- grams, and law enforcement initiatives.

Indirectly, research can provide a cogni- tive backdrop to decisionmaking, sensitizing policymakers to new issues and crystallizing opinion about which social problems have reached intolerable proportions. Social scien- tific analysis has often played a larger role in shaping public perceptionof problems than in directly influencing policy responses to them.

In 1968, for example, President Johnson appointed former Illinois Governor Otto Kerner to chair a commission of social scien- tists to report on the causes of urban civil dis-

orders. Though the Kerner Com- mission’s findings resulted in a lim-

ited number of policy responses, its work fundamentally changed public opinion and played an educational role that, even today, shapes pol- icymakers’ conceptions about urban racial issues.

Another example is the reformulation of urban pover-

ty provided by William Julius Wilson’s 1987 book, The Truly

Disadvantaged.Through quantitative and qualitative analyses, Wilson riveted atten- tion on deindustrialization and the spatial con- centration and social isolation of its victims. His views laid the intellectual groundwork for the comprehensive, place-based empower- ment zone policy that materialized seven years later.

Future Challenges Despite the influence on policy and pub-

lic perception that research has at times wielded, a number of forces threaten to weak- en its future impact. These include external challenges—such as many Americans’ loss of confidence in the public sector, the trend toward devolution without evaluation, and the emergence of morality politics—and internal trends including what I call the “eso- terica fetish” and advocacy research.

Loss of Public Confidence

Much of the public’s loss of confidence in government can be traced to factors having only a tangential link to policymaking itself.

The Challenges for Policy Research in a Changing Environment

George Galster

THE URBAN INSTITUTE

A series on the long-term

forces affecting U.S. social policy

TH E

F U

TU R

E O

F TH

E F

U TU

R E

O F

TH E

PU BL

IC S

EC T

TH E

PU BL

IC S

EC T

O R

No. 7, November 1996

A Research can

prove pivotal in situa- tions where policymakers have reached consensus

about goals but have inade- quate information about

alternative means of reaching them.

These factors include corruption among public figures, negative cam- paign tactics, and the media’s empha- sis on the personal foibles of candi- dates for political office. However, another force—money and special interests—is directly tied to the policy process and, as such, can have a sig- nificant impact on policy research. If in the foreseeable future campaign financing undergoes only rhetorical rather than actual reform, and the influence of monied special interests continues unabated, policy research will be confronted by two growing obstacles.

First, special interests will further corrupt social science by sponsoring more “advocacy research,” an exercise in creating political ammunition by selectively culling or distorting infor- mation to support pre-ordained conclu- sions (more on this below). Second, the public’s increasing skepticism toward government should provide fertile elec- toral ground for across-the- board budget cuts and disman- tling of social programs, rather than targeted budget actions based on the efficacy of such programs. In such an environ- ment of defensive public policy, it is likely that financial or polit- ical support for the public spon- sorship of policy research will diminish, and less notice will be paid to evidence produced by social scientists concerning the effects of public programs.

Devolution without Evaluation

Disillusionment with the ability of public policy to successfully confront complex social issues and to operate in a fiscally responsible manner has led to a loss of confidence in federal poli- cy in particular. The federal govern- ment is widely regarded as being para- lyzed by large political egos that put personal power ahead of the public good; by an inept, bloated bureaucra- cy; and, more recently, by gridlock between Congress and the president. This view, which has been building up for a number of years, is one source of initiatives to devolve power from the federal government to state and local governments. Underlying this devolu- tion are incentives for states to experi- ment with a variety of social program

reforms, backed up by a series of rev- enue block grants allowing expanded latitude for state use of federal funds.

Experimentation at the state level is a practical way to test ideas for reform and potentially can prove valu- able to the nation as a whole. Creating 50 different laboratories, however, will make some kinds of policy research more difficult, unless there is a con- comitant increase in commitment to evaluation. Evaluating a multitude of variations on a particular program theme and applying findings from individual states to other states are truly formidable tasks. It has been expensive and time-consuming for program analysts to evaluate and reach consensus even about major national programs such as CETA employment training and Head Start. It seems unlikely that government will be able to support a similar level of effort to replicate and refine assessments of 50 program variants.

At the same time, it may be unre- alistic to expect states themselves to be able to evaluate their experiments in a way that will be meaningful across states. Many state agencies either lack policy evaluation and research divisions altogether, or use standards for program evaluation that are not comparable to those set by their federal counterparts. The quanti- ty and quality of many state-initiated evaluations of state-sponsored pro- grams may thus prove problematic. The inability to adequately test these state experiments will surely inhibit development of program enhance- ments and may ultimately erode polit- ical support for them.

Morality Politics

The alienation and secularization of large segments of our society have played a role in the growth of what I

call “morality politics,” the emer- gence of single-issue constituencies whose policy positions are primarily grounded on unwavering moral prin- ciples. Efforts to reinvigorate faith- based values and reinforce virtuous behavior have numerous laudable consequences. I worry, however, that in the face of complex contemporary social problems and the increasingly complicated solutions they require, morality politics will lead more peo- ple to “vote with their hearts, not with their heads.” Research findings can be antithetical to cherished values and, as Isabel Sawhill has recently observed, “when research and values collide, values will always win.” What is more disturbing is that the worth of any research findings may eventually be ignored as legislators resort to a politics of morality and opt out of policy debate based on objec- tive information.

The Esoterica Fetish

Beyond the external chal- lenges to policy research lie internal weaknesses that threat- en to erode the credibility and effectiveness of the research community. One of these is social scientists’ proclivity toward technical jargon that results in their communicating with one another rather than with policymakers and the

public. There are few incentives for social scientists, especially those in universities, to popularize their find- ings for a lay audience. The profes- sion typically rewards research that is published in highly technical forums rather than in easily communicated language designed to reach wider audiences. It seems that only social scientists who have already “proven” themselves can afford to write in a more popular vein. Because of the dearth of experts who are also effec- tive communicators, even educated citizens do not always understand the issues, and increasing numbers of the public and of politicians turn to sim- plistic nostrums for “easy” solutions.

Advocacy Research

Legitimate social science can and should provide higher standards of objectivity and methodological sound-

T H

E F

U T

U R

E O

F T

H E

P U

B L

IC S

E C

T O

R N

o .

The inability to adequately test state experiments will surely inhibit development of program enhancements and may ultimately erode political support for them.

ness than those of advocacy research. Advocacy research attempts to mimic certain features of social science, such as use of theory and statistical tech- niques, but fails to be genuinely objec- tive. This is primarily because, whether based on liberal or conserva- tive ideology or simply arising out of a particular cause not linked to any ideology, advocacy research starts out with a firmly held position rather than with a potentially rejectable hypothe- sis. Such research looks for those facts and theories that bolster its position, finding reasons to reject those that fail to support it.

A substantial amount of applied social science has for some time been commis- sioned and funded by govern- ments and special interests who often keenly hope for a particular outcome. Even sup- posedly nonpartisan think tanks and universities have not remained untainted by spe- cial interest groups that may supply research funds. Under such condi- tions, the temptations for bias may be enormous. Seasoned social scientists can typically identify research find- ings that may have been influenced by the sponsor and interpret them accord- ingly. Other users of research often cannot. The news media, for example, may not have the time or ability to judge the soundness of scientific evi- dence on all sides of an issue. For the general public, the task can be even harder. Politicians’ continued reliance on partisan research as artillery for political battles will, I fear, sully the reputation of objective social science.

Maintaining Standards for Social Science Research

The weakening of the role of pol- icy research comes at a time when a strong voice has never been more important. Evaluation research is cru- cial for policymakers who wish to scrap ineffective programs while expanding successful ones. Deciding what to cut and what to keep based on

objective, scientifically sound evalua- tions rather than on the potentially biased assessments of special inter- ests will help restore the public’s faith in the public sector. Even in a regime of morality politics, research remains vital as a means of demonstrating alternative strategies and ascertaining which might best help us reach highly valued goals.

I offer three practical suggestions for helping social science policy research effectively meet the chal- lenges of the future outlined above. First, legislatures should more fre- quently employ blue-ribbon panels of respected social scientists to produce

concise, readable, state-of-the-art briefs on topics of forthcoming legis- lation. Such briefs could also form the core of legislative hearings.

Second, new programs should be legislated with an accompanying requirement and appropriation for eval- uation after a point at which program- matic benefits might be expected to accrue. This procedure has been used in the past. Examples include the Fair Housing Initiatives Program, autho-

rized under the Housing and Com- munity Development Act of 1987, and the Youth Employment Demonstration Project, authorized under the Youth Act of 1977. Federal waivers recently granted to states to experiment with wel- fare reforms have carried similar evaluation requirements. In the future, however, every state will need to evaluate its welfare initia- tives under the 1996 block grant legislation.

Third, proposals solicited from social scientists to conduct

such evaluations should be reviewed by government-convened panels of independent experts, so that to the greatest extent possible, applied social science can be subjected to a peer review process similar to that of the basic sciences.

To avoid becoming marginalized or, worse, a tool of scientific nihilism, policy research must be characterized by open inquiry into all aspects of social problems, a willingness to test conventional conceptions and conclu- sions, the most rigorous and sophisti- cated methods available, accessible reporting of the results, and open dis- cussion of the potential uses and mis- uses of research by future American policymakers.

N o

. 7 T

H E

F U

T U

R E

O F

T H

E P

U B

L IC

S E

C T

O R

Deciding what [programs] to cut and what to keep based on objective, scientifically sound evaluations rather than on the potentially biased assessments of special interests will help restore the public’s faith in the public sector.

George Galster is Hilberry Professor of Urban Affairs at

Wayne State University. He was

director of housing research at the

Urban Institute from 1994 to 1996.