for A-plus writer only

G et

ty Im

ag es

Learning Objectives

In this chapter you will learn to:

• Identify and know how to discuss key concepts in the processes of policy implementation and evaluation.

• Describe various theoretical approaches to implementing and evaluating public policy.

• Identify the key actors and agencies involved in policy implementation and evaluation.

Implementing and Evaluating Policy

6

the80472_06_c06_189-232.indd 189 11/21/12 12:51 PM

Section 6.1 What Is Policy Implementation? CHAPTER 6

Public policy becomes a reality when it is implemented. Simply, implementation of federal policy occurs after a bill becomes a law and involves a number of activities designed to ensure the policy decision’s execution. It is important to understand that policy implementation is not always successful, nor does it always solve the prob- lem for which the policy was adopted. The reasons for this are varied. Often, policy fail- ure results from problems with its original design; sometimes the behavior of the policy actors responsible for the implementation causes it to fail. However, because no rulebook explains how implementation occurs, the failed implementation is often explained by using theoretical approaches and strategies that attempt to identify the obstacles to suc- cessful policy implementation.

In the real world of policy making, success or failure of a policy can be measured through policy evaluation. Policy evaluation occurs after a policy has been implemented; it is the study of what happens as a result of policy implementation. Different types of policy evaluation can be conducted, and each has its own distinctive set of tools and methods. Just as policy implementation faces challenges and obstacles, so does policy evaluation. Over the last 30 years, research into the policy evaluation phase of the policy process has attracted considerable interest among policy makers at all levels. Evaluation provides evidence to policy makers and other policy actors about more than just a policy’s success or failure. It can also provide feedback on what is needed to improve a policy or how the policy goals may be achieved.

6.1 What Is Policy Implementation?

Policy implementation is the stage at which government executes an adopted policy as specified by the legislation or policy action (Anderson, 1997; Dye, 2001; Meter, 1975). It is the series of governmental decisions and actions directed toward putting an already decided mandate into effect. At this stage, various government agencies and departments responsible for the respective area of policy become formally responsible for implementing the particular policy. Until policies are implemented, no insight can be derived about potential effects they may have on a target population or the society, nor can the policy be judged a success or failure. Table 6.1 summarizes the broad components essential to understanding policy implementation. Implementation matters because how well a policy is implemented has much to do with assessing its success or failure.

Table 6.1: Policy implementation

Broad Components of Policy Implementation

• Policy Implementation: Government executes an adopted policy as specified by the legislation or policy action

• Policy action occurs to address a recognized policy problem. • Respective administrative agencies execute policies • Selected instruments are applied that are reflective of the legislative mandate, bureaucratic inter-

pretation, and capacity. • Specified target populations and society experience the first tangible effects of the policy once

implemented.

the80472_06_c06_189-232.indd 190 11/21/12 12:51 PM

Section 6.2 The Relationship Between Policy Design and Policy Implementation CHAPTER 6

Note that the implementation of a policy sometimes requires that those responsible cre- ate programs that will help deliver the goals of a policy. Simply, a policy is an overarching course of action that describes how government is going to deal with a problem; a program is a plan to accomplish a specific goal that will help deliver the policy. In other words, programs are the ways in which policy is often made operational. For example, under President Lyndon B. Johnson in 1964, the government adopted the War on Poverty policy as an attempt to end poverty in the United States. This was a very general and vague goal that said nothing about how the goal would be achieved. Thus, a number of programs were created under the policy that attempted to address different aspects of the problem of poverty. One such program was Head Start, which provides education, health, nutri- tion, and parent involvement services to low-income children and their families.

6.2 The Relationship Between Policy Design and Policy Implementation

As a stage in the policy process, policy implementation is especially important because no adopted policy action—whether a law, an executive order, or a court decree—can in itself alter the nature of the problem that it was designed to solve. Policy implementation is the intervening stage between the desire for change and actual change in the problem. Policy implementation is rarely a linear process, and it usually involves multiple actors who must cooperate with each other if implementation is to be successful (Anderson, 1997; Dye, 2001; Meter, 1975). The relationship between design and implementation is described graphically in Figure 6.1.

Figure 6.1: Relationship between policy design and implementation

Policy implementation is a stage between the desire to change a problem and when the actual change occurs.

Policy ImplementationPolicy Design

+

+

Perceived Policy

Problem

Agency Role

Acceptable Policy

Instruments

Execution of Policy

Intended Impacts

Unintended Impacts

Adopted Policy Design

Problem and Target Pop

Acceptable Solution

Mandate for Implementation

Rules and Regulations

Agency Role

the80472_06_c06_189-232.indd 191 11/21/12 12:51 PM

Section 6.2 The Relationship Between Policy Design and Policy Implementation CHAPTER 6

As Chapter 5’s discussion of policy formulation showed, policy design involves a trade- off between formulating a politically acceptable policy and formulating one that will solve the problem effectively. To be successful, a public policy must be designed to address the problem’s actual nature. Also, a strong policy design, if it is to aid implementation, must assign the appropriate agency and instruments with which to change the target popula- tion’s behavior. Therefore, a poorly designed policy not only may fail to solve the problem but also, once implemented, produce unintended effects.

The relationship between policy design and implementation can be illustrated very sim- ply in a 2-by-2 table. As Figure 6.2 indicates, poor policy design and poor implementation is the worst-case scenario. In this case the design of the policy does not address the logic of the problem, and implementation is poorly executed. In contrast, the policy with a good design and good implementation is the best-case scenario because it achieves the goals of the policy, is executed well, and is by definition effective policy. If a policy is poorly designed but implemented well, it results in unintended impacts or unknown effects. In terms of policy with a good design and poor implementation, that policy will prove inef- fective. In this case, because the policy was poorly implemented, the positive impacts from the well-designed policy solution fail to materialize. In general, such policies will have little or no impact on the specific policy problem.

Figure 6.2: How the relationship between policy implementation and policy design affects the public

A poorly designed policy might fail to solve the problem but might also result in unintended impacts.

Implementation

Policy Design

Very Poor Outcome

Intended and unintended impacts are negative

Poor Good

Poor

Good

Unintended Impacts

Unknown effects

No Impact

Ineffective Policy

Effective Policy

Intended impact achieved

the80472_06_c06_189-232.indd 192 11/21/12 12:51 PM

Section 6.2 The Relationship Between Policy Design and Policy Implementation CHAPTER 6

As Figures 6.1 and 6.2 illustrate, inescapable interrelationships exist among policy design, policy implementation, the impacts of a policy, and the consequences of the policy’s exe- cution. The ideal is to develop a policy that, when implemented, achieves the desired goals and solves the identified problem. The political realities of formulation and adop- tion that result in poor policy design can lead to the policy’s unsuccessful implementa- tion. For example, a policy that is too vague can lead to confusion about the policy’s goals, the implementing agency responsible, and the identity of the target population. In con- trast, a policy that is too specific and restrictive in its design may limit necessary agency discretion and prevent changes that could improve the success of implementation. The most negative outcome for the policy process, however, arises when the solution to be implemented is not appropriate for the identified problem. In other words, the problem is misunderstood, the instruments assigned do not match, and the agency responsible may be incapable or unwilling to execute the policy. For example, many cities have established policies that require food retailers with more than a certain number of locations to display calorie counts for each of their menu items. This way, diners can see how many calories they might consume. This policy obviously is trying to deal with the problem of obesity, but is this an appropriate way to deal with the problem? Obesity is not solely about what people eat away from home; it relates to multiple lifestyle choices.

When policy design cannot alter the problem behavior, successful implementation proves irrelevant (Edwards, 1984). Basically, a poor policy design that is implemented will not miraculously become good policy in the future. A poor policy design and poor imple- mentation merely accentuates the failures of policy. The War on Poverty is an example of poorly designed and poorly implemented policy. It is also an example of a policy with vague goals. The policy’s goal was to end poverty, but what type of poverty? Can a single policy really end poverty? The policy did not discuss jobs or income levels. The policy was pushed through very quickly, and as a consequence policy makers paid little attention to how it would be implemented. Thus, a whole range of programs was established under the policy, all with agency structures that proved unwieldy and made implementation dif- ficult because these new administrative agencies answered to the federal government— even though they were operating at the local level—ignoring potential contributions of local policy actors. The task of the War on Poverty policy—solving poverty in the United States—was immense, and the design provided little direction on how to achieve success. In sum, only a well-designed policy and effective implementation will achieve the policy’s intended impacts. In a perfect world, policy designers would objectively define a problem, determine the best applicable instruments with which to change the target population’s behavior, and assign the policy to the appropriate set of governing agents to implement the solution. The reality of policy design, however, is that policy problems are often speci- fied incorrectly, instruments may be selected chiefly by ideological or political beliefs, the target population may reflect a bias for or against certain groups, and the agency assigned responsibility for the policy may lack the resources to implement it.

To design policy effectively and implement it successfully, those who develop and imple- ment the policy must address some or all of the following questions during the policy process:

• Given the nature of the problem, was the design of the policy appropriate? • Does the design assist or complicate policy implementation? • Is the agency organizationally capable of administering the program or policy?

the80472_06_c06_189-232.indd 193 11/21/12 12:51 PM

Section 6.3 Bureaucracy as the Major Actor in Policy Implementation CHAPTER 6

• Can the instruments be implemented effectively? • Are there political obstacles to effective, efficient implementation? • What impact will the policy solution, once implemented, have on the target

population(s)? • How essential are the administrative actors to the policy’s success? • Have political factors prevented or made policy implementation difficult?

6.3 Bureaucracy as the Major Actor in Policy Implementation

As with every stage in the federal policy process, a variety of institutional and non-institutional actors is involved in shaping the way policy implementation evolves. The role of such policy actors can be roughly divided into two categories: those who implement policy and those who influence the implementation of policy. Among these actors, the bureaucracy is the major actor responsible for implementation. The sec- ond set of actors includes the president, Congress, the courts, and interest groups.

For official decision makers, the bureaucracy represents their executive arm. Laws give these governmental organizations that compose the bureaucracy—and that include such vast agencies as the U.S. Department of Justice or the Social Security Administration—the responsibility to define and execute policy decisions. No governmental decision can be implemented without administration and execution by the bureaucracy, so its importance cannot be overstated (Wilson, 1997).

Bureaucracy’s ideal role is to ensure the effective, efficient implementation of government policies. Bureaucratic implementation is not intended to be a democratic exercise, as that would ensure a political and varying pattern of implementation (Weber, 1997). Instead, bureaucracy is charged with the direct, unemotional, and methodical execution of the policy decision. The power of the bureaucracy, however, is not simply based on its admin- istrative function of implementing policy decisions.

The real power of the bureaucracy lies in its ability to define the details of the policy itself, as well as those details that will affect the method of implementation. It is highly unlikely that any given policy action will define all critical aspects of implementation. Often bureaucratic agencies find they have discretion to decide on what and how a policy mandate is to be executed because the policy or legislation is equivocal (Anderson, 1997). The ambiguity of legislation and the need to establish rules for policy implementation grant the bureaucracy the power of rule making, which is the process by which rules and regulations are subject to the constraints of the Administrative Procedures Act of 1946. According to this law, rule making involves the following:

• Proposed rules: Proposed changes to rules are published in the Federal Register. Public comments and suggestions on the changes are requested.

• Interim rules: Rules and regulations issued on emergency or temporary basis. Public comment is possible before issuance of final rule.

• Final rules: Formal decisions on new rules that alter existing regulations. Public comments introduced previously may result in modifications to rules.

the80472_06_c06_189-232.indd 194 11/21/12 12:51 PM

Section 6.3 Bureaucracy as the Major Actor in Policy Implementation CHAPTER 6

Rule making is significant because seldom—if ever—will an adopted policy action, such as a law, encompass all of the necessary details of how, when, and where the policy is to be implemented. Rules define the basis and conditions in which a policy will be imple- mented by the given agency.

Because of such power over implementation, the bureaucracy enjoys considerable discre- tion in interpreting the actual intent, method, and scope of a policy decision. This dis- cretion means that the bureaucracy can, to some extent, redefine the intent of a policy action—in effect becoming policy makers in the process. Hence, considerable power lies with the 15 major departments that constitute the executive branch and federal bureau- cracy (see Table 6.2).

Table 6.2: Executive departments that are part of the bureaucracy and that have representation in the president’s cabinet

Department and Year Established

Principal Mission Important Subagencies Budget (in Billions of Dollars) 2011*

Percentage of Federal Budget

State (1789)

• Promote peace and stability

• Create and expand markets

• Conduct treaty negotiations

• Coordinate international activities

• Bureau of Arms Control • Bureaus of Political Affairs,

Democracy, Human Rights, and Labor

• Diplomatic Security

$28.9 1%

Treasury (1789)

• Promote prosperous and stable U.S./world economy

• Manage government finances

• Safeguard financial system

• Secret Service • Bureau of Public Debt • Bureau of Engraving and

Printing • Internal Revenue Service

532.3 15

Interior (1849) • Land conservation • Fire management • Service to Native

Americans

• Bureau of Land Management

• Fish and Wildlife Service • National Park Service • Bureau of Indian Affairs

13.0 0.4

Justice* (1870) • Enforce law • Defend interests of United

States • Administer and enforce

immigration laws • Supervise federal

corrections

• FBI • Bureau of Prisons • Immigration and Natural-

ization Service • Civil Rights Division

33.5 1

(continued)

the80472_06_c06_189-232.indd 195 11/21/12 12:51 PM

Section 6.3 Bureaucracy as the Major Actor in Policy Implementation CHAPTER 6

Table 6.2: Executive departments that are part of the bureaucracy and that have representation in the president’s cabinet (continued)

Agriculture (1889)

• Ensure safe food supply • Support sound

development • Expand markets • Reduce hunger

• Food Safety and Inspection Service

• Animal/Plant Health Inspection Service

152.1 4

Commerce** (1913)

• Promote U.S. competitiveness

• Enhance economic competitiveness

• Provide effective manage- ment of nation’s resources

• Bureau of Census • Bureau of Economic

Analysis • National Weather Service • Patent and Trademark

Office • National Oceanic Atmo-

spheric Administration

148.0 0.3

Labor (1913)

• Promote welfare of job seekers

• Improve work conditions Protect retirement and health care

• Promote collective bargaining

• Bureau of Labor Statistics • Occupational Safety and

Health Administration • Pension and Welfare Ben-

efits Administration

148.0 4

Defense*** (1947)

• Deter war and protect the security of the United States

• Departments of Army, Air Force, Navy

• Joint Chiefs of Staff • National Security Agency

739.7 20

Housing and Urban Development (1965)

• Create opportunities for homeownership

• Enforce nation’s fair hous- ing laws

• Help homeless • Spur economic growth

60.8 2

Transportation (1966)

• Ensuring fast, safe, efficient, accessible, and convenient transportation system that meets vital national interests

• Federal Aviation Administration

• Transportation Security Administration

• National Mortgage Association

• Office of Healthy Homes • Public and Indian Housing

Agencies • Office of Fair Housing

79.5 2

(continued)

the80472_06_c06_189-232.indd 196 11/21/12 12:51 PM

Section 6.3 Bureaucracy as the Major Actor in Policy Implementation CHAPTER 6

Table 6.2: Executive departments that are part of the bureaucracy and that have representation in the president’s cabinet (continued)

Energy (1977) • Enhance national security through energy policy

• Ensure integrity and safety of nuclear weapons

• Increase domestic energy production

• Federal Energy Regulatory Commission

• Office of Nuclear Energy • Energy Information

Administration

44.6 1

Health and Human Service**** (1979)

• Protect health of Americans

• Provide essential social health and welfare services

• Food and Drug Administration

• Centers for Disease Control • National Institutes of

Health (NIH) • Administration for Children

and Families

909.7 25

Education (1979)

• Ensure equal access to education

• Promote educational excellence

• Federal Student Aid • Office of Elementary and

Secondary Education • Office for Civil Rights

79.4 2

Veterans Affairs (1988)

• Serve America’s veterans • Provide medical care, ben-

efits, and social support

• Veterans Health Administration

• Center for Minority Veterans

• Veterans Benefit Administration

141.1 4

Homeland Security (2002)

• Manage homeland defense • Respond to domestic

attack or disaster • Immigration

• Customs • Secret Service • Federal Emergency Man-

agement Agency (FEMA) • Bureau of Immigration

48.1 1

*Formed from the Office of Attorney General, 1789

**Formed from Office of Commerce and Labor, 1903

***Formed from Department of War, 1789

****Formed from Department of Health, Education, and Welfare, 1953

Sources: US Census Bureau, 2012; Cato Institute, 2012; USA.gov, 2012.

the80472_06_c06_189-232.indd 197 11/21/12 12:51 PM

Section 6.3 Bureaucracy as the Major Actor in Policy Implementation CHAPTER 6

In addition to the 15 executive departments, other government organizations also com- pose the federal bureaucracy. First, the Executive Office of the President (EOP) repre- sents the various agencies, offices, and councils that lack cabinet status but often serve powerful roles within the executive branch. New offices, for example, will emerge under each president, as occurred with the Office of Faith-Based Initiatives under President George W. Bush.

Second, legislative organizations such as the General Accounting Office (GAO), the Con- gressional Research Service, and the Congressional Budget Office (CBO) provide critical information and expertise to members of Congress and their staff. Such actors can be very helpful in policy design. The Congressional Research Service, for instance, provides critical background and research on policy issues and legislative proposals. The GAO conducts and provides policy analyses and formal program evaluations that help determine prob- lems and weaknesses in policy or program implementation. Similarly, the CBO provides budgetary projections and assistance to Congress as it prepares its yearly federal budget.

Third, independent agencies, regulatory boards and commissions, and public corpora- tions are also examples of the dozens of governmental entities within the bureaucratic realm. Examples of independent executive agencies include the National Aeronautics and Space Administration (NASA) and the Social Security Administration. In addition, independent regulatory boards operate outside the direct authority of the political arena and are created to regulate some aspect of the economic or political arena. These include the Federal Elections Commission (FEC) and the Federal Communications Commission (FCC). Government corporations—unique organizations created to fill a specific mission within the society currently not filled by the private market—include the Tennessee Valley Authority, the U.S. Postal Service, and Amtrak. Figure 6.3 describes the current organiza- tional structure of the federal bureaucracy.

the80472_06_c06_189-232.indd 198 11/21/12 12:51 PM

Section 6.3 Bureaucracy as the Major Actor in Policy Implementation CHAPTER 6

Figure 6.3: Organization of the federal bureaucracy

Interagency conflicts can emerge in this bureaucracy, but the federal bureaucracy is a fundamental part of successful policy implementation.

Adapted from USA.gov, 2012.

Department of Defense

Department of Energy

Department of Homeland

Security

Department of Transportation

Department of Treasury

Department of Commerce

Department of Agriculture

Department of Labor

Department of Housing and Urban

Development

Department of Health

and Human Services

Department of Education

Department of Justice

Department of Interior

Department of State

Department of Veterans

Affairs

U.S. CONSTITUTION

Executive Branch President and Vice President

Executive Office of the President White House Office Domestic Policy Council National Economic Council National Security Council White House Military Office Office of Science and Technology President’s Foreign Intelligence Advisory Board

Office of Administration Office of National Drug Control Office of National AIDS Policy Office of U.S. Trade Rep

Select Independent Establishments and Agencies

Advisory Council on Historic Preservation Commodity Futures Trading Commission Environmental Protection Agency Federal Communication Commission Federal Emergency Management Agency Federal Maritime Commission Federal Trade Commission National Aeronautics and Space Administration National Council on Disability National Endowment for the Humanities National Railroad Passenger Corporation Nuclear Regulatory Commission Office of Federal Housing Enterprise Pension Benefit Guaranty Corporation Selective Service System Tennessee Valley Authority U.S. Arms Control/Disarmament Agency U.S. Postal Service

American Battle Monuments Commission Consumer Product Safety Commission Equal Employment Opportunity Commission Federal Deposit Insurance Corporation Federal Energy Regulatory Commission Federal Reserve System General Services Administration National Archives Record Administration National Credit Union Administration National Indian Gaming Commission National Science Foundation Nuclear Waste Tech Review Board Office of Personnel Management Postal Rate Commission Small Business Administration Thrift Savings Plan U.S. International Trade U.S. Trade/Development Agency

Central Intelligence Agency Corporation for National Service Farm Credit Administration Federal Election Commission Federal Labor Relations Authority Federal Retirement Thrift Board Merit Systems Protection Board National Capital Plan Commission National Endowment for the Arts National Mediation Board NTSB Occupational Safety Health Administration Peace Corps Securities Exchange Commission Social Security Administration U.S. Agency for International Development U.S. Office of Government Ethics Voice of America

Legislative Branch Congress

General Accounting Office Congressional Research Service Library of Congress Congressional Budget Office

Senate House

Judicial Branch Supreme Court

U.S. Federal Courts

the80472_06_c06_189-232.indd 199 11/21/12 12:51 PM

Section 6.4 Other Actors Involved in Policy Implementation CHAPTER 6

The bureaucratic structure described in Figure 6.3 complicates effective administration and implementation by placing a high premium on interagency cooperation. Interagency conflicts emerge, for example, because agencies and departments struggle to define their purpose, clientele, function, and areas for which they are responsible. According the 19th- century political economist Max Weber, the consequence for the implementation of policy is that such “interagency conflicts put a heavy burden on higher officials, particularly the president and department agency heads, who must attempt to coordinate the agencies under their charge” (Fesler & Kettl, 1996, p. 114).

6.4 Other Actors Involved in Policy Implementation

Although the role and structure of the federal bureaucracy are clearly fundamental to the success of policy implementation, the bureaucracy is not the only policy actor that affects and shapes the policy implementation stage; indeed, many other policy actors also play important roles. These actors include the president, Congress, the courts, and interest groups.

The President’s Role in Policy Implementation

One of the president’s broad roles is chief executive of the bureaucracy. As chief execu- tive, the president has the power to appoint a series of officials, from heads of agencies to undersecretaries and deputy undersecretaries. Such appointments represent every presi- dent’s desire to control the method of policy implementation. This control is established, at least in theory, by appointing officials who share similar ideological, political, manage- rial, and policy preferences with the president. As such, a president can achieve some con- trol and influence over bureaucratic implementation through the power to appoint and replace various officials within a given agency. Arguably, the degree of control a president has depends on the policy area.

With defense and national security policy, a president enjoys considerable influence because of his or her position as commander in chief. In these defense-related areas, the president possesses considerable power in directing the implementation of various policy decisions. Nevertheless, internal agendas can emerge within the defense and intel- ligence agencies to weaken presidential authority over implementation. In comparison, within domestic policy the presidential policy agenda must be implemented in a bureau- cratic environment that can be readily co-opted by internal agency divisions, competing agencies and agendas across the government, and domestic political pressures. In reality, however, a presidential preference for successful implementation of any policy or pro- gram—foreign and domestic—requires the cooperation of the entire bureaucracy from top to bottom, including political appointees and professional civil service.

A president gains further control and influence over the bureaucracy and implementa- tion through the preparation of the yearly federal budget proposal. The federal budget proposal represents the fiscal framework for how funding will be allocated to specific fed- eral agencies and departments. Thus, the budget allocation demonstrates how a president

the80472_06_c06_189-232.indd 200 11/21/12 12:51 PM

Section 6.4 Other Actors Involved in Policy Implementation CHAPTER 6

can fiscally force an agency to change its method of implementation. Through cuts in an agency’s resources, the president can diminish the role and capacities of the agency to meet its mandated responsibilities. In comparison, increases in bud- get allocations give an agency additional leeway to execute new programs and initiatives.

In addition to political appointments and bud- getary control, a president can also propose more dramatic management or organizational reforms. Specifically, a particular agency—or the entire bureaucracy—may be restructured or face reform. Such reforms attempt to alter the method and conditions by which bureaucracies function and implement policy decisions. For example, President Ronald Reagan wanted to abolish the Department of Education and the Department of Energy, as well as merge the Commerce and the Labor Departments. Under President Bill Clin- ton’s tenure, management reforms seeking to “reinvent government” were proposed to reduce the minutiae of bureaucratic rules and regula- tions. In 2001 the Bush administration made its management goals quite clear: rethink how gov- ernment operates, reduce waste, and achieve a more effective and efficient government. One of

the largest federal government reorganizations occurred in 2002 with the creation of the cabinet-level Department of Homeland Security.

The difficulty, however, is that such dramatic organizational and management reforms may fail. Reagan’s proposal for abolishing the Energy and Education Departments con- fronted an unsupportive Congress and an unwilling bureaucracy. Clinton’s management initiatives confronted a les-than-cooperative bureaucratic environment. In sum, presiden- tial influence over implementation depends on whether the policy area, the appointees, their preferred budget, and their reforms are able collectively to create the conditions that serve to enhance control over the bureaucracy and implementation.

Congress’s Role in Policy Implementation

The role of Congress includes shaping and overseeing the implementation of policies and programs. From determining the appropriations accorded to each agency, program, and service to the critical oversight that ensures the accountability of the bureaucratic agency, Congress continues to affect how policy decisions are executed. Even though the bureau- cracy is considered an extension of the president and executive branch, bureaucratic over- sight falls very much within Congress’s purview (Fesler & Kettl, 1996). This oversight typifies the significant role that Congress has in affecting the manner and method by which the various bureaucratic organizations implement programs and services.

Getty Images

Homeland Security Secretary Janet Napolitano, pictured, is a member of the cabinet-level department that was created in 2002.

the80472_06_c06_189-232.indd 201 11/21/12 12:51 PM

Section 6.4 Other Actors Involved in Policy Implementation CHAPTER 6

Courts’ Role in Policy Implementation

The role of the courts provides a legal basis by which citizen activists, interest groups, or state and local governments can attempt to regulate the regulators effectively. In par- ticular, the federal Administrative Procedure Act provides for judicial review of both rule making and adjudicative decisions. Rule making involves the establishment of regulatory guidelines that can be enforced on a case-by-case basis or lead to an industry- or sector- wide rule. Adjudication addresses disagreements with an agency decision or citations of alleged violators whose actions contradict existing law and regulations.

The Dodd-Frank Act House of Representatives member Barney Frank discusses the Dodd-Frank Act in an interview. He shares his insights about where lobbying power comes from on these types of bills. He also addresses his role as ranking member of the committee and how things have changed since 2008. Watch the interview here: http://www.youtube.com/watch?v=E142-oJ8sjc.

Critical Thinking and Discussion Questions 1. Does the power of the law reside with those who enforce it? 2. What role does politics play in how this law is implemented? 3. In your opinion, will this law be effectively implemented? Why or why not?

Ideally, congressional oversight is a formal exercise requiring purposeful evaluation to determine the efficiency, effectiveness, and accountability of various bureaucratic agen- cies during policy implementation. The purposes of congressional oversight include: assuring Congress’s intent is followed during implementation, investigating instances of inefficiency and lack of accountability, collecting information for possible change in pol- icy, evaluating effectiveness and impacts, defending agencies and programs supported by congressional members, and reversing unpopular implementation (Fesler & Kettl, 1996).

Congress’s specific oversight role develops within the committees. In general, committees that are organized along policy lines provide oversight as they prepare and formulate new legislation. Hearings addressing various domestic and international issues and the federal agencies’ performance and impact in addressing these issues constitute direct and indirect oversight of how well the bureaucracy is executing its mandate. Budget authorization and appropriations committees, as well as committees on governmental reform, garner added oversight power. Supporting this oversight function are the Congressional Budget Office, Congressional Research Service, and the GAO.

The difficulty confronting Congress, however, is that the power and promise of oversight does not necessarily translate into effective oversight of the bureaucracy. The problem is that oversight often is either irregular or results in members of Congress unduly intruding for their own benefit (Fesler & Kettl, 1996). Simply put, differing political agendas can lead members of Congress to use oversight power as a weapon against unpopular agencies or policies. Finally, political interests such as those presented by constituents, lobbyists, or interest groups can lead members of Congress to attempt to stymie agency implementa- tion. As a result, oversight can be a weapon used for or against effective implementation.

the80472_06_c06_189-232.indd 202 11/21/12 12:51 PM

Section 6.4 Other Actors Involved in Policy Implementation CHAPTER 6

Whether involved in rule making or adjudicative decisions, the courts can and do review agency actions as they affect the public or target populations. Overall, the courts play an instrumental role in regulating and mediating the inevitable clash of interests that emerges as the bureaucracy defines the scope and nature of its regulatory actions during the stage of implementation. In short, the courts review agency actions to determine whether the agency is fulfilling its mission as mandated by the statue or whether agency actions may extend beyond the legal scope of the statute (Fesler & Kettl, 1996).

Interest Groups’ Role in Policy Implementation

The relationship between interest groups and the bureaucracy can at times be either coop- erative or conflictual. Depending on the policy area and the regulatory function of the bureaucratic agency, the array of interest groups can span a wide ideological and policy spectrum. Based on the type of organization and its administrative or regulatory pur- pose within the policy arena, the number and nature of the interactions between interest groups and the bureaucracy can be quite substantial.

The importance of policy implementation, coupled with bureaucratic discretion in deter- mining rules and procedures for implementation, can produce an increasingly close rela- tionship between a given bureaucracy and interest groups. Theories such as iron trian- gles, issues networks, and policy subsystems help explain how various circles of interest groups, political actors, and bureaucracies can develop such relationships. These rela- tionships are basically ensured, given current federal agency procedures.

When developing new initiatives or gathering additional information, various agen- cies may call upon interest groups to offer perspective, guidance, and expertise (Wright, 1996). Further, because of the Administrative Procedure Act’s rule-making procedures, interest groups often become involved by commenting within the 30-day window that follows publication of the pro- posed rules in the Federal Reg- ister (Wright, 1996). Interest groups also become involved, even mobilized, by the release of proposed federal rules in order to persuade regulatory actors to favor decisions and interpretations that benefit their respective interests. In general, bureaucracies operate closely with an array of interest groups whose interests are affected by the respective agency. Again, the overall purpose of interest- group involvement is to create a more favorable regulatory environment during eventual implementation.

Emily Riddell/Getty Images

The Department of Transportation often interacts with several interests groups during the policy process, which includes groups that represent railroads.

the80472_06_c06_189-232.indd 203 11/21/12 12:51 PM

Section 6.5 Theories That Address the Dynamics of Implementation CHAPTER 6

A bureaucracy such as the Department of Transportation will often interact with a number of interest groups from a cross-section of the policy or regulatory sector. Interest groups that attempt to influence and shape how the Department of Transportation implements policy, for example, may include not only corporate interest groups—such as those repre- senting airlines, railroads, shipping, or trucking associations—but also groups who rep- resent local and state governments, trade unions, and environmentalists. Irrespective of the basis for the interest group, each of these actors will attempt to influence regulatory decisions that favor their specific interest and policy agenda. The extent to which interest groups become active during implementation can also serve to influence their future role during subsequent stages of policy implementation (Wright, 1996).

6.5 Theories That Address the Dynamics of Implementation

The challenge implementation poses is in determining how best to execute pol-icy decisions so that they achieve their intended impacts. Three factors affect the implementation stage: the nature of the policy design, the bureaucracy, and the various other policy actors who seek to influence the manner in which policies and programs are executed. Because no blueprint exists for how policies should be imple- mented, practical insights on how to understand and improve the success of implemen- tation derive from theory.

Theoretical approaches have emerged by which to understand the dynamics of and the obstacles to the effective execution of policy decisions. Three such theoretical approaches include the top-down approach, the bottom-up approach, and synthesis. Each of these theories provides differing perspectives on the implementation process, as well as on the factors and dynamics that affect implementation.

Top-Down Approach

Policy implementation under this approach equals planning, hierarchy, and control; it assumes that statutory language is complete and applicable. In short, implementation is a function of the government decision, management, oversight, and resulting bureau- cratic execution. Hence, effective implementation requires an understanding of which factors and dynamics, including nonstatutory forces, influence bureaucracy and govern- ment’s ability to execute decisions effectively (Dye, 2001). Overall, the purpose of the top- down approach is to focus the attention of the policy designers and implementers on those elements that, when recognized and addressed, improve the success of implemen- tation. Simply put, implementation is an administrative function and is a composite of statutory structures, problem tractability, and nonstatutory forces (see Mazmanian & Sabatier, 1989). The top-down approach has been criticized for emphasizing too many fac- tors as relevant and for an overly optimistic belief that managing implementation works from the top. This belief ignores the fact that other actors outside the strict arena of pol- icy designers, decision makers, and government officials are important to the success of implementation. According to political scientist Thomas Birkland, one example of a top- down approach policy is the 1973 legislation that established the 55-mile-per-hour speed limit on freeways as a way to promote energy conservation (Birkland, 2001). This was uniformly implemented across the nation, regardless of traffic flows in different areas.

the80472_06_c06_189-232.indd 204 11/21/12 12:51 PM

Section 6.5 Theories That Address the Dynamics of Implementation CHAPTER 6

Bottom-Up Approach

This approach assumes that statutes are incomplete and focuses on the critical role played by the various actors specifically responsible for or able to influence implementation through the way the policy is delivered (Lester & Stewart, 1996). The bottom-up approach emphasizes the importance of the network of actors directly responsible for policy execu- tion, the explicit bargaining and accommodation that determine the path by which poli- cies are implemented, and the important role played by the front-line or “street-level” agents directly responsible for implementation (Elmore, 1979; Sabatier, 1986).

Criticisms of this approach, not surprisingly, center on what some suggest is an exagger- ated bottom level of implementation coupled with a minimized emphasis on factors such as political management and administrative control, broad socioeconomic and political factors, and the effect of the policy action or statute’s language on the implementation’s success. Block grants, which give money from the federal government to the state and local levels, are an example of the bottom-up approach. For example, the federal govern- ment has a community development policy with the goal of promoting economic growth at the local level. Through the Department of Housing and Urban Development, commu- nity development block grants are distributed nationally to state and local governments, which then have the power to decide how to use the grant money to implement commu- nity development.

Synthesis

Assumed weaknesses with the previously discussed approaches to policy implementa- tion have led to what may be referred to as a synthesis of the two approaches (Lester & Stewart, 1996). A synthesized approach emphasizes the importance of top-level fac- tors, such as policy makers, the design of policy, and the instruments selected; and bot- tom-level factors, such as the nature and capacity of the target population to change its behavior (Elmore, 1985). The advocacy coalition framework model discussed in Chapter 3 represents a synthesis of bottom and top-level aspects to implementation as it focuses on how political, socioeconomic, and policy elements interact with the various actors within a subsystem to affect how policies are implemented (Sabatier & Jenkins-Smith, 1993).

Although no one theoretical approach perfectly explains the process of implementation, the most useful approach to understanding implementation is probably a model that uses synthesis. Such an approach to implementation is not necessarily a better theory, but it is more in tune with the real world of implementation, which is not strictly top-down or bottom-up. The synthesized approach also allows for a discussion of the crucial factors that help explain the success or failure of policy implementation. A good example of such a synthesized approach is Richard Matland’s ambiguity-conflict model. For Matland, factors such as the level of ambiguity of policy goals, the level of ambiguity in the ways and means to achieve the policy goals, and the level of conflict surrounding the policy and the problem it deals with are critical to understanding how any policy is implemented.

the80472_06_c06_189-232.indd 205 11/21/12 12:51 PM

Section 6.6 The Practice of Policy Implementation CHAPTER 6

Challenges of Implementation

The implementation stage, as this chapter discusses throughout, is an action phase of the public policy process. Successful implementation, therefore, requires that day-to-day implementers have a general base of knowledge of what challenges may emerge to hinder the execution of a policy or program. The preceding discussion alluded to some of the major challenges to effective implementation, but they may be summarized as follows.

Dealing With the Clarity or Ambiguity of Policy Goals Clear policy or program goals help specify the policy action’s desired ends or objectives. Ideally, policies should be formulated with consideration of the policy’s actual and spe- cific goals. In reality, policy decisions often lack any direction or specific goals. The differ- ence between specific and ambiguous goals is substantial. An example of a policy with a specific goal is a policy to reduce auto emissions by 10% by 2015. An example of a policy with an ambiguous goal would be one with the aim of improving air quality in the United States through the adoption of programs designed to deliver better air to citizens.

A clear, specific stated goal not only provides direction, but also improves the basis by which policies can be evaluated for accountability, efficiency, and effectiveness. According to political scientists Jeffrey Pressman and Aaron Wildavsky (1994), a clear goal is critical

Implementing Policy Change Through Enforcement Having a good public policy in place is only half the battle. The policy must be put into action to work. Policy implementation is a complicated process that is rarely linear. Enforcement is a key factor when policy change comes in the form of new laws. The following video discusses enforcement to enact pol- icy change for a new law about underage drinking: http://www.youtube.com/watch?v=l8aV6aqPjA0.

Critical Thinking and Discussion Questions 1. How is this particular policy change being enacted through enforcement? 2. What are some other areas of public policy change that might benefit from enforcement? 3. Are there other strategies of implementation aside from enforcement?

6.6 The Practice of Policy Implementation

Although it is important to understand the theories of implementation, in the end, policy implementation remains a practical exercise of the various policy actors’ planning for and executing a policy decision. The practice of implementation can best be understood by looking at the challenges of implementation, the obstacles to implementation, the role of strategic planning, and the start-up and execution planning phases of implementation. Remember, implementation is far more complicated than just putting into operation a policy passed by government. Often it requires that those respon- sible for day-to-day implementation actually create mechanisms through which the goals of the policy they are implementing can be achieved. Such mechanisms for achieving policy goals require the creation of programs, which itself brings a whole set of challenges.

the80472_06_c06_189-232.indd 206 11/21/12 12:51 PM

Section 6.6 The Practice of Policy Implementation CHAPTER 6

to be able to judge the effect and success of policy implementation. Further, as political scientists Daniel Mazmanian and Paul Sabatier (1995) explain, the clarification of objec- tives and priorities is a critical aspect of the statute, having a profound impact on the implementation’s likely success.

Conversely, according to Giandomenico Majone and Aaron Wildavsky (1995), without obvious goals or precise objectives, a policy is unlikely to be easily or successfully imple- mented. The practical challenge, as social scientists Fesler and Kettl (1996) assert, lies in dealing with goals that are seldom clear or specific and that may evolve through the imple- mentation process. However, a policy without a specific goal can never truly fail or suc- ceed, because it operates in a perceptual trap in which the various actors involved define and redefine the goal. Thus, because failure or success cannot truly be mea- sured, no one can say with certainty that the policy has succeed or failed.

Finally, goals—even when specified— do not operate in a static societal or administrative environment. Over time, the goal of a policy can change because of organizational, policy, or political factors. For example, the ini- tial goals of the 1996 Welfare Reform Act were to reduce dependence on welfare. However, as socioeconomic conditions worsened in 2001, the dis- cussion of reauthorizing the 1996 act has led to concerns over whether the goal of reducing the number of wel- fare recipients is achievable or even desirable (Bustillo, 2002). In other words, in times of economic decline and recession—when increasing numbers of the pub- lic are facing reduced financial circumstances—can the government reduce or eliminate aid to those in need?

In the ideal world of administration and policy implementation, a clear or specific goal would provide direction as well as a basis for determining the program’s success or failure. However, ambiguity in goals may persist because a lack of specificity allows for greater political manipulation during implementation. Vaguely defined goals allow a program’s success and failure to be redefined, as well as provide implicit discretion for the agency when implementing (Fesler & Kettl, 1996). The consequence of nonspecific implementa- tion goals, however, is that the program implementation lacks direction as wells as a clear basis for evaluation.

Gathering Information Intelligence Information intelligence refers to the strategic need for constant feedback about how implementation is progressing, as well as, when possible, preliminary assessments of impacts and any problems that occur. Such information is essential for administrative

Getty Images

Policy goals can change over time. The effects of an economic recession influenced initial goals of the 1996 Welfare Reform Act.

the80472_06_c06_189-232.indd 207 11/21/12 12:52 PM

Section 6.6 The Practice of Policy Implementation CHAPTER 6

agents and policy makers if they are to judge the success and impact of implementa- tion. Timely, accurate, and relevant information is also important for enabling strategic changes in implementation as the policy is being executed. Given the increasing reliance on multiple bureaucracies, information and feedback are essential to gauge the degree of interagency and intergovernmental cooperation or conflict, which may be affecting policy implementation. The importance of such information intelligence is highlighted by the enormous administrative and policy responsibilities confronted by the Department of Homeland Security (Homeland Security).

A key responsibility of Homeland Security is managing and coordinating the “imple- mentation of a comprehensive national strategy to secure the United States from terrorist threats or attack” (Department of Homeland Security, 2002). The success of Homeland Security depends on the extent to which the agency can effectively analyze, coordinate, and communicate information from dozens of agencies that are part of it. In addition, effective implementation requires effective communication with a number of additional federal agencies, including the Departments of Defense, Health and Human Services, Transportation, as well as the Central Intelligence Agency (CIA) and the Federal Bureau of Investigation (FBI). Many believe Homeland Security has achieved its identified mission: Since its creation, no successful domestic attack has taken place, in large part because the department has achieved a high level of information intelligence gathering.

Information intelligence requires a high level of communication and feedback not only from the agencies involved in policy implementation but also, when applicable, from the populations affected by the policy action. Implementation can be improved only if the implementers have insight into how a policy or program is affecting a target population. The challenge is in developing a positive path for communication between agencies and the target populations that allows for valuable feedback. However, with certain target populations (such as criminals), no such administrative or political desire for gathering such information may exist.

Identifying Target Groups Those implementing policy need to identify precisely the population(s) that a particular policy is targeting. Implementers need to know how many and what types of groups they should take into account. The more clearly the target groups can be defined and the narrower the size of the population, the greater the likelihood that policy objectives will be achieved (Mazmanian & Sabatier, 1981). Additionally, implementers should also calculate how much compliance from target groups will be sufficient for the policy to be implemented as intended. The required degree of behavioral change of the target group determines whether the intended policy implementation is achieved.

the80472_06_c06_189-232.indd 208 11/21/12 12:52 PM

Section 6.6 The Practice of Policy Implementation CHAPTER 6

Adopting a Strategy or Model Given the intrinsic difficulties in implementing public policy, the model by which a policy is implemented is crucial. Such models of implementation sometimes offer a strategy for executing implementation, provide a blueprint for how a program should be constructed, and may even identify potential obstacles in the implementation process.

Different models of implementation outline the conditions that must be met if the policy is to be actualized (Hogwood & Gunn, 1984). The first is programmed implementation, which tries to remove or control the problems that await implementation by thorough, explicit programming of implementation procedures (Berman, 1980). An example of this implementation model is President John F. Kennedy’s naval blockade of Cuba, in which every detail of how the blockade would occur was written into the policy. Another example of programmed implantation is the Department of Education’s disseminating to school districts packages that explained procedures for adopting other districts’ success- ful innovations (Berman, 1978). Programmed implementation places high value on clarity and rationality and rests on the assumption that the problems that plague implementa- tion are the result of ambiguous policy goals, the involvement of an excessive number of actors, overlapping jurisdictions of authorities, misperceived interests, and conflicts.

The second type of implementation that can occur is adaptive implementation, which tries to improve the process by allowing the original policy mandate to be adjusted as events unfold. Water quality management is an example of adaptive implementation. The overall policy requires that the improvement of water quality and that management of water quality be implemented in a way that measures progress over time and includes a process for adapting the plan and for adjusting strategies, tactics, and timelines accord- ing to feedback from collected performance measurement data. Thus, water quality management allows for implementation at the local level. In contrast to programmed implementation, in adaptive implementation, clarity and specificity are viewed as barriers to implementation because they produce rigidity in the face of shifting political realities.

The last model of implementation is premeditated nonimplementation. In such a model, administrative behavior is deliberately aimed at preventing implementation from occur- ring. Such behavior ensures that the policy will never be more than partially accom- plished. For example, in California in the 1990s, state policy permitted only married couples to adopt children. However, at this time, a number of single women and homo- sexual and lesbian couples were adopting children because the policy’s day-to-day implementers chose not to implement the state policy as written. Rather, when visiting unmarried individuals, the implementers stated the policy and then told the individu- als that they could still adopt, provided a subsequent home study concluded the only reason they were being denied approval was because of the policy restricting adoption to married couples. Unmarried individuals were then instructed to appeal the denial; because no other factors rendered them unfit parents, the adoption was granted.

the80472_06_c06_189-232.indd 209 11/21/12 12:52 PM

Section 6.6 The Practice of Policy Implementation CHAPTER 6

The choice of model will ultimately help determine the implementation’s outcome. Usually, because of the multidimensional nature of the policy process, the type of imple- mentation will combine the programmed and adaptive types, though premeditated nonimplementation often occurs because policy goals and objectives are vague.

Obstacles to Implementation Obstacles to implementation can be grouped into general categories and will vary in the degree to which they prove to be a barrier to implementation according to the context of time and the problem that the policy addresses. Following are the most common catego- ries of obstacles.

• Legal and institutional: These include lack of legal powers to implement a particular policy, and legal responsibilities that are split between implementing agencies, limiting the ability of the governing authority to implement the policy.

• Financial: These include budget restrictions limiting the overall expenditure on the policy’s implementation, financial restrictions on the use of specific policy instruments, and limitations on the flexibility with which revenues can be used to finance the full range of policy instruments needed to implement the policy.

• Political and cultural: These involve lack of political or public acceptance of a policy, restrictions imposed by interest groups, and cultural attributes, such as attitudes to enforcement, that influence the effectiveness of the instruments used to implement the policy.

• Practical and technological: Lack of key skills and expertise can be a significant barrier to successful implementation.

The Strategic Planning Involved in Implementation

The purpose of strategic planning, within the context of policy implementation, is to high- light how important it is to assess an agency’s capacity for meeting specific implementa- tion tasks and goals mandated by the policy decision. Essentially, strategic planning is a tool with which the agency can evaluate its ability to achieve the goals of the policy, as well as plan the policy’s execution. According to Jay Shafritz and E. W. Russell (1999), experts in the study of public administration, strategic planning is also a tool with which an implementer can “match its objectives and capabilities to the anticipated demands of the environment to produce a plan of action that will ensure achievement of objectives” (p. 246). Finally, strategic planning allows for the implementing officials and the agency to assess potential near-term challenges that could affect the success of policy implementa- tion. The basic steps to strategic planning are described in Table 6.3.

the80472_06_c06_189-232.indd 210 11/21/12 12:52 PM

Section 6.6 The Practice of Policy Implementation CHAPTER 6

Table 6.3: Basic steps of strategic planning and implementation

Steps of Strategic Planning Value for Implementation

1. Statement of agency goals, mission, or vision

Define the mission or purpose of the agency.

Clarify the area of administrative responsibility.

2. Adoption of a time frame Define the overall planning horizon for the agency.

What are the critical dates to achieve agency-wide goals?

3. Assessment of president capabilities

Assess the agency’s capabilities.

Critical assessment of financial and human capital strengths or weaknesses.

Determine overall capacity of the agency for successful implementation of programs or services.

4. Assessment of organizational environment

Evaluate the environment confronting the agency.

Will changes in the administrative environment tax or improve future implementation?

5. Develop a strategic plan Outline method by which the agency will achieve goals, mission, or vision.

Provide direction and focus on implementation responsibilities.

Clarify where agency must shift resources or focus in order to achieve implementation.

6. Organizational integration Provide direction and tasks for specific actors within the agency.

Motivate the agency on specific tasks—improves organizational environment.

Implementation’s Start-Up and Execution Phase

The practical realities of implementation require that the implementers understand the necessary steps and actions that must occur to execute a policy decision. The decision to go to war, for example, is not simply an order that leads to action (see Figure 6.4). Rather, with every policy decision, regardless of its scope or purpose, a series of tasks must be executed in some logical order for a policy to be implemented effectively. Basically, in order for a policy to be successfully implemented, or for someone to critique the imple- mentation of a policy, the start-up and execution phases must be known. In each of these phases, implementers must take certain actions (see Figure 6.5).

the80472_06_c06_189-232.indd 211 11/21/12 12:52 PM

Section 6.6 The Practice of Policy Implementation CHAPTER 6

Figure 6.4: The critical path method of implementing a hypothetical decision to go to war

A series of tasks must be executed in a logical order before a policy is successfully implemented. The decision to go to war is one scenario example.

A

C

B E F

D

Event

A. Decision to go to war

B. Forces mobilized/moved to theater

C. Initial target list set

D. Allied coalition achieved

E. Battle plan finalized

F. War

Activity

1. Mobilize military forces/resources to combat theater

2. Develop initial target list

3. Match military resources with target list

4. Move additional forces from other theaters

5. Approach potential allies for coalition

6. Determine allied input/involvement in battle

7. Launch coordinated attack

1

2 3

4

5 6

7

the80472_06_c06_189-232.indd 212 11/21/12 12:52 PM

Section 6.6 The Practice of Policy Implementation CHAPTER 6

Figure 6.5: Tasks undertaken during start-up and execution phases of implementation

Every policy decision encounters hurdles during the start-up phase.

The start-up phase, as it is has been described, is “a traumatic time in the life cycle of a program, since it is at this time that the separate elements considered in formulation must be brought together and begin to function as a system” (Shafritz & Russell, 1999, p. 510). The challenges of this start up phase confront every policy decision. However, a learning curve is possible whereby the administrative agencies and actors consistently responsible for specific program areas and policies learn to overcome the obstacles posed by this ini- tial phase during policy implementation.

The existence of a learning curve assumes that in the initial phase of implementation, the impact of a program may be slow to develop. Agency officials will need some time to develop the necessary level of training and knowledge of the program. Further, the target population itself may be slow to change its behavior or conform to the intent of the program. For example, many states have passed laws that prohibit drivers from talking on cell phones without hands-free devices or from texting while driving. However, driv- ing on any street or freeway in the United States reveals how many individuals flagrantly disregard such laws. With any new program, it is to be expected that the policy’s benefits and impacts may be minimal until implementation reaches an optimal level.

Indentifying key

stakeholders; bargaining

and coalition building;

dealing with opposition;

mobilizing underorganized

stakeholders or beneficiaries.

Constituency Building

Raising awareness; identifying

policy reform advocates;

creating new forums for

discussion; creating bridging

mechanisms; developing

convening authority

Policy Legitimization

Developing plans, expectations

and accountability; identifying

incentives; dealing with

resistance; achieving compliance;

communicating success

Mobilizing Resources

and Actions

Policy Implementation

Place monitoring mechanisms in

place; create analytic capacity;

establish performance standards

Monitoring Progress

and Impact

Developing resource

partnership with other

agencies; creating new

capacities; upgrading

human resources

Resource Accumulation

Buiding implementation capacity;

developing boundaries; fostering

networks and partnerships;

enhancing cooperation and

coordination among

implementing agencies

Organizational Design

and Modification

the80472_06_c06_189-232.indd 213 11/21/12 12:52 PM

Section 6.6 The Practice of Policy Implementation CHAPTER 6

Policy in Practice: The Evolving Status of Wolves and the Politics of Implementation Imagine visiting one of the many National Parks in the United States—Yosemite, the Great Smokey Mountains, Glacier, Denali, or Yellowstone—and never seeing a large, wild animal. Well into the early 20th century, there were few legal protections for any large wild animal species. Long considered threats to livestock, other game species and even people, wolves were actively hunted until the spe- cies had virtually disappeared in the lower 48 states.

In 1973 Congress passed the Endangered Species Act (ESA), which provided protections to animal species that were in danger of becoming extinct or had been removed from their historical habitats. Gray wolves were added to the list of endangered species the following year. Beginning in the late 1980s, an effort emerged to reintroduce gray wolves to Yellowstone. Despite concerted resistance from politicians, ranchers, hunters, and others in the region, several groups of wolves captured in Jasper National Park, Canada, were released into Yellowstone in 1995. Over the next decade wolf populations grew and became self-sustaining. This lead to mounting claims by politicians in Mon- tana, Wyoming, and Idaho that the species was no longer endangered, should be removed from the endangered species list, and should be managed by each of the respective state wildlife agencies. In 2009 the U.S. Fish and Wildlife Service (USFWS) developed an administrative rule

This learning curve is even more complicated if the policy decision deals with an entirely new area of policy making. For example, during the first years of the Department of Homeland Security, the learning curve was expected to be high. One method with which to address issues of how to implement a policy is to plan its execution correctly.

A method of forecasting implementation problems even before the start-up phase is to plan the execution of the policy decision using tools such as the critical path method (CPM) (Huddleston, 2000). The basic value of CPM—also described as network analysis or PERT (for performance evaluation review technique)—is to provide a visual plan of the necessary steps required to implement a policy decision; it helps to forecast poten- tial obstacles before actual implementation so that implementers can better plan for the policy’s execution (Huddleston, 2000). The exercise “identifies the set of events that must occur to implement a policy and links them together in the proper sequence with activity arrows . . . [which] represent the amount of time (or other resources) that must be con- sumed to produce an event” (Huddleston, 2000, p. 307). Again, the analytical value of this exercise is that it forces policy actors to realize the difference between the rhetoric of policy action and the realities of policy action. In other words, implementation is far more dif- ficult to plan than apparent or various policy actors assert. Consider the following simple example of the United States deciding to go to war with another state, and follow just the basic steps involved in implementing this decision.

As the example in Figure 6.4 illustrates, implementing a decision involves a number of critical steps, all of which prove essential to successfully executing the policy decision. While the war example is grossly simplified, it helps to highlight the critical tasks and activities that are associated with such a policy decision. More importantly, it emphasizes that a policy can fail if even one step is not accomplished during implementation. Addi- tionally, if one or two tasks are performed poorly early on, the effects ripple across the entire implementation.

(continued)

the80472_06_c06_189-232.indd 214 11/21/12 12:52 PM

Section 6.7 The Role of Policy Evaluation CHAPTER 6

6.7 The Role of Policy Evaluation

Many policies do fail; they do not achieve their intended effects and have unin-tended outcomes. Thus, concerns over accountability, efficiency, and effective-ness in policy making are intensifying at all levels. Elected officials, policy makers, community leaders, bureaucrats, and the public at large want to know which policies work, which do not, and why. Policy evaluation determines whether an imple- mented program is doing what it is supposed to do. More precisely, policy evaluation is the study of what happens as a result of implementation. The consequences of such poli- cies’ programs are determined by describing their impacts, or by looking at whether they have succeeded or failed according to a set of established standards (Dubnick & Bardes, 1983). Evaluation helps assess whether a policy’s effects are intended or unintended and whether the results are positive or negative benefits for the target population and society as a whole.

U.S. policy makers became more concerned with judging the effects of policies in the 1960s with the advent of President Lyndon B. Johnson’s War on Poverty programs. Spe- cifically, policy makers became concerned with assessing whether the different welfare and antipoverty programs were having the intended effects and whether tax dollars were being spent efficiently and effectively. From the late 1960s, requirements for program

Policy in Practice: The Evolving Status of Wolves and the Politics of Implementation (continued) under the ESA, delisting wolves. The rule was quickly litigated by wolf advocates, and after first deny- ing an injunction against the rule, a federal court required USFWS to relist the wolves. Not wanting to leave the issue to the courts, senators from Montana, Idaho, and Wyoming cosponsored congressio- nal legislation requiring USFWS to reinstate their original rule delisting wolves. Although the legisla- tion passed and was signed into law, and Montana and Idaho have held legal wolf hunts, there is still pending litigation before the federal courts. Wolf management is not yet a settled issue.

Early policy literature assumed, and some policy advocates still desire, a process in which once the politics of policy development and adoption have played out, implementation becomes a largely technical process of carrying out the intent and content of the policy. The reintroduction of wolves into the Yellowstone National Park, under the ESA, shows that implementation is every bit as political as formulation and adoption.

Critical Thinking and Discussion Questions

1. How does wolf management exemplify the models or theories of policy implementation described in the chapter? How does it deviate or differ from these models?

2. What sorts of tools are available to the various stakeholders described in this case, which enable them to impact policy implementation? How do these tools shape the substance and/ or process of implementation?

3. If wolf management were not such a hot-button political issue, what would the implementa- tion process look like? What sorts of information would be used to shape the implementation process?

the80472_06_c06_189-232.indd 215 11/27/12 3:29 PM

Section 6.7 The Role of Policy Evaluation CHAPTER 6

evaluation were written into almost all federal programs (Haverman, 1987). Subsequently, Con- gress established evaluating organizations such as the Congressional Budget Office and the Gen- eral Accounting Office. Funding support for such organizations has varied with each presidential administration. For example, the Reagan admin- istration made deep budgetary cuts that affected the federal government’s ability to conduct pro- gram evaluation (Rushefsky, 1990).

Within the academic field of public policy, many perspectives exist as to the exact nature of evalu- ation and the roles of evaluating organizations. In one perspective, evaluation assesses whether a set of activities implemented under a specific policy has achieved a given set of objectives. Evalua- tors judge the overall effectiveness of the policy (Wholey, Scanlon, Duffy, Fukumotu, & Vogt, 1970). A second perspective views evaluation as any effort that renders a judgment about program quality (Haveman, 1987). The third perspective considers policy evaluation to be information gathering for the purposes of making decisions about the future of a program (Bingham & Fel- binger, 1989). Yet another perspective found in the literature views evaluation as the use of sci-

entific methods to determine how successful implementation and its outcomes have been (Bingham & Felbinger, 1989). Finally, the General Accounting Office (GAO) (1989) defines policy or program evaluation as the provision of sound information about what policies and programs are actually delivering, how they are managed, and the extent to which they are cost-effective.

Although each of the above definitions and perspectives has value, a better definition of policy evaluation may be a process by which general judgments about quality, goal attain- ment, program effectiveness, impact, and costs can be determined. What differentiates policy evaluation from other informal types of assessment? First, the goals of the policy or program are provided to the evaluators. Next, the evaluation’s focus is on outcomes or consequences (Caro, 1977). Third, evaluation occurs after implementation. In other words, the program must have been implemented for a certain period of time. The main purpose of evaluation is to gather information about a particular program’s performance in order to assist in the decision to continue, change, or terminate.

The real value of formal program or policy evaluation is that it allows for accountability to be measured empirically. That is to say, with evaluation, evidence is gathered showing whether a program is successful or not. Conducting an evaluation allows policy makers to be provided with accurate information about key policy questions and costs arising from the implementation of any policy or program. This information is, of course, provided by an evaluation study within a given set of real-world constraints, such as time, budget, ethical considerations, and policy restrictions. Policy makers want to know if programs

Getty Images

Former president Lyndon B. Johnson’s War on Poverty programs marked a change in the way U.S. policy makers judged the impact of policies.

the80472_06_c06_189-232.indd 216 11/21/12 12:52 PM

Section 6.8 Types of Policy Evaluations CHAPTER 6

are being administered and managed in the most efficient way, accountable insofar as it is clear what is going on and why, and effective manner. Evaluation is also useful because it can eventually stimulate change. Finally, conducting an evaluation can discover flaws in a program that policy designers were never aware of in the abstract.

6.8 Types of Policy Evaluations

A variety of models fuse theoretical content with practical guidelines for conducting a program or policy evaluation. Most models arose in the 1960s and 1970s when policy theorists made early attempts to conceptualize what evaluation was and how it should be conducted. Thus, these models offer varying ways to understand the goals of an evaluation, the role of the evaluator, the scope of an evaluation, and how it is organized and conducted. Subsequent theorists have adapted the models to changing times, contexts, and needs. Often, they use two or more models in conjunction. Contem- porary theorists have developed several different types of evaluation models that vary in complexity (Sanders, 1994), and policy evaluators most commonly apply four types: process evaluation, outcome evaluation, impact evaluation, and cost-benefit analysis (see Table 6.4).

Table 6.4: Objectives of the four most common models for policy evaluation

Process evaluation should do the following:

Outcome evaluation determines these elements:

Impact evaluation identifies these elements:

Cost-benefit analysis calculates the following:

• Determine why a program or policy is performing at current levels

• Identify any problems • Develop solutions to

the problems • Improve program

performance by recommending how solutions should be implemented and evaluated once car- ried out

• Legislative intent • Program goals • Program elements

and indicators • Measures of indica-

tors

• Program outcomes and outcome valenc- es (whether they are positive or negative)

• Theoretical goals of the program or policy

• The actual goals Program or policy objectives

• Program or policy results and whether they are intended, unintended, positive, or negative in effect

• Negative costs • Positive benefits • The net benefit

Process Evaluation

This type of policy evaluation focuses on the concrete concerns of program implementa- tion—in short, how a policy program is being administered. Thus, process evaluation assesses how a program or policy is being delivered to target populations or how it is being managed and run by administrators. With this type of evaluation, the focus is not

the80472_06_c06_189-232.indd 217 11/21/12 12:52 PM

Section 6.8 Types of Policy Evaluations CHAPTER 6

on whether the program is meeting specified goals, but on recommending improved implementation procedures. This type of evaluation is best suited to the needs of program managers and has the objective of helping them overcome barriers to achieving the goals of the program policy being implemented. For example, if a new welfare policy is imple- mented that has reducing the number of welfare recipients as its goal, the process evalu- ation would look at how well the policy is being administered. Thus, analysis focuses upon service delivery. In short, the evaluation assesses what administrative measures are in place to reduce the numbers and how such operational measures might be improved.

Outcome Evaluation

This type of evaluation focuses on the degree to which a policy is achieving its intended objectives with regards to the target population. Outcome evaluation is concerned with consequences and whether the policy is producing the intended results. This can lead to assessment of effectiveness, including cost. Outcome evaluation is not well suited to the needs of program level managers because it does not provide operational guidelines on how to improve the implementation of the program. Rather, it best suits the needs of policy designers because it identifies whether there is consistency between policy results and program intent. For example, if a welfare policy’s stated goal to reduce the number of welfare recipients, then an outcome evaluation calculate whether fewer people are receiv- ing welfare benefits after program implementation than did so before.

Impact Evaluation

This type focuses on whether a program is having an impact on the intended target popu- lation. The major difference between an impact evaluation and an outcome evaluation is that the latter is solely concerned with whether the program or policy’s goals and objec- tives are being achieved. In comparison, impact evaluation assesses whether the target population is being affected by the introduction and implementation of the policy. Impact evaluation also focuses on the program’s effect on the original problem being addressed. The benefit of an impact evaluation is it is suited to the needs of both program level man- agers and policy designers, for it is important for both to ascertain whether target popula- tions are appropriately receiving delivery of a program. For example, using welfare policy again, an impact evaluation aims to determine what happened to the people who used to receive welfare benefits and who lost them under the new policy because they were no longer eligible. Are those people working? Are they receiving other means of assistance? Have any turned to criminal activities to generate income?

Cost-Benefit Analysis

This type of evaluation focuses on calculating the net balance of the benefits and costs of a program. Essentially, cost-benefit analysis is a method with which to evaluate and assess the effectiveness of a policy’s costs, benefits, and outcomes. For many, it is a contro- versial evaluation technique because it is difficult to apply to the public sector. It ignores qualitative concerns at the expense of quantitative information. Under a cost-benefit analysis approach to assessing certain policy issue areas, calculating the immediate real dollar costs is easier than measuring the tangible benefits. For example, the tangible cost

the80472_06_c06_189-232.indd 218 11/21/12 12:52 PM

Section 6.9 Stages in the Evaluation Process CHAPTER 6

of a new welfare policy to reduce the number of welfare recipients could be evaluated to include agency operating costs and the actual monetary cost of the benefits provided to welfare recipients. However, the cost-benefit analysis cannot calculate the intangible costs to those individuals who are now ineligible for welfare benefits as a result of the new policy. Additionally, the intangible costs to society are extremely difficult to calculate. These may include increased crime or increased public health-care costs, as well as long- term unemployment of those individuals who might otherwise have been able to pull themselves out of poverty had they still been eligible for welfare assistance. With certain types of programs such as education or the environment, one could argue that the real benefits do not materialize for years or decades. Hence, a cost-benefit analysis may con- clude that a program is inefficient in terms of monetary expenditures, when it may in fact be effective in realizing its long term goals and in delivering benefits that, over the long term, far exceed the dollar costs. Some things—such as quality of life—simply cannot be quantified. When used alone, cost-benefit analysis can color discussion about whether a program or policy is successful. It is most useful as a tool when used in conjunction with one of the other types of evaluation.

6.9 Stages in the Evaluation Process

Evaluation of policy is fairly complex and includes initial activities that must be undertaken to ensure the success of the overall evaluation process. Intrinsic to this success is the evaluator’s duty to communicate findings and conclusions to the cli- ent. Evaluation can be viewed as a three-stage sequence: planning, data gathering, and dissemination. A series of essential activities take place throughout the three stages.

According to policy scientists Sylvia, Sylvia, and Gunn (1997), before any evaluation is undertaken evaluators can confirm whether their evaluation will be planned correctly by following a 10-point checklist:

1. Is the program experimental or is it ongoing? 2. Who is the audience? 3. Are the measures, indicators, and design appropriate for the needs of the

audience? 4. Are we interested in outcome or impact? 5. What is the purpose of the evaluation? 6. Are we trying to build theoretical knowledge or seeing if maximum service is

provided? 7. How will the study affect funding of the program? 8. Can we realistically produce a valid design given our resources? 9. Are we measuring what we are supposed to?

10. What are we doing, how are we doing it, and who cares what we tell them?

Once the checklist is complete, the evaluator is ready to begin the first stage of the evalu- ation sequence.

the80472_06_c06_189-232.indd 219 11/21/12 12:52 PM

Section 6.9 Stages in the Evaluation Process CHAPTER 6

Stage 1—Planning

This stage consists of three steps. Step 1 is gaining familiarity with the program. Step 2 is deciding the focus of the evaluation, and step 3 is developing evaluation measures. In step 1, evaluators must become aware of and familiar with the actual program or policy being evaluated. To accomplish this, evaluators ask themselves a series of questions. The first attempts to clarify the goals and objectives of a program or policy. This is not always easy to do, since the legislative mandate for the policy may have ambiguously expressed goals, multiple goals, or conflicting goals. Next, the evaluator must determine the relationship of the program being evaluated to similar programs and identify the program’s major stakeholders and target populations. Stakeholders are individuals, agencies, or groups who have stakes in the outcome of the evaluation. Target populations are those whom the policy affects. Evaluators should also consider those not in the target groups because they may be affected by the policy (House, 1980). Finally, the evaluator must learn the ongoing and recent history of the program. Once all of these questions have been answered, then the evaluator can move on to the next activity in the planning stage.

In step 2 of the planning stage, evaluators must decide what they are actually assessing. Specifically: What is the focus of the evaluation? Is it the policy’s impact, its outcomes, costs and benefits, or the way the policy is being delivered?

Once the evaluators decide what the focus is, they can conduct step 3, the development of measures for the focus. Such measures should include estimating the cost of the policy in both dollar and nondollar terms.

Stage 2—Data Gathering

Evaluators must collect two types of data. First, they must collect data that allows for better understanding of the program’s overall configuration and structure. This means gathering information about how the program is delivered, to whom, and how many clients are served. The second type of data gathering deals with the degree to which pro- gram goals and objectives are being achieved. The evaluator must also collect data on other effects—both intended and unintended—that can be attributed to the policy. The evaluator’s decision to apply quantitative or qualitative methods determines how data is gathered. Quantitative methods refer to a range of techniques involving the use of statistics and statistical analysis for systematically gathering and analyzing information. Qualita- tive methods aim at understanding underlying behavior, or how and why implementers, clients, and target populations take certain actions. Table 6.5 highlights the differences between quantitative and qualitative methods of analysis.

the80472_06_c06_189-232.indd 220 11/21/12 12:52 PM

Section 6.9 Stages in the Evaluation Process CHAPTER 6

Table 6.5: Differences between quantitative and qualitative analyses

Quantitative Qualitative

Counting Interpreting

Measuring Experiencing

Confirming Understanding

Determining Arguing

Testing Exploring

Observing Listening to others observations

Finding out what’s real Exploring multiple realities

No essential agreement exists on which type of analysis to use. Often the best way to determine which methods to apply is to look at the program’s size and scope, the intended audience for the evaluation, the program’s goals, the evaluator’s own skills, and the resources available to conduct the evaluation. Once they have determined the methods, evaluators must design the research and confirm the instruments (data collection devices) that will be applied (see Table 6.6). Research designs are strategies that help the evaluator improve the validity and reliability of the evaluation. To avoid threats to validity or reli- ability, designs must be rigorous but also appropriately applicable to the complexity and needs of the program. Once the evaluator selects a research instrument and gathers data, he or she may use a number of statistical techniques to analyze and interpret the data. Such techniques allow the evaluator to determine the potential associations or correla- tions of the variables under analysis.

Table 6.6: Research instruments used in policy evaluation

Quantitative Methods Qualitative Methods

Samples Personal observations

Subject knowledge tests Case studies

Attitude surveys Personal interviews

Stage 3—Dissemination

The final stage of policy evaluation involves disseminating the evaluation findings to those who commissioned the evaluation, specifically the client. The client can be an agency or an individual policy actor, such as a mayor, governor, or agency head. In some cases evaluation findings are also forwarded to stakeholders, target groups, or the public at large. The goal of any evaluation is to provide useful information. Usefulness depends upon a number of factors, including timeliness, accuracy, and completeness. All evalua- tion reports should report assumptions, as well as real indicators that affect data inter- pretation. In sum, every evaluation will include the perceptions and assumptions that

the80472_06_c06_189-232.indd 221 11/21/12 12:52 PM

Section 6.10 Who Should Evaluate Implemented Policies? CHAPTER 6

the evaluator derives from the assessment. Additionally, all observed outcomes should have alternative explanations and separate fact from opinion; findings should be clear and unambiguous. Finally, an evaluation should, when appropriate, include recommen- dations for the policy’s continuation, change or termination.

Another aspect critical to effective dissemination is the relationship between the evalua- tor and the client. In many ways clients can influence an evaluation results by applying pressure. For example, a client may have already formed an opinion about the program and may consequently pressure the evaluator to produce an assessment in accord with a predetermined finding. In response to such concerns, professional organi- zations have in recent years clarified the rights and responsibilities of evaluators in pub- lishing standards and guiding principles for program evaluation practitioners (Sanders, 1994). The standards are principles rather than rules to which evaluators must adhere. These standards simply highlight acceptable and unacceptable practices, and thus pro- viding practitioners with benchmarks. Evaluators must decide for themselves which practices are ethical and justifiable.

6.10 Who Should Evaluate Implemented Policies?

The choice for any agency or group seeking policy evaluation is who should conduct it. In many ways this is the most critical decision in the evaluation process. The choice is between internal and external evaluators, and neither alternative is inher- ently better than the other. The key to who should be used as an evaluator depends upon the needs of the entity or organization that is commissioning the study. Internal evalua- tors have an overall advantage of being familiar with the program, the organization, the actors, and the target population. This can save time in the planning stage of the study. For example, a university has an enrollment policy for admissions that has been in place for a number of years, and the university president feels it is time to evaluate the policy. Rather than turn to an external evaluator, the president asks a university administrator from outside of the admissions department to evaluate the current enrollment policy. However, internal evaluators can prove to be a disadvantage if their ties to the organiza- tion make them too close to identify problems, to place blame, or to recommend major changes or termination.

External evaluators are individuals who have no internal connection or ties to the organi- zation being evaluated. They are perceived as outsiders. Organizations often use external evaluators when the evaluation is authorized by an entity other than the organization itself. For example, if the City of Los Angeles wanted to evaluate the Los Angeles Police Department, the city would be an authorizing entity outside of the organization being evaluated. In this case the city is more likely to use external evaluators, assuming that exter- nal evaluators will provide objective information because they have no vested interest or agenda to fulfill. An external evaluation’s major advantage is in its perceived impartiality: Evaluators supposedly have no stake in the outcome of the evaluation. This impartiality is particularly useful when controversial programs or policies are being assessed. A further strength of external evaluators is that they are usually professional consultants trained in the requisite skills and methods of evaluation techniques. In the past this was undoubtedly

the80472_06_c06_189-232.indd 222 11/21/12 12:52 PM

Section 6.11 Potential Obstacles in Policy Evaluation and in the Use of Evaluation Research CHAPTER 6

true. Recently, however, many individuals working in the public sector are educated in administration and manage- ment programs that train their students in both policy analy- sis and program evaluation; students of these programs are therefore also able to conduct an evaluation.

The major disadvantage of opt- ing for an external evaluation is cost, in both money and time. Some also argue that an exter- nal evaluation can prove costly in terms of organizational poli- tics because of its potentially disruptive nature. A further dis- advantage could be that exter- nal evaluators also have an agenda. For instance, they may want to please the client in order to secure future jobs. External evaluations therefore can potentially pose a dilemma. Nonetheless, in theory professional ethics ensure that evaluators—while mindful of client needs—should stay impartial. Another weakness of using external evaluators is they may face resistance from within the organization and among actors and other stakeholders with a vested interest in the evaluation’s outcome.

Political scientist James Q. Wilson (1973a) has formulated two general laws that put into perspective concerns about the evaluation process. Wilson’s first law is that all policy interventions in social problems produce the intended effect—provided the research is carried out by those implementing the policy or by their friends. Wilson’s second law concludes that no policy intervention in social problems produces the intended effects— provided the research is carried out by independent third parties, especially those skeptical of the policy. Wilson’s two laws help explain just how difficult the evaluation process is.

6.11 Potential Obstacles in Policy Evaluation and in the Use of Evaluation Research

As society puts more pressure on government to be more effective and efficient, evaluation will become more important. In addition, those conducting evalua-tions—private consultants, university researchers, and organizations within gov- ernment itself—will be faced with increasing obstacles as the stakes increase because of what evaluations may reveal. Several factors pose serious problems during the policy evaluation (Hogwood & Gunn, 1984). They are as follows:

Time & Life Pictures/Getty Images

Author and political scientist James Q. Wilson formulated two laws that help provide perspective about the policy evaluation process.

the80472_06_c06_189-232.indd 223 11/21/12 12:52 PM

Section 6.11 Potential Obstacles in Policy Evaluation and in the Use of Evaluation Research CHAPTER 6

• Ambiguous or unspecific objectives and goals: Policy objectives and goals are commonly unclear or equivocal; this can cloud the assessment of whether the goals and objectives have been met.

• Stated objectives, but no clear measure of the success of the objective: Most public programs, such as national defense, education, poverty, health care, crime, urban and highway planning, and environmental policy, comprise policy goals that are extremely difficult to measure directly

• Side effects from other policies that interact with the program being evaluated: In essence, the problem is how to weigh outside factors relative to the operation of the program being evaluated.

• Unavailable or unusable needed data. • Interference with the evaluation process by the politics of the situation: For

example, administrators or other policy actors resist an evaluation being conducted or its findings.

• Insufficient resources: The level of available resources will determine the type of evaluation that can be conducted; also insufficient resources often mean the most appropriate type of evaluation cannot be conducted.

• The need for validity: how valid do the evaluators want their findings to be. The obstacles to validity are numerous and range from elements within the environ- ment to methodological errors by the evaluator (Sylvia, Sylvia, & Gunn, 1997).

If evaluators do not pay close attention to such factors in the formulation and conduct of an evaluation, then the findings of the evaluation will be invalidated. Evaluators must be concerned with three broad categories of validity: internal validity, external valid- ity, and programmatic validity. Table 6.7 highlights each of these factors in achieving a valid design.

Table 6.7: The types of validity evaluators seek

Internal Validity External Validity Programmatic Validity

Does evaluation measure what it intends?

Can findings be generalized to other populations?

Does evaluation generate information that is useful to program officials?

Does evaluation require correct identification and measuring of program goals?

Can findings be replicated? Is evaluation designed to be acceptable to all audiences?

Overall, obstacles to evaluation are important factors that can prevent successful evalua- tion and hinder policy makers from using the evaluator’s recommendations. Quite often, because of contextual factors, technical factors, or human factors, decision makers may be prevented from utilizing the results of the study. Contextual factors involve factors within the environment that will be affected in unacceptable ways if decision makers act on the recommendations of the evaluation. Technical factors refer to the problems caused by meth- odological considerations. Human factors are obstacles posed by the personality and psy- chological profile of the decision makers, evaluators, client, and other internal actors. In reality, evaluation is fraught with problems and weaknesses.

the80472_06_c06_189-232.indd 224 11/21/12 12:52 PM

Section 6.11 Potential Obstacles in Policy Evaluation and in the Use of Evaluation Research CHAPTER 6

Case Study: Policy Making and Phthalates By all accounts we are awash in chemicals, many of which may pose long-term health risks. Perhaps 80,000 distinct chemicals go into U.S. industry, from the products we buy to the food we eat to the waste we discard. Phthalates are a class of chemicals known as plasticiz- ers, which are substances that make plastic products more flexible. They are nearly ubiquitous in modern society. One finds them in such common products as toys, vinyl flooring, shower curtains, shampoo, detergents, lubricants, wall-hangings, cosmetics, and adhesives.

The story of phthalates in the United States illustrates the cen- tral role of science in environmental policy making. Good science doesn’t mean good policy in the environmental realm, but good policy is nearly impossible without the steady focus of scientific inquiry. This case also highlights the importance of activist and industry groups in agenda setting, interpreting and publicizing sci- entific knowledge, and in driving policy forward. Nowhere is the interplay among science, advocacy groups, and the media more important than in the formation of environmental policy.

Phthalates account for more than a billion dollars annually in worldwide sales and are essential to the product lines of hundreds of major manufacturers. Another 1,000 or so new chemical com- pounds are introduced each year. Yet the Environmental Protection Agency and the U.S. Depart- ment of Health collect data on less than 7% of the more than 1,400 chemicals most suspected of endangering human health. Rarely are the human health impacts of newly introduced chemicals fully assessed before a chemical’s introduction into the environment. As the number of chemicals in our environment, and our bodies, grows, policy makers are able to react only after the fact when harm is clear, conclusive, and often widespread. But the evolving policy over phthalates is an indica- tion that the burden of proof may be changing.

The cost of fully analyzing thousands of new chemicals for their direct and synergistic impacts is astronomical. But a larger issue goes to how environmental policy is framed in the United States. In the United States, policy makers typically must provide compelling evidence of imminent danger before they can prohibit industrial action or control business decisions. When it comes to chemicals, as with other environmental issues, the burden of proof lies on government to demonstrate harm, rather than on industry to show that their products do no harm. Until this burden shifts, regulators will always be playing catch up, and we will be exposed to a growing diversity of chemicals with often unknown consequences.

Since 1998 phthalates have been publicly suspected of disrupting the human endocrine system. In the late 1990s animal studies began linking phthalates to reduced sperm counts, testicular atro- phy, and structural abnormalities in the reproductive systems of male test animals. Some studies suggested similar dangers to humans, but the evidence was less than certain, as is often the case when extrapolating animal data to humans. Without definitive proof of harm, the chemical industry insisted that it be allowed to continue production. They argued that banning a billion-dollar product on the basis of inconclusive evidence would be reckless and irresponsible.

For a while it seemed that industry would have its way. After a firestorm of publicity in 1998 around the potential dangers of phthalates, several environmental and religious groups petitioned the U.S. Consumer Product Safety Commission (CPSC) to ban vinyl toys for small children. The CPSC is the fed- eral agency responsible for protecting us from potential toxins in consumer products,

Brand X Pictures/Thinkstock

Phthalates are a class of chemicals used to create plastic products.

(continued)

the80472_06_c06_189-232.indd 225 11/21/12 12:52 PM

Section 6.11 Potential Obstacles in Policy Evaluation and in the Use of Evaluation Research CHAPTER 6

Case Study: Policy Making and Phthalates (continued) and it often works closely with the Environmental Protection Agency. The focus on toys by those opposed to phthalates was a shrewd political move, since young children are especially vulnerable to endocrine disrupters, and parents are especially sensitive to risks from the toys they buy their children. But the industry fought back. It drew on scientific experts like former surgeon general Dr. C. Everett Coop to defend the safety of vinyl medical devices and toys. In the midst of this debate, the best the CPSC could do was to plead for more time to assess the issue. It did, however, suggest that parents discard baby-bottle nipples and pacifiers made with vinyl. In 2003, after several alarm- ing but ultimately inconclusive scientific studies, the CPSC voted unanimously to deny the petition to ban phthalates from children’s products, stating that products made with phthalates had no proven health risks.

In trying to safeguard the public good, the CPSC sought to balance the uncertain risk of phthalates against the benefit of the thousands of products made with phthalates. But the anti-phthalate coali- tion of environmental, public health, product safety, and parents groups would have none of this. From 2002 through 2006, a new wave of highly publicized scientific studies reaffirmed the view that phthalates might be dangerous to humans. The evidence was unclear, but the risk was not: Pregnant women who used phthalate-based cosmetics, or parents who exposed their children needlessly to vinyl pacifiers or toys, were taking a chance, though just how large a chance was unclear.

California, with a large economy all its own, was the first to act. In 2002 the state legislature passed legislation prohibiting baby toys containing phthalates. San Francisco became the first city to prohibit the sale, manufacture, and distribution of all products with high phthalate concentrations in 2006. The media covered these events and others like them, fueling protests by parents who otherwise might see themselves as apolitical. In part because of scientific studies and activism in the United States, the European Union passed a law in 2006 regulating the manufacture and use of 30,000 substances, including phthalates. The U.S. phthalate industry found itself under growing pressure.

In 2008 the U.S. Congress stepped in. Emboldened by a National Academy of Science study that raised new concerns about phthalate safety, Congress passed the Consumer Product Safety Reform Act (CPSRA). The CPRSA prohibits phthalates in all children’s products, and requires manufacturers to test their products and certify them as phthalate-free. The American Chemistry Council vigor- ously opposed the law, arguing that “no reliable scientific evidence has found phthalates to cause adverse human health effects.” Some thought this statement portended later lawsuits by the Coun- cil, but more scientific studies in 2009 and 2010 that were critical of phthalates quieted the industry. The chemical industry appears to be slowly moving away from phthalates in response to declining demand in the European Union and California and persistent consumer unease and suspicion.

Congress’s intervention in 2008 to ban phthalates from children’s products is the first major imple- mentation of the precautionary principle in the regulation of potential toxins. This principle, which revolves around the idea of better-safe-than-sorry, suggests that questionable products should be banned until they are proven safe. If the precautionary principle becomes more commonplace in the United States, as it is in the European Union, policy makers may find it easier to study and monitor those chemicals that do make it to market. The burden of proof will have shifted from government to industry.

The role of California and the European Union in this case reminds us of the sometimes surpris- ing connections among the national, state, and international levels in environmental policy making. California’s decision to ban phthalates in children’s products forced manufacturers to (continued)

the80472_06_c06_189-232.indd 226 11/21/12 12:52 PM

Key Points to Remember CHAPTER 6

Summary

Although questions of whether the right policies are being implemented are signifi-cant, it seems just as significant to ask whether the right types of policies are being implemented. Unless future policy implementation becomes more effective, the problems that consistently plague society will continue, as will the likelihood that public policy will continue to fail to address these same problems. Policy actors’ challenge is in appreciating the importance of implementation before policies are simply designed with the assumption that they will be effectively executed. It is also important that policy actors be aware of the challenges that implementation poses. Awareness of the challenges shows an understanding that policy implementation is not easy and can become even more dif- ficult when policy actors pay no attention to these potential pitfalls and subsequent obsta- cles. Good policy design recognizes the value of effective and efficient implementation and will, over time, lead to better policy formulation.

Part of making policy effective is constantly assessing existing policy using evaluation. The type of evaluation carried out and the evaluators’ skill determine the effectiveness of policy evaluation.

Key Points to Remember

• Policy implementation occurs after a bill becomes a law and often involves the development of a program.

• A grasp of policy implementation leads to an understanding of the interrelated- ness of the policy process.

Case Study: Policy Making and Phthalates (continued) remove phthalates from a whole range of products, this in order to eliminate the possibility of cross- contamination in product line. By shaking up the economic logic of plastics manufacturing in 2002, California’s policy made it easier for Congress to act boldly in 2008. California has played a similar role in air pollution policy in the 1960s, energy-efficiency and renewable-energy policy in the 1980s, and water efficiency and electronic-waste disposal in the 2000s. Coming to grips with the hows and whys of national environmental policies often requires a close look at state action and international forces.

Policy making over phthalates hasn’t ended. In early 2010 the Environmental Protection Agency placed phthalates on its watch list of some 1,400 suspicious chemicals most suspected of endanger- ing human health. In August 2012 the advocacy group Center for Health, Environment, and Justice released a report documenting high levels of phthalates in back-to-school gear like backpacks and lunch bags. And New York senator Charles Schumer, among others, announced his intent to push for legislation in the Senate further regulating phthalates, based on the risk they may pose to children. As with so many environmental issues, the uncertainty swirling about the risks continues to fuel policy debate.

the80472_06_c06_189-232.indd 227 11/21/12 12:52 PM

Key Terms CHAPTER 6

• All policy actors must take into consideration the challenges and obstacles that arise during a public policy’s implementation.

• The success of all policies, irrespective of design and goals, depends on the poli- cies’ implementation.

• An interrelationship exists among policy design, implementation, impacts, and consequences.

• Bureaucracy is a major implementation actor, but other actors also affect imple- mentation, including the president, Congress, the judiciary, and interest groups.

• Three main approaches describe implementation—top-down, bottom-up, and synthesized.

• Practice of implementation requires special attention be paid the tasks that must occur over and during implementation.

• Main types of implementation are programmed implementation, adaptive imple- mentation, and premeditated nonimplementation.

• Policy evaluation became more common after 1960s. • Policy evaluation basically assesses whether a policy is working and doing what

it should. • Policy evaluation allows for decisions to be made as to whether policies should

continue, change, or terminate. • There are four main types of evaluation models—process, outcome, impact, and

cost-benefit analysis. Each can be used separately or in conjunction with another. • Evaluation can be viewed as a three-stage sequence:planning, data gathering,

and dissemination. • A critical decision in evaluation process is who does the evaluation—internal or

external evaluators. • Evaluation is fraught with problems and weaknesses.

Key Terms

adaptive implementation Allows original policy mandate to be adjusted as policy is being implemented.

ambiguity-conflict model A synthesized implementation model that posits the level of ambiguity of policy goals, the level of ambiguity in the ways and means to achieve the policy goals, and the level of conflict surrounding the policy and the problem it deals with are critical to under- standing how any policy is implemented.

bottom-up approach An approach to implementation that views policy as incomplete and political because it is car- ried out by local-level administrators who are subject to interaction with a network of actors and groups.

bureaucracy A major actor that is com- posed of governmental agencies, such as the Department of Justice, and has the responsibility to ensure effective and efficient implementation of governmental policies.

challenges of implementation Policy implementer’s general base of knowledge about what challenges may emerge to hin- der the execution of a policy program.

cost-benefit analysis The calculation of the net balance of the benefits and costs of a program.

critical path method (CPM) A visual representation of the steps necessary to implement policy decision.

the80472_06_c06_189-232.indd 228 11/21/12 12:52 PM

Key Terms CHAPTER 6

evaluation process A three-stage sequence of planning, datagathering, and dissemi- nation that ensures a policy evaluation is successful.

external validity A determination that the findings of evaluation can be generalized and replicated.

impact evaluation A type of policy evalu- ation that focuses on whether a program is having an impact on the intended target population by assessing whether that population is being affected by the intro- duction and implementation of the policy.

information intelligence Strategic necessity for constant feedback on how implementation is progressing, as well as—when possible—preliminary assess- ments of impact.

internal validity A determination that an evaluation measures what it intends to measure.

outcome evaluation A type of policy eval- uation that focuses on the degree to which a policy is achieving its intended objectives in regard to the target population.

policy evaluation The study of what hap- pens as a result of policy implementation.

policy implementation The execution of an adopted policy as specified by the legis- lation or policy action.

premeditated nonimplementation A deci- sion to implement policy in such a way that implementation does not occur.

process evaluation A type of policy evaluation that assesses how a program or policy is being delivered to target popula- tions or how it is being managed and run by administrators.

programmatic validity A determination that evaluation generates useful informa- tion to program officials and is acceptable to all agencies.

programmed implementation Explicit programming of policy implementation procedures.

rule making Process by which rules and regulations are subject to the constraints of the Administrative Procedures Act of 1946.

stakeholders Individuals, agencies, or groups who have stakes in the outcome of a policy evaluation.

strategic planning Tool used in imple- mentation to (a) assess whether capabil- ity is present to implement the policy mandate and (b) plan the execution of implementation.

synthesis An approach to implementation that views implementation as both top- down and bottom-up.

target groups Specific population that a particular policy targets.

top-down approach An approach to implementation that views policy as plan- ning, hierarchy, and control; thus, it is an administrative function, because policies are complete and nonequivocal imple- mentation is the enforcement of passed statutes.

the80472_06_c06_189-232.indd 229 11/21/12 12:52 PM

Applied Internet Exercises CHAPTER 6

Applied Internet Exercises: Policy Implementation and Evaluation

Senate Memo: Designed for Implementation

As a new U.S. Senate staffer, you’ve been tasked with determining whether the senator’s new policy idea incorporates a well-thought-out policy design that will make effective implementation possible or not.

Your tasks are as follows:

1. Choose the senator for whom you are working. Use the following link to help you: http:// www.senate.gov/general/contact_information/senators_cfm.cfm.

2. Choose any recent policy that the senator has proposed. You can find these on the senator’s website.

3. Once you have selected your policy proposal, you will need to look at past studies. To aid in your research, you can use current or former studies done on similar legislation. One handy resource is http://www.gao.gov/.

4. Prepare a three-page memo that answers some or all of the following relevant questions: • Was the design of the policy appropriate given the nature of the problem? • Does the design assist or complicate policy implementation? • Is the agency organizationally capable of administering the program or policy? • Can the instruments be effectively implemented? • Are there political obstacles to effective and efficient implementation?

Critical Thinking and Discussion Questions

1. Is the bureaucracy ultimately responsible for the effectiveness of government? 2. Policy actors might not be truly confident about the intended or unintended

impacts of an implemented policy. Given the array of policy actors involved, is effective implementation of policy still the primary goal?

3. Take any recently adopted policy and map out, using the CPM method described in Figure 6.4, the steps and estimated time to effectively implement this policy. Include each step that you can logically deduce is relevant to successful imple- mentation from beginning to end.

4. Design an impact evaluation for a public sector program that provides basic medical services to the homeless.

5. In 2012 the state of California attempted to increase the number of elementary teachers, encourage relocation of teachers to rural areas, and improve the practical teaching experience during training. To accomplish these goals the state opened a new regional campus of its state university and appointed a provost to run it. The first 3 years of the provost’s tenure in office were marked by continual pressure and criticism from the local school board, the teachers’ union, the local community, and the media. In response to such lobbying and criticism, the provost redefined the original goals of the initiative. He developed operational goals of recruiting former teachers as university faculty, established working relationships with all elemen- tary schools in the immediate area, and created programs that enabled currently employed teachers to enhance their teaching skills. How would you evaluate this particular program and why?

(continued)

the80472_06_c06_189-232.indd 230 11/21/12 12:52 PM

Suggested Readings CHAPTER 6

Suggested Readings

Albaek, E. (1996). Why all this evaluation? Theoretical notes and empirical observations on the functions and growth of evaluation. Canadian Journal of Program Evaluation, 11(2), 1–34.

Berman, P. (1980). Thinking about programmed and adaptive implementation: Matching strategies to situations. In H. Ingram & D. Mann (Eds.), Why policies succeed or fail (pp. 205–227). Beverly Hills, CA: Sage.

Bingham, R. D., & Felbinger, C. (2002). Evaluation in practice: A methodological approach. New York: Seven Bridges Press.

Ferman, B. (1990). When failure is success: Implementation and Madisonian govern- ment. In D. Palumbo & D. Calista (Eds.), Implementation and the policy process: Opening up the black box (pp. 39–50). Westport, CT: Greenwood Press.

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. (2004). Program evaluation: Alternative approaches and practical guidelines. Boston, MA: Pearson.

Googin, M., Bowman, A. O., Lester, J., & O’Toole, L. J., Jr. (1990). Implementation, theory, and practice: Towards a third generation. Glenview, IL: Scott, Foresman/Little, Brown.

Matland, R. (1995, April). Synthesizing the implementation literature: The ambiguity- conflict model of policy implementation. Journal of Public Administration Research and Theory, 5, 145–174.

O’Toole, L. J., Jr. (2000, April). Research on policy implementation: Assessments and prospects. Journal of Public Administration Research and Theory, 20, 263–288.

Internet Applied Exercises: Policy Implementation and Evaluation (continued) • What impact will the policy solution, once implemented, have on the target population(s)? • How essential are the administrative actors to the policy’s success? • Have political factors prevented or made difficult policy implementation?

5. Provide a final conclusion of whether or not the senator’s policy is designed to aid implementa- tion. In either case provide an explanation of why you drew this conclusion and your recom- mendations moving forward.

the80472_06_c06_189-232.indd 231 11/21/12 12:52 PM

the80472_06_c06_189-232.indd 232 11/21/12 12:52 PM