Extensive Crime Control Response Needed

Demonstration Exercise 1109

one of

E, ulicy

lone III

[henry ~~ ~~73~,

,,: are

u. ivur-

~~;; title nrnces."

~~nge~

pcas- dl'zed. T rues is that in the ;dam's

plants edam.

xplain i from

~~ad ink the

es with lads, as ~ towns ecrirely p t Fund ih<<~«gh limited fsuper- '~s have :e~~t~ i ~of ~f~~~ the ~n gees,

counties where apple orchards abound (Wall StreetJoa~rnal, December 1, 1977).

Edward J. Moody ... argues persuasively that worship of Satan has the effect of normalizing abnormal people. Thus, to "keep secret" from ordinary people their satanic power and exis- tence, such persons are urged to behave as straight as possible. The effect, of course, is more effective social relations—the goal for which Satan's name has been invoked in the first place! (P. E. Hammond, "Review of Religious Movements in Contemporary America," Science, May 2, 1975, p. 442).

Residents of San Francisco's North Beach areas must now pay $10 for the privilege of parking in their own neighborhood. A residen- tial parking plan was recently implemented to prevent commuters from using the area as a daytime parleing lot. But according to a story in the San Francisco Bay Guardian (March 14, 1978), the plan has in no way improved the residential parking situation. Numbers of commuters from outlying districts of the city have simply been changing their car registra- tions to North Beach addresses. A North

1. Choose a policy issue area such as crime control, national security, environmental protection, or economic development. Use the procedures for stakeholder analysis presented in Procedural

~ Guide 3 to generate, a list of stakeholders who ~ affect or are affected by problems in the issue ~:y area you have chosen for analysis.

After generating the list, create a cumulative frequency distribution. Place stakeholders on the horizontal axis, numbering them from 1 ... n. On the vertical axis, place the number of new (nonduplicate) ideas generated by each stakeholder (the ideas can be objectives, alter- natives, outcomes, causes, etc.). Connect the total new ideas of each stakeholder with a line graph. ■ Does the line graph flatten out? e If so, after how many stakeholders?

Beach resident—now $10 poorer—still spends a lot of time driving around the blocle.

Choose one of these problems and write a short essay on how classification analysis, hier- archy analysis, and synectics might be used to structure this problem.

7. Construct a scenario on the state of one of the following problem situations in the year 2030:

Availability of public mass transit Arms control and national security Crime prevention and public safety Quality of public education State of the world's ecological system

8. Select two editorials on a current issue of public policy from two newspapers (e.g., New York Times, R7asbington Post, The Economist, Le Monde) or news magazine (e.g., Newsweek, The New Republic, National Review). After reading the editorial: a. Use the procedures for argumentation

analysis (Chapter 8) to display contending positions and underlying assumptions.

b. Rate the assumptions and plot them accord- ing to their plausibility and importance (Figure 3.16).

c. Which arguments are the most plausible?

~ What conclusions can you draw about the policyproblem(s) in the issue area?

Compare your worle with Case Study 3.1 at the end of the chapter. After reading Case 3.1, write an essay in which you compare and contrast the process of boundary analysis and estimation in mining and transportation. In your comparison, address these questions: ■ What are the lcey differences in data collec-

tion, represented by the process of group interviewing and content analysis?

■ Why do the cumulative frequency graphs flatten out the way they do?

■ Evaluate the statement: "Boundary analysis is a reliable way to estimate the ̀ universe of problem formulations' in a given policy issue area."

Bibliography ~ 111

tion of Global

on to n New

gal and

New

s. 7 he Tendon

S «A

~,," In Inl 10 reJ by ;New

es for

s. t P, ~~nid

W,

Definition

A stakeholder is a person who speaks for or represents a group that is affected by or affects a policy. Stai<eholders include the president of a legislative assembly or parliament, a chairperson of a legislative committee, or an executive director or members of an organized interest or advocacy group such as the National Rifle Association, the Sierra Club, or Human Rights Watch. Policy analysts and their employers are stakeholders, as are clients who commission a policy analysis. Persons or groups who do not have a stale in a policy (e.g., an uninvolved college professor) are not stakeholders.

ASSUIIiptI011S

• Stakeholders are best identified by policy issue area. A policy issue area is a domain in which stakeholders disagree or quarrel about policies. Housing, welfare, education, and international security are policy issue areas.

• Stakeholders have specific names and titles— forexample, State Senator Xanadi; Mr. Young, chairperson of the House Finance Committee; or Ms. Ziegler, a spokesperson for the National Organization of Women (NOW).

• A sociometric or "snowball" sample such as that described next is an effective way to estimate the "population" of stakeholders.

STEP 1: Using Google or a reference book such as The Encyclopedia of Associations, identify and list about ten stai<eholders who have taken a public position on a policy. Male the initial list as heterogeneous as possible by sampling opponents as well as supporters.

STEP 2: For each stakeholder, obtain a policy document (e.g., a report, news article, e-mail, or telephone

interview) that describes the position of each stakeholder.

STEP 3: Beginning with the first statement of the first stakeholder, list other stakeholders mentioned as opponents or proponents of the policy.

STEP 4: For each remaining statement, list the new stakeholders mentioned. Do not repeat.

STEP 5: Draw a graph that displays statements 1, 2, ...non the horizontal axis, On the vertical axis, display the cumulative frequency of new stakeholders mentioned in the statements. The graph will gradually flatten out, with, no new stakeholders mentioned. If this does not occur before reaching the last stakeholder on the initial list, repeat steps 2 to 4. Add to the graph the new statements and the new stakeholders.

STEP 6: Add to the estimate stakeholders who should be included because of their formal positions (organization charts show such positions) or because they are involved in one or more policy-making activities: agenda setting, policy formulation, policy adoption, policy

implementation, policy evaluation, and policy adaptation, succession or termination.

Retain the full list for further analysis. You now have an estimate of the "population" of I<ey stakeholders who are affected by and affect the policy, along with a description of their positions. on an issue. This is a good basis for structuring the problem.

Complex problems must be structured before they can be solved. The process of structuring a policy problem is the search for and specification of problem elements and how they are the elements are

Policystalceholders, Which stakeholders affect or are affected by a problem?

Policy altet'natives. What alternative courses of action may be taken to solve the problem?

Policy actions, Which of these alternatives should be acted on to solve the problem?

Policy outcomes. What are the probable outcomes of action and are they part of the solution to the problem?

Policy values (ufilities), Are some outcomes more valuable than others in solving the problem?

Most policy problems are messy or ill-structured. For this reason, one or more problem elements can be incorrectly omitted from the definition of a problem. Even when problem elements are correctly specified, relations among the elements may be unknown or obscure. This males it difficult or impossible to determine the strength and significance, practical as well as statistical, of causal relations. For example, many causal processes that are believed to govern relations among atmospheric pollution, global warming, and climate change are obscure. The obscurity of these processes stems not only from the complexity of "nature" but also from the conflicting beliefs of stakeholders who disagree, often intensely, about the definition of problems and their potential solutions. For this reason, the possible combinations and permutations of problem elements—that is, stakeholders, alternatives, actions, outcomes, values—appearto be unmanageably huge.

Under these conditions, standard methods of decision theory (e.g., risk-benefit analysis), applied economics (e.g., benefit-cost analysis), and political science (e.g., policy implementation analysis) are of

79The architecture analogy is from Herbert Simon.

112

limited value until the problem has been satisfactorily defined. This is so because an adequate definition of the problem must be constructed before the problem can be solved with these and other standard methods. Standard methods are useful in solving relatively well-structured (deterministic) problems involving certainty, for example, problems represented as fixed quantities in a spreadsheet. Standard methods are also useful in solving moderately structured (probabilistic) problems involving uncertainty, for example, problems represented as policy outcomes with different probabilities. However, ill-structured problems are of a different order. Estimates of uncertainty, or risk, cannot be made because we do not even Know the outcomes to which we might attach probabilities. Here, the analyst is much like an architect who has been commissioned to design a custom building for which there is no standard plan.79 The adoption of a standard plan, if such existed, would almost certainly result in a type III error: solving the wrong problem.

Public policies -are-deliberate attempts to change

complex systems. The process of making and implementing policies occurs in social systems in which many contingencies lie beyond the control of policy makers. It is these unmanageable contingencies that are usually responsible for the success and failure of policies in achieving their objectives. The contingencies are rival hypotheses that can challenge claims that a policy (the presumed cause) produced one or more policy outcomes (the presumed effects). In such cases, it is usually desirable to test, and when possible eliminate, these rival hypotheses through a process of eliminative induction. Eliminative induction tales this general form: "Repeated observations of policy x and outcome y confirm that x is causally relevant to the occurrence of y. However, additional observations of x, z, and yconfirm that uunmanageable contingency z and not policy x is responsible for the

Case 3.1 ~ 113

occurrence of y." By contrast, enumerative induction tales this general form:"Repeated observations confirm that the policy x is causally relevant to the occurrence of policy outcome y."

Eliminative induction permits a critical examination of contingencies that are beyond the control of policy makers. Because the number of these contingencies is potentially unlimited, the process of identifying and testing rival explanations is never complete. Yet, precisely for this reason, it seems impossible to identify and test an unmanageably huge number of potential rival hypotheses. How is this to be done?

One answer is creativity and imagination. But creativity and imagination are impossible to teach, because there are no rules governing the replication of creative or imaginative solutions. Another answer is an appeal.to well-established theories. However, the bulk of theories in the social sciences are disputed and controversial. "Well-established" theories are typically "well-defended" theories, and rival hypotheses are rarely considered seriously, let alone tested.

Amore appropriate alternative is the use of boundary analysis and estimation to structure problems involving a large number of rival hypotheses. Boundary analysis and estimation look for rival hypotheses in the naturally occurring policy quarrels that tale place among stakeholders. In addition to the policy analyst, these stakeholders include scientists, policy makers, and organized citizen groups. The aim of boundary estimation is to obtain a relatively complete set of rival hypotheses in a given policy context. Although boundary estimation strives to be comprehensive, it does not attempt the hopeless task of identifying and testing all plausible rival hypotheses. Although the range of rival hypotheses is never complete, it is possible to estimate the probable limit of this range.

Assessing the Impact of National Maximum Speed Limit

A boundary analysis was conducted with documents prepared by thirty-eight state officials responsible

for reporting on the effects of the original 55 mph and later 65 mph speed limits in their states. As expected, there were sharp disagreements among many of the thirty-eight stakeholders. For example, some states were tenaciously committed to the hypothesis that speed limits are causally related to fatalities (e.g., Pennsylvania and New Jersey). Others were just as firmly opposed (e.g., Illinois, Washington, Idaho). Of direct importance to boundary estimation is that 718 plausible rival hypotheses were used by thirty-eight stakeholders to affirm or dispute the effectiveness of the 55 mph speed limit in saving lives. Of this tota1,109 hypotheses were unique, in that they did not duplicate hypotheses advanced by any other stakeholder.

Here, it is important to note that from the standpoint of communications theory and language, the information content of a hypothesis is inversely related to its relative frequency or probability of occurrence. Hypotheses that are mentioned more frequently—those on which there is greater consensus—have less probative value than rarely mentioned hypotheses, because highly probable or predictable hypotheses do not challenge accepted I<nowiedge claims.

The rival hypotheses were analyzed according to the cumulative frequency of unique (nonduplicate) causal hypotheses. As Figure C3.1 shows, the cumulative frequency curve of unique rival hypotheses flattens out after the twenty-second stakeholder. Although the total number of rival hypotheses continues to increase without apparent limit, the boundary of unique rival hypotheses is reached within a small and affordable number of observations. This indicates that a satisfactory definition of the problem has probably been achieved. Indeed, of the 109 unique rival hypotheses, several variables related to the state of the economy—unemployment, the international price of oil, industrial production— explain the rise and fall of traffic fatalities better than average highway speeds and the 55 mph speed limit.

(continued)

114 ~ CHAPTER 3 Structuring Policy Problems

120

100

~ 80

U ~a 60

40

20

0

Boundary;Estimate

I

I

1 I I I I

i

i i

I I I

_.,. i

1 4 7 10 13 16 22 23

Stakeholders

100%

75%

50%

25%

0%

FIGURE C3.1

Pareto chart—cumulative frequency of rival causes of traffic fatalities

Evaluating Research on Risl< in Mine Safety

and Health

In 1997, a branch of the U.S. Office of Mine Safety

and Health Research began a process of strategic

planning. The aim of the process was to reach

consensus, if possible, on the prioritization of

research projects that address different aspects of

risk associated with the safety and health of miners.

Priority setting in this and other research

organizations typically seeks to build consensus under

conditions in which researchers, research managers,

and external stakeholders use conflicting criteria to

evaluate the relative merits of their own and other

research projects. Even when reliable and valid data are

available—for example, quantitative data from large-

sample studies of the probabilities of different Kinds of

mine injuries and deaths—it is often unclear whether

"objective" measures of risk, by themselves, provide a

sufficient basis for prioritizing research. The difficulty

is that extra-scientific as wel I as scientific factors affect

judgments about the relative merits of research on risk.

For example, the probabilities of the occurrence of

'o m n m ~.

"black lung" disease and other high-risk conditions

have been intensively investigated. Despite the

importance of the black lung problem, additional

research is not a priority. Accordingly, data on high

expected severity (probability ~ severity) do not alone

provide a sufficient basis for prioritizing research

problems. Because judgments about research priorities

are based on multiple, hidden, and frequently

conflicting criteria, it is important to uncover and

evaluate these criteria as part of the process of problem

structuring. This is a classic ill-structured problem for

which the fatal error is defining the wrong problem.

The problem-structuring process had three major

objectives. The first was to uncover hidden sources of

agreement and disagreement, recognizing that

disagreement is an opportunity for identifying

alternative approaches to priority setting. Second, the

process was designed to generate from stakeholders

the criteria they use to evaluate research on risks

affecting mine safety and health. Third, the process

employed graphs, matrices, and other visual displays

in order to externalize the criteria underlying individual

Case 3.1 1115

100

80

so 'L

+~ 60 ~L

U Y N

40

20

0 1 2 3 4 5 6 7 8 9 10 11 12 13

Stakeholder

120%

100%

80%

m n

60% ~ 't

40%

20%

0%

FIGURE C3.2

Pareto chart—cumulative frequency of criteria for evaluating research

on risi<

judgments. The priority-setting process enabled each criteria could be offered that were not mentioned

team member to understand and debate the varied before. In al I, 84 criteria were generated, none after

reasons underlying the priorities used to evaluate the eleventh presenter. The approximate boundary

research on risk. of the problem was displayed with a Pareto chart to

Stakeholders were prompted to state the criteria show the cumulative frequency of constructs

they use to distinguish among twenty-five accident tFigure C3.2).

and health problems. Each stai<eholder was The process of interactive group priority setting

presented with a form listing pre-randomized sets of not only addressed the "objective" side of risk

three problems. One stakeholder was chosen at research but also captured "subjective" dimensions

random to present criteria to the group. The first that go by such labels as "perceived risk,"

presented 14 criteria for evaluating research on "acceptable risk," "researchable risk," and

risi<, which included two criteria—severity and "actionable risk." The problem was successfully

catastrophic potential of accidents and diseases— bounded by means of an open problem-structuring

which were based on large-sample relative process of generating and discussing criteria that

frequency (probability) data. The second randomly were otherwise tacit, concealed, or implicit. The

chosen team member presented 17 additional (new) method of boundary analysis and estimation ~. criteria. The third team member added 11, the provides a "stop rule" that avoids the infinite

fourth 12, the fifth 6, and so on, until no additional search for problems.

P i

E

S

k