PADM530Wk4

Pitfalls in Evaluating Criteria

Vague and general goal statements or goals that con�ict can make criteria di�cult to use. When there is more than one alternative available,

policymakers must decide the most important consideration when choosing the best one. Having a speci�c and measurable goal statement makes this

easier to do.

There can be no ambiguity in the criteria and their measures. No matter who uses them, the results should be the same. Appropriate measurement

units need to be used for each criterion.

Overview

Welcome to Week 4.

Last week we began our deeper look into the policy analysis process through our

discussion on de�ning/analyzing the problem and constructing alternatives. This

week we consider the next step in the process: developing evaluative criteria. We

will also identify some of the laws and methods used in evaluation. 

Course Objectives

01 CO4: Evaluate the policymaking process. 

02 CO5: Evaluate the processes of implementation and regulation.

The Delphi Method, Impact Assessments, and Policy Implementation

The Delphi Method

The Rand Corporation as a group communication procession developed this technique, named a�er the Greek oracle who could predict the future. Is

a way to reach consensus among a group in a more structured setting.

You can learn more here.

Impact Assessments

Impact assessments measure the e�ectiveness of the activities of organizations. They may be negative, positive, intended, or unintended results.

Impact analysis can predict what might occur with an intervention, noting the potential risks; however, they usually focus on what has already

happened to improve future e�orts. They generally try to determine causes and e�ects by establishing a chain of causation from the policy. This is

generally limited to observed e�ects. While there isn’t a single method for impact assessment, as many sources as possible should be used to evaluate

data.

Policy Implementation

The �nal step is implementation. The behavior of the group must be directly linked to the policy. This means that the objectives cannot be ambiguous.

As we have seen, clarity is vital.

Conclusion

This week we learned about conducting evaluations and using speci�c criteria to

learn how well a policy met its goals and objectives and noted that those goals and

objectives must be clear and speci�c. We reviewed some acts that required

performance reports and the Delphi technique.

Government Performance Acts

The Government Performance and Results Act (GPRA)

The Government Performance and Results Act  (GPRA) required performance reports on federal agencies. These reports were to be released to the

public. The idea was to improve program performance and public con�dence.

Here are some examples of recent examples of these reports. 

Government Performance and Results Modernization Act of 2010

This act created connected plans, programs and performance information. These reports were required quarterly instead of annually.

Introduction

Last week, we continued our examination of policy analysis through the construction of alternatives. This week we will continue the process

beginning with developing alternative criteria.

References EXAMSPM. (2020). Delphi Technique PMP Exam (definition & example provided). Retrieved June 25, 2021, from https://www.youtube.com/watch? v=NVWgeoZPX2U

Government Performance and Results Act (GPRA) Related Materials. (n.d.). Retrieved November 21, 2017, from https://obamawhitehouse.archives.gov/omb/mgmt-gpra/index-gpra

Government Performance and Results Act (GPRA) Reports . (n.d.). Retrieved November 21, 2017, from https://www.federalreserve.gov/publications/gpra.htm

Kaiser, F. M., and McMurtry, V.A. (1998, March 17). Government Performance and Results Act: Proposed Amendments (H.R. 2883). Congressional Research Service. Retrieved from https://digital.library.unt.edu/ark:/67531/metacrs672/m1/1/high_res_d/98- 224gov_1998Mar17.pdf.

Stobierski, T. (2019) How to do a Cost-Benefit Analysis Y Why It’s Important. Harvard Business School Online. Retrieved June 25, 2021, from https://online.hbs.edu/blog/post/cost-benefit-analysis

U.S. Congress. S.20 - Government Performance and Results Act of 1993. Retrieved from https://www.congress.gov/bill/103rd-congress/senate-bill/20.

Criteria

Select each item to learn more.

Open All Panels

Policy analysis usually includes economic criteria. This may include factors such as impact on the economy, expected public revenue and

spending. The most common economic criterion is cost. This may include the costs of borrowing funds, costs to implement the policy,

decreases in net worth, and other costs that cannot be easily quanti�ed as well as �xed costs, maintenance costs, and opportunity costs.

If not all costs are identi�ed and counted, accountability concerns may develop.

• Marginal cost: Marginal cost is the cost of producing one more unit of something that is already being produced.

• Bene�ts: Bene�ts that can be directly ascribed to the policy alternative include things such as interest that is accrued.

Economic Criteria

When examining the fairness criteria, determine who pays for the policy (someone must always pay) and who bene�ts from it. If those who

pay for a policy do not bene�t, it is not likely to be fair. The most popular proposals have been “tax the rich” or tax “business” but this is not

necessarily the best solution.

Most people agree that discrimination based on arbitrary factors such as race, gender, disability is wrong, and that all people need to be

treated the same. Horizontal fairness tells us whether burdens and bene�ts are distributed equally amongst societal groups. Vertical fairness

concerns whether the burdens and bene�ts are shi�ed among unequal groups. Intergenerational fairness concerns whether burdens and

bene�ts are being shi�ed from one generation for another. An example of intergenerational fairness is that the current generations received

COVID-19 stimulus and unemployment enhancement money and future generations are going to have to pay for this and the associated

interest.

Fairness Criteria

E�ectiveness is probably the most o�en used criterion because it measures how well the policy option has solved the problem.

Political Criteria

Elected or appointed o�cials must support policy alternatives in order for approval. For this reason, it is important to know that the

alternative will be acceptable and appropriate to the citizens and legislators. Examining current law determines if further amendments are

required and whether or not constitutionality will help determine the political acceptance.

Administrative Operability

Government agencies tasked with implementing policy, should determine administrative operability and feasibility, in order to ease the

process. Consider whether the agency has the authority, personnel, skills, funding, and training to implement the policy.

Political and Administrative Criteria

Criteria

Select each item to learn more.

Close All Panels

Policy analysis usually includes economic criteria. This may include factors such as impact on the economy, expected public revenue and

spending. The most common economic criterion is cost. This may include the costs of borrowing funds, costs to implement the policy,

decreases in net worth, and other costs that cannot be easily quanti�ed as well as �xed costs, maintenance costs, and opportunity costs.

If not all costs are identi�ed and counted, accountability concerns may develop.

• Marginal cost: Marginal cost is the cost of producing one more unit of something that is already being produced.

• Bene�ts: Bene�ts that can be directly ascribed to the policy alternative include things such as interest that is accrued.

Economic Criteria

When examining the fairness criteria, determine who pays for the policy (someone must always pay) and who bene�ts from it. If those who

pay for a policy do not bene�t, it is not likely to be fair. The most popular proposals have been “tax the rich” or tax “business” but this is not

necessarily the best solution.

Most people agree that discrimination based on arbitrary factors such as race, gender, disability is wrong, and that all people need to be

treated the same. Horizontal fairness tells us whether burdens and bene�ts are distributed equally amongst societal groups. Vertical fairness

concerns whether the burdens and bene�ts are shi�ed among unequal groups. Intergenerational fairness concerns whether burdens and

bene�ts are being shi�ed from one generation for another. An example of intergenerational fairness is that the current generations received

COVID-19 stimulus and unemployment enhancement money and future generations are going to have to pay for this and the associated

interest.

Fairness Criteria

E�ectiveness is probably the most o�en used criterion because it measures how well the policy option has solved the problem.

Political Criteria

Elected or appointed o�cials must support policy alternatives in order for approval. For this reason, it is important to know that the

alternative will be acceptable and appropriate to the citizens and legislators. Examining current law determines if further amendments are

required and whether or not constitutionality will help determine the political acceptance.

Administrative Operability

Government agencies tasked with implementing policy, should determine administrative operability and feasibility, in order to ease the

process. Consider whether the agency has the authority, personnel, skills, funding, and training to implement the policy.

Political and Administrative Criteria

Evaluative Criteria for Public Policy

Following problem identi�cation and clari�cation, development of goals in order to explain what the policy alternative(s) exist will be accomplished. It

is usually broad with a long-term solution or set of solutions. They must be speci�c and measurable. Think SMART goals (speci�c, measurable,

achievable, relevant, and time-based)

Speci�c criteria used to compare how well various alternatives meet goals are as follows:

In addition to the criteria identi�ed above, �ve additional criteria generally be employed in the evaluation process are:

E�ectiveness - measures how well a policy achieves its goal;•

E�ciency - measures the cost of a policy in relation to its expected bene�ts; and•

Fairness- measures the level of fairness in the distribution of costs and bene�ts—that is, who pays the costs and who reaps the bene�ts.•

Ethics - measures how consistent the policy is with cultural norms;•

Political feasibility- measures the level of acceptance of a proposed policy by elected o�cials;•

Social acceptability - measures the level of public acceptance and support of a proposed policy; •

Administrative feasibility- measures the ability to implement a proposed policy; •

Technical feasibility- measures the availability and reliability of the technology needed to implement a proposed policy.•

Maintainability- measures how long the policy can be continued without additional funding sources.•