policy
ueceranz5b
CJ_421_PolicyAnalysis_Winter2018_2.12.2018.ppt"Criminal Justice Policy and Planning 4th edition
Wayne N. Welsh Philip W. Harris"
Policy Analysis defined
“Policy Analysis is the use of social research procedures to systematically investigate the effectiveness of social intervention policies.”
Policy Analysis defined
- "A rule or set of rules or guidelines for how to make a decision."
Planning a Policy Analysis
- There is no blueprint or recipe for conducting a good analysis. Because the term analysis is subject to different interpretations, a policy can be evaluated in a variety of ways.
Adapted from: policy Development and Analysis, Taylor-Powell
Planning a Policy Analysis
- You will discover that policy analysis is more than just collecting information. It involves serious reflection on questions such as:
What is the purpose of the analysis?
What do I want to know?
What do I intend to do with the information?
What is the purpose of the analysis?
- the fundamental purpose of analysis is to create greater understanding.
- what is the purpose(s) of the policy analysis you propose?
Help others understand the policy and its results?
Improve the policy?
Determine if the policy is worth the cost?
Example
Example
National Analysis of WEED & SEED Cross-Site Analysis
U.S. Department of Justice
Office of Justice Programs
Example
A Policy Analysis of the Minneapolis Fire Department Recruit Training Academy
Minneapolis Fire Department
What Counts as Credible Evidence in Applied Research and Analysis Practice? Facts About Applied Research
- Evidence-based research:
Some scientists, conducting basic research, chose subjects of interest to them to advance knowledge
Some scientists, conducting applied research, seek to prevent or solve real world problems
Applied Research
- Descriptive: helps us understand practical problems and their potential solutions;
- Evaluative: improves or determines the effectiveness of actions (e.g., programs and policies) to prevent and solve practical problems
Applied Research
- Focus: credible evidence in applied research and analysis;
- Text addresses problem-based or solution oriented issues, and conducted in “real world” settings as opposed to highly controlled, traditional scientific laboratories
Applied Research
- “Experimenting Society” (Donald T. Campbell);
Hard headed tests … random experiments:
Methodological breakthroughs:
Detailed understandings of threats to validity,
Bias control,
Multiple types of validity,
Implementation of rigorous experimental and quasi-experimental design
- Now we have moved to “Evidence-based” research.
Evidence Based Research….What is it?
Life offers many lessons --- learning experiences that help guide future decisions. Wouldn't it be nice to have these lessons in advance, before venturing along the wrong path? Evidence-based research uses that approach, paying attention to proven signs that simplify and qualify the research process.
Researchers have found a way to use existing theories and ideas to aid their future research. It's a combination of science and experience, exploring the possibilities while respecting sound facts.
(Retrieved from: http://www.ehow.com/about_5118300_evidence-based-research-definition.html)
Evidence Based Research
Medical professionals rely on information that is evidence-based in order to make the right decisions for patient care. What does that mean, really?
Evidence Based Research
Evidence-based means that the information you, as a professional, use is based on sound research, not someone's opinion. Typically, that means you must locate the best, published research studies on the topic under enquiry. The published article (sometimes in paper, but most often in electronic form) presents the actual research results, so that you can see how the conclusions were reached. In addition, the researcher provides data to support those conclusions.
Evidence Based Research
Other professions also rely on such literature, but they use the terms "research-based," "scholarly," "academic," or "peer-reviewed" literature. All these terms mean the same thing as "evidence-based." So if someone uses one of these terms to refer to information, you can be assured they mean evidence-based research.
Evidence Based Research
- Evidence defined:
The basic definition, evidence is proof supporting a theory. It is the smoking gun and bullet taken from the crime scene. In other words, the gun and bullet (evidence) support the victim's death by gunshot wound (theory).
Consider, however, the definition of theory -- a fact-based explanation. This means the victim's death by gunshot wound is a theory ascertained by reviewing many variables, one of which is evidence.
Evidence Based Research
- Defining Research
Research is the investigation, collection and interpretation of facts. However, research is conducted in many ways depending on the field and its accepted methods.
Legal research involves studying previously decided cases, examining evidence and compiling material that will support or disprove a case. The legal field requires case precedence (previously decided cases) as part of research.
In the field of science, research often involves on-site work as well as case studies. Scientific research can be lengthy and involve experimentation. It applies an entirely different research process.
Evidence Based Research
- Evidence-Based Research
Having explored the meaning of evidence and research, these words help define evidence-based research. In most cases, evidence-based research is used to describe a type of research where the researcher is aware of certain evidence before exploring the subject.
In short, the researcher does not enter the project unbiased --- he or she is aware of a theory derived from the evidence and uses research to test its validity.
- Using Evidence-Based Research
Why would an organization want evidence-based research? Most likely, they want to assure the researcher uses qualified material (studies and prior research from industry-recognized sources). In addition, applying previously known theories saves the expense of starting from scratch; the researcher can use what is already known and move forward.
Consider, for example, the Society of Prevention Science, who not only requires evidence-based research, but specifies where to obtain the evidence. They avoid the use of unqualified evidence by specifying reputable sources.
Evidence Based Research
- Considerations
The term "evidence-based" carries different meanings for different industries; however, the common thread seems to be an unbiased approach and the combination of traditional research and factual evidence.
Traditional research supports approaching a project with no preconceived notions. By adding evidence, the approach may not remain unbiased, but valuable knowledge is carried into the project.
Evidence Based Research
Week Review
Model for Planned Change
What is Analysis?
“ the systematic investigation of the merit, worth, or significance of an ‘object’ ”
Michael Scriven
“…the systematic assessment of the operation and/or outcomes of a policy or policy, compared to a set of explicit or implicit standards as a means of contributing to the improvement of the policy or policy…”
Weiss Carol
“ A systematic way to determine the “value” of a policy, policy components, or activity.”
Unknown XXX
Slide 4 Objective: Define Analysis
Speaker:
What is Analysis? Here are three perspectives.
According to Michael Scriven, Analysis is “the systematic investigation of the merit, worth, or significance of an ‘object’.”
Weiss Carol says, it is “the systematic assessment of the operation and/or outcomes of a policy or policy, compared to a set of explicit or implicit standards as a means of contributing to the improvement of the policy or policy.”
Another unknown source said that Analysis is “a systematic way to determine the “value” of a policy, policy components or activity.”
Note Analysis, as indicated, is “systematic” - which means planned, disciplined and objective. Writing Analysis plans is part of the systematizing of Analysis. Another important word is “value” and, unlike research, Analysis cannot be done without judgments of value.
Analysis can mean different things in different situations and as a discipline, there is a lot of flexibility.
The broad definition of Analysis that we’re using and that you’ll find in CDCs Introduction to Policy Analysis for Public Health policies “The systematic collection of information about the activities, characteristics, and outcomes of policies to make judgments about the policy, improve policy effectiveness, and/or inform decisions about future policy development.”
Research vs. Analysis
- Knowledge intended for use
- policy- or funder-derived questions
- Judgmental quality
- Action setting
- Role conflicts
- Often not published
- Multiple allegiances
- Production of generalizable knowledge
- Researcher-derived questions
- Paradigm stance
- More controlled setting
- Clearer role
- Published
- Clearer allegiance
Research
Analysis
Systematic
Methods
Slide 5 Objective: Compare and contrast research and Analysis
Speaker:
Before we talk about what Analysis is, let’s review what Analysis isn’t. For example, research…
Both Analysis and research employ methods that are systematic in their investigations. However, there are fundamental differences between the two.
While research generally is conducted to produce knowledge that is generalizable across different policies, Analysiss are conducted to generate findings that are intended for use by the specific policies under which Analysiss are conducted.
In research, the question for investigation are researcher derived. Questions for Analysis, on the other hand, are derived from the policy itself or its stakeholders
Analysis is “value” based, thus it is judgmental in quality. Unlike Analysis. research is usually conducted without judgments and biases. For this reason, research is normally done under a more controlled setting to prevent biases. Since Analysis findings are to be used for policy improvements, it is conducted in a real-world setting and takes into the account of the environment and the context.
In research, the role of a researcher is more defined and clear. Evaluators, on the other hand, may often have role conflicts as they may be a part of the policy in which the Analysis is being conducted.
Because research and Analysis are conducted for very different reasons – research seeks to add to generalizable knowledge whereas Analysis seeks to generate information useful for policy improvements for a specific policy, research findings are usually published, whereas Analysis findings are often not.
For the same reasons, research had a clearer allegiance, and Analysis has multiple allegiances as it can impact more people and the results are utilized by more people.
For example, TB research findings can be used to develop initiatives that are put in place and implemented in a real world setting. The initiatives can be evaluated using policy Analysis that takes into consideration the setting and context. If the Analysis finds that the initiative is not successful, it may be that the research was fine, but did not work in the policy’s context or the initiative was not implemented properly.
“Research seeks to prove, analysis seeks to improve…”
M.Q. Patton
Slide 6 Objective: Compare and contrast research and Analysis
Speaker:
To put it simply, although research and Analysis are very similar and both use systematic methods, their intent and purposes differ.
As Patton says, “Research seeks to prove, Analysis seeks to improve…” Analysis involves looking closely at the operations of policies or policy initiative, and with the understanding gained from this examination, make recommendations for improving the policy.
It’s important to always keep this difference in mind, and remember that your intention for doing Analysis is to help improve policies and not to blame or criticism or seek to eliminate something.
If the Goal of Analysis is…
… to improve a policy
Then no analysis is good unless findings are used to make a difference
Slide 7 Objective: To emphasize the purpose of Analysis and the importance of its use
Speaker:
The goal of Analysis is to improve policy, thus no Analysis is good unless the results are used.
Often times, when Analysis is completed, reports and papers are written reporting the findings. However, these reports often stay on the bookshelves. If this is case, then the findings are not used to make changes that can really improve the delivery and the outcome of the policy.
It’s important to always keep in mind how the Analysis can be and will be used once completed. And always plan and strategize ways that would encourage and facilitate the use of the Analysis findings. When I talk about the CDC framework, you’ll see that one of the most important parts of it is the dissemination and utilization of the Analysis results.
Introduction
“Homeland security and emergency management are part of a key function of government: protection of its citizens.”
Laws and policies
Homeland Security
operations
Emergency Management
operations
Introduction
“It is vital that those who have homeland security or emergency management responsibilities have an understanding of the government and legal parameters in which homeland security policy and emergency operations function.” (p. 19)
Laws and policies
Homeland Security
operations
Emergency Management
operations
This class will explore:
The meaning of law and policy
Sources of legal authority
Basics of constitutional law
U.S. agencies and government structures
The meaning of law and policy
Laws are rules established by government that require compliance or adherence.
Social Contract
Theory
Coercive powers
of government
Key concepts
Sources of legal authority
Constitution
Statutes
Rules, regulations, policies
Basics of Constitutional Law
Defines structures and functions of government
Bill of Rights sets limits to government powers
Policies – “Policy refers collectively to laws, goals, programs, and other documents and actions that guide decision making and activities in a particular area, such as homeland security policy or flood mitigation policy.
Defines structures and functions of government
*
FIGURE 2.1 The Separation of Powers
*
*
FIGURE 2.2 Checks and Balances
Each of the branches is given some power over the others. This graphic may be useful for an in-class exercise where small groups take one or two of the relationships then discuss them in class. One group can hand over the conversation to the next in following with the diagram.
*
*
Class 2
*
Evaluation
Evaluate the impact in terms of each process and overall goal
Implementation
Ensure that enacted changes becomes rule/processes/budgets
Policy Adoption
Adopt solutions/budget by decision-makers
Problem Rec./Agenda Setting
Clarify the problem & frame/ define it for Policy Agendas
Policy formulation
Conduct analyses to identify a solution to promote
Processes for Adopting or Changing Policies, Environments and Systems
Policy-making Process Competencies in Five Domains
Analyze & articulate the problem
a. Collects, summarizes, & interprets relevant information
b. Defines the problem needing a policy, system, or environmental solution
2. Propose a solution
a. Defines criteria for selecting among various options
b. Records options in clear & concise written statements
c. Estimates the health, fiscal, legal, social, & political implications of each option
d. Predicts the feasibility & expected outcomes of each option
e. Analyzes the options using decision Analysis methods (e.g., cost-benefit)
f. Builds consensus for the chosen course of action
3. Influence the change process
a. Plans a policy/system/environmental change approach
b. Educates decision-makers, media, partners, & the general public by providing relevant information (i.e., become an informational resource)
c. Frames messages & tailors materials to influence the change process
d. Implements policy-advocacy strategies
e. Implements communications strategies to impact social learning, agenda setting, & message framing
f. Monitors the change process & its outcome
.
Monitor the implementation process
a. Predicts how the relevant bureaucratic entities (e.g., agencies, departments) might implement the enacted changes
b. Plans how to monitor & assist each entity as it develops the budgets, rules, guidelines & procedures necessary to implement the enacted change
c. Assists entities with planning for structural & programmatic adjustments
d. Monitors the implementation process to document how the solution is or is not functioning as intended
5. Evaluate the impact
a. Develops mechanisms to monitor policy/system/environmental change
b. Evaluates the impact of the change
c. Incorporates evaluation findings into future planning & analysis efforts
From Public Health Solutions Through Changes in Policies, Systems,
and the Built Environment (www.dhpe.org/HPEC_Comps_Phase_Final.pdf)
Policy-making Process Competencies in Five Domains (con’d)
89,527 Governments in the United States
(and counting . . .)
*
TABLE 3.1 89,527 Governments in the United States
SOURCE: U.S. Census Bureau, www.census.gov (accessed 7/14/08).
New local government entities are on the rise as population surges demand more services including hospital districts, community college districts, school districts, toll road authorities, and utility districts to name a few.
*
Case study: chemical spill West Virginia
Surveillance & Monitoring
vs. Policy Analysis
- Surveillance - tracks disease or risk behaviors
- Monitoring - tracks changes in policy outcomes over time
- Analysis - seeks to understand specifically why these changes occur
Slide 8 Objective: Compare and contrast surveillance, monitoring and Analysis
Speaker:
Surveillance and monitoring are terms frequently used in public health and they are often confused with Analysis.
The term, surveillance, typically refers to the tracking of disease or risk behaviors. TIMS (TB Information Management System) is an example.
Analysis is also more specific than surveillance – it is designed to look at one policy during a specific point in time.
Monitoring, on the other hand, is used for tracking changes in policy outcomes over time.
Both are used for performance measurement and both may also be used in policy Analysis. However, Analysis stands apart from both surveillance and monitoring in that it seeks to understanding why these changes occur.
What Can be Evaluated?
- Direct service interventions
- Community mobilization efforts
- Research initiatives
- Surveillance systems
- Policy development activities
- Outbreak investigations
- Laboratory diagnostics
- Communication campaigns
- Infrastructure-building projects
- Training and educational services
- Administrative systems
– MMWR, 1999
Framework for Policy Analysis in Public Health
*
Slide 9 Objective: Explore what can be evaluated
Speaker:
Any policy or policy initiative can be evaluated and most policies or policy activities can benefit from Analysis.
The various types of public health related activities or initiatives that can be evaluate are listed here. The range of activities includes services and interventions to that of organizational structures and it’s important to how wide the range really is.
The range of activities is important for TB policies. Many of the activities listed here are undertaken by the policies and may be prime candidates for the Analysis you’re planning right now. Note that many of these items are not full policies but initiatives meant to address specific areas where a policy may fall short. Most likely this will be the case with your policy. You’ll be identifying an initiative or part of your policy to Analysis, range than the entire policy.
When to Conduct Analysis?
Conception
Completion
Planning a
NEW policy
Assessing a
DEVELOPING
policy
Assessing a STABLE, MATURE policy
Assessing a policy after it has ENDED
The stage of policy development influences the reason for policy analysis.
Slide 10 Objective: Discuss the appropriate time for conducting Analysis
Speaker:
We now know the purpose of the Analysis and ways Analysis can be used. You are probably wondering when Analysis should be conducted…
Traditionally, most Analysiss occurred at the END of a policy when it’s already too late to make any changes.To be most effective, beneficial and instrumental to the policy, Analysis can be done at all phases of the life of a policy. Analysis can be conducted at any stage of policy development, and the stage of development influences the reasons for Analysis and the type of information gathered.
In the conception phase of a NEW policy, Analysis can be used to assess motivation for starting a new policy, define the need for the policy, and inform decision makers about what a new policy should look like: its objectives and expectations. It can help clarify important factors that are essential in laying down the foundation for a policy.
In a DEVELOPING policy, Analysis can be used to monitor policy progress, and re-define policy activities and expectations. It can identify potential gaps or problems early, so steps can be taken to resolve them.
In a STABLE and MATURE policy, Analysis findings can be used to monitor the policy and provide feedback to management. Analysis can help show policy successes as well as areas for improvement. Outcomes can also be assessed for mature policies because enough time has passed to allow this.
At the END of a policy, Analysis helps assess the value of the policy as well as document lessons learned for the future. In this phase, Analysis assesses what intervention was transferred and adopted, and if the change was sustained. Unexpected outcomes can also be assessed. At this time, you might also evaluate the possibility that an intervention could be replicated in another setting and what factors would encourage this.
For TB policies, most are likely to be stable and mature. Yet an initiative that you’re working on could be new or developing if, for example, you’ve elected to target a new population with an emerging TB problem or you’re trying to promote a treatment protocol among health care providers.
Why Evaluate policies?
- To gain insight about a policy and its operations – to see where we are going and where we are coming from, and to find out what works and what doesn’t
- To improve practice – to modify or adapt practice to enhance the success of activities
- To assess effects – to see how well we are meeting objectives and goals, how the policy benefits the community, and to provide evidence of effectiveness
- To build capacity - increase funding, enhance skills, strengthen accountability
Slide 11 Objective: Discuss reasons for conducting policy Analysis and why it is important
Speaker:
As I mentioned, why you evaluate a policy will change over the life of the policies. Among the many reasons to evaluate, there are 4 basic reasons. “
<TO GAIN INSIGHT>
We evaluate a policy to judge its merit or gain insight about the policy and its operations. This might include providing information concerning the practicality of a new approach or developing policy; and to find out where the policy is and where it is heading. Analysis helps us assess where we are in policy development and helps identify information we need to plan for our next steps. It provides us with information for better decision making, and allows us to manage resources and services more effectively. For example, an Analysis of testing and treatment for LTBI might reveal how many people in high-risk and low-risk groups are being tested. This information could inform decisions about reducing testing in low-risk populations to save money.
<TO IMPROVE PRACTICE>
For this reason, we also evaluate policy for the purpose of making improvement or changing practices. We can use Analysis to improve or enhance what we do, which is appropriate in the implementation stage when established an TB policy seeks to describe what it has done. It tells us what we are lacking and where we need to focus our efforts. Analysis provides us with a means to understand why we achieved our successes, or why we did not meet our objectives. It provides us with information we need to strategize, plan and implement initiatives that enhance the effectiveness of our TB control policy.
It is important to note that Analysis requires that we examines factors objectively, both inside and outside of our policy, for the causes of our performance. Understanding these factors allows us to make better decisions, implement change where appropriate, and improve upon what we have accomplished.
<TO ASSESS EFFECTS>
We also evaluate policy to assess its effects. We conduct Analysis to see how well objectives and goals are being met. Analysis findings can show how effective the intervention was at inducing the intended changes or effects. It helps us demonstrate the value of our efforts. Analysis documents what we do and systematically shows the contributions towards accomplishing our goals. This information can help decision makers at all levels understand the benefits and consequences of what they are doing. At crucial times, findings from Analysis serve to help us advocate for the cause and leverage support.
<TO BUILD CAPACITY>
Lastly, we use policy Analysis as a method for building policy capacity. The process is effective for policies to make self-directed changes, such as to increase funding, develop skills, build infrastructure and/or strengthen accountability. For example, Analysis of contact investigations in a community of recent immigrants might reveal a need to translate informational materials into a new language to enhance the policy’s capacity to serve this emergent group. Analysis also builds on itself – as we learn and gain experiences in conducting Analysiss, we also build Analysis capacity for our policy, and increase policy capacity for self-directed improvements. Finally, Analysis can strengthen accountability by allowing us to demonstrate that we are responsible stewards of the policy’s funding and resources.
CDC Framework for Policy Analysis
– MMWR, 1999
Framework for Policy Analysis in Public Health
Slide 12 Objective: Provide an overview of CDC framework for policy Analysis
Speaker:
This picture summarizes the essential elements of policy Analysis presented in CDC framework for policy Analysis in public health. This framework is intended as a practical, non-prescriptive tool.
The 6 steps of policy Analysis are shown in a circle representing the circular nature of the Analysis process. As depicted, the steps are inter-dependent, and they may be conducted in a non-linear order. Often more than one step is ongoing at any time. The Framework provides the flexibility to go backwards or forwards in the process depending upon the need.
Although the steps can be conducted in a different order or simultaneously, there is a reason for the way the steps have been sequenced – the earlier steps provide the foundation for the subsequent. It is important to note that each one of these steps plays an role and is crucial to the process.
The circle also represents that the process for policy Analysis is iterative. As policy Analysis is conducted, the information or the finding from the Analysis is used to feedback into the policy to make improvements. The process repeats itself in a cycle as a mean to improve the policy and its effectiveness in achieving its objectives.
At the center or core of the circle are the Analysis standards. These four standards are intended to maximize the quality and the effectiveness of Analysis efforts. They are the criteria and standards for which the quality of your policy Analysis efforts can be judged. They are applicable to each of the six steps in the Analysis framework. We will learn more about these standards later.
Standards Utility Feasibility Propriety Accuracy
Steps
Engage stakeholders
Gather credible evidence
Focus the Evaluation design
Describe the program
Ensure use and share lessons learned
Justify conclusions
Steps in Policy Analysis
- Step 1: Engage Stakeholders
- Step 2: Describe the policy
- Step 3: Focus the Analysis Design
- Step 4: Gather Credible Evidence
- Step 5: Justify Conclusion
- Step 6: Ensure Use and Share Lessons Learned
Slide 13 Objective: Describe the steps of the Analysis
Speaker:
Let’s walk through the 6 steps of policy Analysis framework.
Step 1: Engage stakeholders
Stakeholders are those with vested interests in the policy and are potentially affected by the policy and its Analysiss. Stakeholders can be both supporters and skeptics of the policy. Engaging stakeholders in the Analysis process ensures that the Analysis will be useful. When Analysis meets the need of its intended users, the chance of the findings being used to make policy improvements is greatly enhanced.
Step 2: Describe the policy
This step of the Analysis framework allow us to gain a better understanding of the policy and its activities. The need, context, and the target population that the policy serves are defined. Major policy components (i.e., resources, activities, outputs and outcomes) and their relationships with each other are also part of this step.
Step 3: Focus the Analysis design
Step 3 of the Analysis process is crucial as it helps us determines a focus for the whole Analysis. When planning an Analysis, many worthwhile questions are identified. It’s easy to want to evaluate everything and collect all the information available. However, we are bound by limited resources. Thus, it’s important to prioritize and focus the effort on areas where changes can be most valuable.
Step 4: Gather credible evidence
After determining the key questions to examine in the Analysis, the next step is to gather evidence. Prior to gathering any data, indicators of policy performance are identified as are benchmarks or targets that we can use to compare the indicators to. Once you know what indicators to look for, data sources and data collection methods can then be determined.
Step 5: Justify conclusion
After all the data have been collected, the next step is data Analysis and interpretation. The interpretation part of this step is critical. In this phase you judge your findings against the established policy benchmarks. It’s important to involve stakeholders in this step and consider the context in which the policy is operating so that your conclusions are sound, reasonable and objective. Once conclusions have been reached, recommendations can then be make if appropriate.
Step 6: Ensure use and share lessons learned
This step of the framework emphasizes the importance of using findings for policy improvement once the Analysis has been completed. Dissemination of the Analysis findings and sharing lessons learned ensures that all the efforts invested into the Analysis are garnered and will be used.
Underlying Logic of 6 Steps
- No Analysis is good unless… findings are used to make a difference
- No findings are used unless… a market has been created prior to creating the product
- No market is created unless… the Analysis is well-focused, including most relevant and useful questions
Slide 14 Objective: Illustrating the framework
The purpose of this slide is to reinforce the circular nature of the Framework and illustrate how each of the step are interdependent.
One of the basic components of the Framework is using the results. That’s part of step 6 but it’s important to create a market for your Analysis findings. That’s why stakeholders are engaged in step 1.
Finally, part of creating a market and being responsive to it is focusing your Analysis. This is a difficult but important step involving identifying the most important things to evaluate and developing relevant and useful questions.
Establishing the Best Focus Means
- Framework Step 1: Identifying who cares about our policy besides us? Do they define the policy and policy “success” as we do?
- Framework Step 2: What are milestones and markers on the roadmap to the main public health outcomes?
Slide 15 Objective: Using the framework to focus the Analysis
Speaker
Although focusing the Analysis is Step 3 of the Framework, Steps 1 and 2 provide information that will help with the focus. This slide is also meant to illustrate how the steps in the Framework work together to result in a useful Analysis.
For example, engaging stakeholders in Step 1 will help you to focus the Analysis by asking relevant questions of them. Do the stakeholders define the policy and policy success as we do? Further, what do they want to know about the policy? What do they perceive to be policy problems or weaknesses?
Describing your policy – Step 2 of the Analysis – also helps to focus the Analysis by establishing the milestones and markers that lead to the public health outcomes your interested in changes or creating. Tom will talk more about step 2 on Thursday.
Standards for Effective Analysis
*
Slide 16 Objective: Provide an overview of the standards for policy Analysis
Speaker:
Good Analysiss are diligently conducted under the guidance of the standards for policy Analysis provided by the Joint Committee of Standards for Educational Analysis.
There are 4 standards for policy Analysis: Utility, Feasibility, Propriety, and Accuracy.
These standards serve to guide your decision making process at each step of the Framework to ensure that the Analysis stays focused and balanced.
In the next couple of slides, we will go over each standard in depth.
Standards Utility Feasibility Propriety Accuracy
Steps
Engage stakeholders
Gather credible evidence
Focus the Evaluation design
Describe the program
Ensure use and share lessons learned
Justify conclusions
The Four Standards
- Utility: Who needs the information and what information do they need?
- Feasibility: How much money, time, and effort can we put into this?
- Propriety: What steps need to be taken for the Analysis to be ethical?
- Accuracy: What design will lead to accurate information?
Slide 17 Objective: Provide an overview of the standards
Speaker
One of the primary purposes for using the standards is to help design the best Analysis for your policy or situation. There is no one “right” Analysis. Instead when designing the Analysis, you make choices that best suit your particular situation so that the result is the optimal policy Analysis for your policy or initiative.
The four standards – utility, feasibility, propriety, and accuracy – when applied will result in the most worthwhile Analysis for your situation. Under each standard on the slide is a question that can be asked to help design the Analysis. By applying the standards, you have an Analysis that is useful, feasible, proper and accurate for the policy Analysis that you are conducting at this time.
Now we’ll talk about each standard individually.
Standard: Utility
Ensures that the information needs of intended users are met.
- Who needs the Analysis findings?
- What do the users of the Analysis need?
- Will the Analysis provide relevant (useful) information in a timely manner?
Slide 18 Objective: Define Utility Standard
Speaker:
Utility, the first standard of the 4, helps to ensure that the Analysis serves the information needs of its intended users.
As you remember from earlier, Analysis does no good unless it is used to improve policies.
Knowing who needs the information and how the information will be used helps us better tailor and focus the Analysis to address users needs.
When we take the time to find out what is needed and provide people with what they need to do their work or improve their productivity, we ensure the efforts that we put in and the product that we produce will be used, and not wasted.
So, the important questions to keep in mind while planning and conducting Analysiss are:
- Who needs the Analysis results?
- What do they need?
- Will the Analysis provide relevant, or otherwise useful, information in timely manner for the users?
Often the best way to answer these questions is to ask your stakeholders and provide them with the opportunity to be part of the Analysis.
Standard: Feasibility
Ensures that Analysis is realistic, prudent, diplomatic, and frugal.
- Are the planned Analysis activities realistic given the time, resources, and expertise at hand?
Slide 19 Objective: Define Feasibility Standard
Speaker:
Feasibility is another standard to keep in mind while planning and conducting Analysis.
The purpose of this standard is to ensure that the Analysis conducted or to be conducted is realistic, prudent, diplomatic, and frugal.
Resources and manpower in public health policies are limited. Although many of us would love to do everything, there are just not enough resources to do that.
Thus, prioritizing the needs and planning for a feasible Analysis are very important.
When planning an Analysis, we need to ask ourselves, “Are the activities planned realistic given the time, resources, and expertise available?” It is very tempting to want to evaluate everything or a large number of things, particularly with a policy we know well and are invested in. But applying this standard will help us to find focus and channel the resources to the parts or activities of the policy that most need improvement.
Standard: Propriety
Ensures the Analysis is conducted legally, ethically, and with due regard for the welfare of those involved and those affected.
- Does the Analysis protect the rights of individuals and protect the welfare of those involved?
- Does it engage those most directly affected by the policy and by changes in the policy, such as participants or the surrounding community?
Slide 20 Objective: Define Propriety Standard
Speaker:
Propriety ensures that those of us involved in the Analysis, as evaluators or members of the Analysis team, behave legally, ethically, and with due regard for the welfare of those involved and affected by the policy and its Analysis.
Like doing research, Analysis may involve the handling of personal, sensitive information. More specifically in TB policies, Analysis may involve the review of medical records or charts. And the handling of information that patients and providers may regard as highly sensitive.
Although for the purpose of making policy improvements, many states have waived Analysis from having to go through extensive IRB reviews, it is still our responsibility to protect those we serve and respect their rights and privacy.
Couple of questions that help ensure our Analysis is conducted respectfully and according to the propriety standard:
- Does the Analysis or methods employed in conducting Analysis protect the rights of individuals and the welfare of those involved?
- Does the Analysis engage those directly affected by the policy and as well as those affected by potential changes in the policy (i.e., policy participants, patients or the surrounding community)?
- Have we included people in the Analysis process and allow them to have their own voices in making decisions that may affect their health and well being?
Standard: Accuracy
Ensures that the Analysis reveals and conveys technically accurate information.
- Will the Analysis produce findings that are valid and reliable, given the needs of those who will use the results?
Slide 21 Objective: Define Accuracy Standard
Speaker:
Lastly, accuracy is the 4th Policy Analysis standard. It ensures that the Analysis conducted reveals and conveys reliable and valid information.
Having accurate information is the foundation for good decision making and a step toward improving policies.
Thus in planning Analysis it’s important to identify ways or methods that would provide us with the most valid and reliable results. In one of the future sessions when we talk about design and data collection, ways to gather accurate information will be discussed.
Engaging Stakeholders
(Step 1)
*
Slide 22 Objective: Discuss Step 1: Engage Stakeholders
Speaker:
Now that we have a brief overview of the steps in the Analysis framework and have a better understanding of the standards in which we use to guide our Analysis, let’s explore the first step in depth. As I mentioned earlier, the later steps will be covered in subsequent presentations.
For the rest of this session, we will be discussing Step 1: Engage Stakeholders.
We will learn to identify key stakeholders that are essential to the policy and its Analysis, and then figure out ways to include them in the Analysis.
Definition of stakeholder “people and/or organizations that are invested in the policy, are interested in the results of the Analysis, and/or have a stake in what will be done with the results of the Analysis.”
Standards Utility Feasibility Propriety Accuracy
Steps
Engage stakeholders
Gather credible evidence
Focus the Evaluation design
Describe the program
Ensure use and share lessons learned
Justify conclusions
*
“There are five key variables that are absolutely critical in analysis use. They are in order of importance:
people, people, people, people, and people.”
Halcolm
Slide 23 Objective: Discuss the importance of involving stakeholders in Analysis
Speaker:
Some of you may be wondering why engage stakeholders?
Why does the framework recommend us to start with this step?
According to Halcolm, “There are 5 key variables that are absolutely critical in Analysis use. They are in order of importance: people, people, people, people, and people.”
So why are people important in Analysis use? There are many reasons.
First, stakeholders are the people who will implement whatever changes or improvements the Analysis recommends. Engaging them early on in the process increases their “buy-in” and thus, increases the chance that the recommendation made as the result of Analysis will be used.
Stakeholders provide you with reality check. They keep the Analysis grounded
Stakeholders can also generate needed political pressure or support or even resources for the Analysis.
In addition to these reasons, stakeholders:
- Know the policy, and can provide important insights
- Guide Analysis and ensure that Analysis is useful
- Increase credibility of the Analysis
- Assist with dissemination of findings
- Can advocate for the policy
- Can provide future funding
Identifying Stakeholders
*
- Who are the stakeholders?
Persons involved in policy operations
Persons served or affected by the policy
Intended users of Analysis findings
- What is their interest in the policy?
Do they support the policy?
Are they skeptical about or antagonistic toward the policy?
Slide 24 Objective: Explain the process of identifying stakeholders and their interest
Speaker:
So who are the stakeholders?
Stakeholders are persons interested in the policy or are affected by the policy. Generally we look at stakeholders as being from three categories: those involved in policy operations, those served or affected by the policy, and the intended users of Analysis findings.
Once we know who stakeholders are, we need to identify their interest and perspective on the policy. For example,
Do they support the policy? Or are they skeptical about or antagonist toward the policy?
One way to do identify stakeholders systematically is to create a list with the three categories, then under each category, list out the names of the persons who fall into it. Note that some stakeholders will fall into more than one category.
Once you’ve generated the list, then along side each person’s name, list out their interest in the policy and/or the Analysis. Note that this is just the beginning of the exercise. It’s important to seek feedback on this list from others and from the stakeholders themselves. The stakeholders are the best people to tell you what their interests are.
Identifying Stakeholders
*
Be sure to include supporters and skeptics!
- Persons Involved in policy operations
Staff and Partners
- Persons affected or served by the policy
Clients, their families and social networks, providers and community groups
- Intended users of the Analysis findings
Policy makers, managers, administrators, advocates, funders, and others
Slide 25 Objective: Explain how to implement the process of identifying stakeholders
Speaker:
As we just discussed, there are 3 basic categories of stakeholders:
- those involved in policy operations
- those served or affected by the policy, and
- the intended users of the Analysis
Staff, managers, and other partners who are directly involved in day-to-day activities of getting tasks done are those involved in policy operations. This includes public health nurses and TB clinic clerks as well as members of the state policy staff.
Those served or affected by the policy include clients (or patients), their families and friends as well as providers and community groups. For a TB policy or initiative, this group may include TB and LTBI patients, the family and friends of patients, managers or coordinators at schools, prisons, shelters and other congregate settings. Also, representatives of populations disproportionately affected by TB are also part of this group.
The third category, the intended users of the Analysis information, may include policy makers, managers, administrators, advocates, and funders. This includes public health managers and administrators, CDC and advocacy groups.
The final sentence on the slide is urging you to include both policy supporters and skeptics. Both points of view are important and having your skeptics involved in the Analysis will increase the credibility of the findings. The only caveat is that it is important to consider your skeptics point of view and motivations. For example, if funding is in short supply, a skeptic may wish to have your policy funds redirected, so you need to recognize this bias. At the same time, it’s not desirable to exclude critics entirely. They may actually help to better the policy and the Analysis through their skepticism.
Which Stakeholders Matter Most?
Who is…
Affected by the policy?
Involved in policy operations?
Intended users of Analysis findings?
Who do we need to…
Enhance credibility?
Implement policy changes?
Advocate for changes?
Fund, authorize, or expand the policy?
Slide 26 Objective: Determining Key stakeholders
Speaker:
As noted by the explanations in the previous slides, there are many, many potential Analysis stakeholders. What is illustrated by this slide is how you go about identifying the key stakeholders for the Analysis.
The larger box at the top is the “universe of Analysis stakeholders” This is everyone that you and your Analysis team have identified under the three categories of stakeholders. Using this list, then you can begin to narrow it down to stakeholders who are key to the Analysis.
The smaller box low are the key stakeholders, a subset of all stakeholders. Generally, they include those stakeholders that will enhance the credibility of the Analysis, implement policy change that result from the Analysis, advocate for recommend changes to the policy and finally those stakeholders who fund or authorize the policy or who could help expand the policy.
If time allows at the end of the presentation, I can provide a detailed example of how to identify stakeholders and consider their role in the Analysis.
Engaging Stakeholders
Stakeholders should be involved in…
- Describing policy activities, context, and priorities
- Defining problems
- Selecting analysis questions and methods
- Serving as data sources
- Defining what constitutes the “proof” of success
- Interpreting findings
- Disseminating information
- Implementing results
Slide 27 Objective: Emphasize that stakeholders should be involved from the onset and continue being engaged throughout the entire process.
Speaker:
Knowing the importance of engaging stakeholders, the next step is determining how and when to engage them. This slide offers some suggestions, although it is not an exhaustive list. Usually, whoever is leading or coordinating the Analysis takes the lead in identifying and engaging the stakeholders.
It is crucial to involve stakeholders from the onset of the Analysis and continue to engage them throughout the whole Analysis process.
This does not mean that all stakeholders will need to be involved in all stages. A small subset might be but many stakeholders might provide input in only one or two of the stages. The important thing to remember is that at least a subset of key stakeholders should be involved at some level in each step throughout the process.
The suggestions include:
Under Step 2: Describing the policy activities, context, and priority;
Defining problems
Steps 3 and 4: Selecting Analysis questions and methods
Serving as data sources
Defining what constitute the “proof” of success – indicators
Step 5: Interpreting findings
Step 6: Disseminating information
Implementing results
Stakeholder involvement is particularly important in setting values and priorities. And also helping you to address the four standards of Analysis set forth in the Framework. When identifying and engaging stakeholders is properly carried out, we can be sure that the Analysis is being carried out appropriately.
Analysis Methodology
- Evidence based research --- the investigation, collection and interpretation of facts. However, research is conducted in many ways depending on the field and its accepted methods.
- Use credible sources "research-based," "scholarly," "academic," or "peer-reviewed" literature.
What is Inquiry
- https://www.chds.us/coursefiles/research/lectures/research_inquiry_articulate/player.html
Experimental Society
- Donald T. Campbell had a “vision for this utopia involved rational decision making by politicians based on hard-headed tests of bold social policies designed to improve society.”
Experimental Society
- This ideal society would broadly implement social reforms demonstrated to be highly effective by experimental research and Analysis, with the goal of moving at least more, if not most, of the population toward the “good life.”
- “A paradigm typically consists of an ontology (the nature of reality), an epistemology (what is knowledge and who can know it), and a methodology (how one can obtain knowledge).” p. 23
Paradigms
Thomas Kuhn
- Debate topic, “the great quantitative-qualitative debate”:
Qualitative vs. quantitative data
Design choices: quasi-experimental or nonexperimental or mixed method designs
The need to measure “causality” or not to determine policy “impact”
Paradigms
Positivism (quantitative method): One single reality that can be observed by multiple viewers.
Paradigms
Knower
What is to be know
Theory
Conclusion
Cause
Effect
One way relation
Constructivism (ontological relativism): Multiple realities that are subjective and changing. Qualitative method applied.
Paradigms
Knower
What is to be know
Theory
Conclusion
Cause
Effect
Bidirection
- Analysis and applied research are grounded in and are forms of social inquiry;
- Social inquiry is the systematic study of the behavior of groups of individuals in various kinds of social settings, using a variety of methods;
- “Why do people in social groups act as they do?”
Paradigms
Rousseau
Hobbes
Montesquieu
Examples of C.J. interventions
- https://www.chds.us/coursefiles/research/lectures/research_paradigms/player.html
Planned change vs. unplanned change
- Any project, policy, or policy, new or revised, intended to produce a change in some specific problem. The intended change may occur within individuals, groups, organizations, systems of organizations, communities, cities, regions, states, or, much more rarely, within entire cultures or societies.
Planned change vs. unplanned change
- Any project, policy, or policy, new or revised, intended to produce a change in some specific problem. The intended change may occur within individuals, groups, organizations, systems of organizations, communities, cities, regions, states, or, much more rarely, within entire cultures or societies.
Planned change vs. unplanned change
- Unplanned change means that little explicit or proactive planning has been undertaken at all.
Unplanned change often comes about: as a reaction to a crisis,
a dramatic incident publicized by the media,
a political opportunity,
a lawsuit against criminal justice officials,
an untested set of assumptions about a specific problem
Planned change vs. unplanned change
Planned change is limited in scope, and specific.
" It includes a role for consumers."
"A “change agent” guides planned change."
Three approaches to planned change
- Policy
- Program
- Project
Program
- A set of services aimed at achieving specific goals and objectives within specified individuals.
Project
- "A time-limited set of services provided to particular individuals, groups, organizations, or communities, usually focused on a single need, problem, or issue."
Need for planned change
- The need for planned change has been sharpened by three trends
Declining resources
Accountability
Expansion of knowledge and technology
Need for planned change
- "Since 1980, there have been huge cuts in social services spending, especially policies affecting the poor and minorities (e.g., subsidized health care, welfare, daycare for working parents, school lunches, and after-school policies)"
Need for planned change
- As public resources have dwindled, agencies have increasingly been called upon to demonstrate their effectiveness and efficiency in meeting their goals. There has been suspicion by many that public money has not always been well spent.
Need for planned change
- We have greater technological abilities than ever before, and these changes have created both new opportunities and new problems.
- Improvements in computing technology have dramatically increased our information collection, storage, and retrieval capabilities.
- Improved data collection and access mean that our ability to analyze specific needs and problems has improved.
The perils of planned change
Any change to existing procedures and existing conditions carries a certain amount of risk. The proposed change is likely to be resisted by someone, perhaps even its intended beneficiaries
The perils of planned change
- collaborative strategies
- conflict strategies
The perils of planned change
- Conflict strategies: Conflict strategies approach resistance in an adversarial manner. Those who resist the pro- posed change are seen as opponents who must either be persuaded or coerced to change their views.
The perils of planned change
- Conflict strategies produces:
• They create greater resistance;
• They require greater resources;
• They create more unexpected effects; and
• Change tends to be temporary (compliance) rather than long- term.
The perils of planned change
“Collaborative strategies emphasize participation from those affected by change. Individuals, groups, or organizations known to oppose the intervention in part or in toto are included in the design and planning of the intervention.”
A Seven-Stage Model for Planned Change
Analyzing the Problem
• Document the need for change. Collect and analyze data to define what the problem is, where it is, how big it is, and who is affected by it. What evidence of the problem exists?
• Describe the history of the problem. How long has the problem existed? How has it changed over time?
• Examine potential causes of the problem. What causes the problem? What theories do we have? The intervention must target one or more specific causes supported by research.
• Examine previous interventions that have tried to change this problem, identify the most promising interventions, and choose a preferred intervention approach. We need to analyze available information to direct decisions about a possible course of action.
• Identify relevant stakeholders. Do different groups of people have different definitions of the problem? Who is affected by the problem?
• Conduct a systems Analysis. Conduct research on the system within which the problem exists, and determine how the system may create, contribute to, or maintain the problem.
• Identify barriers to change and supports for change. Who is likely to support a certain course of action, and who is likely to resist it?
Setting Goals and Objectives
• Write goal statements specifying the general outcome to be obtained. Goals are abstract statements of purpose.
• Write specific outcome objectives for each goal. Unlike goals, objectives are specific and measurable statements of intended outcomes. Each objective should include a time frame for measuring impact, a target population, a key result intended, and a specific criterion or measure of impact.
• Seek participation from different individuals and agencies in goal setting. Goal setting that includes a broad range of stakeholders helps to ensure that interventions receive necessary resources and support.
• Consider “top-down” versus “bottom-up” approaches.
• Specify an impact model. This is a description of how the intervention will act upon a specific cause so as to bring about a change in the problem. Successful interventions target causes of problems, not problems themselves. An impact model is an explanation of how the intervention will affect the causes and, consequentially, how changes in causes of the problem will affect the problem.
• Identify compatible and incompatible goals in the larger system. Where do values of different stakeholders overlap or conflict?
• Identify needs and opportunities for interagency collaboration. Whose cooperation and participation is needed to achieve the goals of this policy or policy?
Designing the policy or Policy
Choosing an intervention approach involves integrating the information assembled in previous stages to decide how a policy or policy can best achieve the goals that have been specified. How can the information collected at previous stages be used to decide what the substance of an intervention will be? How will specific goals be achieved?
• Major activities for policy design include the following: (1) Identifying the target population: Who is the intervention aimed at? (2) Defining participant selection and intake procedures: How are policy participants recruited and selected for the intervention? (3) Defining policy components: the precise nature, amount, and sequence of activities provided must be specified. Who does what to whom, in what order, and in what amounts? (4) Writing job descriptions of staff, and define the skills and training required in order to deliver services well.
• Major activities for policy design include the following: (1) Identifying the target population of the policy. Which persons or groups are to be affected by the policy? (2) Identifying the responsible authority. Who is required to carry out the policy, and what will their responsibilities be? (3) Defining the provisions and procedures of the policy. Provisions specify the sanctions or services that will be delivered, and the conditions that must be met in order for the policy to be carried out. Individuals responsible for implementing a specific set of rules must also clearly understand the specific sequence of actions to be taken (procedures) to ensure that the policy is carried out consistently.
Action Planning
• Identify resources needed and make cost projections. How much funding is needed to implement a specific intervention? Identify the kinds of resources needed, estimate costs and make projections, and develop a resource plan.
• Plan to acquire or reallocate resources. How will funding be acquired? Identify resource providers, and be prepared for making adjustments to the resource plan.
• Specify dates by which implementation tasks will be accomplished, and assign responsibilities to staff members for carrying out tasks. A “time line” or Gantt chart is particularly useful for this purpose.
• Develop mechanisms of self-regulation. Create mechanisms to monitor staff performance and enhance communication, including procedures for orienting participants, coordinating activities, and managing resistance and conflict.
• Specify a plan to build and maintain support. Anticipate sources of resistance and develop responses.
Policy Implementation and Monitoring
• “Monitoring” refers to the collection of information to determine to what degree the policy or policy design (see Chapter 3) was carried out as planned. How will we know whether the intended target population was reached? Were policy/policy activities or provisions actually carried out as planned? Were appropriate staff or responsible authorities selected and trained, and did they carry out their assigned duties?
• Design monitoring instruments to collect data (e.g., observations, surveys, interviews). Data is collected to find out what is actually being delivered to clients or targets. The purpose is to identify gaps between the “policy/policy on paper” (design) and the “policy/policy in action.”• Fiscal monitoring. Funding sources require regular financial reports detailing how funds were spent. Once the policy or policy is implemented, one may find that adjustments to the budget become necessary.
• Designate responsibility for data collection, storage, and Analysis. Ensure that there is no ambiguity about what information is to be collected, who is responsible for collecting it, or how it is to be collected, stored, and analyzed.
• Develop information system capacities. Information systems may consist of written forms and records that are filed, or fully computerized data entry and storage systems.
• Develop mechanisms to provide feedback to staff, clients, and stakeholders. Depending on the results of monitoring analyses, it may be necessary to make adjustments either to what is being done (“the policy or policy in action”) or to the intended design (“the policy or policy on paper”).
Evaluating Outcomes
• The evidence-based paradigm. Recently, we have seen widespread adoption of a new normative value: that criminal and juvenile justice systems should implement policies that have been proven through rigorous Analysis research to be effective.
• Types of Analysis. Impact, performance, and efficiency. Is the policy or policy achieving its objectives, how do outcomes change over time, and is it worth the investment of resources devoted to its implementation? Impact Analysiss depend on methods that enable us to attribute observed outcomes to the intervention, rather than other potential influences. Additionally, meta-Analysis combines the results of many individual Analysis studies to determine if different intervention approaches or practices work better than others.
• Three prerequisites for Analysis must be met. (1) Objectives must be clearly defined and measurable, (2) the intervention must be sufficiently well designed, and (3) the intervention must be well implemented.
• Evaluability assessment and logic modeling. These are two methods for examining the components of a policy in preparation for Analysis. Both methods also help clarify policy and policy designs.
• Outcome measures should be derived from an intervention’s objectives. Good outcome measures are valid and reliable
.• Identify potential confounding factors (factors other than the intervention that may have biased observed outcomes). Common confounding factors include biased selection, biased attrition, and history. There are three major techniques for minimizing confounding effects: (1) random assignment, (2) nonequivalent comparison groups, and (3) propensity score Analysis.• Specify the research design to be used. Examples include: the simple pre-post design; the pre-post design with a control group; the pre-post design with multiple pretests; the longitudinal design with treatment and control groups; the cohort design; and time series Analysis.• Identify users and uses of Analysis results. Who is the intended audience, and how can results be effectively and efficiently communicated? How will the results be used?
Reassessment and Review
• Planning for failure. Failure of an intervention can occur for a number of reasons, including (1) conflicting or overly ambitious goals, (2) poor design, (3) poor implementation, or (4) failure to maintain support of key stakeholders.
• Planning for success. Ongoing reassessment, learning, and revision are crucial elements of successful interventions. Four tasks increase chances of success: (1) communication with stakeholders, (2) building internal capacities for leading and learning, (3) studying information about policy or policy performance, and (4) increasing the fit between needs and characteristics of the environment and capacities of the policy or policy.
• Learning and adapting. Interventions, their organizations, and surrounding environments change over time, and successful interventions must adapt to change by consciously tailoring the policy or policy.
• A caution about policy and policy survival. Interventions sometimes survive long after they are known to be ineffective. Survival is often dependent on how well the intervention serves the personal goals of key stakeholders.
• The tasks of implementation. Plans from Stages 3 to 6 are finally implemented: Initiating the Design (implementing Stage 3 plans); Initiating the Action Plan (implementing Stage 4 plans); Monitoring policy/Policy Implementation (implementing Stage 5 plans); and Evaluating and Providing Feedback to Users and Stakeholders (implementing Stage 6 plans).
LOGIC MODEL connection
Framework for Analysis
*
Underlying Logic of Steps
- No Analysis is good unless… results are used to make a difference
- No results are used unless… a market has been created prior to creating the product
- No market is created unless…. the Analysis is well-focused, including most relevant and useful questions
Policy Planning Process
Analysis is part of a larger policy planning process. First, you plan the policy. Then you implement it. Then you evaluate it. You use what you learn from the Analysis to improve your policy. And then you start planning your improved policy.
*
Analysis
- Need not be
Expensive
Complicated
Time consuming
- Some Analysis is better than none
- External evaluator is sometimes seen as more objective than internal
- Evaluator should be qualified
- Analysis plan should be meaningful, related to goals and objectives, and be an honest examination of policy
*
Continuous Quality Improvement
(CQI) cycle.
These must be integrated…
- Planning—What actions
will best reach our goals
and objectives.
- Performance measurement—
How are we doing?
- Analysis—Why are we
doing well or poorly?
Analysis Flowchart
GOAL
STATED OBJECTIVE
Activity
Outcome
Performance Indicator
(either quantitative or qualitative)
Types of Analysis
Planning (involves examining the developmental issues prior to setup)
Process or Formative (involves monitoring the “process,” ensuring activities are completed on time and on target, while the policy is ongoing)
Tells you if you’re on track
Points to improvement
Outcome or Summative (involves assessing the outcome at the conclusion of the policy and measures how change that has occurred as a result of the policy)
Shows what impact you have on problem
Helps justify policy
Cost Benefit
*
Class Four
Planning Analysis
- Why is the policy needed?
Identify target population
- Who needs to be involved in the planning?
Identify key players/agencies
- What are the goals of the policy?
Identify goals from perspective of various key stakeholders/agencies
- What resources are necessary?
Identify financial resources
Identify non-financial resources
- What is the timeline?
Determine timeline to policy implementation
*
Analysis Data – Types and Collection
- TYPES: (Which is more useful?)
Quantitative (see, Donaldson page 22)
Qualitative
- How to collect data for process Analysis:
Interviews
Focus groups
Observation
Questionnaires/Surveys
Analysis of existing documents
Outcome Analysis
- Outcome Analysis's are sometimes referred to as “true” or “real” analysis
- Outcome analysis is no more “true” or “real” than process Analysis
- Indeed, a quality process analysis is necessary if one is going to say what it is about the policy that produced the results
Outcome Analysis Data – Types and Collection
- The essence of policy outcome analysis is comparison (i.e. multiple data collection points to compare). Assessment of change over time. Done through utilization of:
Control group
Comparison group
Pre- and post-test
- The data collected is most often quantitative but may be qualitative given the nature of the policy or project.
The Four Standards of Analysis
There is no one “right” Analysis. Instead, best choice at each step is options that maximize:
- Utility: Who needs the info from this Analysis and what info do they need?
- Feasibility: How much money, time, and effort can we put into this?
- Propriety: Who needs to be involved in the Analysis to be ethical?
- Accuracy: What design will lead to accurate information?
*
Step-by-Step Analysis Design
1. Engage stakeholders: Decide who needs to be part of the design and implementation of the Analysis for it to make a difference.
2. Describe the policy: Draw a “soup to nuts” picture of the policy— activities and all intended outcomes.
3. Focus the Analysis: Decide which
Analysis questions are the key
*
Step – by – Step (continued)
Seeds of Steps 1-3 harvested later:
4. Gather credible evidence: Write
Indicators, then choose and implement
data collection sources and methods
5. Justify conclusions: Review and
interpret data/evidence to determine
success or failure
6. Use lessons learned: Use Analysis
results in a meaningful way.
*
Designing an Analysis Plan
- STEP 1: Engage Stakeholders
Who are major stakeholders for our efforts
Where in this model doe they want to see success?
Who needs to be engaged upfront to ensure use of results?
Engaging Stakeholders
- Consider which stakeholders are key to this analysis of your policy or case
- Consider what aspects of the policy and its analysis we must attend to, to keep these stakeholders “at the table”
Designing an Analysis Plan
- STEP 2: Describe the policy
You don’t always need a logic model, BUT you ALWAYS need a policy description
Don’t jump into planning or Analysis without clarity on
The big need your policy is to address
The key target group(s) who need to take action
The kinds of actions they need to take (your intended outcomes or objectives)
Activities needed to meet those outcomes
Causal relationships between activities and outcomes
Proposal Problems
STEP 3: Setting Analysis Focus
Today, Year 1, Year 3, Year 5, where in the model should I be measuring change?
If no change, where should I look for problems?
Key Domains in Analysis Focus
- Implementation (Process)
Is policy in place as intended?
- Effectiveness (Outcome)
Is policy achieving its intended short-, mid-, and/or long term effects/outcomes
- Efficiency
How much “product” is produced for given level of inputs/resources?
- Causal Attribution
Is progress on outcomes due to your policy?
Setting Focus: Some Rules
- Based on “utility” standard
Purpose: Toward what end is the Analysis being conducted?
User: Who wants the information and what are they interested in?
Use: How will they use the information?
Some Analysis Scenarios
- Scenario 1: At year 1, other communities want to adopt your model but want to know “what they are in for?”
Purpose: Examine policy implementation
User: The “other community”
Use: To make a determination, based upon your experience whether or not they want to adopt this project
Analyzing the Problem
Sample flow chart
Analyzing the Problem
• Document the need for change. Collect and analyze data to define what the problem is, where it is, how big it is, and who is affected by it. What evidence of the problem exists?
• Describe the history of the problem. How long has the problem existed? How has it changed over time?
• Examine potential causes of the problem. What causes the problem? What theories do we have? The intervention must target one or more specific causes supported by research.
• Examine previous interventions that have tried to change this problem, identify the most promising interventions, and choose a preferred intervention approach. We need to analyze available information to direct decisions about a possible course of action.
Analyzing the Problem
• Identify relevant stakeholders. Do different groups of people have different definitions of the problem? Who is affected by the problem?
• Conduct a systems Analysis. Conduct research on the system within which the problem exists, and determine how the system may create, contribute to, or maintain the problem.
• Identify barriers to change and supports for change. Who is likely to support a certain course of action, and who is likely to resist it?
Analyzing the Problem: Meth Case Study
“Today’s Native Alaskan and Native American communities face a devastating problem that threatens their way of life and drains their resources. Methamphetamines (meth) are an ever growing problem that is destroying lives in more than 560 independent sovereign tribal communities.”
Donaldson – Representative Democracy
- Government is
Necessary:
- To Maintain Order
To Protect Property
To Provide Public Goods
To Provide Public Services
*
Authority in a Representative Democracy
- Declaration of Independence
- Representative Democracy
Principal-agent concept
Class 1
*
Declaration of Independence
We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness. That to secure these rights, Governments are instituted among Men, deriving their just powers from the consent of the governed, That whenever any Form of Government becomes destructive of these ends, it is the Right of the People to alter or to abolish it, and to institute new Government …….
*
Is Government Necessary?
http://video.foxbusiness.com/v/4632197/stossel-is-government-necessary/
Is Government Necessary?
http://videocafe.crooksandliars.com/heather/real-time-interviews-nj-tea-partiers-who-c/
MAKING SENSE OF GOVERNMENT AND POLITICS
Influencing the Government
- Politics
Conflict and struggle over the leadership, structure, and politics of government
Managing conflict
Deciding who gets what, where, and when
- Power
Influence over a government’s leadership, organization, or policies
*
Who gets what, where, when, and how?
http://mediamatters.org/video/2013/01/22/oreilly-argues-government-doesnt-help-him-succe/192352
Donaldson – Representative Democracy
- “Authoritative such as legislatures, make public policy and policy choices in most cases.” (Henry, p.33)
- “The stated purpose of public policies and policies is to produce a different pattern of social outcomes than currently exist.” (Henry, p. 36)
Gary T. Henry
Donaldson – Representative Democracy
“….when Analysiss are intended to influence the adoption or reform of government policies or policies, randomized experiments and high-quality quasi-experiments are the preferred approaches for Analysis.” (Henry, p.33)
Gary T. Henry
The Concept of a Political System
Inputs
Political
Institutions
Outputs
“the value of Analysis to society is directly related to
the accuracy and credibility of the information produced.”
(p. 38)
*
Goal Setting
What is a Goal Anyway?
- According to Merriam-Webster a goal is: the end toward which effort is directed. Goals are dreams and wants except a goal is more specific.
- “Goals are broad aims of the intervention (e.g., to reduce drug abuse)”;
- “Objectives specify explicit and measurable outcomes (e.g., in a 1-year follow-up of ex-offenders who participated in a drug treatment policy while in prison, researchers predict that participants will have a lower re-arrest rate than nonparticipants).” (Welsh)
The Purpose of Goals
- The purpose of goals is to give you something that you want to enhance your life in some way, so the most important thing you need to know about goals is that you ought to have some.
Types of Goals
- Long Term Goals:
10-year, 5-year and 1-year goals - Short Term Goals:
goals for the next 9 months, 6 months and 3 months - Immediate Goals:
1-30 days from now
How to Set Goals
- Learning how to set goals is as important as knowing what the goal should contain.
- Video: “Setting Goals to Stretch and Grow,” http://ezproxy.wou.edu:2184/p_ViewVideo.aspx?xtid=93377&tScript=0
Identifying Goals and Values
- Goals describe desired future states, some intended change in the problem.
- Goals are broad statements intended to provide direction for change (p. 78)
- Criminal justice example:
Proportionality (punishment fit the crime)
Equity (treat all the same)
Parsimony (least drastic and expensive measures)
Humane treatment (least painful and intrusive measures)
- State specific objectives for each goal
- Seek Participation in goal setting
- Goal setting: top-down v. bottom-up approach
- Specify a impact model
- Identify compatible and incompatible goals in the larger system
- Identify needs and opportunities for interagency collaboration
The Goals of Criminal Sanctions
- Normative values (guiding assumptions held by individuals about how the justice system ought to work).
- Criminal sanctions:
Retribution
Rehabilitation
Deterrence
Incapacitation
Restoration
- State specific objectives for each goal
- Seek Participation in Goal Setting
- Goal setting: top-down v. bottom-up approach
- Specify a impact model
- Identify compatible and incompatible goals in the larger system
- Identify needs and opportunities for interagency collaboration
Video
- Young, Armed, and Dangerous
- http://ezproxy.wou.edu:2184/p_Search.aspx?bc=0&rd=a&q=Young%2C%20Armed%2C%20and%20Dangerous
Normative Values
- Normative values (guiding assumptions held by individuals about how the justice system ought to work).
- Criminal justice example:
Proportionality (punishment fit the crime)
Equity (treat all the same)
Parsimony (least drastic and expensive measures)
Humane treatment (least painful and intrusive measures)
- What are the “Right” Goals and Values?
Normative Values
- Seek Participation in goal setting
- Goal setting: top-down v. bottom-up approach
- Specify a impact model
- Identify compatible and incompatible goals in the larger system
- Identify needs and opportunities for interagency collaboration
- Goal setting works!!!!!!
- Have you ever sat down on a Sunday evening in front of a pile work and asked yourself "What happened to the weekend?"
- When done properly, goal setting helps the agency do those day to day activities that you know you should do to be successful but don't always get done.
- More generally, goal setting helps you to identify what the agency should accomplish and find a way to do it.
Qualities of Effective Goals
- Before you begin your goal setting process it is important to know what qualities your goals should embody. Effective goals have three important qualities. They are realistic, measurable, and reviewed.
Qualities of Effective Goals
- The goal should be SPECIFIC enough so that we know exactly for what we are striving.
Qualities of Effective Goals
- A goal must be MEASURABLE. It should have concrete facts. You should be able to answer very specifically, when and how you will know you attained your goal.
Qualities of Effective Goals
- ACTION-ORIENTED, declaring positive activity that will produce results.
Qualities of Effective Goals
- A goal must be REALISTIC. Challenging yourself and the agency is an important part of goal setting. You want to aim high; however, you also need to be realistic.
Qualities of Effective Goals
- TANGIBLE meaning concrete and not vague.
Also… don’t forget…
Goals must also be…
Qualities of Effective Goals
- REVIEWED.
- Share your goals with colleagues, friends and team members who care about the agency’s success. Remind yourself and the team regularly of the goals you have set for the agency.
- You can write your goals in your calendar, on a grease board, or a desk-wherever you will see them often. Check your progress regularly.
- Ask team members to check on your progress. They might even offer you some additional incentive for accomplishing your goal!
Steps Toward Setting Effective Goals
" THE ESTABLISHMENT OF A CLEAR AND CENTRAL PURPOSE OR GOAL FOR THE AGENCY IS THE STARTING POINT OF ALL SUCCESS." -- BRIAN TRACY
1. Set Goals
Identify what is important to the agency. What do you want the agency to accomplish?
2. Identify possible strategies or objectives to reach each goal.
Make a list of what you feel are the best and most effective ways of reaching the goals you have already identified.
3. Select the best strategies
Now that you have made a list of several ways to achieve your goal, recognize which of these strategies will work best for you.
4. Outline specific plans to accomplish each strategy.
- Once you have narrowed down your strategy list, you can begin to make very detailed and specific plans to accomplish each strategy and ultimately reach your goal.
- Once we have a well-formed Goal Statement we need some direction to follow to achieve this Goal.
"AN UNWRITTEN WANT IS A WISH, A DREAM, A NEVER HAPPEN. THE DAY YOU PUT YOUR GOAL IN WRITING IS THE DAY IT BECOMES A COMMITMENT THAT WILL GUIDE THE AGENCY.
Setting goals is a means of identifying and plotting how the agency will achieve its mission. By writing down your goals you are committing yourself and the agency to accomplishing them; therefore, make sure that your goals are important to the agency in achieving its mission!!!!!
Motivation
- Motivation and commitment are what make us strive to achievement. They give us the push, desire, and resolve to complete all of the other steps in the Goal process.
- This motivation can be obtained by developing a personal statement that creates a high level of emotion and energy that guarantees achievement.
Commitment
- Commitment creates more accountability and is what sets us on direct course to reach our goals. It may create costly negative consequences upon failure to attain a goal.
- Making a commitment might be something like having invested your savings in your new business which you will lose if it fails. Motivation and commitment are specific to your situation and life and only you can form statements that will ensure you reach your goal the quickest.
- The more personal you make your motivation and commitment statements for each goal, the more motivated you will be to accomplish your goal.
BE FLEXIBLE
Be ready to re-adjust your goals and not give up on them. If they are too easy, they won't be worthwhile, and if they are too tough, and you will start thinking that the idea of goal setting is worthless. Remember, small victories lead to big successes.
Why Goals Fail
- The goal was not written down.
- Rewards for achieving the goals were not given.
- The goal was unrealistic or not specific enough.
- The goal is not really believable or little commitment exists.
- Keep changing or switching goals with the weather
- The person who set the goal has not told anyone else for added accountability, help and support.
- The goal was not incorporated into a realistic plan that includes measurements, timelines and resources.
The Benefits of Setting Goals
- Staff suffer less from stress and anxiety
- Staff can concentrate and focus on achieving the agency’s mission
- Show more self-confidence
- Staff perform better
- Staff are happier
Video: Setting Objectives
- http://www.youtube.com/watch?v=0MpfWT92mIU
Objectives
- Objectives are much more specific than goals.
- Objectives should define clearly and concisely exactly what outcome is to be achieved by the intervention.
- Objectives precisely describe the intended results of the intervention in measurable terms.
Objectives
- Objectives, therefore, must always be measurable and specific.
- Objectives should include four major components:
1. A time frame: date by which the objective will be completed.
2. A target population: who will exhibit the intended change?
3. A result: the key outcome intended; a specific change in the problem (or one of its causes).
4. A criterion: a standard for measuring successful achievement of the result.
Stage III: Designing the Policy
III. Designing the Policy
Three Components:
- Choosing an intervention approach
- Major activities for program design
- Major activities for policy design
- Choosing an intervention approach
(1) Choosing a policy involves integrating the information assembled in previous stages.
(2) Review - you have collected and analyzed data, you have an idea of the intervention options that fit the problem or need, and your goals and objectives have been established.
(3) Reassess the systems analysis
(4) Calculate the cost; policy and program options are often weighed in terms of cost.
Native American Cultural Court
- Major activities for program design
Include the following:
(1) Identifying the target population: Who is the intervention aimed at?
(2) Defining participant selection and intake procedures: How are program participants recruited and selected for the intervention?
(3) Defining program components: the precise nature, amount, and sequence of activities provided must be specified. Who does what to whom, in what order, and in what amounts?
(4) Writing job descriptions of staff, and define the skills and training required in order to deliver services well.
- Major activities for policy design
Include the following:
(1) Identifying the target population: Who is the intervention aimed at?
(2) Defining participant selection and intake procedures: How are policy participants recruited and selected for the intervention?
(3) Defining policy components: the precise nature, amount, and sequence of activities provided must be specified. Who does what to whom, in what order, and in what amounts?
(4) Writing job descriptions of staff, and define the skills and training required in order to deliver services well.
Designing a policy
In general, in designing a policy we must answer the following questions as specifically as possible:
- Who does what to whom, where, in what order, how much, and how often?
- How will the policy be set up?
- How are participants selected?
- What activities are delivered, and how?
- What training and qualifications are required for staff?
Designing a policy
By the time you reach this stage of analysis, you should have information about:
- The problem or need to be addressed
- It’s etiology (causes) and theory that explains how these causes work to produce the problem
- Possible interventions, including interventions that have already been tried
- Potential risks, barriers, or sources of resistance to change (discuss - Fault Tree Analysis)
- https://www.youtube.com/watch?v=olMKwMzEcyU with James Vesper
- Goals and objectives (remember that objectives are specific and measurable)
Designing a policy
- If you were acting as a policy “analyst,” you would look at an existing policy and specify in detail exactly what it does.
- When designing a new policy, however, the change agent’s task is to design the “nuts and bolts” of the new policy.
Identify the Target Population of the policy
- Who is to be targeted, or changed?
Identify the Target Population of the Policy
We now need to clarify exactly who or what is the target of change. Are we trying to change:
- Individuals? (e.g., via intensive monitoring, counseling, teaching problem-solving skills)
- A group or groups? (e.g., via support groups, peer groups, family counseling)
- An organization? (e.g., via police training)
- A community? (e.g., via community policing, neighborhood watch, community service)
- Social structural conditions? (e.g., via welfare reform, job training, employment assistance)
Identify the Target Population of the Policy
In order to specify the target population, two major steps are required: defining eligibility and specifying numbers to be served.
Identify the Target Population of the Policy
Define Eligibility:
Who is eligible for the policy?
What kind of individuals is the policy intended for, and which targets are best suited to the intervention approach?
Eligibility is often based on age, residence, income, gender, ethnicity, or other demographic variables.
It is also based on level of need: What is the appropriate population to be targeted, in terms of how significant or urgent their needs are?
Identify the Target Population of the policy
2. Specify Numbers to be Served:
--Given scarce resources, how are policy funds most wisely spent?
--How many people can be served with available resources?
--How many individuals can the policy accommodate over a time period of 6 months? One year?
Assessing Risk and Needs
- One of the most common tools for defining a target population and matching individual needs with appropriate policy is a risk/needs assessment.
- http://www.youtube.com/watch?v=Y-Gvsb5h7wQ&playnext=1&list=PLEtB3msGXciQKm7w1yO1oJuhqVm8iyxcJ&feature=results_main
---Risk typically refers to the likelihood of a negative outcome, like re-arrest;
---Needs pertain to modifiable problems that may be causing them to engage in antisocial behavior.
Defining Participant Selection and Intake Procedures
- Now that we know who the targets of change are, the next question is: How are policy participants recruited and selected for the intervention? Given that targeted individuals and potential referral sources (e.g., police, courts, schools, probation, social services, etc.) are initially unaware of the policy (or perhaps hostile toward it), how will we make them aware of this policy, and how will we encourage them to use it?
- The following five issues should be considered when defining how participants will be selected.
1. Access: How are potential participants “recruited” (i.e., how do they become aware of the policy)? How are they informed of policy operations and activities? Are referrals made to the policy by any outside agencies? If so, by whom and for what reasons? How do referral sources learn about the policy?
2. Screening: How are referrals or applicants screened for eligibility? Is some kind of needs assessment or other assessment tool used? Are application procedures required? How do they work? How is it decided which eligible referrals or applicants will be admitted to the policy and who will be excluded?
3. Intake: Is an intake form used to record basic information when an individual is referred to the policy, such as name, age, source of referral, reason for referral, etc.?
4. Individual Records: Is information to be recorded or stored throughout an individual’s participation in the policy? What kinds of information are needed for agency reporting purposes? For feedback to the individual? For treatment planning?
5. Retention: How long are participants to be retained in the policy? What procedures will encourage them to complete the policy?
Identify the Responsible Authority
- Who is to carry out the policy?
An existing agency?
A new agency?
- What will be the responsibilities of those persons be?
Specifying Policy Provisions and Procedures
- Provisions: What is to be done: the goods, services, opportunities, or interventions that will be delivered to members of the target population.
- Procedures: The steps that need to be followed and the conditions that must be met to apply the policy.
- Vass Bednar, University of Toronto - Making public policy more fun
- https://www.youtube.com/watch?v=wXYSsA5yVSY
Action Planning
Action Planning
- Action planning is Stage 4 of the seven-stage model used in Welsh and Harris’ book. We assume that the first three stages of Analysis have been completed at this point in time:
Stage 1 (problem Analysis),
Stage 2 (goals and objectives), and
Stage 3 (policy or policy design).
Action Planning
Charting the entire sequence of activities and completion dates required to implement the policy or policy design. It involves specifying, in clear and concise detail, the steps required to implement the policy or policy design. It is, in essence, a “blueprint” explaining how to trans- late a vision of the policy or policy into reality.
Developing an action plan is like writing the instructions explaining how to assemble a new computer system.
Action Planning
- Identify resources needed and make cost projections.
- Plan to acquire or reallocate resources.
- Specify dates by which implementation tasks will be accomplished.
- Develop mechanisms of self-regulation.
- Specify a plan to build and maintain support.
http://ezproxy.wou.edu:2120/PortalViewVideo.aspx?xtid=4282&psid=0&sid=0&State=&title=The%20Culture%20of%20Commerce&IsSearch=Y&parentSeriesID=&loid=19398
Review ---
Video: The Culture of Commerce
Action Planning
- Identify resources needed and make cost projections.
How much funding is needed to implement a specific intervention?
Identify the kinds of resources needed, estimate costs and make projections, and develop a resource plan
Action Planning
- Specify dates by which implementation tasks will be accomplished,
and assign responsibilities to staff members for carrying out tasks. A “time line” or Gantt chart is particularly useful for this purpose.
Action Planning
- Develop mechanisms of self-regulation.
Create mechanisms to monitor staff performance and enhance communication, including procedures for orienting participants, coordinating activities, and managing resistance and conflict.
Action Planning
- Specify a plan to build and maintain support.
Anticipate sources of resistance and develop responses.
Action Planning
- Implementation:
The initiation, management, and administration of the action plan (see Chapter 5). Once the policy or policy actually begins, we want to minimize discrepancies between what was planned (i.e., the policy or policy design) and what was actually done (i.e., the “policy or policy in action”).
policies and policies are similar—we need a blueprint
Identify Resources Needed and Make Cost Projections
- One needs to start by developing a resource plan, which enumerates all the specific costs associated with each policy or policy component, including staff salaries, benefits, training, supplies, physical space, and so on.
Identify Resources Needed and Make Cost Projections
- Resource Plan: A comprehensive statement of the specific fiscal, material, and social resources required to implement an intervention. All policy or policy costs are estimated, including personnel, training, equipment, supplies, facilities, travel, and so on.
Resource Plan Goals
- It matches resources to objectives. One must carefully ensure that all the resources necessary to achieve the objectives of the policy or policy are in place. It forces us to impose a test of feasibility: either resources must rise to the level needed to achieve the stated objectives of the policy or policy, or the objectives must be downscaled to match the level of resources available.
Resource Plan Goals
- It identifies the availability of current resources and resources still needed to implement the policy or policy design properly.
Resource Plan Goals
- It attempts to control expenditures, usually by specifying how much money is to be spent over specific periods of time, such as each quarter (a 3-month period) and year.
Resource Plan Goals
- It provides data for monitoring fiscal aspects of the policy or policy and providing feedback to funding sources and other stakeholders (e.g., annual reports, or quarterly grant reports).
Two basic principles of resource planning.
- First, we identify the kinds of resources needed.
- Second, we estimate costs for each type of resource needed.
Two basic principles of resource planning.
- First, we identify the kinds of resources needed.
The resource list should include everything that costs anything. For example:
Staff: how many, with what qualifications?
Staff training: what kind, and how much will be needed?
Supplies: for example, paper, printing and copying, office supplies?
Advertising: for example, brochures, flyers, public service announcements?
Equipment: for example, computers, telephones, copiers?
Rental costs: office space, meeting space, and other specialized space, if required (e.g., private interviewing rooms)?
Utilities: telephone, electrical, heat, and water bills?
Two basic principles of resource planning.
- Second, we estimate costs for each type of resource needed.
It is very important to be realistic about cost estimates. If you estimate too little, the policy or policy is likely to fail. If you estimate too much, the proposal may not get funded, or the agency may face accusations of waste.
A good budget will not only describe all estimated costs, but will provide a clear justification for each expenditure item.
Plan to Acquire or Reallocate Resources
- The task of obtaining funds to implement the policy or policy requires a combination of experience, dedication, persistence, and patience.
- A designee will scour the grant announcements of government, private, and nonprofit agencies, attempting to find some match between the interests of potential funding providers and the type of services the policy is designed to provide.
- The interests of funding providers are usually clearly spelled out in Annual policy Announcements, Grant Announcements, or Solicitations for Proposals.
Plan to Acquire or Reallocate Resources
- Acquiring the necessary resources to implement the policy may involve any or all of a combination of activities:
writing a formal funding application to a government or non-profit agency;
lobbying local, state, or federal politicians for funding;
making informal inquiries, presentations, and solicitations to various agencies; and
familiarizing oneself with the entire funding terrain of potential funding sources.
- The next task, probably the most important one at this stage, is to develop a policy or policy time line, sometimes called a Gantt chart (see Example 4–1), which specifies three elements:
all the specific implementation activities that need to be accomplished,
assignment of responsibility for each specific task to one or more individuals, and
a specific date by which each task is to be completed.
http://ezproxy.wou.edu:2120/PortalViewVideo.aspx?xtid=47484
Specify Dates by Which Implementation Tasks Will Be Accomplished
- Orienting Participants: All participants, both staff and clients, must understand their respective roles. Leaders must:
(1) clearly communicate the policy’s rationale, values, and intent;
(2) clarify the staff’s job descriptions;
(3) spell out behavioral expectations, and specify rewards and punishments; and
(4) allow staff to ask questions.
Develop Mechanisms of Self-Regulation
Managing Change
- Coordinating Activities: like the conductor of an orchestra, the change agent (with assistance from the individuals and agencies comprising the action system described in Chapter 1) must coordinate the activities of several different individuals and groups.
Develop Mechanisms of Self-Regulation
- Managing Resistance and Conflict: Some resistance is inevitable with the start-up of any new policy or policy (recall force field Analysis from Chapter 1).
- Resistance may come from any of the participants involved: clients, targets, even the policy or agency’s own staff (i.e., the action system).
Develop Mechanisms of Self-Regulation
- The policy or agency director is always trying to strengthen sources of support for the policy or policy: within the staff, the community, across other agencies with which the policy links, with his or her superiors, with the funding agency, with the news media, and with clients.
Specify a Plan to Build and Maintain Support
- The Execution Plan
- http://ezproxy.wou.edu:2120/PortalViewVideo.aspx?xtid=39954
- CNBC Titans [electronic resource (video)] : Steve Jobs / CNBC (Television network)
- Reshaping Cities
- Rise of the Superbugs [electronic resource (video)] / Public Broadcasting Service (U.S.)
- Security versus Liberty [electronic resource (video)] : The Other War / Public Broadcasting Service (U.S.)
- Spying on the Home Front [electronic resource (video)] / Public Broadcasting Service (U.S.)
- The Soldier's Heart [electronic resource (video)] / Public Broadcasting Service (U.S.)
- A Conversation with Robert Moses [electronic resource (video)] : From NBC's Wisdom Series / NBC News
- Emergency Management Planning [electronic resource (video)] / TRAINEX (Firm)
- The Coach [electronic resource (video)] / Classroom Video (Firm)
- Managing change in a large organization [electronic resource (video)] / Video Education Australasia
Policy Implementation & Monitoring
Policy Implementation
- “Monitoring”
- Design monitoring instruments to collect data
- Fiscal monitoring
- Designate responsibility for data collection, storage, and Analysis
- Develop information system capacities
- Develop mechanisms to provide feedback to staff, clients, and stakeholders
New Jersey Case Study
- http://www.youtube.com/watch?v=ywSFd_upOZA
Policy Implementation --- Monitoring
- Monitoring refers to the collection of information to determine to what degree the policy or policy design (see Chapter 3) was carried out as planned.
How will we know whether the intended target population was reached?
Were policy/policy activities or provisions actually carried out as planned?
Were appropriate staff or responsible authorities selected and trained, and did they carry out their assigned duties?
Policy Implementation --- Monitoring
- Monitoring defined: An attempt to determine whether policy or policy implementation is proceeding as planned.
- Monitoring is a process that attempts to identify any gaps between the policy or policy on paper (design) and the policy or policy in action (implementation).
Policy Implementation --- Design monitoring instruments to collect data
- Examples: observations, surveys, interviews. Data is collected to find out what is actually being delivered to clients or targets. The purpose is to identify gaps between the “policy/policy on paper” (design) and the “policy/policy in action.”
Outline the Major Questions for Monitoring
- Monitoring relates directly back to the design stage (Chapter 3):
How do we measure whether the critical elements of policy (or policy design) have been implemented properly?
We can specify all the key questions for monitoring in terms of their corresponding policy or policy design features (see Figures 5–1 to 5–3).
- http://www.youtube.com/watch?v=KzgJm0m8B7A
Monitoring
Analysis
page 150
Instruments to Collect Monitoring Data
- Observational Data
By observational data, we mean that evaluators and/or trained observers actually participate in or observe the policy or policy in operation.
(1) the narrative method,
(2) the data guide method, and
(3) the structured rating scheme
- Service Record
DataService record data refers to written, typed, or computerized records that are kept by staff. Many policies require staff to collect certain information on policy clients, service delivery, and staff duties.
- Service Provider
DataService provider data refers to information that the evaluator or change agent obtains from policy or agency staff members directly.
- Participant Data
Participant data refers to information that the evaluator or change agent obtains from clients or targets directly.
- Fiscal Monitoring defined: An additional type of monitoring data is always required by funding sources: regular (usually quarterly or semi-annually) financial reports detailing how funds were spent during the project, policy, or policy implementation.
Policy Implementation --- Fiscal Monitoring
- Fiscal monitoring. Funding sources require regular financial reports detailing how funds were spent. Once the policy or policy is implemented, one may find that adjustments to the budget become necessary.
Policy Implementation
- Designate responsibility for data collection, storage, and Analysis.
Ensure that there is no ambiguity about what information is to be collected, who is responsible for collecting it, or how it is to be collected, stored, and analyzed.
Policy Implementation
- Develop information system capacities.
Information systems may consist of written forms and records that are filed, or fully computerized data entry and storage systems.
Policy Implementation
- Develop mechanisms to provide feedback to staff, clients, and stakeholders.
Depending on the results of monitoring analyses, it may be necessary to make adjustments either to what is being done (“the policy or policy in action”) or to the intended design (“the policy or policy on paper”).
Policy Implementation
- “We need to develop a strategy to observe the policy or policy in action. History has taught us that good intentions alone are insufficient: planning and implementing an intervention are two different things. The example below illustrates how the same strategy can be implemented very differently in different locations.”
Policy Implementation