Quiz

Step 6 Preparing the Evaluation Component

EVERYTHING COMPLETED UP TO THIS POINT in the development of the organization's proposal (problem statement, goals, objectives, and strategies) naturally leads to this component. The evaluation answers critical questions for both the organization and the funder, such as:

· Was the program successful?

· Did it do what it was designed to do?

· What did the organization learn from this experience that can be leveraged?

· What didn't work – and why?

· What's different in the community or the lives of those targeted as a result of the program?

Before writing this section of a grant proposal, it is essential for an organization to plan how it will evaluate what it proposes to do. This step explores learning how to write an effective evaluation plan so that the organization can demonstrate the success of its program and measure program impact – and capture the lessons learned. An exercise will help grantseekers think about what their evaluation plans should contain.

Definitions

Impact: The change an organization creates as a result of its program activities.

Leverage: A method of grantmaking practiced by some foundations, when they give a small amount of money with the express purpose of attracting funding from other sources or providing the organization with the tools it needs to raise other kinds of funds. Sometimes known as the “multiplier effect.”1 (An organization may also use leverage, for instance by enabling other organizations to replicate successful approaches it has developed, thus using one effort to build strength for other efforts.)

Purpose of the Evaluation Component

Evaluation is a process that determines the impact, effectiveness, and efficiency of a program. It reveals what worked and – equally important – what did not. Funders expect to hear from organizations how they define and measure the success of a program, whether the funder explicitly requests an evaluation or not. Yet the primary purpose of an evaluation should not be to satisfy a funder, but to help an organization assess the effectiveness of its work and plan for the program's future. For that reason, how a program will be evaluated must be determined prior to its implementation, so that the organization can build evaluation measurements into the program plan. The organization then will be well-positioned to produce, at the end of a grant period, an organized and objective assessment documenting the return on investment for funders and the realized benefits to the community the organization serves.

Definition

Return on investment (ROI): The amount of benefit (return) based on the amount of resources (funds, hours of work) used to produce it.

The Virtues of Evaluation

First, a good evaluation component strengthens the proposal from the funder's perspective. Grantseekers are asking potential grantmakers to invest in their organizations and programs – and they are asking the funders' staffs to be their advocates. They want the funders to bet on the fact that the world will be improved in some specific way as a result of the nonprofits' proposed programs. Essentially, the programs serve to test a hypothesis: “If we do this, then that will happen.” A solid evaluation component in a proposal reassures funders that the organizations are interested, as the funders are, in learning whether this hypothesis is correct.

Second, through evaluation, organizations will learn about the programs' strengths and areas of weakness. The process of thinking through the evaluation design can strengthen a program before it is even implemented. From there, organizations can take the knowledge gained through an actual evaluation and share it with staff and volunteers to improve programs as they are being implemented. This knowledge may also be shared with others in the field so that they, too, can learn from the lessons of the program's work.

The third benefit is to the public – the impact. Dollars granted from foundations and corporate-giving programs are dollars dedicated to charitable good; therefore, with each grant an organization receives, it becomes a recipient of public trust. Because of that, the organization has an obligation to ensure that its programs are actually having a positive impact on the community or on the organization's target audience. Evaluation is one of the strongest and most effective tools any nonprofit has to verify and document that it is indeed fulfilling its obligation to make a positive impact on the community it serves.

Definition

Hypothesis: The assumed proposition that is tested in a research process.

Internal or External Evaluation

Some foundations will allow organizations to designate a certain amount of money or a certain percentage (generally from 5 to 10 percent, sometimes more) of the total program budget for evaluation; others will not. Therefore, organizations need to consider how they will evaluate their programs for the purpose of documenting results, findings, and lessons learned. There are some organizations that will spend time up front designing their evaluation components with the confidence that they have both the staffing and the expertise in place to objectively handle the evaluation internally. Other organizations will decide to engage an outside evaluator; among the most common reasons for doing so are that the organization (1) lacks expertise among the staff, (2) has the staff expertise but lacks the staff time to dedicate to evaluation, or (3) wants the evaluation to be deemed as objective as possible. Keep in mind that even if an organization hires an outside evaluator, it needs to reserve some staff time for the evaluation. An outside expert cannot conduct an evaluation without significant participation from the organization.

Organizations should provide some background information in the proposal that indicates whether they plan to evaluate the program internally or hire an external evaluator. The proposal budget should also reflect an expense line item for evaluation.

Content of the Evaluation Component

To design an evaluation, one first needs to consider the organization's definition of success – the “so what?” factor. Then one must have a clear understanding of how the activities described in the proposal will lead to the expected outcomes. Finally, one needs to identify the aspects of the program for which it is most important that the organization can assess how well they worked – and why they did or didn't work.

Organizations conduct evaluations

1. Find out whether the hypothesis was correct: Did what the organization proposed actually do what the organization expected it would?

2. Determine whether the strategies that were specified were indeed used and the objectives met.

3. Determine whether the organization's work made an impact on the problem identified.

4. Obtain feedback from the clients served and other members of the community.

5. Make midcourse corrections along the way to increase the program's chances of success.

When preparing the evaluation section of the proposal, answering the following questions will help frame the description:

1. What is the purpose of the organization's evaluation?

2. How will the findings be used?

3. What will the organization know after the evaluation that it does not know now?

4. What will the organization do after the evaluation that it cannot do now because of lack of information?

5. How will the lives of the people or community served by the organization be better?

6. Did the organization use the funder's investment wisely? Were the funds effectively managed or leveraged?

7. Was the program budget accurate, or did the program components end up costing significantly more or less that the organization expected?

This workbook provides a broad overview that can help grantseekers determine the parameters most appropriate for the program. Generally, there are two approaches to data collection: quantitative methods and qualitative methods.

Quantitative methods are methods to quantify (measure or count) data. Using this method, an organization collects data that can be analyzed statistically, via averages, means, percentiles, and the like. It can help an organization quantify how much it did – how many people did it train? How many meals did it deliver? These analyses allow organizations to make statements about cause-and-effect relationships. Employ quantitative methods in order to:

· Understand the quantities or frequency of particular aspects of a program (such as number of enrollees or number of dropouts).

· Determine whether an organization's actions directly caused a particular result.

· Compare two different methods seeking to achieve the same outcomes.

· Establish numerical baselines that can be used in before-and-after comparisons (through such means as pretests and posttests, and quarterly or yearly follow-ups).

Qualitative methods, in contrast, are based on direct contact with the people involved with a program. These methods consist of interviews (group or individual), observation (direct or field), and personal stories told both in writing and in photos or videos, as well as review of selected documents. This approach can help organizations examine what it did and why it mattered. Employ qualitative methods in order to:

· Understand feelings or opinions about a program among participants, staff, or community members.

· Gain insight into how patterns of relationships in the program unfold.

· Gather multiple perspectives to understand the whole picture.

· Identify approximate indicators that clients are moving in the “right” direction.

In other words, pretests and posttests are not the only measures of success. By taking the time upfront to think strategically, an organization can come up with an evaluation design that incorporates both quantitative and qualitative methods.

A good evaluation lets both the funder and the nonprofit know whether the organization's activities are achieving their desired effect. An evaluation that is “just for show” or designed merely to please a funder – and not to answer real questions that organizations should be asking themselves about their work's effectiveness – are a waste of time and resources.

Take a look at the evaluation component prepared for the Alyson Eats organization, which uses both quantitative and qualitative data to assess its long-running programs. The organization is also testing a new program, and will seek outside help to design an evaluation for it.

Sample Evaluation Component

[Quantitative data] Our three core programs have specific annual goals, objectives, and outcomes to achieve during 2020:

· The EAR program seeks to provide 144,000 nutritious meals totaling 180,000 pounds of food for individuals and families in Alyson's south side.

· The EYE program aims to provide lunch five days a week to 1,000 children over the summer, for a total of 60,000 meals; and

· The Sharing Is Caring program strives to recruit 15 new volunteers in order to increase volunteer engagement by 30 percent.

Because we have been conducting these programs for over a decade, we've established a rigorous system of quarterly check-ins to track our progress toward the annual goals.

[Qualitative data] In addition to these quantifiable benchmarks, Alyson Eats regularly seeks input from our partners, volunteers, and the people we serve. These inputs range from quarterly conversations in which volunteers and staff are invited to talk about what they think is working well and what could be improved, to a series of poster boards we display on the walls of our food pantry that enable our clients to tell us, through the use of colored stick-on dots, how they think we're doing and what we could do better. Thanks to our historical and current partnership with the university, particularly the School of Social Work, students frequently work with Alyson Eats to create evaluation instruments and conduct assessments of our programs as part of their field practice. We take all of these evaluations seriously, and continually refine our programs to meet the needs and standards the assessments reveal.

Public Funding: Evaluation

Evaluation requirements for public funding applications vary widely depending on the funding source, type of program being funded, and other factors. For example, many federal – and some state and local – proposals require applicants to include plans to engage outside evaluators to conduct rigorous programmatic assessments, and applicants can and should include these evaluation costs in their proposed budget, as allowable in the Request for Applications and other regulations.

Even in cases in which an outside evaluator is not required, if resources allow, your public funding evaluation section should be developed and/or reviewed by someone with expertise in program evaluation. This could be a staff person, a board member, consultant, or other organizational partner who has this expertise, or even an instructor or graduate level student at a local community college or other institution of higher education.

If you are developing the evaluation section yourself, be sure that the evaluation ties directly to the goals and objectives and is represented in the logic model, workplan, and/or timeline. Some public funders may also ask about your plans to publish or disseminate the findings of your evaluation – be sure to include a detailed plan for this aspect of the evaluation in the logic model, workplan and/or timeline as well.

Answer the questions in Worksheet 6.1 to begin planning the evaluation section. Then review the section with the Evaluation Review Questions.

    WORKSHEET 6.1A: Evaluation Planning Questionnaire

1. What questions will the organization's evaluation activities seek to answer?

2. What are the evaluation plans and time frames?

1. What kinds of data will be collected?

2. At what points?

3. Using what strategies or instruments?

4. Using what comparison group or baseline, if any?

3. If the intention is to study a sample of participants, how will this sample be selected?

4. What procedures will be used to determine whether the program was implemented as planned?

5. Who will conduct the evaluation?

6. Who will receive the results?

7. How does this program or project define success?

Step 7 Developing a Statement of Sustainability

PART OF THE CASE THAT GRANTSEEKERS must make to potential funders is that they have a plan to fully fund their organization and/or program now and in the future. Funders want to know what strategies organizations have developed to build a sustainable pool of grant funding, including other funders who have been secured or identified, individual donors who support the program, and other resources that are being leveraged including in-kind support and partnerships. In this step, grantseekers learn how to develop the sustainability statement and identify, through exercises and examples, potential sources of ongoing support that are best for the program.

Content of the Sustainability Statement

The sustainability statement should reflect whether the proposal is seeking general operating, program, capital or equipment, or capacity-building funding. (Reminder: these types of requests were defined in the Overview Section). Whatever the type of request, the grantseeker should lay out the plan to sustain the organization, program, or capital expenditure in the long term. In the case of a capacity-building grant, the grantseeker should explain how the additional capacity can be implemented or sustained.

Funders pay much more attention to this section than most grantseekers would probably suspect, because they will have a vested interest in the organization's or project's success beyond their funding. Therefore, this section of the proposal should provide a framework that shows how the nonprofit plans to raise money and leverage resources beyond the funder's investment, as well as who on the organization's staff will be responsible for making this plan happen.

· For general operating grants – the sustainability statement should address: the organization's history in meeting revenue needs, future plans for diversifying funding from a variety of sources, the experience of its staff – including any staff specifically dedicated to development – and board in fundraising efforts, and any in-kind or donated resources that count toward its revenue goal (such as donated food and volunteer time in the case of Alyson Eats).

· For project- or program-specific grants – the sustainability statement should address: the history of the funding for the program, or in the case of a new program a rationale for why and how the resources will be raised and by whom. The sustainability statement can also include the organization's success in other fundraising efforts by including the information listed under general operating grants.

· For capital or equipment (major equipment purchases or building renovations and expansion) grants – the sustainability statement should help funders understand the associated costs for operating the new equipment, for maintaining the new building, or for increasing services if building expansion results in program expansion. Funders will want to be sure that the organization can sustain the capital or equipment expenditure that their investment will support. So, for example, if a funder supports the expansion of technology through the purchase of hardware, your sustainability plan should address how that hardware will be maintained and that you have a plan to support the needed software and infrastructure (such as high-speed Internet) to make the hardware usable.

· For capacity-building grants – the sustainability statement should help funders understand how the organization will support the capacity it has grown. For example, a request might be made for a capacity-building grant to increase the organization's fundraising ability by hiring a consultant to create a development plan. Once the plan is created – and the grant is expended – how does the organization plan to pay for the actual implementation of the new development plan? Does the organization have the staff and other resources to implement the plan?

In building the case for sustainability, here are some of the sources grantseekers can discuss in their strategy:

· Grants from other foundations and corporations. A nonprofit can seek continuing support from those foundations and corporations that fund ongoing programs. If your organization or program is effective and you properly cultivate and steward relationships with funders, they will often continue to support organizations beyond their initial investment. Once again, both the funder research along with the relationship building, as outlined in Step 2, have a direct impact on the outcome of continued funding.

· Support from individual donors. Developing support from individual donors is a key strategy for nonprofits, although it takes a particular set of skills and organizational assets to do effectively. Check the Resource section on places to learn more about developing individual donors, including online through crowd-funding and other strategies.

· Fees for service. Some organizations generate revenue in other ways, such as providing consulting services in its areas of expertise or, for example, receiving a fee from a school to provide afterschool tutoring programs. If a nonprofit opts to ask clients to pay fees, the fee scale should be explained and shown in the proposal.

· Sales of items or activities. A nonprofit might be able to set up an income-producing situation, such as a gift shop or thrift store. In addition, it might be able to sell publications, concert recordings, or educational activities. Revenues generated from these sales might cover some costs of the program. If this route is taken, a clear expense and revenue projection should be a part of the proposal. (Please note: Organizations need to check with both legal counsel and accounting counsel to ensure that any revenue-generating ventures launched are set up and monitored in accordance with IRS guidelines.)

A typical mistake that grantseekers make in their proposals is writing a vague sustainability statement, something to the effect of “future funding will come from a mix of sources such as other grants and individual support.” Be sure to take time to develop and write a compelling plan for long-term support that will help funders feel secure in your ability to be around for the long haul.

Tips for Writing the Sustainability Component

Helpful Hint

Sell it! If an organization has examples of other instances in which it successfully continued programs beyond initial funding, this would be the place to share such information, because it speaks to the organization's credibility not only in launching programs but also in maintaining them.

Many funders ask for this component in their grant guidelines; others do not. Whether or not this component is required, grantseekers should include some information on sources of support for the project's future. The more specific grantseekers are in this section, the more confidence they will inspire in potential funders that the project will continue beyond their grant, thereby maximizing the impact of their investment.

Take a look at the Sample Sustainability Component to see what the Alyson Eats has planned for the sustainability of its program.

Sample Sustainability Component for Alyson Eats

Nothing would please Alyson Eats more than to go out of business because there's no longer a need for our services. Short of that, we have a robust fundraising operation.

Our budget for 2020 is $1.3 million. We have commitments totaling 85 percent of this amount. Of our current commitments, approximately 57 percent are in-kind donations of volunteer work, food and facilities; 42 percent are donations from individuals, including our volunteers and their friends, and from institutions and foundations. With few exceptions, these are local and regional funders like the Future Foundation. We take it as a vote of confidence that the people who see our work close-up are the ones most moved to invest in our ongoing efforts. In the past few years, a new population of individual donors has emerged: people who once relied on our services, and now are on firm enough footing economically that they can turn around and donate to our programs. Although we value and appreciate every donation, the gifts from former clients mean the most to us.

Alyson Eats has yet to raise $197,016, which represents about 15 percent of our 2020 budget. A grant at the requested level from the Future Foundation would reduce our revenue gap by approximately 25 percent.

Public Funding

Public funding applications may or may not have a dedicated section that addresses sustainability. Public funders often want to understand how the program will be supported once the grant ends. They also want to understand that public funds are being leveraged to raise dollars from the private sector. If there is a sustainability question(s), the tips provided in this section will be equally helpful for public funding applications. As in the case of foundations, you want to be as specific as possible. And, as always, follow the specific guidelines outlined in the request for application.

Answering the Sustainability Review Questions will assist in developing the component of the proposal that deals with future funding.

Sustainability Review Questions

1. Is it the organization's intent to have the program continue after the initial grant funding is gone?

2. If yes, does the sustainability component of the proposal present a plan for securing future funding for the program?

3. Does it discuss future funding strategies or earned-income strategies?

4. If the organization is requesting a multiyear grant, did it show that the organization will have a decreasing reliance on grant support each year? (Grantmakers are more inclined to make a multiyear grant to nonprofits that assume greater financial responsibility for the project each year, rather than asking the funder to maintain the same level of funding each year.)

The grantseeker has now developed a thoughtful statement of sustainability. It is time to develop a financial document to accompany the grant proposal—the grant budget, which is discussed in Step 8.