What are Evaluation Methods?

Evaluation methods are the criteria for evaluating the success of a program or project.

Evaluation methods are a critical and effective tool for nonprofit organizations and businesses alike. When designed well, an evaluation provides staff, board members, and donors with a clear assessment of whether your organization or program is achieving its goals and objectives.

In this article, we will define evaluation, describe evaluation methods, outline what an evaluation plan is and how to use it, and detail different types of evaluation tools and techniques. We will also provide guidance on putting evaluation techniques into practice when designing and communicating about your program.

What is an Evaluation?

Before developing an evaluation plan, it’s important to define what an evaluation is and how it can be used. So, what exactly is an evaluation?

Evaluation is a systematic process to measure the demonstrable effect of an intervention, for example the activities carried out as part of a project or program. Evaluation can also be a powerful tool to document and communicate the concrete impact of an organization’s work.

When conducting an evaluation, you will need to be open to feedback and prepared to adjust strategies in response to findings. By selecting the right evaluation methods, you’ll be able to consistently increase the impact of your program or project.

The three main types of evaluation methods are goal-based, process-based and outcomes-based. Goal-based evaluations measure if objectives have been achieved (We highly recommend S.M.A.R.T. Goals). Process-based evaluations analyze strengths and weaknesses. Outcomes-based evaluations examine broader impacts and often investigate what greater good was served as a result of the program or project.

How to Choose between Different Types of Evaluation Methods

Choosing the right evaluation techniques for your program or project is not always obvious. This is especially true if you and your organization are new to evaluation. When selecting which types of evaluation methods to use, consider:

How will you utilize findings? For example, will evaluation results inform internal strategies, or will you primarily use results to communicate to external stakeholders? We recommend selecting evaluation methods that allow you to do both.
What information will you need to make decisions based on evaluation results? For example, will your evaluation method need to capture the breadth of your project’s reach (such as number of people reached with information and resources), the depth of its effects (such as the impact on participants based on your intervention), or both?
What information can feasibly be collected and analyzed? Not all information will be available to you. For example, if you are attempting to increase access to and utilization of healthy food in your community, you may be able to track broad trends of food availability in a specific geography and self-reported shopping behaviors of a subset of individuals. But you may not be able to quantify how many households changed their food shopping behavior pre- and post-intervention.
How accurate will the information be? For example, if you are considering pre- and post-intervention surveys, will behavioral changes be self-reported, or will you be analyzing participants’ knowledge on a particular topic?
Will the information be credible to top donors or management? You will want to consider the type of evaluation methods that your peers are using. For example, if you are an advocacy organization looking to create policy change, how can you show impact over a 6-12 month period when policy changes generally happen over a much longer, multi-year timeline?

Common Types of Evaluation Methods

Funding for Good has worked with hundreds of organizations to design programs, develop effective evaluation plans, and communicate both plans and results to donors and other stakeholders.

Some of the most common types of evaluation methods organizations use include:

Testing

Pre and Post Test
Test Against Control Groups

Participation

Attendance
Completion
Certificates
Follow-On Tracking

Data Collection

Surveys
Questionnaires
Interviews
Checklists
Feedback forms

Financial Reports

Cost to budget
Cost per unit of service
On time on budget

Performance

Grades
Graduation
Drop in recidivism
Job placement
Permits, inspections, certifications

Subjective (Qualitative)

Journals
Testimonials
Observations
Photographs
Clippings

What is an Evaluation Plan?

Once you have identified the purpose of your evaluation and the types of evaluation methods you will use, it’s time to create an evaluation plan. At Funding for Good, our definition of an evaluation plan is:

An evaluation plan is a written document that outlines the types of interventions you will undertake, the outcomes you hope to achieve, the criteria you will use to evaluate success, and the evaluation methods you intend to utilize.

A comprehensive evaluation plan should also include:

Methods you will use to collect and monitor data for your evaluation (including who will be responsible for data collection)
Methods you will use to analyze the evaluation information you collected
When the evaluation will be conducted (will you conduct evaluations quarterly, annually, or at project milestones)
Who will be conducting the evaluation (especially their qualifications and credentials)
How you will use the evaluation findings (specify if there is an internal process for reviewing the results and determining how to utilize them)
How and to whom you will disseminate the results of the evaluation (consider too how you will communicate if your evaluation does not show the hoped-for outcomes)

As you prepare an evaluation plan, it’s also important to ensure that your internal stakeholders—meaning board and staff leadership—bring the right mindset.

The end-goal of evaluation isn’t simply a written report. Instead, the goal is creating more effective programming. This means your leadership team must be willing to engage in collective learning and continuous programmatic improvement. It also means committing to receiving and acting upon honest assessments of programs and projects, including those that fail to demonstrate the level of desired impact.

Strengthen Evaluation with Strategic Planning

When designing a new project or program, it’s important to consult your organization’s strategic plan. Here’s how we define a strategic plan:

A strategic plan is a roadmap for where an organization is going, how it will get there, and specific ways to determine if you have “arrived” at the destination.

Your strategic plan will outline your organization’s goals, objectives, strategies, and programs over a 3-5 year period. Using your strategic plan as a planning tool, you can ensure that any new project and program is aligned with your overall goals, will be able to garner strong internal investment from board and staff, and that your organization has the current resources and capacity to sustain the new work.

Your strategic plan will also be grounded in measurable objectives which can be used as a resource in creating your project or program evaluation plan. The more you are able to align evaluation methods across programs, the more economies of scale you will create for the hard work of monitoring and assessing program impact.

If your organization doesn’t have a strategic plan, we recommend investing in one as soon as possible. Organizations with a written strategic plan double their chance of success.

Using Evaluation for Donor Engagement

Creating an evaluation plan means your organization has a powerful new tool available: the ability to tell a compelling story about your work. And you don’t have to wait until your evaluation process is complete to start.

Once you have an evaluation plan in place, you can communicate with donors, members, and other stakeholders about how and why you are undertaking an evaluation. This sends the message that you are serious about increasing the impact of your programs. It also shows that you are committed to using donor dollars in the most effective way possible.
If you are including qualitative techniques in your evaluation, you may be able to begin sharing early results during the evaluation process. For example, say you are launching a new educational program and using participant surveys as one of your evaluation methods. Each survey could include an open-ended question for participants to share reflections on the value of the program and how it could be improved. You can use insightful participant responses in your updates to donors about “what we’re learning so far from our evaluation.” Even critical responses can demonstrate how you are listening to community feedback in order to continually improve programming.
Finally, once your evaluation is complete, you can share your results in a variety of ways. For example, you can prepare donor briefings, written evaluation reports, best practices publications, or membership and donor emails about the impact that your evaluation revealed.

Again, even if your program or project didn’t meet all its goals, there is still a compelling story to tell. For example, perhaps one activity exceeded expectations while the rest underperformed. You could share the results along with a message about how you are going to invest more deeply in what worked best and experiment further on what didn’t.

Learn more: Fundraising Fundamentals: Mighty Metrics

Designing Effective Programs and Program Evaluations

Funding for Good recommends developing your evaluation plan alongside your initial project and program design. In doing so, you will be able to embed evaluation methods into your program, rather than tacking them on after your initiative is already in process.

Often, we see organizations scrambling to develop an evaluation plan after a project is already in process. This can hinder your ability to measure the impact of your efforts.

For example, if you are developing a new community STEM education program, you may want to embed pre- and post-program testing and surveys into your project. You will also need to ensure that staff work plans account for this evaluation activity. If you wait until the program has already started, you will likely miss key pre-intervention data, and staff may spend extra time trying to figure out how to adjust existing workflows.

When designing your program or project and creating your evaluation plan, key questions you should consider are:

Program and project design:

What is the primary purpose of the program or project? For example, is your program focused on education, outreach, prevention, career readiness, economic development, etc.
What are the goals and objectives of the program or project? How to Write Quality Goals and Objectives
Is your program or project part of your strategic plan? It’s critical that all new projects and programs tie into your organization’s long-term vision and mission. Otherwise, you risk mission creep.
Is there a need for this program? Describe the need using statistics or community reports. Notice what data and metrics are used to document need, as these will be invaluable when you set out to evaluate impact.
Who is your target population? We recommend getting as specific as possible, as you will need to know who you are serving in order to measure the effect of your work.
How will your program design help meet the community need you identified?
Will the program model be evidence based? Evidence-based programming means that the program design or intervention has been tested previously and found to have a positive impact.

Program and project implementation:

What will be the frequency and intensity of your program? For example, will you be conducting one round of community outreach or will outreach be part of your day-to-day efforts? Do you want to serve 100 or 10,000 community members?
How will your target population be invited, registered, and retained?
Who will organize and implement your new program? Do you have adequate staff and/or volunteer capacity to support your effort? You will want to ensure that your program’s goals are aligned with your organizational resources. If your program is under-resourced, it may be a challenge to meet more ambitious objectives even if the program design is solid.
What are the true costs associated with program implementation and reporting? You will need to consider personnel (salaries, fringe such as FICA, healthcare, etc.), program materials (such as curriculum creation and dissemination, as well as supplies), marketing materials, travel and transportation costs (for staff and participants), staff development or certifications needed, equipment, space, consultants and independent contractor services (this range from event management to graphic design to evaluation experts), insurance, liability and legal costs, and other miscellaneous expenses. Here’s a tip for those of you creating program and project budgets for grants: Be “Generally Specific”
What risks may be posed by the project? This could include legal, liability, or reputational risks. What policies and procedures will need to be implemented within your organization or project to reduce risks?
What resources are critical for both the short-term and long-term success of the program? You should consider financial resources (will you be charging fees for service, will you seek grant funding, will you be soliciting individual donors), program materials or services, and community partnerships and collaboration.

Program and project evaluation:

What evaluation measures will you use to measure program success? Consulting the list of evaluation methods above, you may want to combine both qualitative and quantitative metrics.
Who will track program data and report to stakeholders and donors? What tools will you use to gather information (written surveys, digital surveys, databases, etc.) and who will be responsible for ensuring data is entered in a timely manner? Will you be relying on staff or volunteers for data management, analysis and reporting? Will you be working with an outside evaluation consultant or conducting all evaluation activities in-house? What additional expertise will you need to ensure you are able to fulfill your evaluation activities?
How will the organization share program impact with the greater community? How frequently will you share impact? This report of the MOVE Culver City pilot program is a great example of how robust evaluation can be used to communicate about the impact of an initiative, raise awareness about the program, and build support for future investments.

Program and project sustainability:

How will the organization sustain this new program? How will the program support your organization’s long-term mission and vision?
What is your organization’s current and long-term capacity? Are your funding streams diversified? Do you have general operating support as well as project support grants and contributions? Does your organization have the appropriate financial systems and procedures in place?
Do you have funders specifically for this project or program? If so, how long are the commitments for and how much of the project or program budget is covered? If you don’t have anchor funders, do you have a fundraising strategy in place? If you don’t secure adequate funds, what will happen to the program?
Does the Board of Directors agree there is a need for the program and commit to supporting it?
Are the project or program goals reflected in your organization’s current strategic plan? If your organization doesn’t have a strategic plan, what is your timeline for developing one?

Read more: Questions to Ask When Designing a Nonprofit Program

FAQs

What are evaluation methods funding for good? ›

The three main types of evaluation methods are goal-based, process-based and outcomes-based. Goal-based evaluations measure if objectives have been achieved (We highly recommend S.M.A.R.T. Goals). Process-based evaluations analyze strengths and weaknesses.