Search Form Controls

Injury Prevention & Control: Division of Violence Prevention

ShareShare

This website is part of

VetoViolence Logo
Hide Toolbar
Skip Navigation | Accessibility Version

Evaluaction Logo

Welcome! Interested in learning about evaluation and how to apply it to your program? Check out this brief video and put EvaluACTION to work for you.

Focus Video
Download Flash if video doesn't play.

Importance of Evaluation

Graph illustrating generic increaces

Hear from the Experts

as they describe the importance of evaluation.

  • Select audio of expert Howard Spivak
  • Select audio of expert Jim Hmurovich
  • Select audio of expert Phyllis Millspaugh
  • Select audio of expert Patricia Cook-Craig

Howard Spivak

Deputy Director and Chief of Staff
National Institute of Justice
(transcript)

Jim Hmurovich, BA, MS Ed

President & CEO, Prevent Child Abuse America (transcript)

Phyllis Millspaugh

Program Administrator
Kentucky Cabinet for Health and Family Services
Clinical Consultation Branch (transcript)

Patricia Cook-Craig

Associate Professor of Social Work
University of Kentucky (transcript)

The Number 1Add Value
TO YOUR WORK

Depending on your view of evaluation, you may see it as invaluable or you may see it as a way to highlight your program's flaws and nothing more. However, evaluation may also uncover areas in which your program truly shines. Or you may find some areas for improvement. When you know what's not working well, you know what to fix. Whether your program is on the right track, needs some help, or could benefit from a reboot, improving your evaluation efforts can help. It's about doing the work as well as you can, whether you're going from good to better—or from better to the best.

money going into a piggy bank

Hear from the Experts

as they discuss using limited resources wisely.

  • Select audio of expert Valerie Spiva
  • Select audio of expert Deborah Gorman Smith
  • Select audio of expert Jim Hmurovich

Valerie Spiva Collins

Training and Technical Assistance Supervisor
FRIENDS National Resource Center (transcript)

Deborah Gorman Smith

Director, Chicago Center for Youth Violence Prevention
Professor, University of Chicago (transcript)

Jim Hmurovich, BA, MS Ed

President & CEO, Prevent Child Abuse America (transcript)

2

Confidence
IN YOUR INVESTMENT

If you are going to invest in a product or program, you want to be confident in your investment, right? Like those infomercials you see on TV-will you really lose 30 pounds in two days? Probably not. What about your program? You want to be able to justify using your resources well: Are you developing your program's activities with appropriate use of resources such as budget and staff time? Does the value or benefit of achieving your program's goals and objectives exceed the cost of producing them?1 Demonstrating you are on the right path and being financially responsible—with good theory supporting your efforts—will help increase your credibility. Evaluation provides checks and balances to help you and your program stay on track, with the evidence to back it up.

trophy

Hear from the Experts

as they explain why accountability matters.

  • Select audio of expert Jim Hmurovich
  • Select audio of expert Dan Whitaker
  • Select audio of expert Valerie Spiva Collins
  • Select audio of expert Jim Hmurovich

Jim Hmurovich, BA, MS Ed

President & CEO, Prevent Child Abuse America (transcript)

Dan Whitaker

Professor of Public Health
Georgia State University (transcript)

Valerie Spiva Collins

Training and Technical Assistance Supervisor
FRIENDS National Resource Center (transcript)

Jim Hmurovich, BA, MS Ed

President & CEO, Prevent Child Abuse America (transcript)

3
ACCOUNTABILITY

A solid evaluation plan can help you feel confident in your investment, and will help you remain accountable. It's crucial to long-term success that you know if your program is working as intended and where any weaknesses are so you can address them. You not only have an obligation to ensure your organization can stand behind the work you're doing, but it's also critical for building and maintaining successful relationships. It's important that you're able to showcase your program to the people and communities you serve and who have a vested interest in your success. And if your evaluation uncovers a success story, an example others can learn from and make their own, even better.

 

1Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide http://www.cdc.gov/eval/guide/index.htm

2Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide http://www.cdc.gov/eval/guide/index.htm

Myths

Do you think evaluation is too hard, too expensive, too time-consuming for you or your organization to handle? Not so fast. Transcript.

Myth 1: It's too complex.

Evaluations can be as different as the programs evaluated. You decide how complex your evaluation will be. It is often a good idea to start small. You have probably already collected valuable information already that could be used in your evaluation.

Myth 1:

It's too
complex. Evaluations can be as different as the programs evaluated. You decide how complex your evaluation will be. It is often a good idea to start small. You have probably already collected valuable information already that could be used in your evaluation.

Click to Solve

Myth 2: It's too expensive.

Ask yourself-can you afford not to do evaluation? Opting out of evaluation doesn't necessarily mean you're saving money. Program evaluation can show you how effectively you are using your resources, and it can be a way to 'sell' your activities or components to stakeholders. Don't forget that you can start small and scale up as you're able. You already track some information for funders and other stakeholders. To start, think about what you can do with that.

Myth 2:

It's too expensive. Ask yourself-can you afford not to do evaluation? Opting out of evaluation doesn't necessarily mean you're saving money. Program evaluation can show you how effectively you are using your resources, and it can be a way to 'sell' your activities or components to stakeholders. Don't forget that you can start small and scale up as you're able. You already track some information for funders and other stakeholders. To start, think about what you can do with that.

Click to Solve

Myth 3: It's too time-consuming.

Time is another scarce resource. So wouldn't you like to know you are using it wisely? Yes-it will take some effort to create a plan and determine how much time you have to spend on evaluation. But on the other hand, as a result of doing an evaluation, you may also find out how to best use that limited time.

Myth 3:

It's too
time-
consuming.Time is another scarce resource. So wouldn't you like to know you are using it wisely? Yes-it will take some effort to create a plan and determine how much time you have to spend on evaluation. But on the other hand, as a result of doing an evaluation, you may also find out how to best use that limited time.

Click to Solve

Myth 4: I don't have enough staff.

Program evaluation often needs to include many types of stakeholders. The good news is everyone won't need to commit the same amount of time and energy to the process. Often a small team-one that is consistently and intensively involved and engaged in the evaluation process-can be an effective way to keep the process manageable.

Myth 4:

I don't have enough staff.Program evaluation often needs to include many types of stakeholders. The good news is everyone won't need to commit the same amount of time and energy to the process. Often a small team-one that is consistently and intensively involved and engaged in the evaluation process-can be an effective way to keep the process manageable.

Click to Solve

Myth 5: I can just tack it on at the end.

It's a common misconception—and, in some cases, an accepted practice—that evaluation is something you do at the end of programming. Although ending violence is a long-term goal of much of your programming, reaching that goal means evaluating your efforts and using that information to continuously improve performance. Even if you are still in the program planning process, you can evaluate who is involved and why.

Myth 5:

I can just
tack it on
at the end. It's a common misconception—and, in some cases, an accepted practice—that evaluation is something you do at the end of programming. Although ending violence is a long-term goal of much of your programming, reaching that goal means evaluating your efforts and using that information to continuously improve performance. Even if you are still in the program planning process, you can evaluate who is involved and why.

Click to Solve

Myth 6: My resources are better used providing activities.

No one will argue that providing activities is a bad way to use your resources. However, a program evaluation shows how well you are using those resources. Are the activities and resources working? Wonderful-now you know for sure. Do they need to be reworked? Have you been duplicating efforts without realizing it? This is also great information to have, so you use your financial resources most efficiently and can identify where to make improvements.

Myth 6:

My resources
are better
used providing activities.No one will argue that providing activities is a bad way to use your resources. However, a program evaluation shows how well you are using those resources. Are the activities and resources working? Wonderful-now you know for sure. Do they need to be reworked? Have you been duplicating efforts without realizing it? This is also great information to have, so you use your financial resources most efficiently and can identify where to make improvements.

Click to Solve

Myth 7: It will make me look bad!

You may find that some of what you are doing could be done better. That's not the end of the world. Learning organizations—those that learn from past experience to continually improve programming—understand that success and failure provide constructive feedback to build upon. Being flexible in this way will show your program is one that changes with the times and is committed to continuous improvement.

Myth 7:

It will make me look bad! You may find that some of what you are doing could be done better. That's not the end of the world. Learning organizations—those that learn from past experience to continually improve programming—understand that success and failure provide constructive feedback to build upon. Being flexible in this way will show your program is one that changes with the times and is committed to continuous improvement.

Click to Solve

e.val.u.a.tion

[ih-val-yoo-ey-shuhn]
noun

CLICK HIGHLIGHTED BOXES TO VIEW DEFINITION

Program Evaluation is a
systematic examination
of the worth, merit, or
significance of an object
or set of organized activities intended to achieve a specific result.3

 

3Framework for Program Evaluation in Public Health (MMWR 9/17/99)

People of Evaluation5

Stakeholders, as the name implies, are people invested in what happens with the program. This label covers a range of individuals and organizations. Program staff members have a professional stake in the organization's success, but many have a connection and passion for the work that make the investment much more personal. Other stakeholders may have a financial investment in the program if they provided funds to support the program, but the individuals and communities that experience the program are also stakeholders because programmatic changes could improve or diminish the components they receive.

CLICK TO EXPLORE ROLES

Survivors

They provide a deeper and broader understanding of the issue. Violence ceases to be abstract when working with them: They will ensure your work is real and relevant.

Community Members

They increase the credibility of your efforts; a more connected community can easily engage families and inspire community participation.

Neighborhood Associations/Networks

They increase credibility of your efforts; a more connected community can easily engage families and inspire community participation.

State and Local Health Departments

They often implement the plans that are central to your effort. They have access to data, specific skill sets, and content knowledge that can supplement your evaluation efforts.

Schools/Educational Groups

They often implement the plans that are central to your effort; this can result in better attendance, grades, and graduation rates. This can improve your relationship with the schools.

Legislators/Policymakers

They can fund/authorize continuation or expansion of your efforts—which, in turn, can help lessen the financial and health burden on the state.

Funders

They can fund your efforts. Contributions of private or public funders can help make communities better—which can help lessen the burden on the state and taxpayers.

Program Evaluators

They may be part of your staff, public health partners, or someone you hire externally. They offer a specific skill set, as they know how to systematically gather credible information to tell the story of your program and work. If you opt to bring in an external evaluator, you may benefit from the different, unbiased perspective they will bring to your efforts.

 

5Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide p. 13

Framework for Program Evaluation in Public Health

CLICK ICONS TO VIEW VIDEO CLIPS

Engage Stakeholders Describe the Program Focus the Evaluation Design Gather Credible Evidence Justify Conclusions Ensure Use Standards

Standards Help you determine if you've designed your evaluation well, in order to reach its full potential.6

1 Utility Ensures you provide useful information

2 Feasibility Ensures the evaluation is doable & practical

3 Propriety Ensures the evaluation is appropriate

4 Accuracy Ensures you provide accurate findings

 

6http://www.cdc.gov/mmwr/pdf/rr/rr4811.pdf

What Can I Afford?7,8

Evaluation plans are not "one size fits all." How you approach evaluation will depend greatly on your organization's specific needs and resources. What are your financial limits? What about time? Do you have staff you can dedicate to this effort? Even if you answer those questions with "Zero dollars, no time, no staff," that doesn't mean evaluation can't work for you. Consider what you're already doing and use that as a starting point. Keep that in the back of your mind as you use the chart below to quickly identify what's feasible based on your situation:

Evaluation Plan Costs
STEP ACTION No Cost Low Cost Moderate CostModerate Cost High CostHigh CostHigh Cost
Take stock of current activities Could be as simple as making a list of what you're currently doing. checked checked checked checked
Develop a logic model There are several free options available online. checked checked checked checked
Monitor your program You can track what program funding is spent on and describe program activities. This may help to justify continued funding and build capacity. checked checked
Self-assessment Programs at this stage of development often use self-assessment or an external evaluator to observe and document program activities. You might obtain numerical counts of activities, participants, activities, or products produced. You may also assess how satisfied participants or recipients were. checked checked
More moderate evaluation efforts You might assess changes in participants' knowledge, attitudes, or behaviors and collect information about implementation of your program.

Programs at this stage of development often use self-assessment or an external evaluator to observe and document program activities. 		checked checked
Moderate to higher cost evaluation efforts With additional resources, you may add a comparison group to help you assess changes in knowledge, attitudes, or behaviors among the participants in your program. Additional costs come in when you want to measure longer term changes because you need to collect additional data. Programs at this stage of development often use external evaluators to observe and document program activities. checked checked

 

7Afterschool Evaluation 101, Harvard Family Research Project, v 1.0, 2011

8Program Manager's Guide to Evaluation, ACF, 2nd ed. 2010

Do's and Don'ts9

Click on the +'s to reveal quick tips on things you should – and shouldn't – do when you're contemplating using evaluation as a tool.

VIEW AS PDF

 

9Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide pp. 13-14
Get Adobe Reader

Recruit your Evaluaction team10

You shouldn’t feel like you have to conduct an evaluation of your program by yourself. In fact, good evaluation requires a combination of skills and is best done with a collaborative team. A broad range of skills is necessary for evaluation success so ensure members of the team that they don't necessarily need to view themselves as "evaluators.” You and your team may already be doing evaluation and not even realize it. You may already be thinking about assessing your program and your roles—whether by asking important questions, determining what information may be needed to answer those questions, or collecting the appropriate information in a thoughtful way.

Any of your colleagues who are involved with the program and critical to its success can be an asset to the team. Maybe they work in your office, are partners at a local university, or are external evaluators you hire to assist your efforts. Regardless of their job or title, building a team with people who have the following skills will help you put your plan into action. And if you’re lacking a member or two, that’s ok—people can wear multiple hats or perhaps the missing role is one you can succeed without. You’ll learn more about how to build your team as you develop your evaluation plan.

 
Swiss Army Knife

Adaptable

You're skilled at working with a variety of stakeholders

handshake

Supporter

You understand the potential pros and cons of evaluation

microscope

Analyzer

You're comfortable with data sources and analysis

scale

Equal Opportunist

You understand the importance of sharing all evaluation findings, positive or otherwise

apple

Educator

You're able to educate program personnel in designing and conducting the evaluation as well as using the results

globe

Experienced

You have experience with the type of evaluation needed

lightbulb

Innovator

You're able to develop innovative approaches to evaluation while considering your program's realities

gears

Incorporator

You know how to integrate evaluation into all program activities

ribbon

Achiever

You're skilled at using what you learn from the evaluation to make program improvements

 

 

10Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide http://www.cdc.gov/eval/guide/index.htm

Build an Evaluation Plan11,12,13

By now, you may realize that you are, in fact, already doing some evaluation of your program. Perhaps you were "thinking evaluatively" all along but didn't realize it. Or maybe it doesn't seem as daunting as it once did and you're ready to dive in. Regardless of where your current activities stand, it's not too late to begin evaluating what you're doing. Evaluation is a process, one dependent on what you're currently doing and on the direction in which you'd like go. Use our Evaluation Plan Builder to help you get started—or to better organize your efforts if they are already under way.

 

So what exactly is an evaluation plan? The Centers for Disease Control and Prevention (CDC) defines one as "a written document that describes how you will monitor and evaluate your program, as well as how you intend to use evaluation results for program improvement and decision making." You can look at it as a road map of sorts. It will help you organize your efforts, ensure you're using your financial and staff resources wisely, help you plan the next steps, and assist you in staying accountable along the way. Although the plan is outlined as a series of steps, you may find that you need to go back and forth as you develop your evaluation plan. That's okay! The CDC Evaluation Framework, which we use here, was designed to be cyclical.

It's important that you prioritize your efforts and streamline your work. It's also essential to know which specific evaluation questions to ask, so you gather information that will help determine if what you're doing is truly making a difference. An evaluation plan will help you get the necessary buy-in from stakeholders and focus your evaluation efforts so you can answer the "so what?" question about your program.

An evaluation plan will help you lay out what your evaluation will and will not entail so you can make the best use of your evaluation resources.

It's important you're able to prioritize your efforts and streamline your work. It's also essential to know which specific evaluation questions to ask, so you gather information that will help determine if what you're doing is truly making a difference. An evaluation plan will help you get the necessary buy-in from stakeholders and focus your evaluation efforts so you can answer the "so what?" question about your program.

 

Keep in mind as you work your way through this tool that you may not be able to answer all questions immediately. Or perhaps you find yourself needing to pause for a moment, so you can discuss parts of the plan with colleagues who may be able to provide the answers you need. No need to worry: if you have a VetoViolence account, you’ll be able to save your work so you can make changes later.

Our Evaluation Plan Builder will provide you with several printable documents: a logic model and your draft evaluation plan. So perhaps you want to start with the logic model and handle the rest later? No problem! You can build your plan step-by-step, moving forward when you're ready.

But before you build your evaluation plan, you'll need to recruit your EvaluACTION team. Why don't we do that now?

Ready...set...let's go!

 

GET STARTED!

Plan Builder


Download a Blank Plan

 

 

11Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide http://www.cdc.gov/eval/guide/index.htm

12http://www.cdc.gov/eval/resources/index.htm

13http://www.cdc.gov/eval/resources/index.htm

Next Steps

Now that you have a better sense of what you’re currently doing and what you would still like to know, check out these resources. They will help you take your efforts to the next level!

CDC's Program Evaluation Resources

This site provides additional evaluation resources, including step-by-step manuals, assistance with specific evaluation steps and logic models, and links to key professional associations and key journals.

Center for Disease Control

CDC's Evaluation for Improvement Manual

This manual provides step-by-step information on how to hire an empowerment evaluator with the goal of building an organization's capacity to conduct evaluations.

Center for Disease Control

CDC's Developing an Effective Evaluation Plan Workbook

This workbook provides detailed information and worksheets for planning evaluations.

Center for Disease Control

HHS's How to Hire an Evaluator

This publication offers some guiding principles for hiring an evaluator who is appropriate to your project's needs.

Department of Health and Human Services

American Evaluation Association (AEA)

This site provides information on evaluation, as well as a searchable database for finding an evaluator.

American Evaluation Association
Policies & Regulations | Accessibility Statement

Date published: Nov 12, 2014