Evaluating the Impact of the Policy Strategy
The policy process is complex, dynamic, and rarely linear. Evaluation can inform all domains of CDC’s Policy Process.[7] Generally, for the purposes of implementing the policy component of the Dating Matters Program you want to distinguish between evaluating the process, impacts, and outcomes related to informing policy and evaluating the process, impacts, and outcomes related to the implementation of a policy.
Evaluate your Efforts to Inform Policy
Once you have developed your policy plan, you can determine specific benchmarks, milestones, or indicators that will demonstrate your progress from your baselines (i.e., where you started) towards your goals and outcomes.[8]
There are a number of indicators and outcomes you can monitor or track to demonstrate your progress in informing policy:
-
Development of relationships and partnerships (e.g., numbers and kinds);
-
Progress of collaborations, including your participation in coalitions or your efforts at coalition building;
-
Development and delivery of key educational messages through identified channels (e.g., how many times does your message show up and where, or how many e-mails sent out by your organization or on your behalf by other organizations);
-
Number of surveys, focus groups, and meetings conducted;
-
Number of meetings with key stakeholders or decision-makers and the results of those meetings; and
-
Statements made in support of the evidence or science by the media, public, or stakeholders.
At significant points in your efforts, such as at the end of a planning period or completion of a series of activities, examine whether you completed your activities and reached your goals and outcomes.[9] See the following for a list of key questions to consider when developing your evaluation plan.
Download Printable PDF of Slider Questions
Evaluate the Impact of the Policy
After a policy is implemented, it is important to evaluate its short-, mid-, and long-term outcomes. The outcome evaluation should determine if the policy goal was achieved and if there were any positive or negative unintended consequences. As you plan your evaluation, consider what types of information will tell you if you have accomplished your goals, who will collect the information for you, and when the data collection and analysis will be completed.
Exactly what you measure will depend on the goals of your policy efforts and the resources you have for evaluation. One place to start helping you think through what you want to measure is a set of six outcome categories compiled from multiple sources by the Annie E. Casey Foundation (Six Outcome Categories of Policy)[10]. Some of these outcomes represent interim steps and infrastructure that create the conditions for successful policy strategies in society and the community environment. Others reflect the end game: inclusion of best available evidence in a policy, funding, or enforcement in various jurisdictions (e.g., local, state, federal).[10]
Once you decide on the outcome categories you wish to measure, you can move to determining specific elements to examine. These are sometimes called benchmarks, milestones, or indicators, and show your progress from your baselines (i.e., where you started) towards your goals and outcomes.[11] Along with identifying what these checkpoints are, you also need to think about what sorts of information will tell you whether you are getting there or not, who will collect the information for you, and when you do the data collection.
Six Outcome Categories of Policy
“The decisions you make now about what to evaluate are not cast in stone. Given the time it may take to actually achieve your policy strategy goals, you may find it necessary to regularly update your evaluation strategies to reflect changes in the community context”.[12]