Evaluating Policy or Advocacy Evaluation
What is policy/advocacy evaluation?
Policy and advocacy projects focus on activities to influence health care policymakers, including elected officials, health systems, payers, and providers. Activities may include public education, capacity building, network formation, relationship building, communications, and leadership development.
Evaluation of advocacy and policy projects measures changes in outputs, short-term outcomes, and long-term outcomes, as well as the impact that can be attributed to the specific policy advocacy strategy or activity.
What types of policy and advocacy outcomes can an evaluation measure?
Policy/advocacy evaluation questions can address:
Outputs: Specific advocacy activities and/or tactics, such as electronic outreach; earned media; paid media; coalition and network building; grassroots organizing and mobilization; rallies and marches; public education; public service announcements; briefing and presentations; polling; issue/policy analysis and research; policymaker education; and relationship-building with decision makers. Examples:
- For briefings and presentations: organization held 10 briefings or presentations for 300 community members.
- For public education activities: organization developed and disseminated educational materials to 5,000 community members.
Short-term outcomes: Changes in organizational capacity, collaboration, organizational visibility, awareness, attitude, and public support. Examples:
- For organizational capacity: staff increased their ability to get and use data; staff improved their media skills and contacts.
- For organizational visibility: organization received increased number of requests for reports or information (including downloads or page views); received increased number of invitations to speak as expert on a particular policy issue.
Long-term outcomes: Changes in policy development; placement on policy agenda; policy adoption; policy blocking; policy implementation; and policy monitoring and evaluation. Examples:
- For placement on policy agenda: policies formally introduced as bills, regulations, or administrative policies.
- For policy implementation: policies implemented or administered in accordance with requirements.
Impacts: Improved services, systems, and social and physical conditions. Examples:
- For improved services and systems: more programs offered; easier access to programs or services; higher quality services.
- For improved social and physical conditions: more community members have a regular source for health care services; improved cholesterol and blood pressure levels among patients with diabetes; fewer emergency department visits.
Note: It is not always feasible to measure long-term outcomes and impacts, which may extend beyond the time frame of the program or be influenced by factors beyond the control of the program.