This presentation describes the findings from The California Endowment Clinic Consortia Policy and Advocacy Program Evaluation, including: activities to increase clinic consortia advocacy capacity; expanded decision maker support for key policies benefiting clinics and their target populations; and expanded access to health care care for vulnerable populations. Summary findings and lessons for evaluators are included.

1. Evaluating The California Endowment Clinic
Consortia Policy and Advocacy Program:
Lessons for Evaluators
Annette L. Gardner, PhD, MPH
Sara Geierstanger, MPH
Claire Brindis, DrPH
Philip R. Lee Institute for Health Policy Studies
University of California, San Francisco
November 4, 2011
University of California
San Francisco

2. Presentation Objectives
 UCSF evaluation design and findings by
outcome
 Monitoring advocacy
 Assessing impact
 Relevance of findings for
researchers/evaluators, advocates and funders

3. The California Endowment
Clinic Consortia Policy and Advocacy
Program
Component of Community Clinics Initiative
(CCI)
Launched in 2001 and refunded two more
Rounds in 2004 and 2007, an investment of
$28 million
14 local/regional clinic consortia and 5
statewide clinic associations representing 800+
primary care clinics
Goals:
1) Increase services to underserved populations
2) Increase access through improved clinic efficiencies

4. Theory of Change:
“Resources and staffing dedicated to policy advocacy and
TA will increase the collective influence of clinics and
strengthen a broad base of long-term support of clinic
policy issues.”

5. Grantees – 18 State, Regional, Clinic
Consortia
Alameda Health Consortium (AHC)
Alliance for Rural Community Health (ARCH)
California Family Health Council (CFHC)
California Planned Parenthood Education Fund (CPPEF)
California Primary Care Association (CPCA)
California Rural Indian Health Board (CRIHB)
Capitol Community Health Network (CCHN)
Central Valley Health Network (CVHN)
Coalition of Orange County Community Clinics (COCCC)
Community Clinic Association of Los Angeles County (CCALAC)
Community Clinic Consortium Serving Contra Costa and Solano
Counties (3C)
Community Health Partnership, Inc. (CHP)
Council of Community Clinics (CCC)
North Coast Clinics Network (NCCN)
Northern Sierra Rural Health Network (NSRHN)
Redwood Community Health Coalition (RCHC)
San Francisco Community Clinic Consortium (SFCCC)
Shasta Consortium of Community Health Centers (SCCHC)

6. The California Endowment
Clinic Consortia Policy and Advocacy
Program
2001 - 03 2004 - 06 2007 - 09
Advocacy
capacity
Technical
Assistance
Partnerships

7. Evaluation Design
2001 - 03 2004 - 06 2007 - 09
Methods
Evaluation
TA
Evaluation
Partnership

8. Policy Change Logic Model:
Measures and Tools
Outcome 1 Outcomes 2, 3 Outcome 4 Outcome 5 Outcome 6
Increase Increased
policymaker Strengthened services for Improved
Expand
awareness clinic underserved health
advocacy
And support operations
capacity and Outcomes
for clinic policy
uninsured
Issues, funding
* Annual Advocacy * Decision Maker
Activities Worksheet Awareness Surveys (2) * Clinic Focus * Policy Case * Policy Case
* Annual Grantee * Interviews with Media Groups (2) Studies (3) Studies (3)
Interviews Representatives (2) * Grantee Case * Grantee Case * Grantee Case
* Sustainability * Policy Outcomes Wrksht Studies (17) Studies (17) Studies (17)
Survey * Funding Secured Wrksht

9. Outcome 1: Expand Advocacy
Capacity
Organizational capacity gains:
 Staffing
 Ability to maintain advocacy
 Technical acumen
Advocacy activities:
 Strategically deploy advocacy “toolkit”, e.g.,
Working with the media
Educating/working with elected officials pre/post passage
Pursuing/leveraging partnerships with stakeholders
Mobilizing grassroots advocates

10. Tool: Annual Assessment of Advocacy
Capacity and Outcomes
13 grant- Feder State Local Effective Outcome 1: Outcome 2: Outcome 3:
funded al (Y/N) (Y/N) ness (1 Increased Achieve a Increased
activities (Y/N) to 4, 4= clinic policy policymaker
“very funding change awareness
effective” (Y/N) (Y/N) (Y/N)
Media
advocacy
Contact
policymaker Analysis:
s directly
1) Determine percent activities at the federal,
state and local levels, and overall
Informal
contacts 2) Calculate effectiveness overall and by
with elected
officials activity
3) Determine percent of activities that
Hosting achieved each Outcome
clinic tours
for 4) Sort by grantee age (pre/post 1990 launch)
policymaker
s
Helping
draft
regulations,
rules

11. Findings: Grantee Advocacy Capacity,
2002 to 2006
Number: engaged in more grant-funded advocacy
activities--increase from 73% to 86% of surveyed
activities.
Target: consistently targeted most activities (80%)
at the local and state levels and less (60%) at the
federal level.
Effectiveness: fairly constant or 3.28 to 3.36 (4
equaled “very effective”
Grantee type: newer (post-1990) consortia were
equally effective in achieving advocacy goals by
2006.

12. Outcome 2: Increase
Policymaker Awareness
Whoever controls the flow of
information dictates our
perceptions and perspectives;
whoever controls the news
shapes our destiny.
- George Clinton
Musician and Funk Philosopher

13. Evaluating Grantee Work With the
Media
Effectiveness of Advocacy Activities Worksheet: # of grantees
media compared to using media at federal, state, local levels;
other advocacy achievement of three Program outcomes
tactics
Partnerships with # and type of media request for information;
media Number and type of media outlets
Coverage secured # of stories in newspapers, radio
Placement of stories, e.g., page, column
Newspaper article type (op-ed, Letter to the
Editor)
Presence of policy Inclusion of specific issues, e.g., role of
issues in the clinics, patients served
media Framing of issues - content analysis
Outreach to media # of press releases developed and distributed
(grantee tracking)

14. Findings-- Media Perspective:
Ef fectiveness of Grantee Strategies
to Obtain Media Coverage
Press Releases
Timing messages
Provide data
2006
Access to clinic 2004
staff/patients
Prompt responses
to media inquiries
Relationships
w/media
0 1 2 3 4 5
5=very effective

15. Policymaker and Community Leader
Familiarity with Grantee and Clinic
Activities
3.7
3.3 3.4 3.5
3.1
2.7
2003 Policymakers
2004 Community
Leaders
Grantee Grantee Clinic
Activities Activities
4 = high familiarity

16. Educating AND Working with
Policymakers
 Advocacy contributes to policymaker
awareness. The challenge is to position yourself
as a credible partner and work with decision
makers
 Watch for opportunities, e.g., bipartisan
support
 Leverage the partnership with decision makers
pre/post passage of policies
 Maintenance of effort required - ongoing
education and visibility

17. Outcome 3: Increased Policymaker Support for
Clinic for Clinic Funding
Tool: Annual Grantee Policy Outcomes Worksheet
Policy Bill Grant-funded policy Target populations Outcomes as of
targeted Number advocacy activities to be affected by 12/07. Choose from
and/or undertaken in 2007. policy. Choose list: a) Passed - signed
title? Choose from list: a) from list: a) clinics; in to law; b) Vetoed; c)
media advocacy; b) in- b) uninsured; c) low- Pending - still being
person meetings with income populations; considered by decision
decision makers; c) d). clinic consortia; e) makers; d) funding
drafting of rules, other special secured; e) Other: list.
regulations; d) materials populations; f) Other:
development; e) grass list
roots activities; f) Other:
list
Federal
Analysis:
State 1. Count number policies and of grantees that
mentioned each policy
2. Code advocacy activities, target populations
Local
3. Code and count outcomes, e.g., “passed”, “vetoed”,
“pending”

19. Evidence of Policymaker Support –
Funding Secured by Grantees (ROI)

20. Funding Secured, by Source

21. Outcomes 4 & 5: Strengthened Clinic Operations
and Increased Services for The Underserved and
Uninsured
Findings: Evidence that more Californians have insurance
coverage and are using the services to which they have
access

22. OUTCOME 6: Improved Health Outcomes for
Targeted Populations
Achievements in three areas :

23. Summary of Evaluation Findings
Advocacy as staffing and toolkit to be deployed
strategically – gains after 2-3 years
Strategic involvement in policymaking process -
desired policy change facilitated by maintenance
of effort, technical acumen and credibility
Crafting working relationships with decision
makers - “pays off”
Achieving a policy change – gains to member
clinics and their target populations
Partnerships with member clinics – critical to
achieving a workable solution

24. Policy Change Logic
Model
Increase Increased
Expand policymaker Strengthened services for Improved
Advocacy awareness clinic underserved health
operations and outcomes
Capacity and
uninsured
support

25. Suggestions for Evaluators
Address different information needs of advocates,
funders, evaluators
Combine rigor with creativity
 Mixed methods
 Triangulate
 Clarify what you’re evaluating re “advocacy” and
“advocacy capacity”
Create a working partnership with advocates,
funder
Communications are key

26. Thank you! For More Information
Annette Gardner, PhD, MPH
Philip R. Lee Institute for Health Policy Studies
University of California, San Francisco
3333 California St., Suite 265
San Francisco, CA 94118
Phone: (415) 514-1543
Email: annette.gardner@ucsf.edu
http://ihps.medschool.ucsf.edu/News/california_endowment.aspx
University of California
San Francisco