we were thrilled to welcome world renown Results Based AccountabilityTM founder Mark Friedman to Vermont for three days of training and policy discussions with over 300 agencies, legislators, and funders. Mark expertly guided everyone through his methodology for answering the three questions:
1) How much did we do;
2) How well did we do it; and
3) Is anyone better off?
Building Skills to Monitor & Evaluate Performance & Outcomes
Vt 2012 09 supplementary slides
1. RBA
Results-Based Accountability
TM
VERMONT SEPTEMBER 2012
SUPPLEMENTARY SLIDES
The Fiscal Policy Studies Institute
www.raguide.org www.resultsaccountability.com
Book - DVD Orders
amazon.com
resultsleadership.org
2.
3.
4.
5.
6.
7.
8. Service: __________________________________
School Hospital Job Training Fire Department
How much did we do? How well did we do it?
Primary customers Unit cost
# students trained
persons
patients
Workload ratio
Primary activity % of ___x___ that
# job courses
alarms
diagnostic
hours of tests happen on time
responded
instruction to
Is anyone better off?
If your service works really well,
how are your customer's better off?
# patients whofully
students
persons to get
% fires keptwhoroom
graduate
jobs
recover
of origin
9. Child Welfare
Quantity Quality
How much did we How well did we do
do? it?
# of
Effort
Workload ratio:
children in No. of children
foster care per worker
Is anyone better off?
# of % of
children in children in
Effect
permanent plcmt permanent plcmt
within 6 months within 6 months
or entering care of entering care
10. Juvenile Justice
Quantity Quality
How much did we How well did we do
do? it?
Effort
Number of Percent of
children in children in
custody community based
(vs. institutional)
care
Is anyone better off?
# of % of
children exiting children exiting
Effect
custody with no custody with no
repeat offence in repeat offence in
6 months 6 months
11. Personnel Department
Quantity Quality
How much did we How well did we do
do? it?
Effort Number of Average
Applicants Recruitment
Processed Period
Is anyone better off?
Workforce
Workforce Turnover Rate
Effect
# New Hires (non-promotions)
Customer *
Satisfaction
* Percent of supervisory staff who report that personnel provides the support they need to do their jobs.
12. Information Technology (MIS)
Quantity Quality
How much did we How well did we do
do? it?
Effort Average response time
Number of to service requests
IT projects % Projects on schedule
on budget
Is anyone better off?
Rate of
# Unscheduled
Effect
Downtime
Customer
# *
Satisfaction
* Percent of staff who report that MIS provides the support they need to do their jobs.
13. Audit
Quantity Quality
How much did we How well did we do
do? it?
Effort
Number of Percent of audits
audits completed on schedule
Is anyone better off?
# Rate of repeat findings
Effect
$ value of corrected
audit findings as
percent of total agency
# budget
14.
15.
16. Next Generation Contracting
Quantity Quality
1. Traditional purchasing
Effort
methods work fine in the
upper quadrants.
BUT
Effect
2. They break down in
the Is anyone better off?
quadrants (because of case mix
differences and perverse incentives).
WHICH MEANS
3. What we purchase in the lower quadrants are not deliverables,
but rather a RELATIONSHIP where funder and grantees work
together to maximize customer results.
17. Next Generation Contracting
Contract Provisions
Provision 1. Specify the 3 to 5 most important performance
measures (from the How well did we do it? and Is anyone
better off? categories).
Provision 2. Specify that the contractor will use a continuous
improvement process (the RBA 7 Questions).
Provision 3. Specify how the funder and contractor will work
in partnership to maximize LR customer results (quarterly
meetings using the 7 questions as the agenda).
Provision 4. Specify that the funder will work with the funding
community to simplify and standardize contracting and
performance reporting.
18. Next Generation Contracting
Contract Provisions
Provision 5: : Clear articulation of role in population/community
well-being. Language of contribution not
attribution.
Provision 6: 10% for quality management and administration.
Provision 7: Multi-year funding using 3 year rolling contracts
Provision 8: Use of targets that are fair and useful.
Provision 9: Fund flexibility and virtual funding pool: transfer
of up to 10% across line items and program lines.
19. Resources
www.raguide.org
www.resultsaccountability.com
RBA Facebook Group
Book - DVD Orders
amazon.com
resultsleadership.org
Notas do Editor
Introduction and the difference between population and performance accountability : We are going to talk about two different kinds of accountability: Accountability for whole populations , like all children in Los Angeles, all elders in Chicago, all residents of North Carolina. This first kind of accountability is not the responsibility of any one agency or program. If we talk for example about “all children in your community being healthy,” who are some of the partners that have a role to play? Notice that the traditional answer is “It’s the health department.” It’s got the word health in it and so it must be the responsibility of the health department. And yet one of the things we have learned in the last 50 years is that the health department by itself can’t possibly produce health for all children without the active participation of many other partners. And that’s the nature of this first kind of accountability. It’s not about the health department. It’s about the kind of cross community partnerships necessary to make progress on quality of life for any population. Now the second kind of accountability, Performance Accountability , is about the health department. It’s about the programs and services we provide, and our role as managers, making sure our programs are working as well as possible. These are two profoundly different kinds of accountability. We going to talk about how to do each one well and then how they fit back together again.
This chart shows in detail the different types of measure we typically find in each quadrant, and the measures that go with the three basic categories of performance measurement: How much did we do? How well did we do it? Is anyone better off? In the upper left, How much did we do? Quadrant, we typically count customers and activities. In the upper right, How well did we do it? Quadrant, there are a set of common measures that apply to many different programs. And there is a set of activity specific measures. For each activity in the upper left, there is one or more measures that tell how well that particular activity was performed, usually having to do with timeliness or correctness. In the lower quadrants, Is anyone better off? We usually have # and % pairs of the same measure. And these measures usually have to do with one of these four dimensions of better-offness: Skills/knowledge, Attitude, Behavior and Circumstance. For each of these measures, we can use point in time measures or point to point improvement measures.
Examples of measures for child welfare.
Examples of measures for a juvenile justice.
Examples of measures for a personnel department.
Examples of measures for an Information Technology division.
Examples of measures for an Information Technology division.
Welsh Epilepsy Unit case study was also posted on the same page and that's the one showing benefits such as:" • The average length of time from seizure to a confirmed diagnosis has decreased by 81 days from 111 days to 30 days • The number of patients who have been seen by a specialist within the NICE guideline of two weeks has increased from 35% to 61% • The average waiting time to see a specialist has decreased from 22 days to 11 days • The number of admissions following a seizure have decreased from 5 a month to 2 a month on average"
Welsh Epilepsy Unit case study was also posted on the same page and that's the one showing benefits such as:" • The average length of time from seizure to a confirmed diagnosis has decreased by 81 days from 111 days to 30 days • The number of patients who have been seen by a specialist within the NICE guideline of two weeks has increased from 35% to 61% • The average waiting time to see a specialist has decreased from 22 days to 11 days • The number of admissions following a seizure have decreased from 5 a month to 2 a month on average"
We have lots of examples of well-established standards in the upper right (How well did we do it?) quadrant, because we know what good service delivery looks like. But standards in the lower right (Is anyone better off?) quadrant are almost always experimental. This is partly because of the different mixes of easy and hard cases in different caseloads or workloads.
We have lots of examples of well-established standards in the upper right (How well did we do it?) quadrant, because we know what good service delivery looks like. But standards in the lower right (Is anyone better off?) quadrant are almost always experimental. This is partly because of the different mixes of easy and hard cases in different caseloads or workloads.