INFORMS Rocky Mtn Presentation 03-17-11

Quantitative Approaches to
Improve Healthcare Access and Quality
Rocky Mountain INFORMS Chapter Meeting

A panel presentation, featuring the work of:
Linda LaGanga, Ph.D.1,3
Steve Lawrence, Ph.D.2
C.J. McKinney, Ph.D. Candidate1,4
Antonio Olmos, Ph.D.1
Michele Samorani, Ph.D. Candidate2

1. Mental Health Center of Denver
2. University of Colorado-Boulder
3. University of Colorado-Denver
4. University of Northern Colorado

1
Rocky Mountain INFORMS, March 17, 2011

Healthcare Issues we address

 To overbook or not?
 If we schedule them, will they come?
 What would Deming do to improve
healthcare?
 To achieve efficiency and effectiveness
of healthcare

2

Where is our work developed and
documented?
 Experience and data from
the Mental Health Center of Denver
 Community mental health center serving over 14,000
people per year
 Surveys and interviews of other healthcare
providers/systems
 Presented at INFORMS annual conferences
 Other conferences:
 Production & Operations Management Society
 Decision Sciences Institute
 Mayo Clinic Conference on OR/Systems Engineering in
Healthcare
 American Evaluation Association

3

Read more about it…

 Decision Science Journal (May, 2007)
 Journal of Operations Management
(2010, in press)
 Conference presentations and proceedings at
http://www.outcomesmhcd.com/Pubs.htm
 Research posters on the wall
opposite this room

4

Appointment Scheduling and
Overbooking
 Clinic Overbooking to Improve Patient Access
and Increase Provider Productivity
 LaGanga, L. R., & Lawrence, S. R. (2007).
Clinic overbooking to improve patient access and
provider productivity.
Decision Sciences, 38(2), 251 – 276.

5

Simple Overbooking Example

6

Model Assumptions
 Number of patients booked, K:
 E(K) = SK = N
 S = Show rate, N = target n of patients
 K = N/S
 Patients scheduled at even intervals throughout
the day
 T = N/K = S
 Inter-appointment times compressed by the show rate
 Patients arrive on time with probability S
 Patient service times deterministic
 Added variability in final version

7

Overbooking: Best Case

10 appointment slots / session; 50% show rate

Regular Time Overtime

Time Slot 1 2 3 4 5 6 7 8 9 10 11 12 13
Start Time 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6

Best Case Expected number of patients (5) arrive, evenly spaced

Arrivals A1 X2 A3 X4 A5 X6 A7 X8 A9 X10

Service D1 D3 D5 D7 D9 No overtime

Waiting No patients wait

5 patients seen; no provider idle time; no patients wait; no clinic overtime

8

Overbooking: Bunched Early


Time Slot 1 2 3 4 5 6 7 8 9 10 11 12 13
Start Time 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6

Case 1 Expected number of patients (5) arrive, bunched early

Arrivals A1 A2 A3 X4 A5 X6 A7 X8 X9 X10

Service D1 D2 D3 D5 D7 No overtime

Waiting W2 W3 W5 W7

5 patients seen; no provider idle time; 4 patients wait; no clinic overtime

9

Overbooking: Late Arrival



Time Slot 1 2 3 4 5 6 7 8 9 10 11 12 13
Start Time 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6

Case 2 Expected number of patients (5) arrive, one late arrival

Arrivals A1 X2 A3 X4 A5 X6 A7 X8 X9 A10

Service D1 D3 D5 D7 I D10 OT

Waiting No patients wait

5 patients seen; 10% provider idle time; no patients wait; 10% clinic overtime

10

Overbooking: Bunched Late



Time Slot 1 2 3 4 5 6 7 8 9 10 11 12 13
Start Time 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6

Case 3 Expected number of patients (5) arrive, bunched late

Arrivals A1 X2 A3 X4 X5 X6 A7 A8 A9 X10

Service D1 D3 I I D7 D8 D9 OT

Waiting W8 W9

5 patients seen; 20% provider idle time; 2 patients waiting; 20% clinic overtime

11

Overbooking: Extra Arrival



Time Slot 1 2 3 4 5 6 7 8 9 10 11 12 13
Start Time 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6

Case 4 More patients arrive (6) than expected (5)

Arrivals A1 A2 A3 X4 A5 X6 A7 X8 A9 X10

Service D1 D2 D3 D5 D7 D9 OT

Waiting W2 W3 W5 W7 W9

6 patients seen; no provider idle time; 5 patients waiting; 20% clinic overtime

12

Overbooking Utility Model

13

Overbooking Utility Model

 Maximize clinic “utility”
 Trade-off
 Patient access (number of patients seen)
 Average patient waiting times
 Expected clinic overtime
 Note that provider productivity is implicit in
this model

14

Relative Benefits and Penalties

  = Benefit of seeing additional patient
  = Penalty for patient waiting

  = Penalty for clinic overtime

 The values of , , and  don’t matter
 Just their ratios or relative importance

15

Utility Function

Expected utility without overbooking
U   SN
Expected utility with overbooking
U O   A  W   O
Expected net utility with overbooking

U N  U O  U   ( A  SN )  W   O
16

Utility Function Described

U N   ( A  SN )  W   O

Utility Benefit of
Less Utility Benefit Patient
Less Less Clinic
Patients that
w/o Overbooking Penalty
Waiting Overtime Penalty
“Show”

Net Utility Benefit from Overbooking (could be negative)
17

Simulation Experiments

 Five clinic size levels N
 N = {10, 20, 30, 40, 50}
 Ten show rates S
 S = {100%, 90%, … , 10%}
 Full factorial experiment
 SN = 5 × 100 = 500 factor levels
 10,000 replications per factor
 500,000 observations

18

Regression Analysis

 Results from simulation analyzed using
regression analysis
 Regression equations obtained
 Expected patient wait times
 Expected clinic overtime
 Expected provider productivity
 All coefficients significant
 R2 = 98%+

19

Sensitivity to Service Uncertainty

40
N50R90
30 N30R90
Average Net Utility

N50R50
20 N30R50
N10R90
10 N10R50
N10R10
0 N30R10
0.0 0.2 0.4 0.6 0.8 1.0 N50R10
-10
Service Time Variability

Average of net utility UN with overbooking as a function of
service time variability cs , with and (=1,  =0.5, τ =1.2)
20

Conclusions

 Overbooking is one solution for
appointment no-shows
 Can significantly improve performance
 Patient access (more patients seen)
 Clinic utility
 But with a cost
 Increased patient waiting & clinic overtime
 Good for some clinics, not for others
21

Directions for Future Work

 Scheduling policies
 Double booking
 Wave scheduling
 Optimal overbooking policies
 Current overbooking policy is not “optimal”
 Dynamic programming
 Nonlinear waiting & overtime functions
 Long waits much worse than short waits

22

Lean Options for Walk-In, Open
Access, and Traditional
Appointment Scheduling in
Outpatient Health Care Clinics
Mayo Clinic Conference on
Systems Engineering & Operations
Research in Health Care
Rochester, Minnesota – August 17, 2009
Linda R. LaGanga, Ph.D. Stephen R. Lawrence, Ph.D.
Director of Quality Systems Leeds School of Business
Mental Health Center of Denver University of Colorado
Denver, CO USA Boulder, CO USA

Additional information available at: http://Leeds.colorado.edu/ApptSched

23
Rocky Mountain INFORMS, March 17, 2011 © 2008 – Linda LaGanga and Stephen Lawrence

Data Mining in Appointment
Scheduling
Michele Samorani
PhD Candidate

Leeds School of Business, University of Colorado at
Boulder

24

Finding Patterns with Data Mining

25

DECISION TREE
Young clients are more likely to keep appointments with no reminder call

26

CLUSTERING
If clients are under the age of 26.3 years old and have low average CRM (<.5),
then they are more likely to keep their appointments

27

Using Data Mining to Schedule
Appointments

28

Overbooking – Shortcomings
Suppose service time = 30 minutes

1 1 0 1 0 1 0
Little waiting time
1 1
and no overtime

11:20

11:40

12:00
11:00
10:00

10:20

10:40
9:40
9:00

9:20

0 0 1 1 0 1 1 1
Some waiting time
1 11:20 and a high overtime
11:40

12:00
11:00
10:00

10:20

10:40
9:40
9:00

9:20

If we could predict which patients show up and which don’t, we could obtain
a more controllable schedule
29

The method
Every time a visit request arrives:
1)A classifier is used to predict if it shows or not (for each day)

2)The visit request is scheduled by solving a stochastic program through
column generation

Non‐controllable parameters Controllable parameters
•Service time •Number of slots K
•Revenue from seeing a patient •Scheduling horizon h
•Clinic overtime cost •Classification
•Waiting time cost performance:
– Sensitivity (sn)
– Specificity (sp)

How good we are at retrieving showing patients

How good we are at retrieving non‐showing patients 30

Productivity vs Punctuality
 Productivity: number of patients seen. It is increased by:

 Punctuality: 1/(overtime + waiting time). It is increased by:

31

Real world case: MHCD
Show rate Same day 1 day 2 days 3 days 4 days R
Low .74 .64 .65 .62 .61 .65
MHCD .87 .74 .75 .72 .71 .76

• Goal: Find the best policy for MHCD in terms of:
– Overbooking
– Open Access
– Data Mining

 After playing for a few hours with the MHCD data set, I can
achieve any of the following classification performances:
 sn = 0.9, sp = 0.5

 sn = 0.7, sp = 0.7

 sn = 0.6, sp = 0.8

32

Data Mining Open Access .
Policy DM OB OA
∗ ∗
(min) (min)
Overbooking
1 No No No 6.39 0.00 0.00 5.99 8 4
2 No No Yes 6.39 0.00 0.00 5.99 8 1
3 No Yes No 7.10 36.22 20.61 8.37 12 4
4 No Yes Yes 7.22 35.33 21.37 8.40 12 1
.6, .8 No No 6.82 0.00 0.00 6.44 8 5
5 .7, .7 No No 6.99 0.00 0.00 6.62 8 4
.9, .5 No No 7.36 0.00 0.00 7.00 8 5
.6, .8 No Yes 6.84 0.00 0.00 6.44 8 1
6 .7, .7 No Yes 6.83 0.00 0.00 6.43 8 1
.9, .5 No Yes 6.66 0.00 0.00 6.27 8 1
.6, .8 Yes No 7.24 21.11 14.96 7.78 12 3
7 .7, .7 Yes No 7.42 29.33 17.88 8.33 12 5
.9, .5 Yes No 7.58 40.78 23.56 9.03 12 2
.6, .8 Yes Yes 7.35 25.00 15.92 8.03 12 1
8 .7, .7 Yes Yes 7.44 28.44 18.51 8.28 12 1
.9, .5 Yes Yes 7.32 35.22 19.83 8.47 12 1
33

.
Policy DM OB OA
∗ ∗
(min) (min)
1 No No No 7.28 0.00 0.00 6.88 8 4
2 No No Yes 7.27 0.00 0.00 6.87 8 1
3 No Yes No 7.47 29.07 15.32 8.39 10 5
4 No Yes Yes 7.52 28.00 15.62 8.39 10 1
.6, .8 No No 7.49 0.00 0.00 7.11 8 5
5 .7, .7 No No 7.56 0.00 0.00 7.18 8 2
.9, .5 No No 7.85 0.00 0.00 7.47 8 2
.6, .8 No Yes 7.56 0.00 0.00 7.17 8 1
6 .7, .7 No Yes 7.59 0.00 0.00 7.19 8 1
.9, .5 No Yes 7.52 0.00 0.00 7.12 8 1
.6, .8 Yes No 7.60 20.73 13.26 8.14 10 2
7 .7, .7 Yes No 7.65 12.11 8.69 7.83 9 5
.9, .5 Yes No 7.86 15.22 9.81 8.18 9 2
.6, .8 Yes Yes 7.62 21.87 13.83 8.20 10 1
8 .7, .7 Yes Yes 7.64 24.87 14.53 8.36 10 1
.9, .5 Yes Yes 7.57 28.13 15.82 8.44 10 1
34

Conclusions
 Data mining can improve appointment scheduling in the
presence of no-shows
 If adopted in conjunction with overbooking, data mining can
either increase punctuality or productivity, depending on
sensitivity and specificity
 In case of low show rate, the advantage obtained by using
overbooking is similar to the one obtained with data mining.
 On the other hand, in case of high show rate, data mining is a
superior technique
 Interestingly, if we can achieve a decent classification
performance, using open access is the worst choice

 Thank you for your attention. Questions?

35

What about the scheduling horizon h?
 h does not have any significant impact by itself:

 But its interaction with sn and sp is significant:

36

High sensitivity classifier

Classifier

37

Driving Clinical Quality
Improvement through Mental Health
Recovery Control Charts
INFORMS Annual Meeting 2009
San Diego, CA
October, 11th, 2009
CJ McKinney, MA*
Antonio Olmos, PhD
Linda Laganga, PhD

Mental Health Center of Denver
Denver, CO, USA

* - Corresponding Author

38

Literature
 Olmos-Gallo, P.A. DeRoche, K.K. (2010, August). Monitoring Outcomes
in Mental Health Recovery: The Effect on Programs and Policies.
Advances in Mental Health (9)1, 8-16. http://amh.e-
contentmanagement.com/archives/vol/9/issue/1/ contact P. Antonio
Olmos for a copy of the publication
 McKinney, C.J., Olmos-Gallo, P.A. McLean, C., LaGanga, L.R. (August
2010). Driving Clinical Quality Improvement through Mental Health
Recovery Control Charts. Presented at the 3rd Annual Mayo Clinic
Conference on Systems Engineering & Operations Research in Health
Care, Rochester, MN. Awarded First Place for Best Poster Presentation.
 Clark, C.R., Olmos-Gallo, P.A. (2007). Performance Measurement: A
signature approach to outcomes measurement improves recovery.
National Council Magazine, 3, 26-28.
 Glover, H. (2005). Recovery based service delivery: Are we ready to
transform the words into a paradigm shift? Australian e-Journal for the
Advancement of Mental Health, 4(3),
www.auseinet.com/journal/vol4iss3/glovereditorial.pdf (accessed 15 May
2009)
 Montgomery, D. C. (2005) Introduction to Statistical Quality Control, Fifth
Edition. Hoboken, NJ: John Wiley and Sons, Inc.
 Olmos-Gallo, P. A., DeRoche, K. K., McKinney, C. J., Starks, R., & Huff,
S. (2009). The Recovery Markers Inventory: Validation of an instrument
to measure factors associated with recovery from mental illness. Working
paper

39

The Heart of Recovery Measurement

40

Act Plan

Continuous
Improvement

Check Do
41

Quality Components in Mental Health Services
Quality Components Relationship to MH Services
How well are MH services working? Are
Performance consumers improving in their recovery?

How often do we see improvements in recovery?
Reliability How consistent are the outcomes across
consumers?

How long does the consumer retain the
Durability recovery-supportive skills and tools taught
through MH services?

How does the consumer perceive our ability to
Perceived Quality
support MH recovery? Community?

Conformance Are we meeting program fidelity standards?
to Standards

42

Quality Control in Mental Health

 Allocate and reallocate clinical resources more
efficiently
 Improve and maintain clinical program fidelity
 Reduce length of treatment, while sustaining same
level of recovery and recovery supportive factors
 Increase the number of consumers served, while
decreasing burden on case managers/therapists
 Identify most effective programs based upon
consumer needs

43

Mental Health Recovery
 Concept of Recovery has taken root
around the world
 Working Definition (MHCD):

“A non-linear process of growth by which people move
from lower to higher levels of fulfillment in the areas of
hope, safety, level of symptom interference, social
networks, and activity.”
 Federal Grant (SAMHSA) for Transformation to
Recovery-Oriented Mental Health Systems
 For information on the Recovery Transformation
Summit, see
http://www.gmhcn.org/files/RRecovery_Newsletter_Fall2010.pdf

44

Mental Health Recovery Outcomes

 MHCD has developed 3 consumer specific recovery
outcomes
 Consumer Recovery Measure – (Consumer Perspective)
Hope, Safety, Activity, Level of Symptom Management,
Social Networks
 Recovery Marker Inventory – (Clinician Perspective)
Housing, Employment, Education, Active Growth,
Participation, and Symptom Management
 Recovery Needs Level – (Clinical Algorithm) Provides for
one of 5 levels of treatment based upon clinical criteria
 The examples in this presentation will utilize the
Consumer Recovery Measure.

45

46

47

Relationship among Recovery Outcomes
(1) Recovery Marker
Inventory (RMI)
(Longitudinal data to support
clinical decision making)

To what degree is
RECOVERY
(4) Recovery happening for
Needs Level consumers at MHCD
(RNL) (Formative and summative
(Appropriate level of services) evaluation of recovery)

(2) Promoting Recovery (3) Consumer Recovery
in Organizations (PRO) Measure (CRM)
(Consumer’s perceptions of how well (Consumer’s perception of their
specific programs and staff are own recovery)
promoting recovery)

48

Consumer Recovery Measure v3.0
 The CRM V3.0 includes the 15 items listed below:
2. Lately I feel like I’ve been making important contributions (active-growth)
4. I have hope for the future (hope)
5. I am reaching my goals (active growth)
7. I have this feeling things are going to be just fine (hope)
8. Recently my life has felt meaningful (hope)
9. Recently, I have been motivated to try new things (active-growth)
11. There are some people who cause me a lot of fear (safety)
12. I get a lot of support during the hard times (social network)
14. In most situations, I feel totally safe (safety)
15. My life is often disrupted by my symptoms (symptom interference)
16. Sometimes I’m afraid someone might hurt me (safety)
17. I have people in my life I can really count on (social network)
18. Life’s pressures lead me to lose control (symptom interference)
19. I have friends or family I really like (social network)
20. My symptoms interfere less and less with my life (symptom interference)
21. When my symptoms occur, I am able to manage them without falling
apart (symptom interference)

49

Quality Control Issues in Recovery

 Multiple sources of variability
 Measurement
 Consumer
 System
 Changing environmental, treatment, and
consumer specific factors affect outcome
measurements.
 Difficulty in detection of small changes due to
large variability within and among consumers

50

Multilevel Modeling and Recovery

 Multilevel modeling allows for the partitioning of
variance among multiple levels of nesting, i.e.
measures within consumers within therapists
 Allows for regression based correction of expected
outcomes for any unit at any level, i.e. conditional
estimates based upon consumer characteristics in
environment or treatment.
 Can be used to simultaneously monitor multiple
aspects of the system from measurements to clinical
sites.
 Based upon Mixed-Effects ANOVA design

51

Example of Multilevel modeling
concepts
Consumer Level Effect
Typical SLR Model System Level Effect
Intercept
Intercept = +
ACT Tx
Intake = +
Intercept
Mood = +
CRM + Disorder ACT Tx
=
Scores
Intercept
Intercept
= +
Time ACT Tx
= +
in Tx
Intercept
Mood = +
Disorder ACT Tx

Higher Level
Effects 52

Multilevel Regression Corrected Control
Charts

 CUSUM for Consumers (between consumer
comparisons)
 Allows for determination of a consumer’s
progress as compared to peers in same
treatment with environmental and
demographic similarities

53

Example MRC-CUSUM
Self Comparison

54

Example MRC-CUSUM
Peer Comparison

55

Utilization of MRC-CUSUM

 Improved allocation of resources – by
allowing consumer comparison to peers
 Identification of factors that may
promote/inhibit recovery
 Provide feedback regarding progress and
relapse more quickly to clinicians

56

Multivariate Control Chart

 Bivariate Control Chart for plotting of
regression parameters (intercept and slopes)
 Corrections may be made based upon
environmental, treatment, and demographic
characteristics

57

I II

III
IV

58 58

Recovery Intercept
BELOW ABOVE
AVG. AVG.
Decreasing Increasing
Recovery Slope

I II

IV III
NOTE: ANY Outlier within a quadrant indicates it is farther away from the
average than would be expected under typical circumstances.

59

Utilization of Bivariate Control Chart

 Identify “outlying” consumers to help determine
aspects of a program that promote self-perceived
recovery, and those aspects that may be a deterrent
to improvement in self-perceived recovery.
 Allow for identification of consumers who may need
further resources or different treatment.
 Allows for overview of consumer progress, where
comparisons over time may allow for evaluation of
process changes and overall consumer effect.

60

Summary of Benefits

 Allow for more efficient allocation of treatment and
resources.
 Identify program aspects that promote or deter
improvement in self-perceived recovery.
 Identify consumer in need of additional treatment or
resources.
 Allow for the identification of consumer and system
factors that affect or interact with consumer
outcomes and program effectiveness.
 Being able to cater to differing needs of the wide
variety of consumers served.
 Identification of Episodes of Care

61

Moving forward in recovery models to drive
quality improvement

Statistical Models

Information
Technology

Knowledge Building
& Dissemination:
Learning Collaboratives
Staff Involvement,Training

62

Future Directions to
Drive Recovery System Improvement

 Identify clinically significant patterns
 Expand to other recovery measures and aspects.
 Coordinate with data mining to identify
relationships between services and recovery
outcomes
 Automate quality control process
 Integrate fully into clinical quality review processes
 Develop accessible reporting and dashboard
systems for clinicians and managers
63

More information

 If you would like to see more information
concerning MHCD’s research and work with
Recovery please visit:
http://www.outcomesmhcd.com/
http://www.reachingrecovery.org/
Or contact Christopher.McKinney@mhcd.org

64

Extra slides that were mentioned but not presented
on 3/17/11 due to time limitations
 From Mayo Clinic Conference on
Operations Research & Systems Engineering in
Healthcare
 Lean Options for Walk-In, Open Access, and Traditional Appointment
Scheduling in Outpatient Health Care Clinics
(LaGanga & Lawrence, 2009)
 Includes further development to appointment scheduling models to
include metaheuristic optimization of overbooking levels
 Comparison of traditional scheduling, open-access,
and walk-in policies
 Lean process improvement to reduce no-shows and expand intake
capacity.
 Condensed slide set. See http://www.outcomesmhcd.com/Pubs.htm
for complete, original presentation.

 Driving Clinical Quality Improvement Through Mental Health Recovery
Control Charts (McKinney, Olmos, McLean, LaGanga, 2010)
 Poster presentation
 First Place Award 65


Additional information available at: http://www.outcomesmhcd.com/Pubs.htm

66
Rocky Mountain INFORMS, March 17, 2011 © 2008 – Linda LaGanga and Stephen Lawrence

1. Background on
Appointment Scheduling

67

Motivation
 Healthcare Capacity
 Funding restrictions
 Demand exceeds supply
 Serve more people with limited resources
 Manufacturing Scheduling
 Resource utilization
 Maximize throughput
 Healthcare Scheduling as the point of
access
 Maximize appointment yield

68

2. Lean Approaches

Rapid Improvement Capacity Expansion (RICE) Team
January, 2008
Article in press, Journal of Operations Management (2010).
Available at http://dx.doi.org/10.1016/j.jom.2010.12.005 69

Lean Approaches
 Reducing Waste
 Underutilization
 Overtime
 No-shows
 Patient Wait time
 Customer Service
 Choice
 Service Quality
 Outcomes

70

Lean Process Improvement in Healthcare
 Documented success in hospitals
 ThedaCare, Wisconsin

 Prairie Lakes, South Dakota

 Virginia Mason, Seattle

 University of Pittsburgh Medical Center

 Denver Health Medical Center

 Influences
 Toyota Production System

 Ritz Carleton

 Disney

 Hospitals to Outpatient
 Clinics run by hospitals

 Collaborating outpatient systems

71

Lean Process Improvement: One Year After
Rapid Improvement Capacity Expansion
RICE Results
 Analysis of the1,726 intake appointments for the one year before and
the full year after the lean project
 27% increase in service capacity
 from 703 to 890 kept appointments) to intake new consumers
 12% reduction in the no-show rate
 from 14% to 2% no-show
 Capacity increase of 187 additional people who
were able to access needed services, without increasing staff or other
expenses for these services
 93 fewer no-shows for intake appointments during the first full
year of RICE improved operations.

72

Lean Process Improvement:
RICE Project System Transformation
Appointments Scheduled
and No-Show Rates
450 20%
400
Appointments

350 15%
300
250
10%
200
150
100 5%
50
0 0%
Mon Tue Wed Thu Fri Mon Tue Wed Thu Fri
Year Before Year After
Lean Improvement Lean Improvement Appointments
No-Show Rate

73

How was this shift accomplished?
 Day of the week: shifted and added
 Tuesdays and Thursdays
 Welcome call the day before
 Transportation and other information
 Time lag eliminated
 Orientation to Intake Assessment
 Group intakes
 Overbooking
 Flexible capacity

74

Lean Scheduling Challenge
 Choice versus Certainty
 Variability versus Predictability
 Sources of Uncertainty / Variability
 No-shows
 Service duration
 Customer (patients’) Demand
 Time is a significant factor
 Airline booking models?

75

3. Response to Overbooking

76

Sample Responses
 Campus reporter’s visit to student health
center
 “Not now and never will”
 Patient waits 15 – 20 minutes
 New administration, new interests
 Morning News Radio
 “Overbooking leading to increased patient
satisfaction? That just doesn’t make any sense!”
 Public Radio Interviewer
 Benefits of increased access at lower cost

77

Other Responses
 Practitioners
 Dentists
 General medicine
 Child advocacy
 How should we overbook?
 Other options
 Lean Approaches
 Open Access (Advanced Access)
 Walk-ins

78

4. Enhanced Appointment
Scheduling Model

20%

15%
Probability

10%

5%

0%
0 1 2 3 4 5 6 7 8 9 10 11 12
Number Waiting (k)

79

Objectives of Research
 Optimize patient flow in health-care clinics
 Traditionally scheduled (TS) clinic
 Some patients do not “show” for scheduled
appointments
 TS clinic wishes to increase scheduling flexibility
 Some capacity allocated to “open access” (OA)
appointments, OR
 Some capacity allocated to “walk-in” traffic

 Balance needs of clinic, providers, and patients

80

Objectives of Research

 Study impact of open access and
walk-in traffic
 When is open access or walk-in traffic
beneficial?
 What mix of TS, OA, and WI traffic is
best?
 What are trade-offs of TS, OA, and WI
on clinic performance?

81

Relative Benefits and Penalties
  = Benefit of seeing additional client
  = Penalty for client waiting

  = Penalty for clinic overtime

 Numéraire of , , and  doesn’t matter

 Ratios (relative importance) are important
 Allow linear, quadratic, and mixed (linear +
quadratic) costs

82

Linear & Quadratic Objectives
 Linear Utility Function
 N k
ˆ  S    A    k    i  1 
N 1,     k
Patient waiting Patient kwaiting N 1,k
U ˆ
jk
ˆ
Benefit from A  j 1 k k i 1  k Clinic overtime
penalties during penalties during
patients served penalties
normal clinic ops clinic overtime
 Quadratic Utility Function
 N k
ˆ  S    A     2k  1     i  12 
U ˆ
jk N 1, k     k 2 N 1,k
ˆ
A j 1 k k i 1  k

83

Heuristic Solution Methodology
1. Gradient search
 Increment/decrement appts scheduled in each slot
 Choose the single change with greatest utility
 Iterate until no further improvement found

2. Pairwise interchange
 Exchange appts scheduled in all slot pairs
 Choose the single swap with greatest utility
 Iterate until no further improvement found

3. Iterate (1) and (2) while utility improves
4. Prior research: Optimality not guaranteed, but
almost always obtained

84

How does Open Access contribute to
leaner scheduling?
1. It provides a more reliable method of
overbooking.
2. It eliminates the uncertainty of demand for
same-day appointments.
3. It guarantees that patients will be seen when
they want.
4. It reduces uncertainty caused by no-shows.
5. It eliminates waste caused by unfilled
appointments.
85

How does Open Access contribute to
leaner scheduling?
1. It provides a more reliable method of
overbooking.
2. It eliminates the uncertainty of demand for
same-day appointments.
3. It guarantees that patients will be seen when
they want.
4. It reduces uncertainty caused by no-shows.
5. It eliminates waste caused by unfilled
appointments.
86

5. Computational Results

10
10
9
9
Net Utility per Provider

8

8
7
7
6
6
5
5
4 Walk-ins
4 Walk-ins
3
3 Open Access
2 Open Access
2
1 -6.19
1 -6.19
0
0 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Open Access (OA) Traffic (% of capacity)

87

Computational Trials
 44 sample problems solved
 Session size N = 12

 Appointment show rate  = 70%

 Number of providers P = {1, 2, 4, 8}

 OA call-in rate  = {0%, 10%, …100%} capacity

 With P = 4 and N = 12, then  = 24 is 50% of capacity

 Walk-in rate  = {0%, 10%, …100%} of capacity

 With P = 4, then  = 2 is 50% of capacity

 Quadratic costs

 Parameters  =1.0,  =1.0,  =1.5

88

Patients Seen
12
12
Patients Seen per Provider
Patients Seen per Provider

Walk-ins
Walk-ins
Open Access
Open Access

11
11

2 Providers (P=2)
2 Providers (P=2)
10
10
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
OA or WI Traffic (% of capacity)

N=12, P=1,  =0.7,  =1.0, =1.0,  =1.0,  =1.5

89

Patient Waiting Time
1.0
1.0
Expected Waiting Time / Patient
Expected Waiting Time / Patient

0.9 Walk-ins
0.9 Walk-ins
0.8 Open Access
0.8 Open Access

0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

N=12, P=1,  =0.7,  =1.0, =1.0,  =1.0,  =1.5

90

Clinic Overtime
2.5
2.5
Expected Provider Overtime
Expected Provider Overtime

2.0
2.0 Walk-ins
(d time units)

Walk-ins
(d time units)

1.5 Open Access
1.5 Open Access

1.0
1.0

0.5
0.5

0.0
0.0
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

N=12, P=1,  =0.7,  =1.0, =1.0,  =1.0,  =1.5

91

Provider Utilization
90%
90%
Expected Provider Utilization

85%
Expected Provider Utilization

85%

80%
80%

75%
75%

70%
70% Walk-Ins
Walk-Ins
65% Open Acess
65% Open Acess

60%
60%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

N=12, P=1,  =0.7,  =1.0, =1.0,  =1.0,  =1.5

92

Net Utility
10
10
9
9

8

8
7
7
6
6
5
5
4 Walk-ins
4 Walk-ins
3
3 Open Access
Open Access
2
2
1 -6.19
1 -6.19
0
0
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

N=12, P=1,  =0.7,  =1.0, =1.0,  =1.0,  =1.5

93

6. Insights and Recommendations

94

Managerial Implications
 TS appointments provide better clinic utility
than does WI traffic or OA call-ins
 Any WI or OA traffic causes some decline in utility
 An all-WI or all-OA clinic performs worse than any
clinic with some TS appointments
 Even a relatively small percentage of scheduled
appointments can significantly improve clinic utility
 Degree of improvement depends on number of
providers
 A mix of TS appointments with some OA or WI
traffic does not greatly reduce clinic performance
(utility)
95

Insights from the Model
 Loss of utility with WI traffic is due to the long
right-tail of Poisson distribution
 Excessive patient waiting & clinic overtime
 Loss of utility with OA traffic is due to uncertainty
about number of OA call-ins
 TS appts reduce patient waiting and clinic
overtime
 Binomial distribution has truncated right tail
 Multiple providers improves clinic utility
 Portfolio effect – variance reduction

96

Questions and comments? linda.laganga@mhcd.org
(laganga@colorado.edu), stephen.lawrence@colorado.edu.
Further information at http://www.outcomesmhcd.com/Pubs.htm
97
© 2008 – Linda LaGanga and Stephen Lawrence

Driving Clinical Quality Improvement Through Mental Health Recovery Control Charts
C.J. McKinney, Pablo A. Olmos, Cathie McLean, Linda R. LaGanga,
Division of Quality Systems, Mental Health Center of Denver, Denver, CO
Presented at the Mayo Clinic Conference on Systems Engineering & Operations Research in Health Care (August 2010), Rochester, MN. Awarded First Place for Best Poster Presentation.

INTRODUCTION RECOVERY ASSESSMENT continued QUALITY CONTROL CHARTS continued
Every community mental health center focuses on clinical quality. Benefits of effective service
delivery support quality through: Recovery Needs Level 3.  The Utilization Review Process: When a consumer is “flagged” by the Change Chart they will be an
The  Recovery Needs Level is a series of indicators  that through an objective algorithm  assigns the automatic candidate for a utilization management review. This review is done by other clinicians
• optimize resource allocation, consumer to an appropriate clinical service level.  The  RNL is completed by the clinician every six months reviewing a consumer’s medical record to determine if a gap in services has occurred and if other
• increase consistency in consumer outcomes, and as needed.   The measure consist of 15 different dimensions such as the GAF, Residence, Case services should be considered. The recommendations from this review are forwarded to the program
• increase service fidelity, Management, Substance Abuse, and Service Engagement. manager for further review and implementation.
• decrease administrative load on clinicians, and
• increase access to consumer services.
Utilization Review Form
This poster presents our development of a set of reliable and valid mental health recovery Promoting Recovery in Organizations Qualitative Identification of
measures, which we combine for a multi‐perspective assessment of recovery progress, which The   PRO survey is completed by the consumer, and consists of 7 sections covering  all major  service
anchors an objective clinical quality control system.   positions at MHCD, i.e. front desk, nursing/medical, case management,  and rehabilitation.   This data is Service Outliers
collected annually through a random sampling of consumers.  The survey summaries are then utilized to
determine how well the teams and system are promoting recovery ideals.

RECOVERY ASSESSMENT
MHCD consistently collects, reviews, and analyzes data across all consumers on four different
recovery‐oriented outcome measurement tools.  The combined data from these assessments
provide multi‐perspective viewpoints for a more comprehensive picture of the consumer’s
recovery experience and what factors may be impacting their recovery.  It also provides
supporting information to ensure the consumer is placed at a level of care that appropriately
reflects their needs.

Recovery Marker Inventory – Clinician Assessment
Assessments are recorded on seven factors associated with recovery:  Employment,
Learning/Education, Activity/Growth Orientation, Symptom  Interference, Participation in
Services, Housing, and Substance Use.
Documentation of this data provides the clinician with a longitudinal perspective – from both an
overall standpoint, as well as more specific recovery dimensions.  These observations can then be
CONCLUSION & FUTURE DIRECTIONS
used to help guide clinical discussion with the consumer, and indicate treatment focus. Consistent with continuous quality improvement, integration of these tools
into the clinical workflow is a constantly evolving process.  We feel the
QUALITY CONTROL CHARTS following are basic needs to meet, and opportunities for operational
The Recovery Outcome Tools have enabled us to develop a quality review system to monitor individual
enhancement:
consumer outcomes and recommend review in cases where the consumer may not be progressing as
expected. We are able to do this in three ways: • Education of Clinical staff, Executive Management, Consumers, and other stakeholders
as to the value of outcomes data collection and analysis and integration into the clinical
1.The Consumer Recovery Profile provides a snapshot of a person’s current mental health recovery
progress. It demonstrates through graphs and tables the current status of a consumer to aid in service
practice
planning. • Technological ability to “communicate” with the Electronic Medical Record ‐ the
Recovery Profile is connected to the Electronic Medical Record, so it can be easily
accessed by clinicians by bringing the information to them, without having to log in or
open other data storage sites
Consumer Recovery Measure – Consumer Assessment
• Integration into the daily clinical work flow – clinicians can review outcomes
With the Consumer Recovery Measure, the consumer rates agreement or disagreement with information with consumers during individual sessions, so as to make the information
statements regarding  their current recovery experience.  These responses gauge consumer more meaningful; it is employed as part of the Peer Review process; and can be used
perspective on five dimensions of recovery:  Symptom Management, Sense of Safety, Sense of
Growth, Sense of Hope, and Social Activity. during six month case reviews
• Automation of Quality Review process – control charts “flag” concerning outcomes
Graphic representation of this data is shared with the consumer to initiate clinical discussion about
changes in these areas, what  the consumer attributes the changes to, and possible relationships outliers and identify them for Utilization Management Review, so as to address and
between categories.  This promotes insight, and empowers the consumer to share their story in a redirect treatment inefficiencies in a timely manner
new and different way.
• Exploration of “super performer” characteristics to identify benchmarks for
2. The Recovery Change Chart automatically identifies consumers needing further review by flagging those
with substantial change in their recovery outcomes. A flag occurs whenever a consumer deviates from teams/programs
their expected outcomes for an extended period of time or if the deviations are large. • Consumer Recovery Portal – consumers will have access to their outcomes data for
increased engagement in the recovery process
Self‐Comparing Control Chart Peer‐Comparing Control Chart
•Integrate physical and mental healthcare

•Maximize outcomes to improve human lives! mental
For more information about research or
health recovery at MHCD, please view conference
presentations on our website: 98
www.outcomesmhcd.com

INFORMS Rocky Mtn Presentation 03-17-11

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (20)

INFORMS Rocky Mtn Presentation 03-17-11