1. White Paper
Physician Performance Improvement:
An Analytical Approach — Part One
Robert Sutter, RN, MBA, MHA
Brian Waterman, MPH
Michael Udwin, MD, FACOG
Center for Healthcare Analytics
Truven Health Analytics
July 2012

3. Table of
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Physician Variability . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Contents
Physician Performance . . . . . . . . . . . . . . . . . . . . . . . . . 3
Physician Performance Classification . . . . . . . . . . . . . . . . . 4
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

5. Introduction Healthcare reform has strengthened the link
between performance and reimbursement,
exemplified by value-based purchasing and
accountable care organizations. This has led
to heightened interest in improving physician
performance.
However, providing actionable assessments of physician performance is not as
straightforward as it may seem. Many organizations, for example, launch physician
performance improvement initiatives without addressing relevant aspects of
performance, such as the following:
§§ What proportion of total performance variability is attributable to physicians?
§§ Are there statistically significant differences in physician performance?
§§ How is physician performance distributed across the outcome categories of “better
than expected,” “as expected,” and “worse than expected”?
Consequently, it is not uncommon for a physician improvement initiative to be
launched without establishing relevant, quantifiable objectives. Furthermore, and
more importantly, the strategy to realize physician improvement may not be clearly
defined. Subsequently, after many meetings and hours of expended work, the
initiative may be abandoned due to lack of direction and progress.
Fortunately, however, there is a more thoughtful approach that enhances the success
of physician improvement initiatives.
Physician Performance Improvement: An Analytical Approach — Part one 1

6. Harnessing the data at your disposal and conducting analytics to answer the
questions posed in the aforementioned bullet points will provide the knowledge
required to successfully engage physicians and improve organizational performance.
In this white paper, we address how to approach answering these questions. In a
subsequent white paper titled, “Physician Performance Improvement: Case Studies,”
we will provide case studies applying these principles and using the information to
formulate a performance improvement strategy that engages physicians.
Physician Variability
All healthcare outcomes (length of stay, cost, mortality, etc.) have two components
that contribute to the overall variability in the achieved performance:
§§ Organizational factors, such as policies, procedures, staffing, etc.
§§ Physician practice patterns
Quantifying what percentage of the total variability that physician practice patterns
comprise is strategically valuable information. With this knowledge, a performance
improvement strategy can be formulated.
Figure 1 depicts the percentage of total variability in risk-adjusted excess length
of stay (defined as observed length of stay minus expected length of stay) that
is attributable to physician practice patterns for clinical conditions that have an
opportunity for improvement.
Figure 1: Physician Variability Contribution
Back and Neck Procedures 93.7
Circulatory Disorders 77.8
Appendectomy 64.7
Infectious Disease 62
Sepsis 41.2
Red Blood Cell Disorders 38.6
Respiratory Failure 25.3
Metabolic Disorders 20.0
Seizures 17.0
Gynecology Procedures 15.7
Cellulitis 8.6
COPD 6.2
Pneumonia 5.4
Newborn 3.8
Rehabilitation 3.1
Asthma 0.0
Cardiac Arrhythmia 0.0
Physician Variability Percentage
2 Physician Performance Improvement: An Analytical Approach — Part ONe

7. Engaging physicians to explore and standardize practice patterns to reduce
variability in risk-adjusted excess length of stay for “Back and Neck Procedures” is
the strategy of choice for this clinical condition, since physician practice patterns
account for 93.7 percent of the total variability. On the other hand, if “Pneumonia”
is the clinical condition selected for improvement, focusing the improvement efforts
on organizational factors is the strategy that will yield the greatest benefit. That is
because organizational factors represent 94.5 percent of the variability and physician
practice patterns represent 5.4 percent.
One can readily see the value this information provides for successful performance
improvement and physician engagement. Without this information, one may pursue
exploring physician practice patterns for pneumonia with the hopes of performance
improvement only to be disappointed with the results and potentially incurring
resentment and disengagement among physicians in the process.
Note: The physician variability percentage is the intraclass correlation coefficient
derived using hierarchical regression techniques. By assessing the degree to which
measured outcomes are correlated within physicians, the intraclass correlation
provides an estimate of the degree to which outcome variation can potentially be
explained by variation in physician practice patterns. Several currently available
statistical software packages provide this capability. See references in the footnotes
for further reading on the subject.1, 2
Physician Performance
Determining if statistically significant differences in physician performance exist
provides another piece of information that assists in deriving the improvement
strategy. If the goal is to improve risk-adjusted excess length of stay, the question
to answer is: Is risk-adjusted excess length of stay significantly different among
physicians? If the answer is yes, then reducing variability for risk-adjusted excess
length of stay among physicians will likely yield meaningful improvements in
hospital performance. On the other hand, if the answer is no, then variability in
physician performance does not exist and any attempt to reduce this variability will
likely not yield meaningful results.
To illustrate, let us take two examples of physician performance variability and
use the common p-value of ≤ 0.05 to determine whether statistically significant
differences exist in physician performance.
Figure 2 below depicts the median risk-adjusted excess length of stay for attending
physicians treating patients undergoing vascular procedures. The graph depicts
performance that ranges from a median of -0.7 to 1.9 days. The one-way test of
significance for these data yields a p-value of 0.0001. Since this value is less than
0.05, the answer to the above question — Is there a statistically significant difference
in risk-adjusted excess length of stay among physicians — is yes. Therefore,
reducing the variability in physician performance will likely produce meaningful
improvement.
Physician Performance Improvement: An Analytical Approach — Part one 3

8. Figure 2: Vascular Procedures: Attending Physician Performance by Risk-Adjusted Excess Days
2 -0.7 39
5 -0.4 36
3 -0.4 54
12 -0.3 24
13 -0.1 31
14 -0.1 58
Physician ID
6 0.1 33
10 0.4 27
8 0.6 33
4 0.8 27
9 0.8 36
7 0.9 36
11 1.1 33
1 1.9 33
Median Excess Days Case Count
P<0.05
The scenario depicted in Figure 3 for renal failure depicts variation in median
performance that ranges from a -0.4 to 1.4 days, and the one-way test of significance
for these data yields a p-value of 0.4873. Given that this probability is considerably
greater than 0.05, the answer to the above question: — Is risk-adjusted excess
length of stay significantly different among physicians — is no. Simply put, there
is insufficient evidence in the data to conclude that there is true variability in
physician performance. Given this, one could not expect that attempts to reduce
physician performance variability would yield meaningful improvement results.
Figure 3: Renal Failure: Attending Physician Performance by Risk-Adjusted Excess Days
4 -0.4 45
3 -0.4 17
-0.0 19
Physician ID
5
1 0.6 50
7 0.5 21
6 1.3 19
2 1.4 22
Median Excess Days Case Count
P<0.05
Physician Performance Classification
The last piece of information in evaluating physician performance is the distribution
of performance across the performance categories of “better than expected,”
“as expected,” and “worse than expected.” The “expected” component of this
measurement is the predicted risk-adjusted outcome based on the severity of illness
among the physician’s patient population. In our aforementioned examples, we
derived risk-adjusted excess length of stay by subtracting a patient’s “expected”
length of stay from their observed length of stay. Negative differences represent a
shorter length of stay than expected, zero differences represent a length of stay that is
equal to expected, and positive differences signify a length of stay that is longer than
expected. By statistically summarizing these differences across a group of patients
attributed to a specific physician, one can determine whether or not systematic
departures from risk-adjusted expected length of stay are present.
4 Physician Performance Improvement: An Analytical Approach — Part ONe

9. Classifying physician performance involves deriving risk-adjusted confidence
intervals and comparing each physician’s confidence interval to an appropriate
reference. Before we delve into examining performance using confidence intervals,
let’s first understand what a confidence interval is. A confidence interval is a range of
values that are likely to occur given the variability present in the data. Performance
measurements are not static; they vary from one time period to the next. Therefore,
to accurately measure physician performance, this variability must be accounted for
in the measurement system. When deriving confidence intervals, one can select the
level of precision desired. Typical levels are 99 percent, 95 percent, and 90 percent,
with the most common being 95 percent.3
Let’s use an example to demonstrate how to interpret confidence intervals and derive
an improvement strategy. Figure 4 depicts 95-percent confidence intervals of the
median risk-adjusted excess length of stay. The left-most bar of the interval is the
lower confidence limit (LCL), the right-most bar is the upper confidence limit (UCL),
and the median is depicted by the dot. The appropriate reference here is the line
located at the zero value, which represents performance that is as expected.
The physicians at the bottom of the graph, highlighted in orange, are performing
better than expected, since their confidence intervals do not intersect the reference
line, and the UCL lies to the left of the reference line. The physicians depicted
in green are performing as expected, since their confidence intervals intersect
the reference line. And the physicians depicted in red are performing worse than
expected, since their confidence intervals do not intersect the reference line and the
LCL lies to the right of the reference line.
In this scenario, a viable improvement strategy is to examine the practice patterns of
the physicians with better than expected performance and disseminate the findings
to the other physicians. A standardized clinical protocol could also be developed
based on the findings; and garnering the cooperation of the other physicians in
utilizing the protocol will likely yield meaningful improvement.
Figure 4: Diabetes: Attending Physician Performance by Risk-Adjusted Length of Stay (LOS) Comparison
Attending Physician
Better Than Expected
As Expected
Worse Than Expected
-4 -3 -2 -1 0 1 2 3 4 5 6 7 8
Risk-Adjusted Median Excess LOS Confidence Interval
Physician Performance Improvement: An Analytical Approach — Part one 5

10. Figure 5 depicts a scenario where all physicians are performing as expected.
Hence, there are no physicians who can be used as role models for performance
improvement purposes. The improvement strategy in this case consists of
researching and implementing best practices and practice guidelines.
Figure 5: Renal Failure: Attending Physician Performance Risk-Adjusted Length of Stay Comparison
Attending Physician
Better Than Expected
As Expected
Worse Than Expected
-3 -2 -1 0 1 2 3 4 5 6 7 8 9
Risk-Adjusted Median Excess LOS Confidence Interval
Summary
Deploying an effective strategy that will engage physicians in performance
improvement requires a comprehensive understanding of physician performance.
There are three questions that facilitate a comprehensive understanding of physician
performance:
§§ What proportion of total performance variability is attributable to physicians?
§§ Are there statistically significant differences in physician performance?
§§ How is physician performance distributed across the outcome categories of “better
than expected”, “as expected” and “worse than expected”?
Once these questions are answered an appropriate strategy to engage physicians
in performance improvement can be derived that will likely yield meaningful
performance improvement. Deploying a physician performance improvement
strategy by relying on only one measurement of performance is less likely to result in
meaningful performance improvement. With this approach, one is hoping they have
selected an appropriate performance improvement strategy.
In part two of this white paper, we will apply these concepts to three commonly
encountered scenarios. In practice, other scenarios will be encountered, however, it
is our hope that these scenarios will provide some general guidance on deploying an
effective physician performance improvement strategy.
1. Harman, JS, et al; “Profiling Hospitals for Length of Stay for Treatment of Psychiatric Disorders.” The
Journal of Behavioral Health Services & Research. 2004;31(1):66-74.
2. Snijders TAB, Bosker RJ. “Multilevel Analysis: An Introduction to Basic and Advanced Multilevel
Modeling.” Sage Publications Inc: 2012.
3. Martin JG, Altman DG: “Statistics With Confidence: Confidence Intervals and Statistical Guidelines.”
British Medical Journal: 1989.
6 Physician Performance Improvement: An Analytical Approach — Part ONe

11. Physician Performance Improvement: An Analytical Approach — Part one 7

12. For more information
Send us an email at
info@truvenhealth.com
ABOUT TRUVEN HEALTH ANALYTICS
Truven Health Analytics delivers unbiased information, analytic tools, benchmarks, and services to the healthcare industry. Hospitals, government agencies,
employers, health plans, clinicians, pharmaceutical, and medical device companies have relied on us for more than 30 years. We combine our deep clinical,
financial, and healthcare management expertise with innovative technology platforms and information assets to make healthcare better by collaborating with our
customers to uncover and realize opportunities for improving quality, efficiency, and outcomes. With more than 2,000 employees globally, we have major offices
in Ann Arbor, Mich.; Chicago; and Denver. Advantage Suite, Micromedex, ActionOI, MarketScan, and 100 Top Hospitals are registered trademarks or trademarks of
Truven Health Analytics.
truvenhealth.com
| 1.800.366.7526
©2012 Truven Health Analytics Inc. All rights reserved. All other products names used herein are trademarks of their respective owners. HOSP 11377 0712