1. Improving Your Performance in PI-CME
Harvey C. Parker, Ph.D., CCMEP
National Association for Continuing Education (NACE)
Gregg Sherman, MD
National Association for Continuing Education (NACE)
B. Stephen Burton, MS
CE Outcomes LLC
Jeremy C. Lundberg, MSSW
DLC Solutions, LLC
Michael S. Adler
myCME
Alliance for Continuing Education in the Health Professions
38th Annual Conference • San Francisco, California
January 31, 2013

2. Disclosures
Harvey C. Parker, Ph.D., CCMEP
National Association for Continuing Education (NACE)
Nothing to Disclose
Gregg Sherman, MD
National Association for Continuing Education (NACE)
Nothing to Disclose
B. Stephen Burton, MS
CE Outcomes LLC
Nothing to Disclose
Jeremy C. Lundberg, MSSW
DLC Solutions, LLC
Nothing to Disclose
Michael S. Adler, BA
myCME
Financial Interest in Haymarket/myCME

3. Learning Objectives
1. Identify barriers to successful implementation
of PI-CME
2. Discuss importance of linking performance
measures to interventions
3. Use multiple outcomes strategies to evaluate
your PI-CME
4. Design a customized IT platform for your PI-
CME
5. Summarize strategies to increase the success
of your PI-CME

4. Learning Objective
1. Identify barriers to successful implementation
of PI-CME
Harvey C. Parker, Ph.D., CCMEP
Gregg Sherman, MD

5. Tailor Education to Fit the Learner
George Bernard Shaw once said…..
“The only man who behaves sensibly is my tailor; he
takes my measurements anew every time he sees me, while
all the rest go on with their old measurements and expect me
to fit them.”
Like a good tailor, the CME provider must offer activities that meet
the specific needs of learners and not use a one-size-fits-all model
of program design and delivery.
Individualized, performance-based education is precisely the goal
of PI-CME.

6. Making the Case for PI-CME
2005
AMA approved guidelines by which AMA PRA
Category 1 CreditTM could be awarded—up 20
AMA PRA credits for participation in structured
performance improvement CME (PI CME)
activities.

7. 3 Stages of PI-CME
Performance Improvement CME Credit for Performance
Improvement
Find evidence-based measures Stage A
relevant to the identified Learning from current practice—
practice problem (Plan) performance assessment
Implement interventions Stage B
identified to improve Learning what PI to apply to current
performance (Do) patient care
Re-measure to assess Stage C
improvement (Study) Learning from the evaluation of the PI
effort
Standardize the improvement Restart the process
(Act)

8. Barriers to Successful Implentation of PI-CME
Learner Barriers CME Professional
Barriers
Time Time
Effort Steep learning curve for
methodology to deliver
content and measure
outcomes
Self-directed Tracking learner involvement
Sustained effort over months Recruitment and education of
learners about PI CME
Data collection required Expense

9. Activity Start and End Dates: August 1, 2010 to July 31, 2011
Collaborating Organizations
National Association for Continuing Education
DLC Solutions, LLC
CE Outcomes LLC
myCME
Canadian ADHD ADD Research Association (CADDRA)
Children and Adults with Attention Deficit Hyperactivity Disorder (CHADD)
Supported by Eli Lilly
Grant ID 100204775

10. Statement of Need and Purpose
The vast majority of adult patients with ADHD remains undiagnosed and,
therefore, untreated.
•Close to half of PCPs (48%) do not feel confident to diagnose ADHD in adults.
•A large majority (85%) indicate that they would take a more active role in diagnosing
and treating adult ADHD if there were an easy-to-use, validated screening tool for
diagnosing adults.
•A large majority of PCPs (75%) indicate that they would take a more active role in
diagnosing and treating adult ADHD if there were effective prescription medicines that
were not stimulants or controlled substances.
•Approximately 13% of PCPs refer adult patients with ADHD to specialists specifically
because many of the existing treatment options include controlled substances, which
they are uncomfortable prescribing.
Kessler RC, Adler L, Barkley R, Biederman J, Conners CK, Demler O, Faraone SV, Greenhill LL, Howes MJ, Secnik K, Spencer T, Ustun TB, Walters EE,
Zaslavsky AM. (2006). The Prevalence and Correlates of Adult ADHD in the United States: Results from the National Comorbidity Survey Replication. Amer.
Jour. of Psychiatry, 163: 724-732.
Barkley, R.A., Murphy, K.R., & Fischer, M. (2008). ADHD in Adults: What the science says. Guilford Press, New York.
Adler, L.A. (2004). ADHD: a survey of 400 primary care physicians. Presented at 51st Annual Meeting of the American Academy of Child and Adolescent
Psychiatry; October 19-24, 2004; Washington, DC.

11. Survey Results: Knowledge of Target
Conditions
PCPs indicated that they are significantly more knowledgeable about depression than they are
about all other target conditions. They are also significantly more knowledgeable about GAD than
they are about bipolar disorder, ADHD and OCD.
100% 92%
% Very / Extremely Knowledgeable
83%
80%
(top 2 box)
60%
40% 36% 34% 34%
20%
Mean=
0%
4. 4. 3. 3. 3.
2a
Depression (n=336) 0b
GAD (n=331) 3c
Bipolar disorder 2 d
ADHD (n=137) 2e
OCD (n=134)
(n=144)
Base: Qualified Respondents (n=400)
Q400 Please indicate how knowledgeable you consider yourself to be about the following conditions?
Scale: 1=Not At All Knowledgeable; 2=Not Very Knowledgeable; 3=Somewhat Knowledgeable; 4=Very Knowledgeable; 5=
Extremely Knowledgeable
Adler, L.A. (2004). ADHD: a survey of 400 primary care physicians. Presented at 51 st Annual Meeting of the American Academy of Child and Adolescent
Psychiatry; October 19-24, 2004; Washington, DC.

12. Survey Results: Level of Clinical Instruction
Relative to other mental health conditions, PCPs report having received the least thorough
clinical instruction regarding ADHD. They report receiving the most thorough instructing
regarding the diagnosis and treatment of depression.
60% 57%
% Very / Extremely Thorough
40%
40%
(top 2 box)
20%
20% 15%
13%
Mean=
0%
3. 3. 2. 2. 2.
6a
Depression (n=227) GAD 2
b
(n=159) 7
c
Bipolar disorder
(n=78)
6d
OCD (n=58) 3e
ADHD (n=53)
Base: Qualified Respondents (n=400)
Q410 Please indicate the level of clinical instruction you received regarding diagnosing and treating the following conditions.
Scale: 1=Not At All Thorough; 2=Not Very Thorough; 3=Somewhat Thorough; 4=Very Thorough; 5= Extremely Thorough
Adler, L.A. (2004). ADHD: a survey of 400 primary care physicians. Presented at 51st Annual Meeting of the American Academy of Child and
Adolescent Psychiatry; October 19-24, 2004; Washington, DC.

13. Survey Results: Refer to Specialist for Dx
PCPs indicate that they most frequently refer to a specialist for the diagnosis of ADHD and bipolar
disorder. The vast majority of PCPs diagnose depression (98%) and GAD (97%) themselves.
OCD Depression
2%
47% ADHD
54%
98%
35%
GAD Bi-polar Disorder
65%
3%
33%
Diagnose Myself
68%
97%
Refer to a specialist for diagnosis
Base: Qualified Respondents (n=400)
Q505 For each condition, please indicate whether you most frequently make the diagnosis yourself, or refer adult patients to a
specialist for diagnosis.
Adler, L.A. (2004). ADHD: a survey of 400 primary care physicians. Presented at 51 st Annual Meeting of the American Academy of Child and Adolescent
Psychiatry; October 19-24, 2004; Washington, DC.

14. Survey Results: Reasons for Working with
Specialists
Inexperience / Lack of Inexperience / lack of confidence 52%
confidence (52%) is
the primary reason Difficult to treat & diagnose / No clear
22%
why PCPs refer to diagnostic criteria
specialists when
diagnosing and Second opinion / confirm diagnosis 20%
treating adults with
ADHD. Many treatment medications are
13%
controlled substances
Many physicians also
cite difficulty
e Other 11%
diagnosing and
treating adult ADHD
(22%) and a general Time constraints 4%
desire for second
opinions to confirm 0% 10% 20% 30% 40% 50% 60%
ADHD diagnoses % Answering
(20%). Base: Refer to Specialist for Diagnosis of ADHD (n=261)
Q507 Please explain why you sometimes collaborate with or defer to specialists when diagnosing adult ADHD.
Adler, L.A. (2004). ADHD: a survey of 400 primary care physicians. Presented at 51 st Annual Meeting of the American Academy of Child and Adolescent
Psychiatry; October 19-24, 2004; Washington, DC.

15. NACE Convened a Panel of ADHD Experts to Develop
Performance Measures for Adults ADHD Dx and Tx
1. Performs a comprehensive interviews to assess ADHD
history and symptoms.
2. Uses adult ADHD rating scales to assess symptom
frequency and severity.
3. Uses interviews or rating scales to corroborate patient self-
report and assess functional impairment.
4. Initiates a pharmacologic treatment plan for adult ADHD
patients when indicated
5. Recommends non-pharmacologic treatments and
providing patient education for adult ADHD patients.
6. Provides timely and consistent follow-up care.

16. Baseline Self-Assessment of Utilization of
Performance Measures by PCPs
N=94 (45 Physicians(top row); 22 NPs, 24 PAs, 3 Others(bottom row))
Performance Measures Not at Somewhat Moderately More Than Very
All Moderately Much
1. Conduct an interview 21% 32% 16% 21% 11%
33% 21% 18% 26% 3%
2. Use ADHD Rating Scales 50% 24% 11% 16% 0%
69% 10% 10% 8% 3%
3. Gather corroborating information 26% 39% 21% 11% 3%
and assess functional impairment
44% 28% 5% 15% 8%
4. Plan pharmacologic therapy to treat 18% 21% 16% 37% 8%
ADHD symptoms
47% 8% 8% 21% 16%
5. Provide or recommend psychosocial 20% 20% 28% 18% 15%
interventions
47% 16% 5% 26% 5%
6. Schedule regular follow-up visits to 16% 21% 21% 26% 16%
monitor treatment safety and efficacy
for ADHD interventions 24% 24% 18% 21% 13%

17. Baseline Self-Assessment of Confidence in
the Use of Performance Measures by PCPs
N=94 (45 Physicians(top row); 22 NPs, 24 PAs, 3 Others(bottom row))
How confident are you in your ability to: Not at All Somewhat Moderately More Than Very
Moderately Much
1. take a patient history that will identify the 15% 17% 34% 27% 7%
essential components of an ADHD
complaint? 33% 29% 16% 16% 6%
2. employ clinical assessment tools (e.g., 21% 24% 31% 21% 2%
ADHD symptom checklists, structured
diagnostic interview, etc.) in the diagnosis
31% 35% 20% 6% 8%
of ADHD in your adult patients?
3. identify co-morbid psychiatric disorders 7% 17% 43% 29% 5%
(e.g., depression, anxiety, substance
use/abuse disorders) in your adult patients
27% 20% 20% 24% 8%
presenting with symptoms of ADHD?
4. discuss appropriate ADHD medication 10% 33% 21% 21% 14%
options with your adult ADHD patients?
39% 29% 12% 16% 4%
5. select the appropriate pharmacotherapy for 12% 24% 31% 31% 2%
treating ADHD, taking into account your
patient’s overall health status?
37% 29% 18% 14% 2%
6. determine when psychosocial interventions 19% 38% 26% 17% 0%
such as cognitive behavioral therapy (CBT)
or workplace or academic accommodations 33% 35% 20% 10% 2%
are indicated as a part of an ADHD
treatment plan for your patients?

18. Program Overview
Stages A-B-C of Our PI-CME: Making Exam Room
Decisions for Adults with ADHD
Adults with ADHD: Making Exam Room Decisions
Participants evaluated existing practices in providing care to
Participants evaluated existing practices in providing care to
Stage A adults with ADHD by completing a self-assessment survey
adults with ADHD by completing a self-assessment survey
and answering chart-review questions. Participants selected
and answering chart-review questions. Participants selected
Pre-Assessment one or more Quality Improvement Plans (QIP) to improve care
one or more Quality Improvement Plans (QIP) to improve care
to adults with ADHD.
to adults with ADHD.
Participants implemented one or more of the QIP selected in
Participants implemented one or more of the QIP selected in
Stage B Stage A by using the materials found in the NACE Adult ADHD
Stage A by using the materials found in the NACE Adult ADHD
Toolkit. Comprehensive educational and informational
Toolkit. Comprehensive educational and informational
Action Stage resources were provided at this stage with online and printed
resources were provided at this stage with online and printed
materials.
materials.
Participants re-evaluated performance by again completing
Participants re-evaluated performance by again completing
the self-assessment survey and answering chart-review
the self-assessment survey and answering chart-review
Stage C questions on aasecond set of 55patients. Feedback provided to
questions on second set of patients. Feedback provided to
Post-Assessment participants explained the improvements that were made in
participants explained the improvements that were made in
caring for adults with ADHD.
caring for adults with ADHD.

19. Audience Generation and Participation
Participants were recruited by invitations that were both surface and
electronically mailed, recruitment at NACE Emerging Challenges in
Primary Care symposia, and through online advertising (e-newsletters
and banner on myCME.com)

20. Steps Taken To Increase Participation
• Aggressive marketing for audience generation
• E-newsletters to NACE members announcing PI CME activity
• Mailed invitations to PCPs in cities throughout the US
• Listing with myCME.com
• Announcements and recruitment at NACE live symposia
• Weekly postcard and e-mail reminders to move from one stage to
another
• Dedicated CME manager to call or email participants to remind them to
continue activity
• Sent hard copy of NACE Adult ADHD Toolkit when learners completed
Stage A and moved to Stage B (Action Stage) and this motivated them
to continue
• After hours phone support

21. Learning Objective
2. Discuss importance of linking measures to
interventions
Harvey C. Parker, Ph.D., CCMEP
Gregg Sherman, MD

22. Our 6 Performance Measures
1. Performs a comprehensive interviews to assess ADHD
history and symptoms and co-morbid disorders.
2. Uses adult ADHD rating scales to assess symptom
frequency and severity.
3. Uses interviews or rating scales to corroborate patient self-
report and assess functional impairment.
4. Initiates a pharmacologic treatment plan for adult ADHD
patients when indicated
5. Recommends non-pharmacologic treatments and
providing patient education for adult ADHD patients.
6. Provides timely and consistent follow-up care.

23. NACE Adult ADHD Toolkit
The National Association for Continuing Education ( NACE) developed this Adult ADHD
Toolkit to assist clinicians in the assessment, diagnosis and treatment of adults with
Attention-Deficit/ Hyperactivity Disorder ( ADH D) . This toolkit was prepared as a resource
for an accredited medical education per formance improvement activity entitled, Making
Exam Room Decisions for Adults with ADH D.*
The toolkit contains practical information and tools to help clinicians and
members of the adult ADH D care team incorporate evidence-based and
consensus-based guidelines into office practice. T he toolkit contains:
www.lillygrantoffice.com.
Available Online: http://www.naceonline.com/AdultADHDtoolkit/QI1.php

24. Performance Measures Linked to Interventions
Contained in the NACE ADHD Toolkit
ASSESSMENT TOOLS PROFESSIONALEDUCATION PATIENT EDUCATION TOOLS
TOOLS
Adult ADHD Self-Report Scale-V1.1 Canadian ADHD Practice Guidelines Diagnosis of ADHD in Adults PM 1, 2, 3
(Symptom Checklist (English and PM 1, 2, 3, 4, 5, 6
Spanish) PM 2
Adult ADHD Self-Report Scale-V1.1 FDA Approved Medications for Adults Facts About Approved Medication for
Screener PM 2 with ADHD PM 4 Adults with ADHD (Monograph) PM 4
Barkley Quick Check for Diagnosis of ADHD Coding Fact Sheet Succeeding in the Workplace
ADHD in Adults (Monograph) PM 6
PM1, 2
Brief Semi-Structured Interview for DSM-IV TR ADHD Symptoms Managing Money (Monograph) PM 6
ADHD in Adults PM 1, 2 PM 2
Weiss Functional Impairment Rating Understanding ADHD in Adults Social Skills in Adults with AD/HD PM 6
Scale Self-Report PM 3 (Webcast and Monograph)
PM 1, 2, 3
ADHD Medication Side Effects Checklist Assessment of Adult ADHD (Webcast A Guide to Organizing the Home and
PM 4 and Monograph) Office PM 6
PM 1, 2
Medication Response Form Pharmacotherapy of Adult ADHD and ADHD Coaching for Adults (Monograph)
PM 4 Co-morbid Conditions PM 4, 5 PM 6
CAGE Questionnaire PM 1 Maintenance and Follow-up Care for Legal Rights: Higher Education and the
Adults with ADHD (Webcast and Workplace (Monograph) PM 6
Monograph) PM 4, 5
Hamilton Depression Rating Scale PM 1 Psychosocial Interventions for ADHD in Women and ADHD (Monograph) PM 6
Adults (Webcast and Monograph) PM 6
ADHD in Adults—Differential and Adult ADHD Resources (Monograph)
Coexisting Diagnosis (Webcast and PM 6
Monograph) PM 3

25. Stage A—Pre-Assessment
• Participants evaluated existing practices in providing care
to adults with ADHD by completing a self-assessment
survey and answering chart-review questions on 5
patients.
• Participants selected one or more Quality Improvement
Plans (QIP) to improve care to adults with ADHD.
• QIP 1 Assessment of ADHD in Adults
• QIP 2 Treatment of ADHD in Adults
• QIP 3 Patient Education
• Toolkit was sent to all participants who completed Stage
A.
• 5 credits awarded.

26. Stage B—Action Stage
• Participants implemented one or more of the QIPs
selected in Stage A by using the materials found in the
NACE Adult ADHD Toolkit.
• Comprehensive educational and informational
resources were provided at this stage with online and
printed materials.
• Participants must wait at least 45 days before starting
Stage C.
• 5 credits awarded.

27. Professional Education Tools

28. Assessment Tools

29. Patient Education Tools

36. Stage C—Post-Assessment
• Participants re-evaluated performance by again
completing the self-assessment survey and answering
chart-review questions on a second set of 5 patients.
• Feedback provided to participants explained the
improvements that were made in caring for adults with
ADHD.
• Participants evaluated their performance in the PI-
CME activity.
• 5 credits awarded for completing Stage C and 5 more
for completing final evaluation.

37. Follow Up Education for ADHD in Adults
Take a DVD of the NACE Adult ADHD Toolkit
with you today and share it with your primary
care provider.
In 2012 NACE educated an additional 1600
PCPs on adult ADHD and introduced them to the
NACE Adult ADHD Toolkit through our Emerging
Challenges in Primary Care symposia held in 15
cities. We hope to extend this reach in 2013.

38. Learning Objective
3. Use multiple outcomes strategies to evaluate
your PI-CME
B. Stephen Burton, MS

39. Outcome Measures for Adult ADHD PI CME
1. Learner self-assessment at Stages A and C
• Frequency with which each measure is used in
practice
• Degree of confidence learner has with the use of each
measure
• Case-based knowledge/competence for diagnosing
and treating ADHD
2. Chart Review Data Collection
• Data collected from 5 patients at Stage A and 5 new
patients at Stage C on use of each measure
• Drill down on details regarding each measure
3. Evaluation of program and self-assessment of learning

40. Self-Assessment Stages A and C
Changes in the Use of Performance Measures
N = 92 P < .001 for each item
Graph displays those that selected “always” on a five point frequency scale from “never” to “always”

41. Self-Reported Changes in Confidence in Diagnosing and
Treating Adults with ADHD from Stages A to C
N = 92 P < .001 for each item
Graph displays those that selected “very confident” on a five point confidence scale from “not at all confident” to “very confident”

42. Chart Abstraction Tool
Patient History - as obtained by you at initial assessment or confirmed
during review of patient's medical records
Was a patient history obtained or confirmed that covered all of the following elements:
developmental/academic history, past medical & psychiatric history, medication use,
substance use, and family history?
 Yes
 No
Was the patient assessed for significant impairments in social, academic and
occupational functioning?
 Yes
 No
Was the patient assessed or screened for common comorbid conditions including
learning disability, depression, anxiety and bipolar disorder?
 Yes
 No

43. Chart Review Data
Changes in the Use of Performance Measures
P < .001
P < .01
P = .078
P = .093 P = .384
P = .015
Stage A, N = 464
Stage C, N = 462

44. Learning Objective
4. Design a customized IT platform for your PI-
CME
Jeremy C. Lundberg, MSSW
DLC Solutions, LLC

45. Learning Objective
• Design a customized IT platform for your PI-
CME
1. Leverage mature, CE-oriented learning management system
(i.e., EthosCE.com)
2. Design easy-to-use, step-wise learner workflow to complete
module.
3. Use email reminder triggers to encourage completion and
mitigate against attrition.
4. Share individual and aggregate benchmark reporting with
learners to demonstrate variance.
5. Pilot test to ensure engaging, intuitive learning experience.

58. Learning Objective
5. Describe strategies to increase success in PI-
CME
Audience Generation
Michael S. Adler, BA

59. Disclosure
Disclosure
The views expressed in this presentation
are my own and do not represent those
of my current employer.
I do have a financial interest in
Haymarket Medical Education/myCME.
59
59

60. myCME Partnership with NACE
myCME Partnership with NACE
• Extended Reach of myCME--a Preferred Choice of
NACE and Other Medical Education Providers
– Experienced distribution site that delivers a cost-effective
learner solutions for their educational offerings
– Superior placement of educational offerings
– Aggressive marketing/recruitment campaign
– High guarantees on readership and certification
• 2010: myCME Chosen to Partner with NACE on PI-
CME Activity
– Adults with ADHD: Making Exam Room Decisions
– Promotion and distribution over an 8-month period
To this day, NACE continues its partnership with myCME
To this day, NACE continues its partnership with myCME
60
60

61. Adults with ADHD Activity on
Adults with ADHD Activity on
myCME
myCME
61

62. myCME Recruitment
myCME Recruitment
E-Mail/Online Campaigns
E-Mail/Online Campaigns
• Executed throughout the myCME Network and tailored to the specific audience
• E-mail
– Featured spot in the myCME “Weekly Update” and “Monthly Update” eNewsletters
– Dedicated e-mail blasts specific to the activity
– CME Spotlight of The Clinical Advisor monthly newsletter - the #1 NP/PA medical
journal
– CME Spotlight of MPR Weekly News Brief eNewsletter
• Online Recruitment
– “Featured CME Course” image rotator on the myCME home page
– “Latest Courses” section on the myCME home page
– “Relevant Specialty” home pages (Psychiatry, Family Medicine, et al.): 24/7 fixed
placement
myCME delivered more than 4,800 referrals to the
myCME delivered more than 4,800 referrals to the
activity, which yielded a 29% participation rate!
activity, which yielded a 29% participation rate!
Proprietary and Confidential
62
62

63. New CME/CE Programs from myCME
New CME/CE Programs from myCME
Weekly Update
Weekly Update
63
63

64. Learning Objective
5. Summarize strategies to increase success in
PI-CME
Harvey C. Parker, Ph.D., CCMEP
Gregg Sherman, MD

65. Strategies to Improve Your Performance in PI-CME
• Collaborate with partners that have PI-CME
experience
• Have an aggressive marketing plan for
audience generation
• Develop systems to keep learners involved
(i.e. e-mail reminders, postcards, phone
contact, etc.)
• Use evidence-based performance measures
• If your activity is web-based use a platform
that is easy to access and navigate

66. Strategies to Improve Your Performance in PI-CME
• Make certain your PI CME platform can track learner data and
can provide feedback to your learners so they can assess
their performance
• Develop meaningful interventions that provide value to
learners and tools that can be immediately implemented into
practice based on your learner’s specific needs
• Keep data collection as simple as possible
For more information on Making Exam Room Decisions for Adults
with ADHD and outcome data see:
Parker, H., et al, Improving the Diagnosis, Treatment and Follow-
up of Adult ADHD Patients in Primary Care. CE Meas.
2012;6:3-12.
Our thanks to Derek Dietz at Improve CME for his invaluable
assistance in reviewing this manuscript.

67. Questions and Discussion
Thank You For Your Attention!