This document discusses revalidation in New Zealand for non-certified general practitioners. It defines key terms and explains why revalidation was implemented, including ensuring doctor competence and standardizing continuing professional development. The revalidation program includes requirements like an e-portfolio, annual and triennial activities, and a Regular Practice Review involving practice visits. Early results found most doctors viewed the review positively and many reported immediate practice changes. Follow-up surveys found later participants saw greater benefits including improved patient care. As the program continues, results are expected to provide firmer conclusions about its effectiveness and value.
The outcomes & effectiveness of revalidation in New Zealand for non-certified general practitioners
1. Andrew Connolly
Chair, MCNZ
The outcomes & effectiveness of
revalidation in New Zealand for
non-certified general
practitioners
2. Definitions
• “Non-certified” are those general registrants
who have not achieved vocational (specialist)
registration and are not actively in an
accredited specialist training program
– Not restricted to General Practice, but vast
majority are working in this field
• Does not apply to PGY-1
• Will not apply to PGY-2 from 2016
3. Why do it?
• Council legally responsible for ensuring
practicing doctors are competent
• Long-established CPD requirements but a
lack of standardisation, consistency
• College programs all accredited via
Australian Medical Council & MCNZ
• Council recognised the lack of a general
registrant mandated program was a risk
4. Council decisions:
• Electronic format (“e-portfolio”)
• Mandatory unless “grandfathered” by existing
successful participation in an accredited
scheme
• Mix of annual and triennial requirements
– RPR & MSF three-yearly
• Prospective research evaluation by
independent company
5. The Program
• Introduced in May 2012
– Essentials quiz
– Collegial relationship with meetings mandated
– Professional development plan
– Audit
– CPD record
– Regular practice review: began July 2013
– Multi-source feedback
6. The numbers
• Currently some 1800 participants
• Very few non-compliant but Council has
escalation policy including suspension - has
been necessary in a very small number
• Most radical has been the Regular Practice
Review (RPR)
• RPR is prime focus of research
• Numbers are small, but knowledge is growing
7. • A formative assessment via the Best Practice
Advocacy Centre’s Inpractice programme
• Focus is on quality improvement by helping individual
doctors identify areas of improvement and build them
into professional development plan
- A mandatory requirement for doctors registered in a
general scope of practice
- Reviewers visit the participating doctors in their
practices for a one day visit involving review of notes,
observation and discussion
• Will be prospectively studied
The Regular Practice Review (RPR)
Programme
9. Methodology
• Survey at 2 weeks post RPR
• Repeated at 12 months post RPR
– Early cohort of follow-ups
– Later cohort of follow-ups
• “Early” and “Later” cohorts refer to RPR pre
or post July 2014
– Early cohort did not have 2 week survey
– Considerable on-going communication with the
profession about purpose & importance
– Later cohort also saw emphasis on reviewer skills
10. Doctors participating in the evaluation
• Surveys of participants two-weeks and twelve-months post-RPR
Later cohort
17 of 31 doctors
(55% response)
Early cohort:
32 of 45 doctors
(71% response)
Early cohort
Completed
RPR Jan-May
2014
Later cohort
Completing
RPR from June
2014 - ongoing
Two-week post-RPR
survey
Twelve-month post-RPR survey
Did not complete RPR
post survey - not yet
implemented
Completed twelve-
month survey
Closed
Completed two-week
survey
159 of 236 doctors
(67%) - Ongoing
Completed twelve-
month survey
Ongoing
11. The Participants
Doctor characteristics
Two-week
respondents
(n = 159)
12 month
early cohort
(n = 32)
12 month
later cohort
(n = 17)
Role
General
practice 68% 91% 76%
Other 32% 9% 24%
Trained in
New Zealand 36% 34% 35%
Other 64% 66% 65%
Years in
practice
0-10 45% 47% 18%
11-20 43% 38% 76%
30+ 12% 16% 6%
English
first
language
Yes 77% 84% 100%
No 23% 16% -
12. Improvements for patients: 2 weeks
• Just under half strongly agreed or agreed that participating in
RPR had improved the care they deliver to their patients
(44%) and improved their practice in other ways (49%)
13. Changes to practice: Two weeks after
RPR
• Nearly half (46%) said they had already made changes to
their practice as a result of RPR
• A further 13% said they intended to make changes
14. Overall views on RPR: 2 weeks
• The majority (71%) agreed that the practice visit
was a positive experience
• Over half found the report useful (63%) and would
positively recommend RPR to colleagues (56%)
15. Examples of immediate changes
• Improved records and note taking – most commonly mentioned
Ensuring appropriate documentation of clinical notes. Going
deeper into patient history beyond presenting complaint.
• The consultation – style and interaction with patients
Tried to change consultation style, trying to prioritise patient
questions.
Communicating more effectively with patients
• Review of prescribing and ordering lab tests
I am a bit more critical about which lab tests I order.
•Improving cultural competence
Taking specific interest in Māori and Pacific cultural aspects of
patients and trying to integrate them in consultations.
16. Changes to professional development plans
(PDPs) at two weeks
• Doctors evenly divided between those who had and
had not made changes to their PDP
• One-third (34%) had discussed their PDP with the
reviewer
• Just over half (52%) agreed that their professional
development plan is a useful tool to improve their
practice
17. Changes to practice: Twelve months after
RPR
59%
19%
I have made changes to my practice
Early cohort
Later cohort
• Early group lacked knowledge of the process & of the
importance of the program; later reviewers were more skilled
• Improved relationship with collegial supervisor
• Small numbers, but encouraging
18. Improvements for patients: Twelve
months
• The later cohort were much more likely to agree that RPR had
improved care (53% compared to 18%)
19. Changes to PDP: 12 months – Early cohort
• Similar proportions – about one-fifth – reported making
changes to their PDP, the way they manage it, and that
changes have made it more useful
20. Changes to PDP: 12 months – later cohort
• Later cohort was much more likely to agree that RPR led
them to make changes to their PDPs and that those changes
made their PDPs more useful
21. Overall views on RPR: Twelve
months
• Later cohort much more likely to view RPR as
useful or very useful - 53% compared to 31%
• Later cohort much more likely to recommend RPR
to their colleagues – 65% compared to 28%
22. What about the RPR reviewers?
• Almost every reviewer provided an example of a benefit to
themselves. Some benefits were personal:
Seeing a variety of different GPs in their practices and consulting
has been extremely interesting.
• Online survey and interviews in 2014 – to be repeated in
2015/16. All nineteen reviewers responded in 2014.
23. Reviewer comments
• I learn from the positives I observe in sitting through
consults of my colleagues and use it in my own
practice.
• It has also been an opportunity to chew the fat over
both ordinary and contentious issues and to hear fresh
and diverse perspectives. I too feel under the looking
glass by those I review so it puts pressure on me to
ensure I am really up to date so I appear credible.
24. Summary
• Early experience is encouraging
• Better knowledge of the purpose and
importance of revalidation appears crucial to
sustained success
• Reviewer skills and understanding are
important
25. Other Groups
• Obstetrics & Gynaecology – voluntary
• General Practice specialists – voluntary
• Orthopaedics- mandatory
– 95% pre-RPR were opposed
– 95% post-RPR were in favour
• Greatest benefits in work/life balance, scope of practice
26. Conclusions to Date
• Early indications of value of RPR
– higher proportions reporting sustained benefits
• Positive influence on professional
development
• Improved collegial relationships
• Positive influence on practice
• Positive influence on patient care
27. Next Steps
• Continue to highlight the importance of
revalidation
• Continued 2-week and 12 month follow-ups
• As numbers grow we will be able to firm our
conclusions
– Including begin to look at competence concerns
• Further survey of reviewers
• Map progress into vocational training
• Ultimately see processes adopted by other
vocational groups