Catholic Relief Services partners with Dimagi Inc. and Vatsalya to implement the Reducing Maternal and Newborn Deaths (ReMiND) Pilot Project in 2 blocks of Kaushambi District in Uttar Pradesh (India). ReMiND has pioneered a supervision approach that uses Dimagi's CommCare technology and active use of real-time data generated from community health workers' and supervisors' use of mobile phones to improve the quality and timeliness of government community health workers' (called ASHAs) home visits to pregnant women, newborns and young children.
2. SUPPORTIVE SUPERVISION & MONITORING
Analysis &
use of
ASHAs’
real-time
data
Outcome:
Maternal, newborn &
child health outcomes
CommCare
HQ
Supportive
Supervision
Application
Quality:
IPC & counseling
skills, Mobile skills
Analysis &
use of real-
time
supervision
data
Active Data
Management
Reports
Output:
Frequency &
timeliness of ASHA
home visits
SMS
reminders
& missed
visit alerts
2
7. 76%
78%
91%
61%
79%
98%
50%
59%
59%
24%
58%
85%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
ASHA encouraged woman to use next health
service
ASHA waited for client to respond before
moving on
ASHA expanded on ICT audio message
Client asked any questions
ASHA encouraged client to speak or ask
questions
ASHA greeted woman
Trends in quality of ASHA home visits
Sep-12 Mar-13
8. HEALTH OUTPUTS & OUTCOMES
Indicator (Data) Jan – March
2013
Number of new pregnancy cases registered in CommCare
in quarter
1,084
Total number pregnant women tracked with CommCare in
quarter
2,286
Total number pregnancy cases closed during the quarter 918
% CommCare clients who were registered with ANM before
the fourth month of their pregnancy
79.8%
% CommCare clients giving birth at a health facility 74.4%
8
9. 9
PERFORMANCE FEEDBACK RESEARCH
Conducting study with Harvard Business School and University of
Washington on the effect of personalized feedback graphs
provided to the ASHAs on their Performance
10. SUMMARY OF BENEFITS
10
Improved Accountability:
- Real time monitoring of daily activity of each ASHA
- Active Data Management gives key information to Supervisors to
improve workforce performance
- Notifications to Supervisors of missed visits
Improved Access to Information:
- More timely data
Improved Experience
- Improved credibility of ASHA
- Multimedia to improve usability for all workers
Scalability:
- Supervision applications are transferrable to ASHA Facilitator cadre
- The ReMiND intervention is scalable, given the ASHA
Facilitator/Supervisors are in place, motivated, and effective.
ReMiND is exploring ways to leverage mobile technology to strengthen ASHA supportive supervision and monitoring. This is currently done at 3 levels:Output level monitoring and support focuses on tracking and improving the frequency and timeliness of ASHA home visits. This is done through weekly and monthly Active Data Management Reports. With the planned roll-out of the postpartum & infant applications, a system will be introduced that includes SMS reminders to ASHA to conduct home visits with missed visit alerts to ASHA supervisors if the visits are not conducted on time.ReMiND is also using a CommCare-based mobile application for supportive supervision that includes tools for monitoring the quality of ASHA’s interpersonal communication and counseling skills during home visits, as well as their mobile use skills. This application is currently used by ReMiND Project staff, but is easily transferrable to ASHA Facilitators once that cadre is in place and to other ASHA supervisors. Real-time data generated from the use of the supportive supervision data can then be used to target support low performing ASHA and recognize those who are high performing.At the highest level, ASHAs’ use of their CommCare-based job aids (mobile application) generates real-time health outcome data for every pregnant woman, newborn and infant they have registered. The project is currently doing quarterly analysis of health outcome data, but via CommCare HQ (Dimagi’s cloud-based server) the data can be accessed and analyzed at any time.
Active Data Management (or ADM) reports are generated sub-center-wise on a weekly and monthly basis. They are available in Hindi and in English It draws from time-print data (i.e., when and for how long the ASHA uses the mobile application) generated automatically from the ASHA’s use of the CommCare application on her phone. Based on that, it is possible to know: Number of active/inactive ASHAs in the last 7 days, last 30 daysSummary of client follow up: number of active/inactive cases, client follow-up the last 30, 60, 90 days, late visits.CONTINUE SUMMARY OF ADM INFO ON THE NEXT SLIDE
ADM information summary (continued):Which ASHAs have been recently inactive (i.e., not visited any clients)Which ASHAs have low or high coverage of their registered clients (i.e., % of registered clients recently visited) Which ASHAs have registered new pregnancy or newborn/infant cases during the last week.SATISH – PLEASE ADD SOME BRIEF DETAIL ABOUT HOW THE PROJECT CURRENTLY USES THE ADMS…ESPECIALLY WHEN/HOW IT’S SHARED WITH GOVERNMENT. I’VE CUT AND PASTED THE INFORMATION BELOW FROM THE ORIGINAL SLIDES, BUT FEEL FREE TO ADD/ADAPT THE FACILITATION NOTES AS REQUIRED.ADM Reports are currently used for:Reflection and planning at ASHA monthly meetings, sector level ASHA meetings, project planning meetingsNeed based planning by Sector Facilitators, prioritizing poor performing ASHAsProject Reporting
The SMS reminder and supervisor missed-visit alert system that ReMiND is planning to introduce with the roll-out of the its postpartum and newborn/infant applications is based on a CommCare-based system that was originally tested with Community Health Workers in Tanzania.This graph shows the impact that SMS reminders and missed visit alerts had on the frequency of on-time visits by the CHWs:A steady decrease is seen in the number of on-time visits for CHWs who received no SMS reminders with no supervisor missed visit alerts (control)In Study 1, CHW who received SMS reminders with missed visit alerts to supervisors showed high and more stable trend in on-time visits.In Study 2, missed visit alerts to supervisors were dropped for some CHW. The drop in the frequency of on-time visits is noticeable in the SMS-only group while the CHWs who continued to get both the SMS reminders with missed visit alerts to their supervisors maintained the higher frequency of on-time visits.The researcher (Brian DeRenzi) who did the study is currently part of the operations research team working with the ReMiND project to explore ways that mobile technology can be used to strengthen community health worker (ASHA) performance (i.e., performance feedback graphs).
The supportive supervision and monitoring tool application that ReMiND is using allows supervisors to first register each ASHA in her area and then to individual follow-up and monitoring of each. The application contains the following forms/tools:ASHA Follow up Form : Reports details of follow up visits with ASHA Home Visit Observation Form : Tracks quality of ASHAs interpersonal communication and counseling skills with clients based on an observation checklist that the supervisor uses while accompanying ASHAs on home visits.Mobile Experience survey: Assesses ASHAs’ skills at using the mobile application on her phone and other mobile use skills.Technology Issue form : Gives picture of reported and resolved technological issues of ASHA’s phones From the Technology Issue form is a Tech Issue Follow-up form that can be filled to report the status and resolution of the tech issues.Meeting Form: Captures information about the type, time, place, participants and decisions made in different meetings attended (e.g., ASHA monthly meeting, ANM meetings, etc.)All of the data the supervisors collect using this supportive supervision application is uploaded onto CommCare HQ where it can be regularly analyzed to track the quality of ASHA performance (home visit observation) and identify areas for strengthening.This data is used in:Discussion with ASHAs by HEO during meetingsPlanning of SFs – to prioritize support to poor performing ASHAs) Project Reporting
This graph summarized data submitted through project staff’s use of the Home Visit Observation form during monthly follow-up with ASHAs. It reflects only a selection of the observation points in the checklist. Based on the trends in monthly observations, ASHAs were given specific feedback on how they could improve the quality of their IPC and counseling during home visits. The quality improvements are noticeable between September 2012 (when CommCare was just being scale-up to ASHAs in Manjhanpur Block) and March 2013.
I’D LIKE TO ADD IN MORE DATA POINTS, JUST HAVEN’T HAD TIME TO DO IT YET.IF THIS NEEDS TO GO FOR PRINTING AT NOON, JUST DELETE THE EXTRA ROWS BEFORE PRINTING. I THEN STILL MIGHT TRY TO ADD IN MORE INFO TONIGHT.
In collaboration with researchers from Harvard University and the University of Washington, operations research is currently be conducted on how technology can be used to provide feedback to ASHAs and whether/which kind of feedback has the biggest impact on improving ASHA performance.A LITTLE MORE DETAIL (NOTES, NOT SLIDES) ABOUT THE 3 KINDS OF GRAPHS WOULD BE GOOD…
Feel free to edit modify – first pass.MH: I shifted the improved work quality points to presentation 1…nice summary of the benefits of the ASHA app. Have added the ‘scalability’ bit trying to adapt from the scalability point previously in presentation 3…feel free to further refine.
Matt: Not sure if this fits here, but is a very simple explanation of the supervision workflow. Feel free to remove/edit/deleteMH: I’m not quite sure what to put in here for facilitation notes. Maybe it’s better to not include in the main body of the presentation? Though we can leave it tacked on here at the end as an “extra” slide should it be useful during the discussion time???