The rapid expansion of mobile phone usage in low-income and middle-income countries has created unprecedented opportunities for applying AI to improve individual and population health. In benshi.ai, a non-profit funded by the Bill and Melinda Gates Foundation, the goal is to transform health outcomes in resource-poor countries through advanced AI applications. We aim to do so by providing personalized predictions and recommendations to support diagnosis to medical care teams and frontline workers, as well as to nudge patients through personalized incentives towards an improvement in disease treatment management and general wellness. To this end, we have built an operational machine learning platform that provides personalized content and interventions real-time. Multiple engineering and machine learning decisions have been made to overcome different challenges and to build an experimentation engine and a centralized data and model management system for global health. Databricks served as a cornerstone upon which all our data/ML services were built. In particular, MLflow and dbx (an opensource tool from Databricks) have been crucial for the training, tracking and management of our end-to-end model pipelines. From the data science perspective, our challenges involved causal inference analysis, behavioral time series forecasting, micro-randomized trials, and contextual bandits-based experimentation at the individual level. This talk will focus on how we overcome the technical challenges to build a state-of-the-art machine learning platform that serves to improve global health outcomes.

1. Towards Personalization in Global Digital Health
28 May 2021 África Periáñez, PhD
Data + AI Summit 2021 Founder & CEO benshi.ai
ai for equitable healthcare

2. Accelerating and democratizing behavioral
machine learning for low- and middle- income
countries to reduce health inequalities

3. Mission
▸ Provide real-time and just in time personalized incentives and
recommendations to frontline health workers and patients
▸ Utilize data-driven insights and predictions for individual
behaviors towards shaping strategies for collective behaviors

4. ▸ Computationally operationalize large-scale digital
traces from mobile health devices
▸ Turn behavioral logs into robust personalized
scientific results and move towards causal analysis
beyond correlations
▸ Leverage fine-grained health logs to advance
science through a better understanding of
behavior and health
Challenges

5. Provide interactive &
actionable data-driven
insights for individual and
collective behaviors
Apply machine
learning models and
experimentations
Reduce global
healthcare
inequities
Deliver personalized
recommendations and
incentives to users
Receive frontline worker
and patient data from
mHealth providers &
local partners
Optimize healthcare
systems to reach
underserved populations
Combine
behavioral,
health &
contextual data
1
2
3
4
5
6
Model of
Change

6. Our
Global
Team
Our passionate team of scientists, engineers, and
creative minds works closely with our partners to
push the frontiers of AI and global health

7. Our
Global
Team
17 Team Members
14 Nationalities

8. Our
Global
Team LEADERSHIP
STAFF MEMBERS

9. Game data
science
Health apps in low
income settings
A machine learning journey from game design to global health impact.

10. Processing layer
▸ Model training
▸ Feature engineering
Machine learning module
Storage layer
▸ Database
▸ Data Lake
Ingestion layer
▸ Data connectors
ML service
▸ Deploy
▸ Monitor
▸ Manage
Experiment service
▸ Define
▸ Track
▸ Compare
Scalable computation
APIs / SDKs
Centralized ML services
mHealth
provider
patients
frontline
workers
health
authority
data insights
Frontend
Dashboards
Impact analysis
Interactive insights
Actionable
recommendations

11. ▸ More than half of all maternal and
newborn deaths happening due to poor
quality of care
▸ Safe Delivery App, an online learning
app, to improve skilled birth attendant
training and support
▸ Goal: personalized and adaptive
learning journey

12. ▸ Predictions on Learning Progress Among Safe
Delivery App users in Ethiopia
▸ Users that predicted to make a significant
progress in the learning feature are midwives

13. ▸ Predicted progress typically increases
with predicted connected time
▸ Most users predicted to progress above
level 5 will spend at least 3 hours using
the app

14. ▸ Tracks model runs
▸ Centralized model registry
API nodes Model nodes
▸ Real-time monitoring
▸ Better UX
▸ Deployment
model management system
▸ Data pipelines
▸ Model training
data lake
log
models
fetch
models
▸ Model management
▸ Define experiments
▸ Monitoring
System Schema
Frontend
API
API

16. Model Management System

17. ▸ dbx makes local development running on Databricks easily
▸ Easy access and manipulation of data in the cloud
▸ For production pipeline we need full control (reliable, right time, right output): dbx2
▸ In dbx2 we added other in house customizations to suit our development needs
Data Pipeline Development

18. benshi ai platform: Machine Learning

19. benshi ai platform: Statistical Analysis

20. Experimentation

21. benshi ai platform: XP Engine

22. MRTs & Contextual Bandits
▸ MRTs involve multiple randomizations and enable causal modeling of
proximal effects of the randomized intervention components
▸ Evaluation of when and for whom interventions are effective, and what
factors moderate the intervention effects
▸ In contextual bandits settings, the system learns the optimal decision
rules and is able to adapt
▸ Bridge connecting experimentation to a continuously adapting system
providing personalized and contextualized interventions

23. Summary
Our goal: to boost health outcomes in resource-poor
countries through personalized incentives to nudge
healthcare teams and patients
▸ Machine Learning platform focusing on: individual behavioral
predictions, recommendations and reinforcement learning
experimentation
▸ Democratizing behavioral machine learning: ease-of-use
integrations, API/SDK, to reach existing health care teams
and frontline workers

24. Thank
you!
Do you want to join us to push the boundaries of
machine learning and global health?
@benshi_ai
We are hiring!
benshi.ai