Slides presented by Eric C. Schneider, MD, at the 2016 Grantmakers in Aging annual conference on Oct. 27, 2016. Eric C. Schneider, MD, MSc, FACP, is the Senior Vice President for Research and Policy at The Commonwealth Fund. The Commonwealth Fund is a private foundation that aims to promote a high performing health care system that achieves better access, improved quality, and greater efficiency, particularly for society's most vulnerable, including low-income people, the uninsured, minority Americans, young children, and elderly adults.

1. Evaluation in an Era of
Digital Technology Innovation
Reflections for Philanthropy
Eric C. Schneider, MD, MSc, FACP
Senior Vice President for Research and Policy
@ericschneidermd

2. Outline
• Challenges in evaluating digital health technology
• A human-centered measurement approach
• Concluding thoughts

3. Evaluating digital technology:
Are these devices?
• Clinical trials testing
• Five-phase model
• Reliance on randomized, controlled trials
• Highly-selected patient samples
• Clinical outcomes/surrogate biological markers
• 12-18 years from lab to marketing

4. Digital health technologies:
Many value propositions
• Increase patient engagement
• Close communication gaps between patient and providers
• Improve providers’ ability to tailor services to population
segments
• Enable convenient and cost-effective care delivery
• Improve decision making
4
M. Hostetter, S. Klein, and D. McCarthy, Taking Digital Health to the Next
Level: Promoting Technologies That Empower Consumers and Drive Health
System Transformation, The Commonwealth Fund, October 2014.

5. Innovation on Three Different Time Horizons
Unknown
Known
Partially
known
Existing Business
Model: Process
Innovation
Execute
New/Disruptive
Business Model
Search
New Opportunities
via Business Model
Innovation
Execute/Search
Source: Steve Blank, Lean Innovation Management – Making Corporate
Innovation Work
Digital technology innovation is unpredictable

6. Dependencies:
VIVIFY Health and CHRISTUS St. Michael Health
• Goal: Reduce 30-day readmissions and reduce
Care Transition Nurse case load
• Implement a remote patient monitoring solution
• Transmits patient data using personal health devices
(tablets, weight scales, blood pressure cuffs, etc.)
• If patient data indicates a health concern, device alerts
Care Transition Nurse via text/email

7. Measurement Myopia:
VIVIFY Health Evaluation Results
• Pilot study: patients with chronic congestive heart failure
who consented to participate in care transition program
• Results
• 44 patients
• Decrease in cost of care from $12,937/per patient to $1,231/per
patient
• ROI of $2.44, including cost of technology
• A winner?

8. Evaluating Smarter:
VIVIFY Health’s Qualitative Insights
•Connectivity issues
•Patients outside of cellular
service areas
•Equipment issues
• Tablets overheated
•Bugs!
• Packaging promoted egg-laying
Ambar Kulshreshtha JCK, Abhinav Goyal, Elkan F. Halpern, and Alice J
Watson. Use of Remote Monitoring to Improve Outcomes in Patients
with Heart Failure: A Pilot Trial. International Journal of Telemedicine
and Applications. 2010;2010(870959):7

9. A New Approach: Human-Centered Design
www.sagehealthadvisor.com
EMOTIONAL
NEEDS
PERSONAL
NEEDS
FUNCTIONAL
NEEDS
MEDICAL
NEEDS

10. Measuring the impact of digital health technologies
Technology
•Feasibility
•Functionality
•Unexpected
Bugs
Workflow
•Take up
•Use Patterns
•Stickiness
•Safety
•Unintended
Consequences
•Costs
Clinical
•Adherence to
Treatment
•Use of Health
Services
•Quality of
Care
•Patient
Engagement
•Patient
Experience
•Provider
Experience
Person/Life
•Social Relationships
•Quality of Life
•Caregiver experience
•Self-Efficacy
•Functioning
•Health Status
•Emotional Status

11. Conclusion: Evaluation Approach
•Start with a “person/life”
holistic view
•Assess technology
compatibility with human-
centered design principles
•Beware of workflow and
adjacent technologies
•Expect the unexpected!

12. References
1. Jones SS, Heaton PS, Rudin RS, Schneider EC. Unraveling the IT productivity
paradox--lessons for health care. N Engl J Med. 2012;366(24):2243-2245.
2. HIMSS. Case Study: Decreasing Costs and Improving Outcomes Through
Community-Based Care Transitions and Care Coordination Technology
Healthcare Information and Management Systems;March 2014.
3. Broderick A. Partners HealthCare: Connecting Heart Failure Patients to
Providers Through Remote Monitoring. Online: The Commonwealth Fund;2013.
4. Ambar Kulshreshtha JCK, Abhinav Goyal, Elkan F. Halpern, and Alice J Watson.
Use of Remote Monitoring to Improve Outcomes in Patients with Heart Failure:
A Pilot Trial. International Journal of Telemedicine and Applications.
2010;2010(870959):7.
5. Rudin et al. The value of health information technology: filling the knowledge
gap. Am J Managed Care. 2014; eSP1-eSP8.
12