SlideShare uma empresa Scribd logo
1 de 26
Baixar para ler offline
© Nuffield Trust22 June 2015
Matched Control Studies:
Methods and case studies
Cono Ariti
cono.ariti@nuffieldtrust.org.uk
© Nuffield Trust
Predictive risk
modelling
Resource
allocation
Descriptive
studies Evaluations
Integrated
care
pilots
nuffield trust
Nuffield Trust Research team – data linkage projects
Risk
sharing
for CCGs
nuffield trust
Combined
predictive
model
nuffield trust
Person
based
resource
allocation
nuffield trust
Social
care at
end of life
nuffield trust
Cancer
and social
care
nuffield trust
Predicting
social
care
costs
nuffield trust
Virtual
Wards
nuffield trust
WSD
nuffield trust
British Red
Cross
nuffield trust
© Nuffield Trust
Need for evaluation
Need to know what works
• In a practical setting – “real world evaluation”
• Clarify the debate
• Likely impacts – unbiased results
• Link to qualitative work
Refine programs
• Obtain feedback and learnings – the pain of
implementation
• Explore sub-groups – where did it work? Where
could it work?
© Nuffield Trust
Issues with evaluations
Randomised control trials
• “Gold standard”
• May not be feasible or ethical
• Inclusion and exclusion rules can limit generalisation
• Are still subject to poor implementation – can induce
bias
• Potentially expensive!
Observational studies
• Typically no “natural” experiment exists
• Often no comparable control group to provide a fair
assessment
© Nuffield Trust© Nuffield Trust
Matched Control Studies -
Methods
© Nuffield Trust
Matched Control Studies
The basic idea
• Match controls to those treated based on measured
characteristics in existing datasets
• The control group and treated group should look
similar “on balance”
• Mimics the idea of an RCT
• Based on propensity score theory (Rubin &
Rosenbaum, 1983) and earlier work on matching
(Cochran, 1965)
© Nuffield Trust
Matched Control Studies
Matching
• Prognostic risk score
• Demographics – age, gender, deprivation, ethnicity
• Prior acute care service use – admissions, OP and
A&E attendances
• Prior diagnoses, targeted chronic conditions
Balance
• In this case all matching variables
• Additional variables such as length of stay, additional
diagnoses and longer service use history
• Assures comparability between the groups
© Nuffield Trust
Matching Algorithm
Algorithm
• Exact match not possible
• Computer intensive “genetic algorithm”
• Uses a weighted Mahalanobis “distance” to
determine closest match
• Automatically assesses balance and moves to an
improved solution
Assessing Balance
• On overall group similarity
• Compares means and distribution of variables in the
two groups
© Nuffield Trust
Analysis of matched control studies
• Standard statistical methods to estimate the difference
in the two groups
• Regression models, difference in difference analysis
• By including matching variables in the statistical
adjustment remaining imbalances can be reduced –
“doubly robust”
• Methods exist for sensitivity analysis – impact of
unobserved variables
• Some controversy around accommodating the
matching in analysis
© Nuffield Trust© Nuffield Trust
Case Study 1: Telehealth
Programme
© Nuffield Trust
Case Study 1: Telehealth program
Intervention:
• Remote monitoring for patients with long term conditions
Nuffield commissioned to evaluate impact:
• Primary: Reduction in emergency hospital admissions?
• Secondary: Reductions in Emergency attendances, outpatient
attendances, mortality
Methods:
• Retrospective matched control study – use of already existing
administrative data
© Nuffield Trust
Description: Telehealth program
© Nuffield Trust
Matched control studies – broad aim
>30,000 individuals – resident in local area June
2010 to March 2013, did not receive telehealth
and were eligible for matching
(local controls)
Aim to find 716 individuals who match
almost exactly on a broad range of
characteristics
Use this group as study control group
716 individuals – enrolled June 2010 to March 2013
& received Telehealth intervention & eligible for
matching
© Nuffield Trust
Datasets available
Telehealth Nuffield trust
N = 716
• person details
• dates of service
• type of service
Identifiers:
Names, DOB,
Addresses, etc
• dates & place
of death for all
people in
England,
• associated
hospital (HES)
records
Identifiers:
Nuffield Trust
specific HESID
Administrative data ONS deaths Hospital inpatient, outpatient, AE
© Nuffield Trust
1
2
3
4
5
6
7
8
9
. .
1
2
3
4
5
6
7
8
9
. .
1
2
3
4
5
6
7
8
9
. .
Telehealth Data Linkage Service Nuffield Trust
New Identifier New Identifier New Identifier
(NHS no) (NHS no)
Names Names
Address Address
DOB DOB
HESID HESID
Telehealth person identifiers (File A)
© Nuffield Trust
Final datasets available for analysis
Nuffield trust
Identifiers:
HESID on all
ONS deaths Hospital inpatient, outpatient, AETelehealth data - desensitised
Use all this
info to carry
out matched
control
analysis
© Nuffield Trust
Control group – how well matched?
© Nuffield Trust
Control group – how well matched?
Telehealth Controls
© Nuffield Trust
Control group – how well matched?
Telehealth Controls
© Nuffield Trust
Key Result 1: Risk of admissions or death
© Nuffield Trust
Key Result 2: Changes in admissions or attendances
(six months pre and post intervention)
© Nuffield Trust
Results
• Telehealth patients tended to be admitted for an
emergency admission earlier than control patients
• There was no difference in mortality between the
telehealth and control groups
• There were no statistically significant reductions in
hospital admissions when comparing the period six
months before and after the telehealth intervention
• In summary the Telehealth program did not have a
significant impact on acute care outcomes
• Sensitivity analysis showed little evidence of an
important unobserved variable
© Nuffield Trust© Nuffield Trust
Matched Control Studies:
Summary
© Nuffield Trust
Matched Controls: Summary
Benefits
• Makes full use existing data, with relative ease
• Techniques applicable to many different types of
services and datasets
• Decisions on what seems to work (and what may
not) based on more robust analyses leading to better
informed decisions
Caveats
• If important unobserved variables exist results may
be biased
• The routine data sources must contain the relevant
data
© Nuffield Trust
Implementing locally – key enablers
Do you have … Can you …
• Access to data that contains
the outcomes relevant to your
evaluation?
• Access to data containing
relevant matching
characteristics?
• Do you have consent to
access/link the data?
• Analysis tools to apply
statistical methods to the data?
• Skilled analysts to analyse the
data?
• Link multiple sources of data?
• Handle large amounts of data
(millions of observations)?
• Identify recipients of the
intervention?
• Transform and augment that data
with bespoke variables?
• Apply sophisticated matching
algorithms routinely to this data?
• Analyse the data with a variety of
statistical methods and interpret
the results appropriately?
© Nuffield Trust
www.nuffieldtrust.org.uk
Sign-up for our newsletter
www.nuffieldtrust.org.uk/newsletter
Follow us on Twitter:
Twitter.com/NuffieldTrust
© Nuffield Trust
Thank you

Mais conteúdo relacionado

Mais procurados

Shaping Informatics for Allied Health - Refining our voice
Shaping Informatics for Allied Health - Refining our voiceShaping Informatics for Allied Health - Refining our voice
Shaping Informatics for Allied Health - Refining our voiceHealth Informatics New Zealand
 
Knowledge synthesis for policy and practice
Knowledge synthesis for policy and practiceKnowledge synthesis for policy and practice
Knowledge synthesis for policy and practiceSax Institute
 
SURE: Helping get the most out of longitudinal data
SURE: Helping get the most out of longitudinal dataSURE: Helping get the most out of longitudinal data
SURE: Helping get the most out of longitudinal dataSax Institute
 
Providing actionable healthcare analytics at scale: A perspective from stroke...
Providing actionable healthcare analytics at scale: A perspective from stroke...Providing actionable healthcare analytics at scale: A perspective from stroke...
Providing actionable healthcare analytics at scale: A perspective from stroke...Nuffield Trust
 
Effectiveness of the current dominant approach to integrated care in the NHS:...
Effectiveness of the current dominant approach to integrated care in the NHS:...Effectiveness of the current dominant approach to integrated care in the NHS:...
Effectiveness of the current dominant approach to integrated care in the NHS:...Sarah Wilson
 
Safety, Sleuthing and Students: A Novel Collaborative MedRec Event
Safety, Sleuthing and Students: A Novel Collaborative MedRec EventSafety, Sleuthing and Students: A Novel Collaborative MedRec Event
Safety, Sleuthing and Students: A Novel Collaborative MedRec EventCanadian Patient Safety Institute
 
New Models of General Practice: Practical and policy lessons
New Models of General Practice: Practical and policy lessonsNew Models of General Practice: Practical and policy lessons
New Models of General Practice: Practical and policy lessonsNuffield Trust
 
Introduction - The Emergency Laparotomy Collaborative
Introduction - The Emergency Laparotomy CollaborativeIntroduction - The Emergency Laparotomy Collaborative
Introduction - The Emergency Laparotomy CollaborativeInnovation Agency
 
Health IT Summit in Seattle 2014 – “Think Big, Act Small” with Deborah Dahl, ...
Health IT Summit in Seattle 2014 – “Think Big, Act Small” with Deborah Dahl, ...Health IT Summit in Seattle 2014 – “Think Big, Act Small” with Deborah Dahl, ...
Health IT Summit in Seattle 2014 – “Think Big, Act Small” with Deborah Dahl, ...Health IT Conference – iHT2
 
Engage and Retain Patients in Long-term Observational Studies
Engage and Retain Patients in Long-term Observational StudiesEngage and Retain Patients in Long-term Observational Studies
Engage and Retain Patients in Long-term Observational StudiesJohn Reites
 
Alicia O' Cathain: qualitative research and evaluation
Alicia O' Cathain: qualitative research and evaluationAlicia O' Cathain: qualitative research and evaluation
Alicia O' Cathain: qualitative research and evaluationNuffield Trust
 
Preparing for the Board Assurance framework for 7DS with guest speaker from U...
Preparing for the Board Assurance framework for 7DS with guest speaker from U...Preparing for the Board Assurance framework for 7DS with guest speaker from U...
Preparing for the Board Assurance framework for 7DS with guest speaker from U...NHS England
 
Collaborative Solutions eHealth Event - CSIRO
Collaborative Solutions eHealth Event - CSIROCollaborative Solutions eHealth Event - CSIRO
Collaborative Solutions eHealth Event - CSIROCollaborative Solutions
 

Mais procurados (20)

Shaping Informatics for Allied Health - Refining our voice
Shaping Informatics for Allied Health - Refining our voiceShaping Informatics for Allied Health - Refining our voice
Shaping Informatics for Allied Health - Refining our voice
 
Cordio
CordioCordio
Cordio
 
Making surgical practice improvement easy
Making surgical practice improvement easyMaking surgical practice improvement easy
Making surgical practice improvement easy
 
Knowledge synthesis for policy and practice
Knowledge synthesis for policy and practiceKnowledge synthesis for policy and practice
Knowledge synthesis for policy and practice
 
Surveillance of social media: Big data analytics
Surveillance of social media: Big data analyticsSurveillance of social media: Big data analytics
Surveillance of social media: Big data analytics
 
SURE: Helping get the most out of longitudinal data
SURE: Helping get the most out of longitudinal dataSURE: Helping get the most out of longitudinal data
SURE: Helping get the most out of longitudinal data
 
Providing actionable healthcare analytics at scale: A perspective from stroke...
Providing actionable healthcare analytics at scale: A perspective from stroke...Providing actionable healthcare analytics at scale: A perspective from stroke...
Providing actionable healthcare analytics at scale: A perspective from stroke...
 
Effectiveness of the current dominant approach to integrated care in the NHS:...
Effectiveness of the current dominant approach to integrated care in the NHS:...Effectiveness of the current dominant approach to integrated care in the NHS:...
Effectiveness of the current dominant approach to integrated care in the NHS:...
 
Safety, Sleuthing and Students: A Novel Collaborative MedRec Event
Safety, Sleuthing and Students: A Novel Collaborative MedRec EventSafety, Sleuthing and Students: A Novel Collaborative MedRec Event
Safety, Sleuthing and Students: A Novel Collaborative MedRec Event
 
New Models of General Practice: Practical and policy lessons
New Models of General Practice: Practical and policy lessonsNew Models of General Practice: Practical and policy lessons
New Models of General Practice: Practical and policy lessons
 
Introduction - The Emergency Laparotomy Collaborative
Introduction - The Emergency Laparotomy CollaborativeIntroduction - The Emergency Laparotomy Collaborative
Introduction - The Emergency Laparotomy Collaborative
 
Health IT Summit in Seattle 2014 – “Think Big, Act Small” with Deborah Dahl, ...
Health IT Summit in Seattle 2014 – “Think Big, Act Small” with Deborah Dahl, ...Health IT Summit in Seattle 2014 – “Think Big, Act Small” with Deborah Dahl, ...
Health IT Summit in Seattle 2014 – “Think Big, Act Small” with Deborah Dahl, ...
 
Engage and Retain Patients in Long-term Observational Studies
Engage and Retain Patients in Long-term Observational StudiesEngage and Retain Patients in Long-term Observational Studies
Engage and Retain Patients in Long-term Observational Studies
 
ED Safety Checklist Masterclass Presentation
ED Safety Checklist Masterclass PresentationED Safety Checklist Masterclass Presentation
ED Safety Checklist Masterclass Presentation
 
Medicines Breakthrough Collaborative 1
Medicines Breakthrough Collaborative 1Medicines Breakthrough Collaborative 1
Medicines Breakthrough Collaborative 1
 
Alicia O' Cathain: qualitative research and evaluation
Alicia O' Cathain: qualitative research and evaluationAlicia O' Cathain: qualitative research and evaluation
Alicia O' Cathain: qualitative research and evaluation
 
Qipp increasing productivity using existing resources
Qipp increasing productivity using existing resourcesQipp increasing productivity using existing resources
Qipp increasing productivity using existing resources
 
QIPP: An Overview
QIPP: An OverviewQIPP: An Overview
QIPP: An Overview
 
Preparing for the Board Assurance framework for 7DS with guest speaker from U...
Preparing for the Board Assurance framework for 7DS with guest speaker from U...Preparing for the Board Assurance framework for 7DS with guest speaker from U...
Preparing for the Board Assurance framework for 7DS with guest speaker from U...
 
Collaborative Solutions eHealth Event - CSIRO
Collaborative Solutions eHealth Event - CSIROCollaborative Solutions eHealth Event - CSIRO
Collaborative Solutions eHealth Event - CSIRO
 

Destaque

Changes to the Taxation of Residential Real Estate - What is "Residential Pro...
Changes to the Taxation of Residential Real Estate - What is "Residential Pro...Changes to the Taxation of Residential Real Estate - What is "Residential Pro...
Changes to the Taxation of Residential Real Estate - What is "Residential Pro...Christopher Connors
 
คำพิพากษาศาลปกครองสูงสุดกรณีมาตรา 12
คำพิพากษาศาลปกครองสูงสุดกรณีมาตรา 12 คำพิพากษาศาลปกครองสูงสุดกรณีมาตรา 12
คำพิพากษาศาลปกครองสูงสุดกรณีมาตรา 12 Pairat Dam
 
Resume (1) patricia stibor
Resume (1) patricia stiborResume (1) patricia stibor
Resume (1) patricia stiborPatricia Stibor
 
Waseem Al-Ezz Resume
Waseem Al-Ezz ResumeWaseem Al-Ezz Resume
Waseem Al-Ezz Resumewaseem alezz
 

Destaque (10)

Cabincrewdirect.com
Cabincrewdirect.comCabincrewdirect.com
Cabincrewdirect.com
 
достижение Final
достижение Finalдостижение Final
достижение Final
 
Changes to the Taxation of Residential Real Estate - What is "Residential Pro...
Changes to the Taxation of Residential Real Estate - What is "Residential Pro...Changes to the Taxation of Residential Real Estate - What is "Residential Pro...
Changes to the Taxation of Residential Real Estate - What is "Residential Pro...
 
คำพิพากษาศาลปกครองสูงสุดกรณีมาตรา 12
คำพิพากษาศาลปกครองสูงสุดกรณีมาตรา 12 คำพิพากษาศาลปกครองสูงสุดกรณีมาตรา 12
คำพิพากษาศาลปกครองสูงสุดกรณีมาตรา 12
 
Resume (1) patricia stibor
Resume (1) patricia stiborResume (1) patricia stibor
Resume (1) patricia stibor
 
Waseem Al-Ezz Resume
Waseem Al-Ezz ResumeWaseem Al-Ezz Resume
Waseem Al-Ezz Resume
 
work
workwork
work
 
Document1
Document1Document1
Document1
 
TCS Certificate
TCS CertificateTCS Certificate
TCS Certificate
 
Czech language test - A2
Czech language test - A2Czech language test - A2
Czech language test - A2
 

Semelhante a Cono Ariti: matched control studies

Evaluating Service Innovations for Older People - Martin Bardsley, Nuffield T...
Evaluating Service Innovations for Older People - Martin Bardsley, Nuffield T...Evaluating Service Innovations for Older People - Martin Bardsley, Nuffield T...
Evaluating Service Innovations for Older People - Martin Bardsley, Nuffield T...Ian Brown
 
Dr Martin Bardsley: Use of Retrospective Matching Methods 30 June 2014
Dr Martin Bardsley: Use of Retrospective Matching Methods 30 June 2014Dr Martin Bardsley: Use of Retrospective Matching Methods 30 June 2014
Dr Martin Bardsley: Use of Retrospective Matching Methods 30 June 2014Nuffield Trust
 
AIRAEvaluation_20150423
AIRAEvaluation_20150423AIRAEvaluation_20150423
AIRAEvaluation_20150423Lauren Snyder
 
Seven Day services - Healthcare Science presentation (Blower)
Seven Day services - Healthcare Science presentation (Blower)Seven Day services - Healthcare Science presentation (Blower)
Seven Day services - Healthcare Science presentation (Blower)AHCS
 
Martin Bardsley: integration and innovation in health
Martin Bardsley: integration and innovation in healthMartin Bardsley: integration and innovation in health
Martin Bardsley: integration and innovation in healthNuffield Trust
 
Operational research to increase the efficiency of ART initiation in Africa
Operational research to increase the efficiency of ART initiation in AfricaOperational research to increase the efficiency of ART initiation in Africa
Operational research to increase the efficiency of ART initiation in AfricaSydney Rosen
 
Dr. Martin Bardsley Digital Health Assembly 2015
Dr. Martin Bardsley Digital Health Assembly 2015Dr. Martin Bardsley Digital Health Assembly 2015
Dr. Martin Bardsley Digital Health Assembly 2015DHA2015
 
Xavier Chitnis: the effects of telehealth on service use
Xavier Chitnis: the effects of telehealth on service use Xavier Chitnis: the effects of telehealth on service use
Xavier Chitnis: the effects of telehealth on service use Nuffield Trust
 
Ian Blunt & Martin Bardsley: Cause for concern
Ian Blunt & Martin Bardsley: Cause for concernIan Blunt & Martin Bardsley: Cause for concern
Ian Blunt & Martin Bardsley: Cause for concernQualityWatch
 
Holly Holder & Ian Blunt: Integrated care pilot evaluation
Holly Holder & Ian Blunt: Integrated care pilot evaluationHolly Holder & Ian Blunt: Integrated care pilot evaluation
Holly Holder & Ian Blunt: Integrated care pilot evaluationNuffield Trust
 
Elizabeth Fisher: Tracking the quality of care
Elizabeth Fisher: Tracking the quality of careElizabeth Fisher: Tracking the quality of care
Elizabeth Fisher: Tracking the quality of careQualityWatch
 
Transforming outpatient services - Nuffield Trust/NHS Improvement Event
Transforming outpatient services - Nuffield Trust/NHS Improvement EventTransforming outpatient services - Nuffield Trust/NHS Improvement Event
Transforming outpatient services - Nuffield Trust/NHS Improvement EventNuffield Trust
 
Redesigning care - Nigel Edwards
Redesigning care - Nigel EdwardsRedesigning care - Nigel Edwards
Redesigning care - Nigel EdwardsNuffield Trust
 
How to Define Effective and Efficient Real World Trials
How to Define Effective and Efficient Real World TrialsHow to Define Effective and Efficient Real World Trials
How to Define Effective and Efficient Real World TrialsTodd Berner MD
 
How to design effective and efficient real world trials TB Evidence 2014 10.2...
How to design effective and efficient real world trials TB Evidence 2014 10.2...How to design effective and efficient real world trials TB Evidence 2014 10.2...
How to design effective and efficient real world trials TB Evidence 2014 10.2...Todd Berner MD
 
Our vision for using patient insight and feedback in the nhs, 12.00, pop up u...
Our vision for using patient insight and feedback in the nhs, 12.00, pop up u...Our vision for using patient insight and feedback in the nhs, 12.00, pop up u...
Our vision for using patient insight and feedback in the nhs, 12.00, pop up u...NHS England
 
Dr Alisha Davies: local evaluation
Dr Alisha Davies: local evaluation Dr Alisha Davies: local evaluation
Dr Alisha Davies: local evaluation Nuffield Trust
 
Using simulation to drive changes in health and social care
Using simulation to drive changes in health and social careUsing simulation to drive changes in health and social care
Using simulation to drive changes in health and social careNHS Improving Quality
 
Clinical information systems
Clinical information systemsClinical information systems
Clinical information systemsMark Wardle
 

Semelhante a Cono Ariti: matched control studies (20)

Evaluating Service Innovations for Older People - Martin Bardsley, Nuffield T...
Evaluating Service Innovations for Older People - Martin Bardsley, Nuffield T...Evaluating Service Innovations for Older People - Martin Bardsley, Nuffield T...
Evaluating Service Innovations for Older People - Martin Bardsley, Nuffield T...
 
Dr Martin Bardsley: Use of Retrospective Matching Methods 30 June 2014
Dr Martin Bardsley: Use of Retrospective Matching Methods 30 June 2014Dr Martin Bardsley: Use of Retrospective Matching Methods 30 June 2014
Dr Martin Bardsley: Use of Retrospective Matching Methods 30 June 2014
 
AIRAEvaluation_20150423
AIRAEvaluation_20150423AIRAEvaluation_20150423
AIRAEvaluation_20150423
 
Seven Day services - Healthcare Science presentation (Blower)
Seven Day services - Healthcare Science presentation (Blower)Seven Day services - Healthcare Science presentation (Blower)
Seven Day services - Healthcare Science presentation (Blower)
 
Martin Bardsley: integration and innovation in health
Martin Bardsley: integration and innovation in healthMartin Bardsley: integration and innovation in health
Martin Bardsley: integration and innovation in health
 
Operational research to increase the efficiency of ART initiation in Africa
Operational research to increase the efficiency of ART initiation in AfricaOperational research to increase the efficiency of ART initiation in Africa
Operational research to increase the efficiency of ART initiation in Africa
 
Dr. Martin Bardsley Digital Health Assembly 2015
Dr. Martin Bardsley Digital Health Assembly 2015Dr. Martin Bardsley Digital Health Assembly 2015
Dr. Martin Bardsley Digital Health Assembly 2015
 
Xavier Chitnis: the effects of telehealth on service use
Xavier Chitnis: the effects of telehealth on service use Xavier Chitnis: the effects of telehealth on service use
Xavier Chitnis: the effects of telehealth on service use
 
Ian Blunt & Martin Bardsley: Cause for concern
Ian Blunt & Martin Bardsley: Cause for concernIan Blunt & Martin Bardsley: Cause for concern
Ian Blunt & Martin Bardsley: Cause for concern
 
Seven Day Services - Crawley event
Seven Day Services - Crawley event Seven Day Services - Crawley event
Seven Day Services - Crawley event
 
Holly Holder & Ian Blunt: Integrated care pilot evaluation
Holly Holder & Ian Blunt: Integrated care pilot evaluationHolly Holder & Ian Blunt: Integrated care pilot evaluation
Holly Holder & Ian Blunt: Integrated care pilot evaluation
 
Elizabeth Fisher: Tracking the quality of care
Elizabeth Fisher: Tracking the quality of careElizabeth Fisher: Tracking the quality of care
Elizabeth Fisher: Tracking the quality of care
 
Transforming outpatient services - Nuffield Trust/NHS Improvement Event
Transforming outpatient services - Nuffield Trust/NHS Improvement EventTransforming outpatient services - Nuffield Trust/NHS Improvement Event
Transforming outpatient services - Nuffield Trust/NHS Improvement Event
 
Redesigning care - Nigel Edwards
Redesigning care - Nigel EdwardsRedesigning care - Nigel Edwards
Redesigning care - Nigel Edwards
 
How to Define Effective and Efficient Real World Trials
How to Define Effective and Efficient Real World TrialsHow to Define Effective and Efficient Real World Trials
How to Define Effective and Efficient Real World Trials
 
How to design effective and efficient real world trials TB Evidence 2014 10.2...
How to design effective and efficient real world trials TB Evidence 2014 10.2...How to design effective and efficient real world trials TB Evidence 2014 10.2...
How to design effective and efficient real world trials TB Evidence 2014 10.2...
 
Our vision for using patient insight and feedback in the nhs, 12.00, pop up u...
Our vision for using patient insight and feedback in the nhs, 12.00, pop up u...Our vision for using patient insight and feedback in the nhs, 12.00, pop up u...
Our vision for using patient insight and feedback in the nhs, 12.00, pop up u...
 
Dr Alisha Davies: local evaluation
Dr Alisha Davies: local evaluation Dr Alisha Davies: local evaluation
Dr Alisha Davies: local evaluation
 
Using simulation to drive changes in health and social care
Using simulation to drive changes in health and social careUsing simulation to drive changes in health and social care
Using simulation to drive changes in health and social care
 
Clinical information systems
Clinical information systemsClinical information systems
Clinical information systems
 

Mais de Nuffield Trust

13 reasons to spend more on health and social care
13 reasons to spend more on health and social care 13 reasons to spend more on health and social care
13 reasons to spend more on health and social care Nuffield Trust
 
Energising your workforce in the face of adversity
Energising your workforce in the face of adversityEnergising your workforce in the face of adversity
Energising your workforce in the face of adversityNuffield Trust
 
Shifting the balance of care: great expectations
Shifting the balance of care: great expectations Shifting the balance of care: great expectations
Shifting the balance of care: great expectations Nuffield Trust
 
Automation, Employment, and Health Care
Automation, Employment, and Health Care Automation, Employment, and Health Care
Automation, Employment, and Health Care Nuffield Trust
 
Public perspectives on the NHS and social care
Public perspectives on the NHS and social carePublic perspectives on the NHS and social care
Public perspectives on the NHS and social careNuffield Trust
 
Evaluation of the Integrated Care and Support Pioneers Programme
Evaluation of the Integrated Care and Support Pioneers ProgrammeEvaluation of the Integrated Care and Support Pioneers Programme
Evaluation of the Integrated Care and Support Pioneers ProgrammeNuffield Trust
 
Ensuring success for new models of care
Ensuring success for new models of careEnsuring success for new models of care
Ensuring success for new models of careNuffield Trust
 
Effectiveness of the current dominant approach to integrated care in the NHS
Effectiveness of the current dominant approach to integrated care in the NHSEffectiveness of the current dominant approach to integrated care in the NHS
Effectiveness of the current dominant approach to integrated care in the NHSNuffield Trust
 
Local and national uses of data
Local and national uses of dataLocal and national uses of data
Local and national uses of dataNuffield Trust
 
Applied use of CUSUMs in surveillance
Applied use of CUSUMs in surveillanceApplied use of CUSUMs in surveillance
Applied use of CUSUMs in surveillanceNuffield Trust
 
Coping with change for a sustainable future
Coping with change for a sustainable futureCoping with change for a sustainable future
Coping with change for a sustainable futureNuffield Trust
 
Helen Parker: Lessons from New Zealand
Helen Parker: Lessons from New ZealandHelen Parker: Lessons from New Zealand
Helen Parker: Lessons from New ZealandNuffield Trust
 
The Primary Care Home: A solutions focused new care model provider
The Primary Care Home: A solutions focused new care model providerThe Primary Care Home: A solutions focused new care model provider
The Primary Care Home: A solutions focused new care model providerNuffield Trust
 
Early findings from Collaborate
Early findings from CollaborateEarly findings from Collaborate
Early findings from CollaborateNuffield Trust
 
Effective governance of multi-site general practice organisations
Effective governance of multi-site general practice organisationsEffective governance of multi-site general practice organisations
Effective governance of multi-site general practice organisationsNuffield Trust
 
Extended Primary Care Access in Southwark
Extended Primary Care Access in Southwark Extended Primary Care Access in Southwark
Extended Primary Care Access in Southwark Nuffield Trust
 
Arvind Madan, NHS England
Arvind Madan, NHS England Arvind Madan, NHS England
Arvind Madan, NHS England Nuffield Trust
 
Has clinical commissioning found its voice? GP perspectives on their CCGs
Has clinical commissioning found its voice? GP perspectives on their CCGsHas clinical commissioning found its voice? GP perspectives on their CCGs
Has clinical commissioning found its voice? GP perspectives on their CCGsNuffield Trust
 
Revolutionising the workforce for child health services - Hilary Cass
Revolutionising the workforce for child health services - Hilary CassRevolutionising the workforce for child health services - Hilary Cass
Revolutionising the workforce for child health services - Hilary CassNuffield Trust
 

Mais de Nuffield Trust (20)

13 reasons to spend more on health and social care
13 reasons to spend more on health and social care 13 reasons to spend more on health and social care
13 reasons to spend more on health and social care
 
Energising your workforce in the face of adversity
Energising your workforce in the face of adversityEnergising your workforce in the face of adversity
Energising your workforce in the face of adversity
 
Shifting the balance of care: great expectations
Shifting the balance of care: great expectations Shifting the balance of care: great expectations
Shifting the balance of care: great expectations
 
Automation, Employment, and Health Care
Automation, Employment, and Health Care Automation, Employment, and Health Care
Automation, Employment, and Health Care
 
Public perspectives on the NHS and social care
Public perspectives on the NHS and social carePublic perspectives on the NHS and social care
Public perspectives on the NHS and social care
 
Evaluation of the Integrated Care and Support Pioneers Programme
Evaluation of the Integrated Care and Support Pioneers ProgrammeEvaluation of the Integrated Care and Support Pioneers Programme
Evaluation of the Integrated Care and Support Pioneers Programme
 
Ensuring success for new models of care
Ensuring success for new models of careEnsuring success for new models of care
Ensuring success for new models of care
 
Effectiveness of the current dominant approach to integrated care in the NHS
Effectiveness of the current dominant approach to integrated care in the NHSEffectiveness of the current dominant approach to integrated care in the NHS
Effectiveness of the current dominant approach to integrated care in the NHS
 
Local and national uses of data
Local and national uses of dataLocal and national uses of data
Local and national uses of data
 
Applied use of CUSUMs in surveillance
Applied use of CUSUMs in surveillanceApplied use of CUSUMs in surveillance
Applied use of CUSUMs in surveillance
 
Engaging with data
Engaging with dataEngaging with data
Engaging with data
 
Coping with change for a sustainable future
Coping with change for a sustainable futureCoping with change for a sustainable future
Coping with change for a sustainable future
 
Helen Parker: Lessons from New Zealand
Helen Parker: Lessons from New ZealandHelen Parker: Lessons from New Zealand
Helen Parker: Lessons from New Zealand
 
The Primary Care Home: A solutions focused new care model provider
The Primary Care Home: A solutions focused new care model providerThe Primary Care Home: A solutions focused new care model provider
The Primary Care Home: A solutions focused new care model provider
 
Early findings from Collaborate
Early findings from CollaborateEarly findings from Collaborate
Early findings from Collaborate
 
Effective governance of multi-site general practice organisations
Effective governance of multi-site general practice organisationsEffective governance of multi-site general practice organisations
Effective governance of multi-site general practice organisations
 
Extended Primary Care Access in Southwark
Extended Primary Care Access in Southwark Extended Primary Care Access in Southwark
Extended Primary Care Access in Southwark
 
Arvind Madan, NHS England
Arvind Madan, NHS England Arvind Madan, NHS England
Arvind Madan, NHS England
 
Has clinical commissioning found its voice? GP perspectives on their CCGs
Has clinical commissioning found its voice? GP perspectives on their CCGsHas clinical commissioning found its voice? GP perspectives on their CCGs
Has clinical commissioning found its voice? GP perspectives on their CCGs
 
Revolutionising the workforce for child health services - Hilary Cass
Revolutionising the workforce for child health services - Hilary CassRevolutionising the workforce for child health services - Hilary Cass
Revolutionising the workforce for child health services - Hilary Cass
 

Último

"Radical excision of DIE in subferile women with deep infiltrating endometrio...
"Radical excision of DIE in subferile women with deep infiltrating endometrio..."Radical excision of DIE in subferile women with deep infiltrating endometrio...
"Radical excision of DIE in subferile women with deep infiltrating endometrio...Sujoy Dasgupta
 
AORTIC DISSECTION and management of aortic dissection
AORTIC DISSECTION and management of aortic dissectionAORTIC DISSECTION and management of aortic dissection
AORTIC DISSECTION and management of aortic dissectiondrhanifmohdali
 
Male Infertility, Antioxidants and Beyond
Male Infertility, Antioxidants and BeyondMale Infertility, Antioxidants and Beyond
Male Infertility, Antioxidants and BeyondSujoy Dasgupta
 
Male Infertility Panel Discussion by Dr Sujoy Dasgupta
Male Infertility Panel Discussion by Dr Sujoy DasguptaMale Infertility Panel Discussion by Dr Sujoy Dasgupta
Male Infertility Panel Discussion by Dr Sujoy DasguptaSujoy Dasgupta
 
SGK NGẠT NƯỚC ĐHYHN RẤT LÀ HAY NHA .pdf
SGK NGẠT NƯỚC ĐHYHN RẤT LÀ HAY NHA    .pdfSGK NGẠT NƯỚC ĐHYHN RẤT LÀ HAY NHA    .pdf
SGK NGẠT NƯỚC ĐHYHN RẤT LÀ HAY NHA .pdfHongBiThi1
 
How to cure cirrhosis and chronic hepatitis naturally
How to cure cirrhosis and chronic hepatitis naturallyHow to cure cirrhosis and chronic hepatitis naturally
How to cure cirrhosis and chronic hepatitis naturallyZurück zum Ursprung
 
blood bank management system project report
blood bank management system project reportblood bank management system project report
blood bank management system project reportNARMADAPETROLEUMGAS
 
Physiology of Smooth Muscles -Mechanics of contraction and relaxation
Physiology of Smooth Muscles -Mechanics of contraction and relaxationPhysiology of Smooth Muscles -Mechanics of contraction and relaxation
Physiology of Smooth Muscles -Mechanics of contraction and relaxationMedicoseAcademics
 
AUTONOMIC NERVOUS SYSTEM organization and functions
AUTONOMIC NERVOUS SYSTEM organization and functionsAUTONOMIC NERVOUS SYSTEM organization and functions
AUTONOMIC NERVOUS SYSTEM organization and functionsMedicoseAcademics
 
Using Data Visualization in Public Health Communications
Using Data Visualization in Public Health CommunicationsUsing Data Visualization in Public Health Communications
Using Data Visualization in Public Health Communicationskatiequigley33
 
SGK RỐI LOẠN KALI MÁU CỰC KỲ QUAN TRỌNG.pdf
SGK RỐI LOẠN KALI MÁU CỰC KỲ QUAN TRỌNG.pdfSGK RỐI LOẠN KALI MÁU CỰC KỲ QUAN TRỌNG.pdf
SGK RỐI LOẠN KALI MÁU CỰC KỲ QUAN TRỌNG.pdfHongBiThi1
 
MedMatch: Your Health, Our Mission. Pitch deck.
MedMatch: Your Health, Our Mission. Pitch deck.MedMatch: Your Health, Our Mission. Pitch deck.
MedMatch: Your Health, Our Mission. Pitch deck.whalesdesign
 
Different drug regularity bodies in different countries.
Different drug regularity bodies in different countries.Different drug regularity bodies in different countries.
Different drug regularity bodies in different countries.kishan singh tomar
 
DNA nucleotides Blast in NCBI and Phylogeny using MEGA Xi.pptx
DNA nucleotides Blast in NCBI and Phylogeny using MEGA Xi.pptxDNA nucleotides Blast in NCBI and Phylogeny using MEGA Xi.pptx
DNA nucleotides Blast in NCBI and Phylogeny using MEGA Xi.pptxMAsifAhmad
 
Role of Soap based and synthetic or syndets bar
Role of  Soap based and synthetic or syndets barRole of  Soap based and synthetic or syndets bar
Role of Soap based and synthetic or syndets barmohitRahangdale
 
Generative AI in Health Care a scoping review and a persoanl experience.
Generative AI in Health Care a scoping review and a persoanl experience.Generative AI in Health Care a scoping review and a persoanl experience.
Generative AI in Health Care a scoping review and a persoanl experience.Vaikunthan Rajaratnam
 
Mental health Team. Dr Senthil Thirusangu
Mental health Team. Dr Senthil ThirusanguMental health Team. Dr Senthil Thirusangu
Mental health Team. Dr Senthil Thirusangu Medical University
 

Último (20)

"Radical excision of DIE in subferile women with deep infiltrating endometrio...
"Radical excision of DIE in subferile women with deep infiltrating endometrio..."Radical excision of DIE in subferile women with deep infiltrating endometrio...
"Radical excision of DIE in subferile women with deep infiltrating endometrio...
 
AORTIC DISSECTION and management of aortic dissection
AORTIC DISSECTION and management of aortic dissectionAORTIC DISSECTION and management of aortic dissection
AORTIC DISSECTION and management of aortic dissection
 
American College of physicians ACP high value care recommendations in rheumat...
American College of physicians ACP high value care recommendations in rheumat...American College of physicians ACP high value care recommendations in rheumat...
American College of physicians ACP high value care recommendations in rheumat...
 
Male Infertility, Antioxidants and Beyond
Male Infertility, Antioxidants and BeyondMale Infertility, Antioxidants and Beyond
Male Infertility, Antioxidants and Beyond
 
Male Infertility Panel Discussion by Dr Sujoy Dasgupta
Male Infertility Panel Discussion by Dr Sujoy DasguptaMale Infertility Panel Discussion by Dr Sujoy Dasgupta
Male Infertility Panel Discussion by Dr Sujoy Dasgupta
 
SGK NGẠT NƯỚC ĐHYHN RẤT LÀ HAY NHA .pdf
SGK NGẠT NƯỚC ĐHYHN RẤT LÀ HAY NHA    .pdfSGK NGẠT NƯỚC ĐHYHN RẤT LÀ HAY NHA    .pdf
SGK NGẠT NƯỚC ĐHYHN RẤT LÀ HAY NHA .pdf
 
How to cure cirrhosis and chronic hepatitis naturally
How to cure cirrhosis and chronic hepatitis naturallyHow to cure cirrhosis and chronic hepatitis naturally
How to cure cirrhosis and chronic hepatitis naturally
 
blood bank management system project report
blood bank management system project reportblood bank management system project report
blood bank management system project report
 
Physiology of Smooth Muscles -Mechanics of contraction and relaxation
Physiology of Smooth Muscles -Mechanics of contraction and relaxationPhysiology of Smooth Muscles -Mechanics of contraction and relaxation
Physiology of Smooth Muscles -Mechanics of contraction and relaxation
 
AUTONOMIC NERVOUS SYSTEM organization and functions
AUTONOMIC NERVOUS SYSTEM organization and functionsAUTONOMIC NERVOUS SYSTEM organization and functions
AUTONOMIC NERVOUS SYSTEM organization and functions
 
Using Data Visualization in Public Health Communications
Using Data Visualization in Public Health CommunicationsUsing Data Visualization in Public Health Communications
Using Data Visualization in Public Health Communications
 
Biologic therapy ice breaking in rheumatology, Case based approach with appli...
Biologic therapy ice breaking in rheumatology, Case based approach with appli...Biologic therapy ice breaking in rheumatology, Case based approach with appli...
Biologic therapy ice breaking in rheumatology, Case based approach with appli...
 
SGK RỐI LOẠN KALI MÁU CỰC KỲ QUAN TRỌNG.pdf
SGK RỐI LOẠN KALI MÁU CỰC KỲ QUAN TRỌNG.pdfSGK RỐI LOẠN KALI MÁU CỰC KỲ QUAN TRỌNG.pdf
SGK RỐI LOẠN KALI MÁU CỰC KỲ QUAN TRỌNG.pdf
 
Rheumatoid arthritis Part 1, case based approach with application of the late...
Rheumatoid arthritis Part 1, case based approach with application of the late...Rheumatoid arthritis Part 1, case based approach with application of the late...
Rheumatoid arthritis Part 1, case based approach with application of the late...
 
MedMatch: Your Health, Our Mission. Pitch deck.
MedMatch: Your Health, Our Mission. Pitch deck.MedMatch: Your Health, Our Mission. Pitch deck.
MedMatch: Your Health, Our Mission. Pitch deck.
 
Different drug regularity bodies in different countries.
Different drug regularity bodies in different countries.Different drug regularity bodies in different countries.
Different drug regularity bodies in different countries.
 
DNA nucleotides Blast in NCBI and Phylogeny using MEGA Xi.pptx
DNA nucleotides Blast in NCBI and Phylogeny using MEGA Xi.pptxDNA nucleotides Blast in NCBI and Phylogeny using MEGA Xi.pptx
DNA nucleotides Blast in NCBI and Phylogeny using MEGA Xi.pptx
 
Role of Soap based and synthetic or syndets bar
Role of  Soap based and synthetic or syndets barRole of  Soap based and synthetic or syndets bar
Role of Soap based and synthetic or syndets bar
 
Generative AI in Health Care a scoping review and a persoanl experience.
Generative AI in Health Care a scoping review and a persoanl experience.Generative AI in Health Care a scoping review and a persoanl experience.
Generative AI in Health Care a scoping review and a persoanl experience.
 
Mental health Team. Dr Senthil Thirusangu
Mental health Team. Dr Senthil ThirusanguMental health Team. Dr Senthil Thirusangu
Mental health Team. Dr Senthil Thirusangu
 

Cono Ariti: matched control studies

  • 1. © Nuffield Trust22 June 2015 Matched Control Studies: Methods and case studies Cono Ariti cono.ariti@nuffieldtrust.org.uk
  • 2. © Nuffield Trust Predictive risk modelling Resource allocation Descriptive studies Evaluations Integrated care pilots nuffield trust Nuffield Trust Research team – data linkage projects Risk sharing for CCGs nuffield trust Combined predictive model nuffield trust Person based resource allocation nuffield trust Social care at end of life nuffield trust Cancer and social care nuffield trust Predicting social care costs nuffield trust Virtual Wards nuffield trust WSD nuffield trust British Red Cross nuffield trust
  • 3. © Nuffield Trust Need for evaluation Need to know what works • In a practical setting – “real world evaluation” • Clarify the debate • Likely impacts – unbiased results • Link to qualitative work Refine programs • Obtain feedback and learnings – the pain of implementation • Explore sub-groups – where did it work? Where could it work?
  • 4. © Nuffield Trust Issues with evaluations Randomised control trials • “Gold standard” • May not be feasible or ethical • Inclusion and exclusion rules can limit generalisation • Are still subject to poor implementation – can induce bias • Potentially expensive! Observational studies • Typically no “natural” experiment exists • Often no comparable control group to provide a fair assessment
  • 5. © Nuffield Trust© Nuffield Trust Matched Control Studies - Methods
  • 6. © Nuffield Trust Matched Control Studies The basic idea • Match controls to those treated based on measured characteristics in existing datasets • The control group and treated group should look similar “on balance” • Mimics the idea of an RCT • Based on propensity score theory (Rubin & Rosenbaum, 1983) and earlier work on matching (Cochran, 1965)
  • 7. © Nuffield Trust Matched Control Studies Matching • Prognostic risk score • Demographics – age, gender, deprivation, ethnicity • Prior acute care service use – admissions, OP and A&E attendances • Prior diagnoses, targeted chronic conditions Balance • In this case all matching variables • Additional variables such as length of stay, additional diagnoses and longer service use history • Assures comparability between the groups
  • 8. © Nuffield Trust Matching Algorithm Algorithm • Exact match not possible • Computer intensive “genetic algorithm” • Uses a weighted Mahalanobis “distance” to determine closest match • Automatically assesses balance and moves to an improved solution Assessing Balance • On overall group similarity • Compares means and distribution of variables in the two groups
  • 9. © Nuffield Trust Analysis of matched control studies • Standard statistical methods to estimate the difference in the two groups • Regression models, difference in difference analysis • By including matching variables in the statistical adjustment remaining imbalances can be reduced – “doubly robust” • Methods exist for sensitivity analysis – impact of unobserved variables • Some controversy around accommodating the matching in analysis
  • 10. © Nuffield Trust© Nuffield Trust Case Study 1: Telehealth Programme
  • 11. © Nuffield Trust Case Study 1: Telehealth program Intervention: • Remote monitoring for patients with long term conditions Nuffield commissioned to evaluate impact: • Primary: Reduction in emergency hospital admissions? • Secondary: Reductions in Emergency attendances, outpatient attendances, mortality Methods: • Retrospective matched control study – use of already existing administrative data
  • 12. © Nuffield Trust Description: Telehealth program
  • 13. © Nuffield Trust Matched control studies – broad aim >30,000 individuals – resident in local area June 2010 to March 2013, did not receive telehealth and were eligible for matching (local controls) Aim to find 716 individuals who match almost exactly on a broad range of characteristics Use this group as study control group 716 individuals – enrolled June 2010 to March 2013 & received Telehealth intervention & eligible for matching
  • 14. © Nuffield Trust Datasets available Telehealth Nuffield trust N = 716 • person details • dates of service • type of service Identifiers: Names, DOB, Addresses, etc • dates & place of death for all people in England, • associated hospital (HES) records Identifiers: Nuffield Trust specific HESID Administrative data ONS deaths Hospital inpatient, outpatient, AE
  • 15. © Nuffield Trust 1 2 3 4 5 6 7 8 9 . . 1 2 3 4 5 6 7 8 9 . . 1 2 3 4 5 6 7 8 9 . . Telehealth Data Linkage Service Nuffield Trust New Identifier New Identifier New Identifier (NHS no) (NHS no) Names Names Address Address DOB DOB HESID HESID Telehealth person identifiers (File A)
  • 16. © Nuffield Trust Final datasets available for analysis Nuffield trust Identifiers: HESID on all ONS deaths Hospital inpatient, outpatient, AETelehealth data - desensitised Use all this info to carry out matched control analysis
  • 17. © Nuffield Trust Control group – how well matched?
  • 18. © Nuffield Trust Control group – how well matched? Telehealth Controls
  • 19. © Nuffield Trust Control group – how well matched? Telehealth Controls
  • 20. © Nuffield Trust Key Result 1: Risk of admissions or death
  • 21. © Nuffield Trust Key Result 2: Changes in admissions or attendances (six months pre and post intervention)
  • 22. © Nuffield Trust Results • Telehealth patients tended to be admitted for an emergency admission earlier than control patients • There was no difference in mortality between the telehealth and control groups • There were no statistically significant reductions in hospital admissions when comparing the period six months before and after the telehealth intervention • In summary the Telehealth program did not have a significant impact on acute care outcomes • Sensitivity analysis showed little evidence of an important unobserved variable
  • 23. © Nuffield Trust© Nuffield Trust Matched Control Studies: Summary
  • 24. © Nuffield Trust Matched Controls: Summary Benefits • Makes full use existing data, with relative ease • Techniques applicable to many different types of services and datasets • Decisions on what seems to work (and what may not) based on more robust analyses leading to better informed decisions Caveats • If important unobserved variables exist results may be biased • The routine data sources must contain the relevant data
  • 25. © Nuffield Trust Implementing locally – key enablers Do you have … Can you … • Access to data that contains the outcomes relevant to your evaluation? • Access to data containing relevant matching characteristics? • Do you have consent to access/link the data? • Analysis tools to apply statistical methods to the data? • Skilled analysts to analyse the data? • Link multiple sources of data? • Handle large amounts of data (millions of observations)? • Identify recipients of the intervention? • Transform and augment that data with bespoke variables? • Apply sophisticated matching algorithms routinely to this data? • Analyse the data with a variety of statistical methods and interpret the results appropriately?
  • 26. © Nuffield Trust www.nuffieldtrust.org.uk Sign-up for our newsletter www.nuffieldtrust.org.uk/newsletter Follow us on Twitter: Twitter.com/NuffieldTrust © Nuffield Trust Thank you