A New Approach at Conducting Post-Market Drug Safety Surveillance

A New Approach at Conducting Post-Market
Drug Safety Surveillance
Production of Real-World Evidence at the hospital level to provide accurate and
timely insight into the safety and efficacy of marketed drugs
Author: Veronika Valdova, DVM
Reviewed by: Jonathan Fishbein, MD
2/25/2018
Stakeholders in the healthcare ecosystem need accurate, comprehensive and timely medical evidence. An
integrated solution to provide this evidence through analysis of real-world data can be designed and
developed. Utilizing master study protocols, such a service would employ data analysis and integration
platforms that would process real-world data within each contributing hospital. By producing standardized and
layered medical evidence, valuable insight into the safety and efficacy of marketed drugs can be obtained and
shared.

ARETE-ZOE, LLC: Redesigning Post-Market Drug Safety Surveillance
2/25/2018 Confidential Page 1 of 1
Abstract
Monitoring post-market drug safety represents a significant public health challenge. The types of data screened
during post-market surveillance activities include spontaneous and voluntary reporting of adverse events into
national, regional and local databases, the Sentinel Initiative, scientific literature, voluntary and mandatory
post-marketing studies, healthcare databases and active surveillance systems. The main limitations of current
pharmacovigilance systems that rely on these sources include drastic under-reporting of adverse events (1-
10%), reporting bias, and varying quality of reports. These systems cannot provide accurate assessments of
prevalence and incidence because the numerator is uncertain and the denominator can only be projected from
drug utilization data. To appraise the total body of evidence of harm can be a complex and daunting task when
many post-authorization studies have not reported their data, when study designs, objectives, endpoints,
inclusion and exclusion criteria vary widely, and when study reports are of inconsistent quality. Post-
authorization studies are typically conducted with long delays and typically have no results available for public
scrutiny. When study results are available, publication bias and complexity of methodologies decrease the
confidence in the overall assessment and strength of recommendations. Additional confounding variables that
complicate evidence synthesis are introduced due to all too common off-label use, use in populations the drugs
were not intended for, or administration in combination with other medications. Endpoints and biomarkers
used in the drug development stage often do not translate into parameters routinely collected in clinical
practice, so they provide little value in post-marketing assessments. It is no wonder that signal detection in
post-market drug surveillance is forced to rely on individual case review and disproportionality analysis.
Yet, in the period from 2001 to 2010, one third of newly approved drugs in the U.S. were subjected to a
withdrawal (3), boxed warning or a safety communication. These high-impact decisions were based upon a
pharmacovigilance reporting system that can only provide evidence from individual case reports. In fact, of the
462 products withdrawn from the market in at least one country since 1950, more than 70% decisions for drug
withdrawal were based on case reports and case series. The median time lapse between first identification of a
safety problem and eventual market withdrawal was 6 years, and the time has not substantially shortened
since the 1950s.
This paper describes an integrated solution to provide accurate, comprehensive and timely medical evidence
through analysis of adverse events culled from real-world data that can be designed and developed. Utilizing
master study protocols, such a service would employ data analysis and integration platforms that would
process real-world data within each contributing hospital. By producing standardized and layered medical
evidence, valuable insight into the safety and efficacy of marketed drugs can be obtained and shared.

Contents
Abstract ......................................................................................................................................................1
Analysis.......................................................................................................................................................4
Pharmacovigilance reporting as part of post-market safety surveillance..............................................4
Signal detection methods in pharmacovigilance....................................................................................6
Post-market safety events as a public health challenge ........................................................................8
System responsiveness: safety withdrawals ..........................................................................................9
Evidence from post-authorization studies .......................................................................................... 10
Level of evidence, total body of evidence........................................................................................... 11
Outcome measures, endpoints ........................................................................................................... 12
Novel Biomarkers ................................................................................................................................ 13
Communicating results........................................................................................................................ 14
Publication bias, cognitive biases........................................................................................................ 15
Reporting format................................................................................................................................. 16
Data security in healthcare.................................................................................................................. 17
Data breaches in healthcare................................................................................................................ 19
Solution Design........................................................................................................................................ 21
From RWD to RWE............................................................................................................................... 21
5W ....................................................................................................................................................... 24
Stakeholders........................................................................................................................................ 26
Main uses of post-market data ........................................................................................................... 27
Boyd Cycle (O-O-D-A Loop).................................................................................................................. 28
The Intelligence Cycle.......................................................................................................................... 29
Post-Market Surveillance Redesigned................................................................................................. 30
Stakeholders’ vital interests and critical information needs............................................................... 34
Expected improvement from the use of Real-World Evidence........................................................... 36
Abbreviations........................................................................................................................................... 39
Appendices .............................................................................................................................................. 42
1 Signal detection methods in pharmacovigilance.............................................................................. 43
2 Programs using data from Sentinel .................................................................................................. 44
3 Healthcare Cost and Utilization Project (H-CUP).............................................................................. 71
4 Summary of Metrics Related to Anticoagulant ADEs Collected by Federal Surveillance Systems
(DHHS, National Action Plan for ADE Prevention, 2014)..................................................................... 73
5 Breakdown of Phase 4 studies by the availability of results (Source: CLinicalTrials.gov) ................ 74

6 Fulfilled CDER and CBER Post-market commitments and requirements (FDA, 2018) ..................... 75
7 Levels of Evidence........................................................................................................................... 104
8 Example Summary of Findings Table (GRADE, Cochrane).............................................................. 106
9 Diversity of outcome measures in phase IV studies: Sample of Phase IV, Interventional trials
that study “diabetes” ........................................................................................................................ 107
10 Elements of the COU Statement for Biomarker Qualification (FDA, 2014).................................. 109
11 Pharmacogenomic biomarkers, level of evidence and guidelines for interpretation (CPIC)....... 112
12 Disclosure of study findings.......................................................................................................... 126
13 Detecting publication bias............................................................................................................ 128
14 The RECORD statement extended from STROBE ......................................................................... 130
15 Data breaches in healthcare......................................................................................................... 132
16 Static drug safety control structure.............................................................................................. 135
17 Information environment: Model of the dynamics of drug prescription (Leveson et al., 2012) . 136
References ............................................................................................................................................ 138

Analysis
Pharmacovigilance reporting as part of post-market safety surveillance
Risk management of pharmaceuticals in Europe and in the U.S. is a process aimed at identifying, characterizing,
evaluating, monitoring, communicating and mitigating risks associated with the use medicinal products within
the context of medical care. The objective of post-market surveillance is to detect less frequent adverse drug
experiences, define patient populations that are at higher risk of developing ADRs, determine safety during
long-term use, detect drug-drug and drug-food interactions, note any increased severity or frequency of known
ADRs, identify issues with misuse and abuse, and spot prescribing errors.
The primary means of detecting safety problems in approved drugs is the adverse effect reporting system
FAERS in the U.S.
1
Adverse effect reporting in the U.S. is mandatory for manufacturers and distributors but
voluntary for medical personnel and consumers
2
. The EU counterpart to FAERS is EudraVigilance
3
. Individual
European countries have their own national pharmacovigilance systems. The World Health Organization (WHO)
also collects information from national databases worldwide
4
. The WHO issued a document that describes
minimum requirements for pharmacovigilance systems
5
and guidance how to assess them
6
. The WHO
pharmacovigilance database VigiBase is searchable online and available to anyone with an internet connection
via public interface VigiAccess
7
.
Activities.
The types of data screened during PMS activities include spontaneous and voluntary reporting of cases
into national (FDA MedWatch), local and regional (Joint Commission Requirement) databases,
scientific literature, voluntary and mandatory post-marketing studies, both randomized controlled
trials and observational studies, including automated healthcare databases and active surveillance
such as e.g. the Drug-Induced Liver Injury Network (DILIN) and the Sentinel Initiative
8
.
REMS.
Since the passage of the FDA Amendments Act of 2007 (FDAAA), the FDA has the authority to mandate
Risk Evaluation and Mitigation Strategies (REMS)
9
, post-approval studies
10
and post-market
surveillance studies under the Section 522
11
to evaluate known and emerging serious risks of approved
drugs
12
. Penalties for delays can include fines, warning letters, and withdrawal of a product deemed
misbranded
13
. In Europe, every application for market authorization
14
requires a detailed description
of the pharmacovigilance system to be used for capturing adverse effects, and can include a risk
management plan (RMP) where appropriate..
Post-authorization studies.
Pre-approval clinical trials often fail to capture the breadth of adverse effects that might be observed
in a real world cohort, owing to their small enrollment, short duration, restrictive inclusion and
exclusion criteria, and permissibleconcomitant medications. For this reason Market Authorization
Holders in Europe are now encouraged to implement multi-regional post-authorization safety studies
PASS as part of post-market surveillance activities. In the U.S., post-marketing studies are conducted
as part of post-authorization requirements and commitments (PMRs/PMCs). The objective of
conducting these studies is to gather additional safety information, assess patterns of drug utilization,
and measure the effectiveness of a risk minimization activity. The responsibility for the evaluation of
post-market drug safety surveillance, including studies, belongs to the FDA’s Center for Drug
Evaluation & Research (CDER).

Sentinel Initiative.
Sentinel allows screening of vast amounts of healthcare data such as electronic health records, insurance claims
and registries, as well as dispensing data. The Mini-Sentinel database contains 99 million individuals, 2.9 billion
prescription drug dispensings and 2.4 billion unique medical encounters, including 38 million acute inpatient
hospital stays
15
.
“Sentinel queries may be undertaken to assess potential medical product safety risks, they may also be
initiated for various other reasons. Some examples include determining a rate or count of an identified
health outcome of interest, examining medical product use, exploring the feasibility of future, more
detailed analyses within Sentinel, and seeking to better understand Sentinel capabilities. Data obtained
through Sentinel are intended to complement other types of evidence such as preclinical studies,
clinical trials, post-market studies, and adverse event reports, all of which are used by FDA to inform
regulatory decisions regarding medical product safety” (Sentinel Modular Program Report)
16
.
ADRs are dramatically underreported for a wide variety of reasons, but primarily because the system is largely
voluntary. Additionally, the healthcare community and patients poorly understand the process and the vital
role it plays in protecting the public health. Other factors that dissuade the reporting of adverse effects include
the nature of the ADR/ADE, type of product and the length of time it has been on the market, the extent and
quality of the manufacturer’s reporting system, prescription status (OTC vs. Rx) and reporting regulations
17
.
These vary significantly depending on the country, making data for certain type of products difficult to
compare.
The main limitations of the current reporting system are under-reporting (estimated 1-10%), reporting bias,
and varying quality of reports and duplication of reports. The system does not provide information on
incidence and prevalence because the numerator is uncertain and denominator can only be projected from
drug utilization data. The Sentinel Initiative, once fully implemented, will solve some of the limitations of the
current post-market surveillance system.

Signal detection methods in pharmacovigilance
In Europe, signal detection methods are defined in guideline Good pharmacovigilance practices (GVP) Module IX.
The term “Safety signal” refers to a new or known adverse event with suspected causality that requires further
investigation. The responsibility for detecting and managing safety signals belongs to the European Medicines
Agency, national competent authorities and market authorization holders. The sources of information include
EudraVigilance and national databases, literature and clinical studies. The new EudraVigilance system launched in
November 2017 enables access to the database for market authorization holders. For some substances, a lead
member state is appointed, for all others, the responsibility is shared among all Member States. The
Pharmacovigilance Risk Assessment Committee (PRAC) is responsible for prioritization and assessment of safety
signals at EU level
18
.
The methods of routine signal detection and interpretation are discussed in guidance EMA/849944/2016. Absolute
performance of signal detection methods varies between databases. Signal detection in pharmacovigilance relies
heavily on the review of individual case reports and case series and disproportionality analysis of data reported
ADR databases. The creation of other signal detection methods besides disproportionality statistics is a matter for
ongoing research19
. The foundational concept for many disproportionality methods is the proportional reporting
ratio (PRR) that means the degree of disproportionate reporting of an adverse event for a product of interest
compared to this same event for all other products in the database. Statistical association does not imply causality.
Change-point analysis (CPA) detects changes in either the slope or variability in a time series or sequence in very
large databases. Text mining is used to analyze unstructured data
20
. Schematic flowchart of the detection of safety
signals in pharmacovigilance can be found in Appendix 1.
FDA’s national medical product monitoring system Sentinel mandated by Section 905 of the FDAAA of 200721
(Appendix 2A) is designed to complement the current ADR reporting system by allowing the FDA to access securely
large quantities of electronic healthcare data such as insurance claims, registries, and electronic health records.
Section 905 mandates the establishment of the postmarket risk identification and analysis methods and system for
the purpose of advanced analysis of drug safety data from a variety of disparate sources. The Act mandates the FDA
to cooperate with public, academic and private entities to develop methods to obtain access to disparate data
sources, develop validated methods for the establishment of postmarket risk identification and analysis and
convene a committee of experts that would develop tools and methods for the ethical and scientific use and
communication of postmarket data. The system should allow prompt investigation of drug safety questions,
perform advanced research and analysis of identified drug safety risks, and focus post-approval studies and clinical
trials on cases for which standard signal detection is not sufficient.
The Sentinel Initiative was announced in May 2008 and Report to Congress
22
mandated by Section 905(c) was
submitted in August 2011. The Sentinel Initiative utilizes a distributed data system that maintains data in local
environments as opposed to a centralized approach. The vision, as communicated to the U.S. Congress in 2011,
shall serve as an active surveillance system that would support signal generation, signal refinement, and signal
evaluation. The initial efforts focused on signal refinement to minimize the likelihood of false positives. According
to the 2011 report, the system will involve the use of sophisticated statistical and epidemiological methods to
search for patterns in defined population derived from insurance claims databases and electronic health records
systems. The system was intended to enhance the FDA’s surveillance capabilities by the ability to identify and
evaluate safety issues in near-real time, and expand the capacity to access data on population subgroups, long-term
trends, and identify low intensity signals on high-noise background (i.e. myocardial infarction or fractures) that are
generally not expected to be linked to medications. The Mini-Sentinel Pilot Program is giving the FDA the
opportunity to test statistical and epidemiological methodologies in the assessment of postmarket safety issues.
The participating partners will transform their data into a standardized format, enabling the Coordinating Center to

write a single program of analytical code for a given safety question, with each partner running the query
separately. According to the Report, Sentinel was not intended for the sole use by the Agency but shall be
developed as a national resource for all stakeholders in the healthcare system interested in the safety of medicinal
products and healthcare quality. Sentinel Initiative milestones are included in Appendix 2B.
The Mini-Sentinel database contains data from nearly 100 million patients. Surveillance tools include Active Risk
Identification Analysis (ARIA), routine querying tools and SAS Macro toolkits that can check logs, stratify age
groups, create data subsets and de-identify data. The capabilities of ARIA include the production of descriptive
analyses and unadjusted rates (Level 1, current), adjusted analyses with sophisticated confounding control (Level 2,
current) and sequential adjusted analyses with sophisticated confounding control (Level 3, future). Unlike
pharmacoepidemiology studies, ARIA does not utilize customized programming nor does not use protocols to
analyze data. Modular program queries can calculate background rates of chosen events such as exposure,
outcome or condition; exposures and follow-up time, identify most frequently observed ICD-10 codes (diagnoses)
evaluate uptake of new molecular entities, identify exposures and follow-up time and perform propensity score
matching and establish self-controlled risk interval design23. Summary of analyses conducted by the FDA in
Sentinel's Active Risk Identification and Analysis (ARIA) system and in Mini-Sentinel can be found in Appendix 2C
and 2D, respectively. A list of 15 Safety Analyses conducted by the FDA to date can be found in Appendix 2E. In
addition to that, the FDA conducted 217 exploratory analyses of Sentinel data in the period from 2012 to 2018. The
FDA communicates its interpretation of Sentinel data through FDA press announcements, MedWatch alerts, and
Drug Safety Communications. The FDA issued three safety communications based on the results24
. No postmarket
commitments and/or requirements (PMRs/PMCs) currently utilize Sentinel data. Sentinel is currently not utilized
for the evaluation of effectiveness of REMS. Search of scientific library PubMed for term “Sentinel database”
identified 24 articles published since 2013 (See Appendix 2F). The authors voiced numerous methodological
concerns that limit the practical use of Sentinel, most importantly as a result of varying quality of administrative
data.
The main limitations of the current pharmacovigilance signal detection system are underreporting and reporting
bias, varying quality of reports, varying quality of manufacturers’ reporting systems and the lack of information
on incidence and prevalence, and the reliance on case review and disproportionality analysis. Sentinel has yet to
prove its worth in the detection and assessment of new and emerging safety concerns. The examination of
Sentinel data is not part of any post-market commitments and requirements. Also, the safety and exploratory
analyses seem to be a lot more laborious than originally intended. Sentinel was not designed to establish causal
relationships between exposures and outcomes.

Post-market safety events as a public health challenge
Multiple databases and information collection systems exist that collect and monitor different aspects of quality of
care and patient safety. The Healthcare Cost and Utilization Project (HCUP) database collects longitudinal hospital
care data from State data organizations, hospital associations, private data organizations, and the Federal
government to create a national information resource of encounter-level health care data (Appendix 3).
The highest number of adverse drug events in inpatient settings can be attributed to insulins, anticoagulants, and
opioid analgesics. Appendix 4 provides an example of metrics from HCUP related to anticoagulant ADEs collected
by federal surveillance systems. The National Action Plan for Adverse Drug Event Prevention (2014) focuses on the
reduction of the rate of ED visits for injury associated with therapeutic use of opioid analgesics, insulins, and oral
anticoagulants. To calculate the rates of these incidents, the number of ED visits for injury from a specific drug is
used as a numerator, and the number of patients receiving the drug serves as a denominator
25,26
.
 Adverse Drug Events: All harms that occur during medical care directly caused by the drug including
medication errors, ADRs, allergic reactions, and overdoses
 Adverse drug reaction: “Harm directly caused by a drug at normal doses”
 Medication error: “Inappropriate use of a drug that may or may not result in harm”
While pharmacovigilance and quality management systems in
inpatient and outpatient healthcare settings certainly
overlap, they differ significantly owing to the nature and
purpose of information collected. While pharmacovigilance
systems are designed to collect information on adverse drug
reactions, healthcare provider quality management systems
were developed to collect information that measures the
quality of care, which can include all adverse experiences
relating to drugs, including transcribing and dispensing errors.
High-Impact Adverse Drug Events27
Measure Numerator Denominator
Rate of visits to U.S. hospital EDs
for injury from oral anticoagulants
Number of visits to U.S. hospital EDs
for injury from oral anticoagulants
Number of patients receiving
dispensed oral anticoagulants in U.S.
retail outpatient settings
for injury from insulin
for injury from insulin
dispensed insulin in U.S. retail
outpatient settings
for injury associated with
therapeutic use of opioid analgesics
for injury associated with
therapeutic use of opioid analgesics
dispensed opioid analgesics in U.S.
retail outpatient settings
At present, pharmacovigilance reporting systems (ADRs) and hospital quality management systems (ADEs) are
disconnected and any adverse drug experiences processed depend on the relevant regulatory/ internal
requirements compliance tracks and utilize separate systems. The list of FDA analyses of Sentinel data (Appendix
2C, 2D, 2E and 2F) shows poor alignment of priorities between different systems.

System responsiveness: safety withdrawals
In the period from 2001 to 2010, the FDA approved 183 novel pharmaceuticals and 39 biologics. One-third of the
newly approved therapeutics was affected by at least one post-market safety event: 3 withdrawals, 61 boxed
warnings, and 59 safety communications. The most affected groups were psychiatric medications and biologics
28
.
In a study published in 2016, Onakpoya and colleagues identified 462 products that were withdrawn for safety
reasons in the period from 1950 to 2014 in at least one country. The researchers evaluated the quality of evidence
that led to these withdrawals. In 72% of cases, the supporting evidence that led to market withdrawal consisted of
anecdotal reports (level 4). Less than 10% of the identified products were withdrawn worldwide and 39% were
withdrawn in one country only. The median interval between first reported reaction and the year of withdrawal in
the first country that acted on the issue was 6 years (ranging from 1 to 15 years). Evidence to support the
withdrawal of drugs from the market can include anecdotal reports, observational studies, clinical trials, systematic
reviews and animal data. The authors argue that there is insufficient research into the evidence that leads to
market withdrawals. The most common reason leading to product withdrawal was hepatotoxicity. The median
interval between the first reported adverse reaction and the year of first withdrawal was 6 years. The interval
between first detection and authorization withdrawal has not significantly shortened since 195329
.
Levels of evidence used to justify post-marketing withdrawal of medicinal products (Onakpoya et al., 2016)
Level of evidence Number (%) of withdrawals
All marketed drugs (n = 462) Marketed drugs launched since
1950 (n = 286)
Level 1: Systematic reviews 6 (1.3) 6 (2.1)
Level 2: Randomized studies 27 (5.8) 25 (8.7)
Level 3: Non-randomized studies 43 (9.3) 30 (10.5)
Level 4: Case reports 330 (71.4) 189 (66.1)
Level 5: Mechanism-based reasoning 56 (12.1) 36 (12.6)
The most comprehensive resource on drugs withdrawn and discontinued drugs for safety reasons is WITHDRAWN
database maintained by the Structural Bioinformatics Group at Charité
30
. The database lists therapeutic targets, off-
targets, toxicity types and biological pathways. The database also lists genetic variations associated with
therapeutic targets and off-targets to shed light on common genetic variations that could lead to toxicity in specific
populations
,31
.
The key problems with current pharmacovigilance reporting systems is the quality of evidence they can provide,
and the time lapse between first identification of a safety concern and regulatory action. Level 4 evidence is
routinely used to make high-impact decisions including product withdrawals and other regulatory actions. The
average time from detection of a safety signal to eventual product withdrawal on the basis of safety is six years.
This time lag did not substantially improve since the 1950s. Toxicity in specific populations that is the result of
abnormal metabolism is not detectable in the current pharmacovigilance reporting system since data on CYP450
variant genotypes and other common variations known to be associated with drug toxicity is not routinely
collected in clinical practice.

Evidence from post-authorization studies
Post-authorization safety studies are relevant for drugs, devices, and biologics, as well as behavioral and dietary
interventions. In the case of medical devices, the FDA may require a post-approval study as a condition of a
premarket approval (PMA), humanitarian device exception (HDE), or product development protocol (PDP)
application. The objective is to ensure continued safety and efficacy of marketed drugs, biologics, and devices. Post-
marketing commitments (PMCs) are studies or clinical trials that a sponsor has agreed to conduct after the launch
of a drug or a biologic. Post-marketing requirements (PMRs), on the other hand, are studies that are required under
law. These studies can be mandated under FDAAA to assess a known serious effects relating to the use of the drug,
assess signals of serious risk, and identify unexpected serious risks32,33
.
 Post-marketing studies/clinical trials of drugs approved under the accelerated approval program
(Accelerated approval requirements: 21 CFR 314.510 and 21 CFR 601.41)
 Deferred pediatric studies required under the Pediatric Research Equity Act (21 CFR 314.55(b); 601.27(b)
 Studies/clinical trials under the Animal Efficacy Rule (21 CFR 314.610(b)(1) and 601.91(b)(1))
 Drugs: Database of post-approval drug studies lists 306 ongoing PMC and PMR studies
34
 Devices: Database of post-approval studies of medical devices lists 248 studies
35
As of December 20, 2017, the ClinicalTrial.gov registry lists 874 observational phase IV studies. Results are available
in a mere 69 of the 626 completed studies and for 11 of the 61 terminated studies
36
. Similarly, of the more than
23.000 phase IV interventional studies found in the database, only less than 4000 have results available to the
public
37
. The majority of post-marketing studies do not have any published results at all, or the results become
publicly available long after completion of the study. For breakdown of phase 4 studies by type and the
availability of results, see Appendix 5. Phase IV studies only partially overlap with PMRs/PMCs. Section 921 of the
Food and Drug Administration Amendments Act (FDAAA) of 2007 requires the FDA to review the backlog of
unfulfilled PMRs and PMCs and determine which of them require revision and which can be dropped from the list,
and submit the report to Congress. In the ninth review (data lock point December 30, 2016), the FDA identified
1.636 PMRs and PMCs, of which 1.553 were required by CDER and 83 by CBER
38
. Detailed breakdown of fulfilled
PMRs/PMCs can be found in Appendix 6. Information required by the FDA for PMRs/PMCs is an example of system
Critical Information Requirements and Priority Information Requirements in post-market surveillance.
CDER PMR/PMC Statuses for Annual Review (FDA 9
th
Report to Congress, 2017)
The majority of PMRs/PMCs do not have results available for public scrutiny. Phase IV studies only partially
overlap with PMRs/PMCs. Priority Information Requirements as defined in PMRs/PMCs minimally overlap with
Sentinel analyses. No PMRs/PMCs use Sentinel as source data for analysis.

Level of evidence, total body of evidence
The quality of medical evidence produced in pre-clinical, clinical and post-authorization studies varies depending on
study design, number of participants, statistical power, research objectives, outcome measures, study endpoints
and other factors. Systematic reviews of randomized controlled trials (RCTs), considered the gold standard in
medical evidence, are irreplaceable in clinical research. Case series obtained from the current pharmacovigilance
databases are the equivalent of level 4 evidence. Individual non-interventional studies that generate real-world
evidence based on real-world data can reach the level of evidence 2b/2c, while a systematic review of such studies
would be at the 2a level
39
. Level of Evidence table can be found in Appendix 7.
Assessment of the total body of evidence is a complex and daunting task. Cochrane GRADEing Methods Group
40
developed GRADE, an approach for grading quality of evidence and the strength of recommendations in clinical
guidelines and systematic reviews. The framework is instrumental for specifying healthcare questions, choosing
outcomes of interest, rating their importance, evaluating available evidence and bringing the evidence together in
order to produce recommendations. The quality of evidence for an outcome across all studies is then rated using
the GRADE approach, in order to create a “Summary of Findings” (SoF) table
41
. The overall body of evidence is the
result of a systematic review of outcomes of available studies. The total body of evidence for a specific topic is the
key output crucial for informed decision-making in clinical practice as well as in the drug development and
commercialization process. For an example of evidence synthesis in the Summary of Findings table, see Appendix 8.
The most important limitations of the current post-authorization surveillance are
 Reliance on level 4 evidence in signal detection to support high-impact decisions such as product
withdrawals
 Limited number of post-authorization studies that generate a higher quality of evidence
 PMRs/PMCs only fulfilled with long delay, results only available to the FDA, accessible to the public
through labeling updates
 Analysis of data from Sentinel and other integrated systems (H-CUP) not part of PMRs/PMCs
 Results of most Phase IV studies, both interventional and observational, are not posted in the registry
 Synthesis of evidence from pre-approval and post-market studies is a complex task, often hindered by
unavailability of results and diversity of evidence formats
 The real-world performance of approved drugs, their clinical usefulness in real-life conditions such as
treating comorbidities and co-medications, comparison with other available interventions in the same
therapeutic group, comparative safety, efficacy, and cost-effectiveness are only known after a long delay,
and if at all, often after any issues have already become self-evident.
The post-authorization phase of the product lifecyclelacks adequate insight into the drugs’ true clinical utility. An
information environment that lacks transparency creates significant ambiguity about the true clinical value of
individual treatment options. It is extremely difficult to compare clinical utility, safety and efficacy profiles and
cost-effectiveness of individual treatment protocols in real-world patients in the context of standardly delivered
care and routinely monitored parameters. Accurate and timely comparison of multiple treatment options in real-
world conditions is currently not available in a standardized and accessible format that would provide adequate
support to decision makers in the healthcare ecosystem. Lack of accurate insight into the true performance of
treatment options is a source of misplaced incentives for clinicians, healthcare providers, and insurers, as well as
investors in the drug development and commercialization enterprise. Without a doubt, the current post-
authorization surveillance system , cumbersome and fraught with delays, does not adequately serve the public
good and may result in unnecessary morbidity and mortality.

Outcome measures, endpoints
Post-authorization studies add to the body of evidence already available from clinical trials. These studies are
purposefully designed to answer specific safety and efficacy questions, utilizing clinically relevant biomarkers in
populations in which the drugs are typically used. The evaluation of the total body of evidence from clinical trials
and post-authorization studies has many challenges, specifically when assessing a wide range of outcome
measures, inconsistently used biomarkers, and diverse study endpoints. Endpoints in clinical trials are the means of
measuring the patient’s disease state. To be relevant, these endpoints need to be consistently used in clinical trials
as well as clinical practice.
Between 2008 and 2012, two-thirds of 54 new cancer drug approvals were based on a surrogate endpoint. Several
years later, there is still not enough evidence that 31 (86%) of them improve clinically relevant endpoints
42
. This
reliance on surrogate endpoints in the clinical research phase increases the need to continue assessing the drugs’
performance, safety, and efficacy, as well as cost-effectiveness after launch.
 A validated surrogate endpoint: supported by a clear mechanistic rationale; strong clinical evidence that
the surrogate predicts a clinical benefit; can be used to support drug approval
 A reasonably likely surrogate endpoint: supported by clear mechanistic and/or epidemiologic rationale;
clinical data insufficient; can be used for accelerated approval (drugs)/ expedited access (medical devices)
 A candidate surrogate endpoint: under evaluation for its ability to predict clinical benefit
43
. Systematic
reviews synthesize evidence from studies that use a variety of designs, outcome measures and endpoints
in a variety of population subgroups that may or may not adequately represent the population treated.
Study endpoints employed to demonstrate benefit of a drug often do not correspond with information that is
routinely collected in clinical practice. For example, outcome measures of phase IV studies conducted in patients
with diabetes include a wide range of laboratory values collected under a variety of conditions. Even the most
standardly used biomarkers such as HbA1c, fasting plasma glucose, post-prandial glucose, mean amplitude glucose
excursions (MAGE) index and symptomatic hypo- and hyperglycemic episodes are inconsistently used across studies
(see Appendix 9). The effect of individual treatments on other relevant parameters such as blood lipid profile,
cardiovascular outcomes, major adverse cardiovascular events (MACE), hepatic fat, BMI or body fat is even more
difficult to compare due to missing data. Common comorbidities and complications complicate the data collection
even more. Advanced laboratory markers of insulin resistance are underutilized in clinical practice and their use in
clinical studies is inconsistent and still relatively rare. The situation is even more complex in conditions that typically
require the involvement of multiple different specialties. For example, the involvement of an additional specialty, a
pulmonologist, in already complex diabetes patients was a major factor in the failure of inhaled insulin products
Exubera (Pfizer) and Affrezza (Sanofi)
44
.
The diversity of endpoints and outcome measures makes it very challenging to synthesize evidence from clinical
studies in order to provide strong, relevant and actionable clinical recommendations with high level of
confidence. Quantification of clinical benefit is problematic due to the diversity of outcome measures used in
studies, extensive use of surrogate endpoints, and the disconnect between parameters used in studies and those
routinely collected in clinical practice. The safety and efficacy profiles and clinical utility of individual
interventions or their combinations cannot be quantified without a highly sensitive post-market surveillance
system.

Novel Biomarkers
The application of novel biomarkers is the current area of focus for many healthcare stakeholders, including the
drug development and commercialization enterprise, health authorities and patient advocacy groups. Novel
biomarkers have the potential to improve the efficiency of the drug development process. The main problem is the
lack of standardized methods for their validation, lack of reliable evidence about their performance, lack of
evidentiary standards for qualifying new biomarkers in the context of clinical use, and inadequate coordination of
resources to areas of the greatest unmet need
42
.
Biomarkers and clinical outcome assessments are discussed in FDA guidance for industry on Drug development
tools. Biomarkers, as indicators of normal and pathological biological processes or responses to interventions are
used in clinical practice as well as drug development to characterize disease state, a disease process, or a response
to treatment. Their consistent use would reduce uncertainty regarding the appropriateness of selected
interventions for a particular condition and helps adjust doses or detect pathological responses. In drug
development, biomarkers include diagnostic, prognostic, predictive and pharmacodynamics measurements.
Biomarkers can be used to monitor drug effect including potential toxicity, select sub-populations that are likely to
respond to treatment, or to guide dosing regimens. Some biomarkers are used surrogate endpoints (Hb1Ac, blood
pressure, LDL cholesterol). Robust evidence is required to accept a biomarker as a surrogate endpoint because of
the public health implications of surrogate endpoints that misrepresent the disease state and do not correspond
with clinical outcomes
45
. Context of Use Statement in the biomarker qualification contains a concise biomarker use
statement and a comprehensive description of conditions for qualified use (Appendix 10).
 A diagnostic biomarker is a disease characteristic, categorizes a specific physiological or
pathophysiological state or disease.
 A prognostic biomarker is a baseline characteristic that categorizes patients by degree of risk for disease
occurrence or progression.
 A predictive biomarker is a baseline characteristic that categorizes patients by their likelihood of response
to a particular treatment.
 A pharmacodynamic (or activity) biomarker shows that a biological response has occurred in a patient
following intervention; the magnitude of the change is pertinent to the response.
Pharmacogenomic (PGX) biomarkers are one novel area of interest especially in the field of psychiatry, where they
are well mapped
46
(see Appendix 11). For example, variant CYP2D6, CYP2C9 and CYP2C19 genotypes that interact
with the metabolism of SSRI antidepressants have been linked to akathisia-related violence including homicides
47
.
Important differences in genetic variants and gene expression that code the specific subtypes of serotonin
receptors, transporter mechanisms and metabolism determine how individuals respond to medications in relation
to impulsive and aggressive behavior
48
. Individual variant metabolism of psychiatric medications potentially
threatens the affected patient as well as people around. Biomarkers used in drug development, i.e. PGX
biomarkers, need to be adopted in clinical practice where such guidance exists, to respect the distinctions between
patients who would and would not benefit from a specific treatment as established in clinical trials. The new EU
draft guideline on Good PGX Practices outlines principles for validation of PGX biomarkers
49
. PGX biomarkers of
variant metabolism are a typical example of slow uptake of a well-established biomarker.
Biomarkers used in the drug development stage often do not correspond with biomarkers that are routinely
monitored in clinical practice. The lack of continuity between drug development and clinical practice makes it
difficult to evaluate the true clinical utility of approved drugs in real world population.

Communicating results
Disclosure of study findings. The timely disclosure of clinical trial findings in a consistent manner is a critical unmet
need in the medical information ecosystem. Selective disclosure of results of clinical trials provides an incomplete
and misleading picture of a studied products’ benefit:risk profile
50
. According to STAT News, trial sponsors greatly
improved at disclosing clinical trial results to the ClinicalTrials.gov database, compared to the situation two years
ago. However, the database holds more than 250.000 registered studies, of which only 18.700 are required to have
results posted online. The law does not require results to be posted for Phase 1 trials, any trials completed prior to
2008, and trials that are exempt by the FDA to protect proprietary information. For eligible trials, the results have
to be posted within 1 year since completion
51
(See Appendix 12).
Interpretation and amplification. The interpretation of medical evidence is a science in itself. While levels of
evidence are universally understood in the medical research community, the value of individual papers in terms of
quality of evidence is not always obvious to other healthcare professionals including many clinicians. Low-level
evidence published in a journal with high impact factor can generate significantly higher response than what would
be proportionate to its actual value. One such example is a Letter to Editor published in 1980 NEJM. The five-
sentence letter stated that the risk of addiction was low in patients with no history of addiction who were
prescribed opioids in hospital settings. The letter was widely and non-critically cited. Leung et al. (2017) in their
article “A 1980 Letter on the Risk of Opioid Addiction” argue that the citation pattern greatly contributed to the
current opioid crisis
52
. Nowadays, marginal findings can be amplified by social media, which has the power to make
anecdotal evidence viral. The wide spread of misinterpreted findings further complicates perception of value of
evidence, as seen at the example of the opioid crisis: the original statement “addiction risk is low in hospitalized
patients” morphed into widely accepted inference that “the risk of addiction is low in chronic patients in outpatient
setting” due to amplification of a distorted message through extensive citation.
Leung et al. identified 608 citations of this
letter. Other standalone letters in the same
journal were cited 11 times on average.
Selective disclosure of only a small subset of study findings makes a significant portion of medical evidence
unavailable for review. Synthesis of materials with different levels of evidence requires specialized training.
Physicians and other healthcare professionals are not immune to misinterpretation of evidence. The high impact
factor of a reputable journal can contribute to disproportionate amplification of otherwise marginal findings.

Publication bias, cognitive biases
Publication bias exists when the studies that are included in the analysis differ systematically from the total sum of
all studies that should be included in the analysis. The problem of publication bias affects meta-analyses and
systematic reviews as well as other forms of research synthesis
53
. Methods were developed to minimize the effect
of publication bias, such as the funnel plot or the forest plot (Appendix 13). These methods, however, rely on
assumptions such as the fact that the resulting pattern is due to bias and that bias skews results in a specific
manner. Published studies are more likely to be included in meta-analyses and systematic reviews as opposed to
unpublished studies and other grey sources. Most studies in the ClinicalTrial.gov registry do not have any
publication on record. Publication bias is a major problem especially when only a small minority of qualifying
studies present in the registry has results available in a form that can be considered for inclusion.
 Missing studies: works that were not identified during the search, i.e. due to the limitation to electronic
media or language or exclusion of paid articles. Studies with significant findings are more likely to be cited
and therefore easier to find. Studies that were conducted but the results were never made available are
more likely to be negative or null and their omission is likely to lead to bias in the summary effect.
 Significance of findings: Studies with significant substantive findings that are statistically significant are
more likely to be published. Studies that are negative or inconclusive are less likely to be published and
the delay is typically longer. Selective publication of positive results from study subsets is also common.
 Upward bias occurs due to omission of negative results
 Inflation turns small studies or marginal findings into positives with exaggerated overall impact
Additional bias is introduced by reviewers who have to simplify study eligibility and findings into a binary yes or no
answer that does not account for signal strength. Furukawa in his thesis “Unbiased publication bias: theory and
evidence”
54
argues that even unbiased researchers will inflate some of the marginal findings and dismiss noisy, null
and negative results. Finally, medical researchers, as any other analysts, are vulnerable to a variety of cognitive
biases such as the cause and effect or estimation of probabilities. Cognitive biases, like optical illusions, remain
even when the researchers are fully aware of their nature55
.
 Vividness: information that is personal and vivid impacts our thinking more than abstract evidence.
 Absence of Evidence: key information in intelligence, as well as medicine, is often lacking, decreasing
confidence in final recommendations. Fault tree analysis can be used to mitigate its impact.
 Oversensitivity to Consistency: while internal consistency is characteristic for logically consistent
scenarios, there are exceptions to the rule, namely when most/all of the available information comes from
a single source. Typical example would be a single study published multiple times in different journals.
 Evidence of uncertain accuracy: Analysts tend to employ rule of thumb to complicated probabilistic
relationships and evidence of uncertain reliability to simplify the answer to a binary yes or no judgment.
 Discredited evidence: impressions tend to outlast evidence that has been fully discredited. Humans tend
to interpret new information in the context of pre-existing impressions even when the evidence, on which
the original impression was based, has been fully discredited.
Biases relating to the perception of cause and effect are even more complex. The explanation of causal
relationships comes from inference that is based on the juxtaposition of events in time and space in
combination with scientific theory and logical explanation. In search of coherence, people tend to favor causal
explanations as opposed to random events.
Selective availability of studies and complex methodology required to appraise and synthesize such evidence
makes the entire process prone to human error.

Reporting format
The variety of formats in which study results are available further complicates interpretation of the total body of
medical evidence. The form of reporting results is equally important to facilitate understanding and correct
interpretation. Randomized controlled trials are the gold standard in the evaluation of medical evidence. To enable
objective assessment, the report on study findings needs to be accurate, complete and transparent. In 2010, the
CONSORT Group (CONsolidated Standards Of Reporting Trials) published an updated version of reporting standards
for randomized clinical trials, with extension checklists for each specific design type. CONSORT checklist specifies
information that should be included in a proper RCT report to enable accurate assessment. The standards were
adopted by some journals that require the checklist along with paper submission. CONSORT has been the most
commonly used standard for the presentation of results of randomized controlled trials. Standards were also
developed for the reporting of different types of studies including non-interventional
56
(See Appendix 14).
CONSORT standards are not mandatory for disclosure of study results in the ClinicalTrials.gov database and are very
rarely used to publish study results. Accurate evaluation of studies listed in the registry is, therefore, difficult or
impossible even in instances when the study findings are posted.
 Title and abstract: RCT, summary of design, methods, results and conclusions
 Introduction: Background and objectives
 Methods: design, important changes, participants, settings, location, interventions, outcomes, sample size
 Randomization: sequence generation, allocation concealment, implementation
 Blinding
 Statistical methods used for primary and secondary outcomes, additional analyses
 Results: participant flow diagram, recruitment, baseline demographics, numbers of participants, results for
primary and secondary outcomes, estimated effect size and precision, ancillary analyses and harms
 Discussion: limitations, generalizability and interpretation
 Registration number and name on trial registry
 Location of the full trial protocol, funding
The Enhancing the Quality and Transparency of Health Research (EQUATOR) Network provides information on all
reporting guidelines in clinical research57
. Guidelines exist for reporting most types of studies in a wide variety of
therapeutic areas. Again, the use of these standards is not mandatory and are rarely used.
 Randomized trials: CONSORT
 Observational studies: STROBE
58
 Systematic reviews: PRISMA
59
 Quality improvement studies: SQUIRE
60
and another more than 400 design-specific reporting guidelines.
The main limitations of the current information environment in healthcare and its linkage to post-market
surveillance system include selective disclosure of study findings both in the registry and in published literature,
and low and inconsistent quality of the majority of study reports. The probability of human error during
collection, appraisal and synthesis of evidence increases with the share of unreported and unpublished studies,
lack of information required for accurate appraisal, and complexity of methodology that needs to be employed
to account for all the intelligence gaps, biases and inadequate reporting. The lack of timely and accurate
feedback regarding clinical utility of approved drugs contributes to the lack of responsiveness of the healthcare
system. This includes the ability to accurately incorporate new evidence from real-world use into the total body
of evidence that comes from clinical studies.

Data security in healthcare
Patient privacy rights, expectations, attitudes and research
Medical privacy and breaches of personal health information (PHI) have been a hotly debated topic for several
years. Successful adoption of electronic health records greatly depends on the willing cooperation of hospital staff.
The quality and consistency of electronic health records including coding patterns are essential for the future
utilization of EHRs in research. Perceived effectiveness of regulatory and technological mechanisms positively
impacts trust and perceived privacy control
61
.
Patient attitudes toward sharing their health data with third parties (Grill et al., 2017).
Survey administered to cognitively normal participants who previously participated in Alzheimer’s Disease
prevention research in the UC Irvine ADRC. The patients completed an annual follow-up visit and agreed to be
contacted about future AD studies. 42% of participants reported they would be likely to enroll in a shared
information registry while 55% would enroll in the honest broker model
62
.
Concerns about sharing sensitive information with industry, insurers, and employers are among the important
factors that hinder trial recruitment. Carlisle et al. (2014) evaluated phase 2 and 3 trials registered in the National
Library of Medicine and closed in 2011 to find out how many were terminated because of unsuccessful
recruitment. Of the 2579 identified trials, 481 (19%) were either terminated for failed accrual or were completed
but the enrollment was lower than 85% of target enrollment
63
. The problem of recruitment is particularly limiting in
cancer trials where, according to the Institute of Medicine (IOM), more than 70% of phase 3 trials approved by the
National Cancer Institute closed without meeting their accrual targets. Unfortunately, only less than 7% of cancer
patients participate in clinical trials. Generalizability of findings depends on sufficient number of patients enrolling
in clinical trials64
.
Terminated trials due to insufficient recruitment result in missed opportunities, waste of resources and time and
NDA submission delays. A public-private partnership the Clinical Trials Transformation Initiative (CTTI) conducted a
systematic review of literature to prepare a survey for key stakeholders in order to examine barriers to trial
recruitment. Answers from responders suggest that one of the most pressing problems is finding eligible patients

who meet both inclusion and exclusion criteria, and that the most effective ways how to mitigate this problem is to
screen electronic health records and hospital-based registries for eligible patients
65
.
Registries that contain information that is sensitive and confidential represent a unique set of challenges that
impact patient population enrolled in clinical trials. In an “honest-broker” model, the registry shares only de-
identified data with investigators. For example, Brain Health Registry asks the potential participants to complete a
series of online tests, allow access to medical records, and provide blood samples and saliva for genetic tests
66
.
Balancing patient privacy and research in care delivery, including the use of electronic health records (EHRs) for
screening and research, is an ongoing challenge. Survey regarding patient attitudes to handling sensitive patient
information conducted in 2007 by Privacy Consulting Group revealed that patients hold very strong concerns about
the handling of their data, and especially when it comes to any use of data not directly related to patient care
67
.
Medical records are often used for research without explicit consent of patients. These records include wide range
of information including rich-content clinical genomic data that can still be used for research without the patient’s
consent as long as the information is de-identified and shared under data use agreements with other HIPAA-
covered entities. Kulynych and Greely (2017)
68
explored the consequences of proliferation of electronic health
records in the context of genetic privacy risks. As the cost and practical utility of gene sequencing decreases, its
uptake in clinical practice continues to increase. Genome researchers increasingly seek electronic health records as
an inexpensive source of population wide data on genome, health and phenotype. This type of research often
occurs without the patients’ consent and knowledge. This practice is in stark contrast with patient expectations of
privacy and control over their data. Under the Privacy Rule, it is possible to utilize electronic medical records for a
variety of purposes, including billing, quality improvement, or public health functions. These disclosures are
permitted without patient’s consent, although patients may request an account of instances of PHI disclosures,
including public health, during the past six years. Very few patients exercise this right, arguably because of lack of
awareness.
Overall, patient expectations of privacy and control over their data do not seem to be aligned with the current
state of affairs in medical research and the use of medical data, and the level of protection offered by HIPAA.

Data breaches in healthcare
In the U.S., summary details of private health information breaches that involved more than 500 individuals are
available at the HHS OCR portal called Wall of Shame
69
for the public to view
70
. Disclosure obligations
in HIPAA
71
made the problem of data breaches in healthcare transparent (Appendix 15). European legislators rely
extensively on administrative measures implemented by national competent authorities. Although specific and
detailed EU-level legislation exists, specific information about data breaches, cases and incidents, volume and type
of affected data, root causes and analysis of consequences is unavailable.
Affected individuals by state (period from 2009 to 2016)
The ANTHEM Breach in March
2015 affected 78,8 million
patients due to a Hacking/IT
incident that involved the
insurer’s Network server.
Incidents by state (period from 2009 to 2016)

New EU legislation
72
on data privacy provides for numerous exceptions for handling PHI for a variety of legitimate
purposes including scientific research. Appropriate measures against unauthorized disclosure, theft or loss of data
shall include organizational measures, certification, secrecy clauses in contracts, and codes of conduct, the design
of applications, and data pseudonymization and encryption. Policy 0070
73
on the publication of clinical trial data
covers clinical reports and individual patient data as well as data submitted after approval. To access
the database
74
, users are forbidden to download, save, edit, photograph, print, distribute or transfer the clinical
reports, and will not seek to re-identify the trial subjects or other individuals, or face stiff penalties.
Exploitation of medical data for nefarious purposes is on the increase. The black market value of electronic health
records exceeds credit card data
75
. Medical identity theft (MIT) and record tampering can be life-threatening Theft
of PHI in order to gain access to health treatment or file for reimbursement qualify as identity theft. The
consequences include financial loss if the PHI are used to obtain medical services as goods. Life-threatening
situations can occur if medical records are changed, absent or erroneous as a result of the theft. Paper records
limited the volume that could be stolen during an incident. Electronic records exponentially increase the number of
records stolen during an incident
76
.
Even worse, health data controllers and processors have a limited ability to detect data breaches in real time.
According to a Verizon report, two-thirds of healthcare data breaches go undiscovered for months or even years.
Consequently, people are withholding information – including critical information – from their healthcare providers
because they are concerned that there could be a confidentiality breach of their records
77
. Electronic systems
make confidential data more easily and rapidly accessible to a wider circle of recipients than paper systems,
with greater potential for breaches of confidentiality. Patients’ willingness to participate in research is partly
rooted in their trust that data management systems will preserve confidentiality and personal health information
will not be shared inappropriately.
State actors, specifically from China, also target healthcare information. In August 2017, the FBI arrested Yu Pingan
a.k.a. GoldSun for distributing Sakula malware to 147 unique U.S. IP addresses, including the Office of Personnel
Management (OPM) and health insurer Anthem
78
. The ANTHEM Breach in March 2015 affected 78.8 million
patients due to a Hacking/IT incident that involved the insurer’s Network server. The OPM breach exposed
information on 25.7 million Americans.
Centralization of healthcare information systems, digitalization, merging previously disparate and compartmented
data pools, and the combination of clinical trial data with inpatient, outpatient, A&E and administrative records in
interconnected databases, increase substantially the value of such records to any threat actors. Vulnerability
assessments of information systems need to take into account all human-machine interfaces, user behavior,
awareness and training, and breach detection mechanisms, as well as historical experience and its impact
on patients’ trust and consequently recruitment of subjects in clinical trials. Opportunity for exploitation increases
exponentially with the number of individuals having legitimate access to any one of these interconnected
compartments, as well as the number of entities involved in access control.
The most relevant challenges are the attractiveness of health data for cyber criminals, loss of trust in the ability
of providers to keep the data safe, the reliance on designs that require pooling of sensitive data, and systems
that require/allow PHI sharing for research and analysis.

Solution Design
An integrated solution that would utilize real-world data for the production of medical evidence can be designed
and developed to answer the information needs of stakeholders in the healthcare ecosystem in a more accurate,
comprehensive and timely manner. A service that utilizes a variety of data analysis and integration platforms and
is based on the processing of real-world data at a hospital level utilizing Master study protocols in order to
produce standardized and layered medical evidence that would provide valuable insight into the safety and
efficacy of marketed drugs.
From RWD to RWE
The existing post-market surveillance system is a complex process that follows multiple compliance and commercial
tracks depending on individual stakeholders’ vital interests. Signals detected through standard pharmacovigilance
methods such as pharmacovigilance reporting, database queries or clinical reports may result in the detection of
safety concerns that need to be investigated further to determine causal relationships. Post-market requirements
(PMRs) and commitments (PMCs) are designed to answer these safety questions and result in product label update
where necessary (Appendix 6). Priorities defined in PMRs/PMCs are an example of well-defined critical and priority
information requirements (CIR/PIR). At hospital level, it would be possible to answer some of the important
research questions in drug safety, efficacy and comparative effectiveness using real-world evidence derived from
real-world data.
High-quality medical evidence is a critical system need. The Evidence-based Practice Center (EPC) Program
established by the Agency for Healthcare Research and Quality (AHRQ) is one the most recognized producers of
evidence reviews in medicine today. The program has developed considerable expertise in performing systematic
reviews of interventions in a wide range of therapeutic areas. In their report A Framework for Conceptualizing
Evidence Needs of Health Systems, Schoelles at al. evaluated the evidence needs of health systems that would
enable informed decisions about acquiring new technologies, implementation, and expansion of service offerings,
and the selection of governance, finance and delivery system models. The authors argue that the stakeholders’
preference is for evidence syntheses that are succinct, layered and easy to understand and available in a timely
manner. Trustworthiness in terms of content and methodology is paramount. The lack of linkage between
researchers as the creators of knowledge and decision-makers who are its key users remains a significant
challenge79
.Improved quality and efficiency of postmarket drug safety benefit-risk analysis is spelled out in Section
905 of the FDAAA of 2007 (Appendix 2A). The current pharmacovigilance system cannot produce the quality of
evidence that would meet these criteria.
From compliance to Priority Information Requirements. The current pharmacovigilance post-market surveillance
system satisfies compliance requirements but does not serve the information needs of other stakeholders in the
system. Per protocol analysis of RWD can produce level 2B/2C evidence for individual non-interventional studies
and level 1B/1C for randomized controlled trials, and 1A for systematic reviews. This is a significant improvement
compared to reliance on solely level 4 evidence produced by the current national PV reporting systems. PASS/PAES
studies produce evidence of level 2 (observational) or up to 1B (RCTs) but these are relatively few and their results
are mostly unavailable. Hence, the acquisition of RWD can become the basis for the production of high-volume,
high-quality RWE, with the potential to cover entire therapeutic areas in a systematic and consistent manner to
satisfy specific information needs of non-industry stakeholders in the system including clinicians, healthcare
providers, insurers, and investors.

From data to evidence. Accurate and readily available information is critically important to all stakeholders in the
healthcare ecosystem. RWE can be used to evaluate risks and benefits of treatments and procedures in the
context of clinical care, compare their safety, efficacy and cost-effectiveness and support new indications and
extensions to additional population subgroups. To mitigate potential biases, studies leveraging RWD data sources
have to be carefully designed. The FDA issued guidance that elaborates on the conditions of utilization of RWE to
support regulatory submissions
80
. Real-world data include patients with comorbidities and co-medications,
compare brand products as well as generics, and allow mapping of entire therapeutic areas in a systematic and
consistent manner. Independently produced high-volume real-world evidence would enable near real-time insight
into the safety and efficacy of marketed drugs in a more efficient and cost-effective manner than what we see in
the post-market surveillance ecosystem today. The ability to generate evidence from real world use in the context
of routine care from population samples represented by specific healthcare facilities will add depth and accuracy to
the current information ecosystem in the post-marketing drug lifecycle.
Evidence interpretation and synthesis. RWD can be used to support a variety of clinical trial designs to produce
RWE to satisfy the information needs of a variety of stakeholders in healthcare. The total body of evidence from
consistently produced studies will be easier to synthetize and interpret due to the use of standard Master
Protocols, consistently used outcome measures and study endpoints. Methodology necessary to synthesize such
evidence is less complex and easier to communicate as trustworthy.
 Data sources. The main sources of RWD typically include electronic health records (EHRs), pharmacy
information systems (PIS), computerized prescriber order entry (CPOE) systems, laboratory and imaging
information, and data from disease and product registries. Data capture is a critical component of the
process.
 Analysis of selected streams of RWD in a consistent and systematic manner using standardized pre-
approved Master Study Protocols would produce medical/scientific evidence of consistent quality across
the entire therapeutic group.
 Synthesis of evidence that consistently uses outcome measures and endpoints and utilizes the same
laboratory methods and biomarkers is methodologically significantly easier. The result is higher confidence
in practice recommendations than in instances when the reviewers have to synthetize evidence from a
variety of diverse source studies.
 Standardization of the format of study results (e.g. CONSORT, STROBE) will make the results of individual
studies significantly more accessible and understandable to the professional public. Layered information
presented in the form comparative charts and interactive visuals will add value to stakeholders without
formal education in medical statistics.
Real-time insight into the true performance and benefit:risk profile of marketed drugs is in the interest of all
stakeholders within the healthcare ecosystem. Professional societies, physicians, and pharmacists can use RWE to
update clinical guidelines and adjust clinical practice with greater accuracy and confidence. For insurers, the
patterns of use and risk:benefit profile of medications/procedures contribute to the evaluation of overall cost-
effectiveness. For hospitals and hospital administrators, post-market surveillance studies of marketed drugs are an
inherent part of the monitoring of quality of care. Unlike clinical trials, RWD-derived studies provide insight into a
much broader population that is receiving the concerned drugs under real-life conditions.

Enhanced system transparency will reduce the liability of providers for a drug-related patient injury that will
become easier to detect and prevent. Drug injury becomes subject to litigation and mass tort suits, especially if
products involved were deemed adulterated or misbranded. It will become easier to attribute causal relationships
between a drug and patient injury in the light of provided information and guidance. This capability is also relevant
in instances of aggressive behavior that may be linked to variant metabolism of commonly prescribed drugs.
Detection of counterfeit and substandard products will become easier due to the linkage to pharmacy information
systems including package level identifiers and the instant availability of product descriptors to patient outcomes.
An integrated solution designed to produce high-volume, high-quality medical and scientific evidence from real
world data collected in the context of routine medical care, can become an indispensable feature of an evolving
post-market surveillance information ecosystem.
Security by design
Surveys regarding handling patient sensitive information clearly show that patients expect high level of privacy,
security and control over their PHI. The awareness of implications of medical identity theft and other risks linked to
compromised health data increased due to widely publicized major healthcare data breaches. Although significant
portion of secondary research occurs without the explicit consent of research subjects, the nature and extent of
data processing this program would require would not be possible without informed consent. The broad consent of
patients as data subjects with the processing of their data for research is essential for the program’s success.
Willing participation is only achievable if the patients have a high level of confidence and trust in the system’s
security and integrity.
Analysis on site. Real-world data processing and analysis in situ, with onsite dedicated staff, without the need to
off-shore and outsource any part of PHI processing outside the physical location of the hospital makes patient data
significantly safer than designs that rely on the pooling of PHI data sources for exploratory analysis by third parties.
In addition, RWD analysis and RWE production on site respects the current reality of hospital information
environments, the diversity of treatment protocols, guidelines, laboratory methodologies and other practices
including local regulations and policies and prices of medications and equipment.
Share results, not raw data. The program is based on assumption that the majority of patients who enter the
facility would be willing to make their data available for research as long as their individual PHI is never made
available to any third parties. This model is possible only if the whole data collection, processing and analysis takes
place on site. Because the product is the analytical output or a study report, there is no need to share the actual PHI
beyond the hospital door. Information shared with third parties that are not directly involved in patient care will
only be presented in an aggregate form that by design does not contain any PHI.

5W
Who would benefit?
 The pharmaceutical industry, namely Market Authorization Holders (MAHs) and Sponsors in the pursuit of
compliance with pharmacovigilance obligations, specifically post-authorization studies; evidence to
support REMS and evaluate it effectiveness, and to generate evidence for new indications and the use in
special populations, populations that were not tested in clinical trials and during long-term use.
 Hospitals and healthcare professionals, for continuous monitoring of the quality of provided care; to
compare the treatment outcomes of individual treatment protocols for specific situations and patient
populations, to monitor the safety and efficacy of off-label use of approved medications, to monitor
trends and patterns of ADRs and ADEs in specific populations and in commonly used drug combinations;
and to monitor the safety/efficacy of drugs used in contingencies during drug shortages.
 Insurers and payers, to conduct pharmaco-economic studies, compare the cost of individual interventions
and treatment protocols in specific patient populations, validate the cost-effectiveness of specialized, i.e.
pharmacogenomic, testing, and decrease the costs associated with avoidable patient injury;
 Investors, to gain near real-time insight into the actual clinical performance of newly approved drugs
compared to existing alternatives in real-life conditions; reducing potential for liability for harm caused to
patients and mapping new opportunities for investment;
 Regulators, to seek new ways of obtaining the required information in more timely and comprehensive
manner, to verify claims made in clinical trials, compare multiple interventions within the same indication
group, and to gain real-time insight into the benefit:risk profile of marketed products when used under
real-life conditions.
 Patient organizations, the public, consumers, patients, to gain better insight into the value provided by
individual interventions, and be able to make informed decisions regarding their own care including
insurance plans and co-pays.
What would be the benefits?
 The detection of prevalence, incidence and rate of ADRs and ADEs, patterns of use, and trends, near-real-
time, utilizing data from actual use at the level of a healthcare facility or network
 The analysis of relationships between real-world data (RWD), namely product data, patient characteristics,
key indicators of disease, selected biomarkers, interventions such as medications and their combinations,
and treatment outcomes including complications, ADRs and ADEs using a series of pre-approved
standardized Master study protocols
 The production of real-world clinical evidence (RWE) regarding the patterns of use and potential benefits
and risks of medicinal products and treatment protocols and derived from analysis of RWD, systematically
covering entire therapeutic areas and other topics of interest.
 Post-authorization studies into safety and efficacy (mandatory, voluntary); health technology assessments,
evaluation of cost-effectiveness of new drugs compared to standard of care;
 Evaluation of effectiveness of risk evaluation and mitigation strategies (REMS)

When would data be collected?
 After approval of new drugs
Where would data be collected?
 Hospitals, healthcare provider networks
How would data be collected, processed and communicated?
 Per protocol analysis of RWD in randomized trials (large simple clinical trials, pragmatic trials) and
observational trials (prospective and retrospective; case-control, cohort); utilizing pre-approved Master
protocols for a series of studies to make evidence comparable and consistent across the whole therapeutic
group;
 Consistency and standardization would be maintained across outcome measures and study endpoints
utilizing pre-approved Master study protocols. Variables such as individual interventions, population
subgroups, diagnostic biomarkers etc. would be modified/rotated to systematically map entire therapeutic
areas and to answer specific study questions.
 Research results would be communicated in a standard format (CONSORT, STROBE) and layered analytical
outputs (visuals, graphs, charts, interactive tables) without disclosing individual patient information
(including de-identified PHI) to any third parties that are not directly involved in patient care.
Why (The Objective):
 To produce consistent medical evidence in real-world treatment setting that is comparable across the
whole therapeutic/indication group that reflects actual use
 To complement evidence produced in Phase I – III clinical trials and phase IV studies
 To produce RWE usable to suppo rt regulatory submissions (post-market safety surveillance,
PASS/PAES/PMRs/PMCs, new indications, new populations); the evaluation of risk-benefit profile;
evaluation of effectiveness of REMS; clinical recommendations and analyses of cost-effectiveness
 To improve the quality of evidence and increase the speed of generating and evaluating such evidence,
which has the potential to substantially increase system responsiveness to prevent patient injury.

Stakeholders
 Manufacturers/Sponsors: For pharmaceutical research sponsors and pharmaceutical manufacturers who
typically sponsor post-market surveillance efforts, the primary objective is to produce datasets that satisfy
regulatory demands to continue sales and ensure return on investment. Other objectives include identification
of new target populations and new indications to expand the potential for product use.
 Clinical Research Organizations (CROs): The actual conduct of clinical trials is oftendelegated to CROs. The
business objective of CROs is to conduct the research according to Sponsor’s specifications and supply the
data/results, ensuring ROI on provided service.
 Regulators: According to its mission statement, the Food and Drug Administration is responsible for
“protecting the public health by ensuring the safety, efficacy, and security of human and veterinary drugs,
biological products, and medical devices; and by ensuring the safety of our nation's food supply, cosmetics,
and products that emit radiation”. FDA is also “responsible for advancing the public health by helping to speed
innovations that make medical products more effective, safer, and more affordable and by helping the public
get the accurate, science-based information they need to use medical products and foods to maintain and
improve their health”
81
 Healthcare providers: Hospital mission statements vary from one institution to another, often depending on
the form of ownership. The dominant value featured in mission statements is “quality” of the provided care
(65%), followed by “access” (21%), “cost” (21%) and “community benefit” (24%)82
.
 Centers for Medicare & Medicaid Services: CMS's mission is to serve Medicare & Medicaid beneficiaries. The
CMS vision is to become the most energized, efficient, customer friendly agency in the Government. CMS vows
to strengthen the health care services & information available to Medicare & Medicaid beneficiaries & the
health care providers who serve them. CMS’s goals include the creation of a culture of responsiveness
83
.
o Insurers and payers: Cost-effectiveness of the provided care at population and individual level and
prevention of avoidable patient injury, including drug-related injury, are among their key interests
 Investors: The primary objective is a return on investment from drug development projects, early detection of
critical issues that will result in program failure, and precise direction of research efforts to meet critical unmet
needs.
 Patients: Access to safe, effective and affordable care, and the ability to participate in decision-making relating
to own treatment, including co-pays and costs involved.

Main uses of post-market data
 Pharmacovigilance compliance activities such as ICSR reporting, collection of data for clinical post-market
commitments and requirements, and Risk Evaluation and Mitigation Strategies (REMS)
 Active monitoring of safety concerns - complement to nation-wide data from Sentinel and FAERs
 Efficacy evaluation to support new indications/new target populations supplement submissions, compare
new therapies against mainstay options, introduction of low-cost alternatives
 Health Technology Assessments to support acquisition of new technologies, procedures, interventions,
policies and payment models
 Quality of Care monitoring – complications, readmissions, ADEs, medication errors, hospital-acquired
conditions, polypharmacy-relating complications, off-label use patterns, contingencies during drug
shortages, sick day/return to work, QALY/QADY scores, need for home care etc.
 Safety, efficacy and cost-effectiveness of various interventions including non-pharmacological (dietary,
exercise, psychotherapy, combined approaches), comparison of treatment protocols, interactions
 Cost-effectiveness, pharmaco-economic studies
 Specific reporting – i.e. opioid overdose, antibiotic resistance patterns
 Decision support for research funding, direction of future research
 Detection of new safety and efficacy concerns, intensive monitoring of known issues
 Analysis of patterns of use – safety, efficacy, long-term effects, cost-effectiveness, patient satisfaction,
healthcare and system related costs (contact with providers, testing)
 Observational research, outcomes research, pharmaco-epidemiological studies
 Randomized controlled trials, pragmatic trials
“Physicians need up-to date and independent information about drug safety and efficacy in order to adequately
protect their patients”
Wealth of data is routinely collected during the post-authorization product life cycle in order to comply with a
variety of local, state and national regulations and to satisfy information requirements of a variety of stakeholders
involved in routine care, research, monitoring and enforcement. The data serve a variety of stakeholders whose
information needs differ from one another. Leveson et al. (2012)
84
in their paper “Applying System Engineering to
Pharmaceutical Safety” analyzed the current pharmaceutical safety control structure on the example of the market
withdrawal of cox-2 inhibitor Vioxx. Broad definition of control includes design and process controls as well as
regulatory, professional standard, cultural, social and self-interest incentives. The team examined system goals and
system hazards, identified major components of the system, and modelled the static safety control structure.
Model of the dynamics of drug prescription expresses physician’s likelihood to prescribe a drug. The team also
modeled how this model changes if the information environment improves (Appendix 16 and 17).
Improved information environment affects the orientation, decisions and actions of all stakeholders in the
system, with ultimate impact on the patterns of use of prescription drugs.

Boyd Cycle (O-O-D-A Loop)
Post-market surveillance is no different from any other information feedback loop. Accurate and timely feedback
from real-world use is essential for all interested stakeholders in the healthcare information ecosystem.
In the observation phase, the system needs to collect information that is relevant in terms of quality, volume and
granularity level to enable meaningful processing and support informed decision at appropriate levels.
Information that is collected and processed needs to be available in a form that is understandable and actionable.
Standard format of new evidence that is presented in multiple layers of granularity is essential to enable timely and
accurate orientation that can be consistently performed by target audiences after appropriate training. Orientation
is the key phase of the Boyd cycle, essential for informed decision and action.
Long delays in the availability and lengthy and complex processing of collected data into an actionable form (study
reports or publications) are typical for the current post-market surveillance system. These extensively long delays
effectively disable this key feedback mechanism that is essential to the function of healthcare information
ecosystem.
Scientific/medical/technical evidence produced by the industry to support regulatory submissions and enable
commercialization of products is currently difficult or impossible to verify independently. This is especially true if
the product use is expanded to the entire population and when the patterns of use significantly change in the post-
approval stage of product lifecycle compared to the use documented in clinical trials. Accurate and timely insight
into the safety and efficacy of individual treatment options in different subpopulations is essential for all
stakeholders in healthcare information ecosystem to support informed decisions.
OBSERVE ORIENT DECIDE ACT

The Intelligence Cycle
The emphasis of the pharmaceutical industry on compliance obligations in terms of limiting information collection
to pharmacovigilance purposes and other compliance functions only addresses one category of threat rather than
the whole spectrum. While the risk of compliance failure is indeed important, other threats such as the risk of
litigation, consumer rejection, or loss of competitive edge have to be part of the intelligence cycle as well. Each
stakeholder in the healthcare ecosystem has its unique set of information needs that are necessary to fulfil their
mission. These unique needs can be translated into Critical Information Requirements (CIR) and Priority
Information Requirements (PIR).
The intelligence cycle is driven by the organization’s needs as defined by its Critical Intelligence Requirements
(CIR) and Priority Intelligence Requirements (PIR).

Post-Market Surveillance Redesigned
Research Planning and Direction
In the planning stage, CIR and PIR have to be established based on mission analysis, compliance and reporting
obligations, internal performance indicators and operational needs of the enterprise. A holistic approach to
intelligence direction and planning ensures that evolving needs of the organization are periodically reviewed and
the intelligence cycle is adjusted as appropriate to reflect the operational environment in a timely, accurate and
cost-effective manner. Scientific and technical research in healthcare has to be timely, relevant and responsive to
stakeholders’ information needs. While it is indeed possible to repurpose research results produced by other
stakeholders to support their commercial interests, it is vital to have the capability to generate own
scientific/medical/technical evidence independently to provide timely, accurate and relevant feedback that is
responsive to each stakeholder’s own specific needs.
 Research strategy ensures end-to-end design of the entire data collection, collation and analysis process
to the production of the final intelligence product
 Define reporting requirements and design means of fulfilling these obligations
 Define internal customer information needs
 Systematically search and periodically review foreseeable external customer information needs
 Select and design data collection and processing infrastructure
 Built existing and foreseeable information requirements in the data collection and processing design,
allowing for significant data stream selection flexibility
 Seek broad consent from data subjects for internal data analysis and external sharing of analytical outputs
and study results
 Plan to collect subsets of relevant data from all applicable hospital information systems, and built in ability
to turn individual data streams on/off.
 Build on gold standard treatment guidelines, laboratory practices and biomarker use and consider local
circumstances and internal policies
 Align internal capabilities with PIR/CIR defined in external documents (PMRs/PMCs, REMS, action plans)
 Ensure robust validation of algorithms in AI systems
 Define optimal output formats for each type of product
 Plan sequence from easy to implement individual components (e.g. microbial resistance
genotype/phenotype studies) to more complex undertakings (observational studies, large simple trials)
 Define relevant and obtainable outcome measures and clinically relevant endpoints, align them with drug
development outputs where practicable
 Define data core that is collected continuously, regardless any other requirements
 Define optional variables that enable adding additional data streams for specific research projects
 Define routine data collection tracks (core and optional) and distinguish them from enhanced data
collection efforts for research purposes, including additional components of informed consent
 Avoid collecting unnecessary information that does not contribute to the research objectives

 Design, develop, validate and implement electronic data capture methods and define processes
 Seek feedback and ensure continuous refinement of data capture methods to ensure accuracy and timely
processing without hindering care delivery
 Define QC/QA processes to ensure high quality of data at entry
 Design and develop training materials and deliver training to ensure high-quality data capture
 Preparation of a set of model study designs and template Master protocols
 Design, develop, validate and implement processes for the production of RWD-based RWE
 Seek early feedback during all stages of design, development and implementation
Information Collection
Information collection represents the systematic screening of defined sources as specified in the Intelligence Plan.
In the context of RWD for the production of RWE, this includes specific subsets of data from pharmacy information
systems, computerized prescriber order entry (CPOE), electronic health records, laboratory data, medical devices
(infusion pumps, life function monitors), pharmacy barcode scanning system, treatment outcomes and other
information as appropriate and relevant.
 Information collection infrastructure needs to be robust enough to accommodate the volume of data
generated by routine data collection efforts, in addition to any anticipated research needs.
 The system requirements correspond with current and anticipated CIR/PIRs of internal and external
customers, data format and quality parameters, privacy safeguards, and output requirements.
 Verification/validation of the information and quality assessment occurs at this stage. Quality of routinely
collected data in healthcare vary greatly from one facility to another depending on the systems used,
training of personnel, workload, and quality management systems employed. The quality and consistency
of collected data is of paramount importance for the quality of the final product.
 Assessment of relevance of the material included and documentation of information excluded from
assessment is essential for evaluation and quality management of the final product. Selective inclusion and
exclusion of individual data streams enable flexibility necessary to produce relevant outputs as defined by
core requirements and additionally added research protocols.
 Anonymization/de-identification of PHI occurs at this stage.
 Informed Consent for collection and processing data for research and quality management purposes shall
be obtained from all participating patients. Very high standards of patient privacy and a policy that does
not allow PHI sharing with third parties that are not directly involved in patient care makes it easier to
obtain consent, as opposed to models that rely on PHI pooling and processing by external researchers.
 Master Protocol Approval, including a model for rotating variables has to be obtained before the research
can be commenced.
 Ethics Committee/Institutional Review Board approval is necessary for research on human subjects.

A New Approach at Conducting Post-Market Drug Safety Surveillance

A New Approach at Conducting Post-Market Drug Safety Surveillance

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a A New Approach at Conducting Post-Market Drug Safety Surveillance

Semelhante a A New Approach at Conducting Post-Market Drug Safety Surveillance (20)

Mais de Arete-Zoe, LLC

Mais de Arete-Zoe, LLC (20)

Último

Último (20)

A New Approach at Conducting Post-Market Drug Safety Surveillance