EHR mining, claims databases, PheWAS phenome-wide association, and Kaplan-Meier survival analysis to evaluate drug effectiveness in real patient populations beyond clinical trial populations
Real-world evidence (RWE) is clinical evidence derived from the analysis of real-world data (RWD) — data collected from sources outside traditional clinical trials. As electronic health records have become universal in developed countries, an enormous new resource has emerged: the complete medical history of entire populations, enabling studies of drug effectiveness and safety at a scale and in populations impossible to achieve in randomized controlled trials.
Real-world data for drug effectiveness studies comes from five primary source types:
1. Electronic Health Records (EHR): • Clinical notes (unstructured text): diagnoses, symptoms, procedures by physician — NLP required • Structured data: ICD-10 diagnosis codes, CPT procedure codes, NDC drug codes, LOINC lab codes • Major vendors: Epic (38% US market), Oracle Cerner (25%), MEDITECH (15%) • Key limitation: data completeness limited to treating health system — no out-of-network care
2. Administrative claims (insurance): • Pharmacy dispensing records: NDC code, days supply, quantity dispensed — highly reliable medication exposure • Medical claims: ICD-10 procedure + diagnosis codes for every visit, procedure, hospitalization • Databases: IBM MarketScan (44M lives), Optum Clinformatics (85M), Medicare/Medicaid CMS (145M) • Advantage over EHR: complete care capture across all providers for insured population
3. Patient registries: • Disease-specific: SEER cancer registry (100% US cancer incidence), ACC-NCDR cardiology • Post-marketing surveillance: PSUR, REMS, FAERS (1M adverse event reports/year) • Patient-reported outcomes: PRO collected in EHR (PROMIS) or direct-to-patient (wearables)
4. Genomic + biobank linkage: • UK Biobank: 500,000 participants, WGS + EHR linkage — pharmacogenomics RWE • VA Million Veteran Program: 900,000 veterans — largest pharmacogenomics dataset • All of Us (NIH): 1M US participants — diverse ancestry pharmacogenomics
5. OMOP Common Data Model (CDM) standardization: • Maps all local coding systems to standardized concepts (SNOMED, RxNorm, LOINC) • Enables federated analyses across 800M+ patient records without data sharing • OHDSI network: distributed analysis — each site runs code locally, shares aggregate statistics • Key tables: PERSON, DRUG_EXPOSURE, CONDITION_OCCURRENCE, MEASUREMENT, VISIT_OCCURRENCE
Data quality challenges: • Coding completeness: diagnosis codes present only when reimbursement claimed — miscoding ~15% • Surveillance bias: patients on frequent monitoring have more diagnoses recorded (immortal time bias risk) • Healthy user bias: patients taking a new drug tend to be healthier than comparable patients who do not • Immortal time bias: time from cohort entry to drug dispensing incorrectly attributed to treatment exposure
The fundamental challenge in observational pharmacoepidemiology is that patients who receive a drug differ in systematic ways from patients who do not — confounding by indication. A diabetic patient prescribed SGLT2 inhibitors may be healthier overall than one on sulfonylureas; the apparent cardiovascular benefit might reflect that healthier selection rather than drug effect. The active comparator new user design, combined with propensity score methods, is the closest observational approximation to a randomized trial.
Study design elements that minimize confounding in RWE:
1. New User Design (Ray 2003): • Include only patients with NO prior exposure to study drug (new users = "incident users") • Eliminates prevalent user bias: prevalent users are survivors (tolerated the drug) → biased toward benefit • Baseline: ALL covariates measured in washout period before first prescription • Mimics RCT enrollment: patients in same clinical state at initiation
2. Active Comparator Design: • Compare to another drug in same class or same indication (not untreated patients) • Reduces confounding by indication: both drugs prescribed for same clinical reason • Example: SGLT2 inhibitors vs. DPP-4 inhibitors (both add-on to metformin for T2DM) • Exclusion: cannot compare to placebo/no treatment — patients not treated are fundamentally different
3. Propensity Score (PS) estimation: • PS = P(treatment | covariates X) = probability of receiving treatment given all observed confounders • Estimated by logistic regression, gradient boosting, or neural network on baseline covariates • Covariates included (typically 50–200): demographics, comorbidities at ICD level, all prior medications, hospitalizations, labs (if available), healthcare utilization proxies • PS summarizes all observed covariates into single score → conditional on PS, treatment is approximately random
4. PS matching: • 1:1 matching (nearest neighbor, caliper 0.01 SD): match treated to control with closest PS • Caliper: maximum allowed PS difference — prevents poor matches • Check balance: standardized mean difference (SMD) = (mean_T - mean_C) / pooled_SD • Goal: ALL covariates — including interactions — SMD <0.1 (i.e., <10% of SD difference) • Loss of population: matching typically reduces cohort by 40–60% (those who cannot be matched dropped)
5. Alternative PS methods: • PS stratification: divide into 5 quintiles, adjust in outcome model • Inverse probability weighting (IPTW): weight = 1/PS(treatment) + 1/(1-PS)(control) — retains full cohort • Doubly robust estimator: IPW + outcome model — consistent if EITHER model correctly specified
6. Residual confounding assessment: • E-value (VanderWeele, Ann Intern Med 2017): minimum strength of unmeasured confounder needed to explain away observed association • Negative control outcome: test drug vs. outcome impossible to be caused by drug → if HR≠1, indicates residual confounding • Negative control exposure: inactive ingredient, placebo-like comparison → validation • Falsification endpoints: conditions biologically unrelated to drug → p-value distribution should be uniform
A PheWAS tests a single exposure (drug or genetic variant) against hundreds to thousands of clinical phenotypes simultaneously. Rather than testing one hypothesis at a time, PheWAS explores the complete phenotypic landscape of a drug — discovering both anticipated effects, unexpected side effects, and drug repurposing opportunities in a single unbiased scan. The technique has become a standard tool for post-marketing surveillance and repurposing signal generation.
PheWAS execution framework:
1. Phenotype coding: • ICD-9 and ICD-10 codes mapped to "phecodes" (Denny 2013) — collapsing related diagnosis codes • Example: I21.x (all STEMI types) + I22.x (subsequent MI) → phecode 411 "MI" • Case definition: require ≥2 phecode occurrences (reduces false positive diagnosis codes; accidental coding, rule-out diagnoses) • Control definition: patients without phecode, but in same healthcare system • Exclusion: patients with rare phecode in exclusion range (e.g., for MI analysis, exclude angina phecode from controls)
2. Statistical model: • Binary logistic regression: Phecode ~ Drug_exposure + covariates + offset(log(observation_time)) • Covariates: age, sex, race/ethnicity, practice site, calendar year • OR reported (equivalent to RR for rare outcomes) • Beta-blocker PheWAS across 250K patients in BioVU (Roden 2011): first pharmacogenomic PheWAS
3. Visualization — Manhattan plot: • X-axis: phecodes grouped by organ system (circulatory, neoplasms, neurological, infectious...) • Y-axis: -log10(p-value) • Color: organ system category • Horizontal line: Bonferroni threshold (-log10(2.75×10⁻⁵) = 4.56) • Bubble size: effect size (OR)
4. Drug PheWAS signals for our compound: Protective associations (OR <1): • Renal failure (phecode 585): OR=0.61 (p=8×10⁻⁹) — unexpected nephroprotection • Peripheral vascular disease (phecode 443): OR=0.72 (p=4×10⁻⁷) — vascular benefit • Atrial fibrillation (phecode 427): OR=0.81 (p=7×10⁻⁶) — rhythm stabilization
Adverse associations (OR >1): • Drug-induced liver disorder (phecode 573): OR=1.58 (p=2×10⁻⁶) — hepatic signal • UTI (phecode 599): OR=1.32 (p=4×10⁻⁵) — class effect (SGLT2 glucosuria)
5. PheWAS for drug repurposing: • Metformin PheWAS: protective associations with multiple cancers (colon, pancreatic, endometrial) • Statins PheWAS: protective association with sepsis, dementia, AMD • SGLT2 inhibitor PheWAS: heart failure hospitalization reduction; chronic kidney disease — led to DAPA-HF trial • Finasteride PheWAS: repurposing signal for prostate cancer (replicated in PCPT trial RCT)
6. VigiBase WHO pharmacovigilance: • 30+ million spontaneous adverse drug reaction reports globally • Disproportionality analysis: reporting odds ratio (ROR) = (A×D)/(B×C) in 2×2 table • Drug-adverse event combinations above threshold → regulatory signal → DHPC letter or label update
The Kaplan-Meier estimator and Cox proportional hazards model are the statistical workhorses of outcomes research. They answer the question: among patients who have not yet experienced an event, how does drug treatment change the rate of experiencing it over time? Unlike simple proportions, survival analysis correctly handles censoring — patients who are lost to follow-up or who have not yet experienced the event at study end — making it the only valid approach for time-to-event data.
Kaplan-Meier survival analysis:
Survival function S(t): S(t) = Π_{j: t_j ≤ t} (1 - d_j / n_j)
Where: • t_j = ordered event times • d_j = number of events (deaths/MACE) at time t_j • n_j = number at risk just before time t_j (includes censored patients until censoring time) • Censoring: patient did not experience event — either follow-up ended, they were lost to follow-up, or study ended
Key properties of K-M estimator: • Non-parametric: no assumption about shape of hazard function • Handles censoring correctly: at censoring time, remove patient from risk set • 95% CI: Greenwood's formula: Var[S(t)] = S(t)² × Σ_{j: t_j ≤ t} d_j / (n_j × (n_j - d_j)) • At-risk table below K-M plot: shows n_j decreasing over time
Log-rank test: • Test H₀: S₁(t) = S₂(t) (no difference between groups) • Test statistic: χ² = (Σ(O₁ - E₁))² / Var(Σ(O₁ - E₁)) • O₁ = observed events in treatment group; E₁ = expected events under H₀ • Most powerful when hazard ratio is constant over time (proportional hazards) • Weighted versions: Wilcoxon (early differences), Harrington-Fleming (late differences)
Cox proportional hazards model: h(t|X) = h₀(t) × exp(β₁X₁ + β₂X₂ + ... + βₚXₚ)
• h₀(t) = baseline hazard (unspecified, non-parametric) • HR = exp(β₁) = hazard ratio for treatment vs. control, adjusted for all covariates • HR interpretation: at any point in time, treatment patients have HR × (event rate) of control patients • Proportional hazards assumption: HR is constant over time — tested by Schoenfeld residuals • When PH violated: time-dependent covariate, stratified Cox, or restricted mean survival time (RMST)
Number Needed to Treat (NNT): • NNT = 1 / (ARR) = 1 / (CIR_control - CIR_treatment) • CIR = cumulative incidence risk at 3 years (1 - K-M) • Treatment 3-year MACE risk: 17.6%; Control 3-year risk: 23.2% • ARR = 23.2% - 17.6% = 5.6% • NNT = 1/0.056 = 18 (18 patients treated for 3 years to prevent 1 MACE)
Competing risks: • MACE endpoint: patient who dies from non-cardiovascular cause is a competing event • Fine-Gray subdistribution hazard model: directly models cumulative incidence in presence of competing risks • Cause-specific hazard (Cox): censors competing events — convention for primary causal analysis • Cumulative incidence function (CIF): preferred for absolute risk estimation in presence of competing events
The 21st Century Cures Act (2016) and subsequent FDA guidance explicitly encourages use of real-world evidence to support regulatory decisions. The Bayesian framework provides a mathematically principled way to combine evidence from multiple sources — using RWE as an informative prior informed by RCT results, or using a power prior that downweights RWE based on its similarity to RCT conditions. This approach is now used to support label expansions, pediatric approvals, and rare disease indications.
Bayesian evidence synthesis framework:
1. Power Prior (Ibrahim & Chen 2000): • Likelihood for historical data (RWE) downweighted by α ∈ [0,1] • π(θ | D_0) ∝ L(θ | D_0)^α × π₀(θ) D_0 = historical (RWE) data; α = discounting factor; π₀ = initial prior • α=0: completely ignore RWE (non-informative prior) • α=1: treat RWE as equally reliable as new RCT data • α chosen based on historical/current trial similarity: Bhattacharyya coefficient, covariate overlap measure
2. Commensurate Prior (Hobbs et al. 2011): • More flexible: strength of borrowing adapts based on observed similarity between RCT and RWE • Hierarchical model: θ_new | θ_hist ~ N(θ_hist, τ²) • τ² (heterogeneity) estimated from data: large τ → little borrowing; small τ → strong borrowing • Preferred by FDA for adaptive borrowing when prior-data conflict possible
3. Meta-analytic predictive (MAP) prior: • Synthesize k available historical studies: μ_1..μ_k → predictive distribution for new study θ_new • Normal-normal hierarchical model: μ_i | μ, τ² ~ N(μ, τ²) • Robust MAP: mixture of MAP + vague component — protects against prior-data conflict • EMA "reflection paper" (2018): recommends MAP for rare diseases, pediatric extrapolation
4. Current study results integration: Prior (from RWE): HR ~ LogNormal(log(0.77), 0.09²) derived from observational analysis Likelihood (from RCT): log(HR_new) | log(HR_true) ~ N(log(HR_new), SE²) Power prior (α=0.3): π(HR) ∝ L_RWE(HR)^0.3
Posterior HR: 0.79 (2.5th–97.5th percentile: 0.71–0.87) Bayes factor (BF): • BF₁₀ = P(data | H₁: HR<1) / P(data | H₀: HR=1) = 12.4 • Interpretation: data 12.4× more probable under H₁ than H₀ → "strong evidence" • Jeffreys scale: BF>10 = strong; BF>30 = very strong; BF>100 = decisive
5. FDA regulatory use of RWE (2024 landscape): • Ibrance (palbociclib): RWE supported label expansion to male breast cancer (N=27 in RCT, N=13,000 in RWE) • Keytruda (pembrolizumab): synthetic control arm from RWD for rare tumor indications • CDRH medical devices: 67 device submissions used RWE 2015–2020 per FDA report • Accelerated approval confirmatory trials: RWE allowed as one source of evidence for conversion
6. Regulatory considerations and limitations: • FDA RWE framework (2018): fit-for-purpose RWD; reliability of study design; relevance to regulatory question • EMA RWE methodology guidelines: Appendix 1 (methodological standards); DARWIN EU network • Key limitation: unmeasured confounding cannot be fully eliminated — E-value analysis required • Replication: FDA expects RWE finding replicated in ≥2 independent databases before regulatory acceptance
The first FDA approval supported by a real-world synthetic control arm (instead of a randomized placebo group) occurred in 2019 for pembrolizumab in TMB-high tumors. The RWE synthetic control used 85,000 advanced solid tumor patients from the Flatiron Health database to substitute for a randomized control arm — allowing approval with N=158 single-arm trial patients. This precedent has opened the door for RWE-supported approvals in rare diseases where randomized trials are impractical.