🩺 Real-World Evidence (RWE)

Real-World Data Sources — The Digital Footprint of 300 Million Patients

Real-world evidence (RWE) is clinical evidence derived from the analysis of real-world data (RWD) — data collected from sources outside traditional clinical trials. As electronic health records have become universal in developed countries, an enormous new resource has emerged: the complete medical history of entire populations, enabling studies of drug effectiveness and safety at a scale and in populations impossible to achieve in randomized controlled trials.

• >96%: US EHR adoption rate (hospitals; >86% physicians (ONC 2023))
• 800M+ patients: OMOP CDM networks (worldwide; 50+ countries in OHDSI)
• 2018: FDA RWE guidance (21st Century Cures Act)
• 145M beneficiaries: Medicare/Medicaid database (complete US claims linkage)

The RWD ecosystem — data sources, standards and linkage approaches

Real-world data for drug effectiveness studies comes from five primary source types:

1. Electronic Health Records (EHR): • Clinical notes (unstructured text): diagnoses, symptoms, procedures by physician — NLP required • Structured data: ICD-10 diagnosis codes, CPT procedure codes, NDC drug codes, LOINC lab codes • Major vendors: Epic (38% US market), Oracle Cerner (25%), MEDITECH (15%) • Key limitation: data completeness limited to treating health system — no out-of-network care

2. Administrative claims (insurance): • Pharmacy dispensing records: NDC code, days supply, quantity dispensed — highly reliable medication exposure • Medical claims: ICD-10 procedure + diagnosis codes for every visit, procedure, hospitalization • Databases: IBM MarketScan (44M lives), Optum Clinformatics (85M), Medicare/Medicaid CMS (145M) • Advantage over EHR: complete care capture across all providers for insured population

3. Patient registries: • Disease-specific: SEER cancer registry (100% US cancer incidence), ACC-NCDR cardiology • Post-marketing surveillance: PSUR, REMS, FAERS (1M adverse event reports/year) • Patient-reported outcomes: PRO collected in EHR (PROMIS) or direct-to-patient (wearables)

4. Genomic + biobank linkage: • UK Biobank: 500,000 participants, WGS + EHR linkage — pharmacogenomics RWE • VA Million Veteran Program: 900,000 veterans — largest pharmacogenomics dataset • All of Us (NIH): 1M US participants — diverse ancestry pharmacogenomics

5. OMOP Common Data Model (CDM) standardization: • Maps all local coding systems to standardized concepts (SNOMED, RxNorm, LOINC) • Enables federated analyses across 800M+ patient records without data sharing • OHDSI network: distributed analysis — each site runs code locally, shares aggregate statistics • Key tables: PERSON, DRUG_EXPOSURE, CONDITION_OCCURRENCE, MEASUREMENT, VISIT_OCCURRENCE

Data quality challenges: • Coding completeness: diagnosis codes present only when reimbursement claimed — miscoding ~15% • Surveillance bias: patients on frequent monitoring have more diagnoses recorded (immortal time bias risk) • Healthy user bias: patients taking a new drug tend to be healthier than comparable patients who do not • Immortal time bias: time from cohort entry to drug dispensing incorrectly attributed to treatment exposure

Confounding and How to Fight It — The Active Comparator New User Design

The fundamental challenge in observational pharmacoepidemiology is that patients who receive a drug differ in systematic ways from patients who do not — confounding by indication. A diabetic patient prescribed SGLT2 inhibitors may be healthier overall than one on sulfonylureas; the apparent cardiovascular benefit might reflect that healthier selection rather than drug effect. The active comparator new user design, combined with propensity score methods, is the closest observational approximation to a randomized trial.

• 2–5×: Confounding bias in RWE (unadjusted vs. adjusted HR)
• 50–200: Propensity score variables (typical PS model covariates)
• <0.1: SMD threshold for balance (standardized mean difference)
• —: E-value confounding bound (minimum HR for unmeasured confounder)

Propensity score methods and the active comparator new user design

Study design elements that minimize confounding in RWE:

1. New User Design (Ray 2003): • Include only patients with NO prior exposure to study drug (new users = "incident users") • Eliminates prevalent user bias: prevalent users are survivors (tolerated the drug) → biased toward benefit • Baseline: ALL covariates measured in washout period before first prescription • Mimics RCT enrollment: patients in same clinical state at initiation

2. Active Comparator Design: • Compare to another drug in same class or same indication (not untreated patients) • Reduces confounding by indication: both drugs prescribed for same clinical reason • Example: SGLT2 inhibitors vs. DPP-4 inhibitors (both add-on to metformin for T2DM) • Exclusion: cannot compare to placebo/no treatment — patients not treated are fundamentally different

3. Propensity Score (PS) estimation: • PS = P(treatment | covariates X) = probability of receiving treatment given all observed confounders • Estimated by logistic regression, gradient boosting, or neural network on baseline covariates • Covariates included (typically 50–200): demographics, comorbidities at ICD level, all prior medications, hospitalizations, labs (if available), healthcare utilization proxies • PS summarizes all observed covariates into single score → conditional on PS, treatment is approximately random

4. PS matching: • 1:1 matching (nearest neighbor, caliper 0.01 SD): match treated to control with closest PS • Caliper: maximum allowed PS difference — prevents poor matches • Check balance: standardized mean difference (SMD) = (mean_T - mean_C) / pooled_SD • Goal: ALL covariates — including interactions — SMD <0.1 (i.e., <10% of SD difference) • Loss of population: matching typically reduces cohort by 40–60% (those who cannot be matched dropped)

5. Alternative PS methods: • PS stratification: divide into 5 quintiles, adjust in outcome model • Inverse probability weighting (IPTW): weight = 1/PS(treatment) + 1/(1-PS)(control) — retains full cohort • Doubly robust estimator: IPW + outcome model — consistent if EITHER model correctly specified

6. Residual confounding assessment: • E-value (VanderWeele, Ann Intern Med 2017): minimum strength of unmeasured confounder needed to explain away observed association • Negative control outcome: test drug vs. outcome impossible to be caused by drug → if HR≠1, indicates residual confounding • Negative control exposure: inactive ingredient, placebo-like comparison → validation • Falsification endpoints: conditions biologically unrelated to drug → p-value distribution should be uniform

Phenome-Wide Association Study — Unbiased Scan of All Drug Effects in One Analysis

A PheWAS tests a single exposure (drug or genetic variant) against hundreds to thousands of clinical phenotypes simultaneously. Rather than testing one hypothesis at a time, PheWAS explores the complete phenotypic landscape of a drug — discovering both anticipated effects, unexpected side effects, and drug repurposing opportunities in a single unbiased scan. The technique has become a standard tool for post-marketing surveillance and repurposing signal generation.

• 1,817: Phecodes in standard PheWAS (mapped from ICD-10 codes)
• 2.75×10⁻⁵: Bonferroni p-threshold (for 1,817 tests at α=0.05)
• ~12%: Drug repurposing signals (PheWAS replicated in RCT)
• >1M/year: FAERS adverse event reports (FDA MedWatch (spontaneous))

PheWAS methodology — from ICD codes to phenome-wide association signals

PheWAS execution framework:

1. Phenotype coding: • ICD-9 and ICD-10 codes mapped to "phecodes" (Denny 2013) — collapsing related diagnosis codes • Example: I21.x (all STEMI types) + I22.x (subsequent MI) → phecode 411 "MI" • Case definition: require ≥2 phecode occurrences (reduces false positive diagnosis codes; accidental coding, rule-out diagnoses) • Control definition: patients without phecode, but in same healthcare system • Exclusion: patients with rare phecode in exclusion range (e.g., for MI analysis, exclude angina phecode from controls)

2. Statistical model: • Binary logistic regression: Phecode ~ Drug_exposure + covariates + offset(log(observation_time)) • Covariates: age, sex, race/ethnicity, practice site, calendar year • OR reported (equivalent to RR for rare outcomes) • Beta-blocker PheWAS across 250K patients in BioVU (Roden 2011): first pharmacogenomic PheWAS

3. Visualization — Manhattan plot: • X-axis: phecodes grouped by organ system (circulatory, neoplasms, neurological, infectious...) • Y-axis: -log10(p-value) • Color: organ system category • Horizontal line: Bonferroni threshold (-log10(2.75×10⁻⁵) = 4.56) • Bubble size: effect size (OR)

4. Drug PheWAS signals for our compound: Protective associations (OR <1): • Renal failure (phecode 585): OR=0.61 (p=8×10⁻⁹) — unexpected nephroprotection • Peripheral vascular disease (phecode 443): OR=0.72 (p=4×10⁻⁷) — vascular benefit • Atrial fibrillation (phecode 427): OR=0.81 (p=7×10⁻⁶) — rhythm stabilization

Adverse associations (OR >1): • Drug-induced liver disorder (phecode 573): OR=1.58 (p=2×10⁻⁶) — hepatic signal • UTI (phecode 599): OR=1.32 (p=4×10⁻⁵) — class effect (SGLT2 glucosuria)

5. PheWAS for drug repurposing: • Metformin PheWAS: protective associations with multiple cancers (colon, pancreatic, endometrial) • Statins PheWAS: protective association with sepsis, dementia, AMD • SGLT2 inhibitor PheWAS: heart failure hospitalization reduction; chronic kidney disease — led to DAPA-HF trial • Finasteride PheWAS: repurposing signal for prostate cancer (replicated in PCPT trial RCT)

6. VigiBase WHO pharmacovigilance: • 30+ million spontaneous adverse drug reaction reports globally • Disproportionality analysis: reporting odds ratio (ROR) = (A×D)/(B×C) in 2×2 table • Drug-adverse event combinations above threshold → regulatory signal → DHPC letter or label update

Survival Analysis — Time-to-Event Statistics for Drug Effectiveness

The Kaplan-Meier estimator and Cox proportional hazards model are the statistical workhorses of outcomes research. They answer the question: among patients who have not yet experienced an event, how does drug treatment change the rate of experiencing it over time? Unlike simple proportions, survival analysis correctly handles censoring — patients who are lost to follow-up or who have not yet experienced the event at study end — making it the only valid approach for time-to-event data.

• 18.4: Log-rank test χ² (our study) (p=1.8×10⁻⁵)
• 0.68–0.87: HR confidence interval (95% CI; excludes 1.0 → significant)
• 18: Number Needed to Treat (18 pts treated 3y prevent 1 MACE)
• 1958: K-M estimator (original) (Kaplan & Meier, JASA)

Kaplan-Meier estimator and Cox proportional hazards model

Kaplan-Meier survival analysis:

Survival function S(t): S(t) = Π_{j: t_j ≤ t} (1 - d_j / n_j)

Where: • t_j = ordered event times • d_j = number of events (deaths/MACE) at time t_j • n_j = number at risk just before time t_j (includes censored patients until censoring time) • Censoring: patient did not experience event — either follow-up ended, they were lost to follow-up, or study ended

Key properties of K-M estimator: • Non-parametric: no assumption about shape of hazard function • Handles censoring correctly: at censoring time, remove patient from risk set • 95% CI: Greenwood's formula: Var[S(t)] = S(t)² × Σ_{j: t_j ≤ t} d_j / (n_j × (n_j - d_j)) • At-risk table below K-M plot: shows n_j decreasing over time

Log-rank test: • Test H₀: S₁(t) = S₂(t) (no difference between groups) • Test statistic: χ² = (Σ(O₁ - E₁))² / Var(Σ(O₁ - E₁)) • O₁ = observed events in treatment group; E₁ = expected events under H₀ • Most powerful when hazard ratio is constant over time (proportional hazards) • Weighted versions: Wilcoxon (early differences), Harrington-Fleming (late differences)

Cox proportional hazards model: h(t|X) = h₀(t) × exp(β₁X₁ + β₂X₂ + ... + βₚXₚ)

• h₀(t) = baseline hazard (unspecified, non-parametric) • HR = exp(β₁) = hazard ratio for treatment vs. control, adjusted for all covariates • HR interpretation: at any point in time, treatment patients have HR × (event rate) of control patients • Proportional hazards assumption: HR is constant over time — tested by Schoenfeld residuals • When PH violated: time-dependent covariate, stratified Cox, or restricted mean survival time (RMST)

Number Needed to Treat (NNT): • NNT = 1 / (ARR) = 1 / (CIR_control - CIR_treatment) • CIR = cumulative incidence risk at 3 years (1 - K-M) • Treatment 3-year MACE risk: 17.6%; Control 3-year risk: 23.2% • ARR = 23.2% - 17.6% = 5.6% • NNT = 1/0.056 = 18 (18 patients treated for 3 years to prevent 1 MACE)

Competing risks: • MACE endpoint: patient who dies from non-cardiovascular cause is a competing event • Fine-Gray subdistribution hazard model: directly models cumulative incidence in presence of competing risks • Cause-specific hazard (Cox): censors competing events — convention for primary causal analysis • Cumulative incidence function (CIF): preferred for absolute risk estimation in presence of competing events

Integrating RCT and RWE — The Bayesian Framework for Regulatory Submission

The 21st Century Cures Act (2016) and subsequent FDA guidance explicitly encourages use of real-world evidence to support regulatory decisions. The Bayesian framework provides a mathematically principled way to combine evidence from multiple sources — using RWE as an informative prior informed by RCT results, or using a power prior that downweights RWE based on its similarity to RCT conditions. This approach is now used to support label expansions, pediatric approvals, and rare disease indications.

• ~20%: FDA Bayesian clinical trials (of recent device submissions use Bayesian)
• 0–1: Power prior weight α range (0=non-informative; 1=full weight)
• >10: Bayes factor interpretation (strong evidence; >100 decisive)
• 2019: FDA real-world submission (first oncology RWE label expansion (EU))

Bayesian meta-analysis and FDA regulatory framework for RWE integration

Bayesian evidence synthesis framework:

1. Power Prior (Ibrahim & Chen 2000): • Likelihood for historical data (RWE) downweighted by α ∈ [0,1] • π(θ | D_0) ∝ L(θ | D_0)^α × π₀(θ) D_0 = historical (RWE) data; α = discounting factor; π₀ = initial prior • α=0: completely ignore RWE (non-informative prior) • α=1: treat RWE as equally reliable as new RCT data • α chosen based on historical/current trial similarity: Bhattacharyya coefficient, covariate overlap measure

2. Commensurate Prior (Hobbs et al. 2011): • More flexible: strength of borrowing adapts based on observed similarity between RCT and RWE • Hierarchical model: θ_new | θ_hist ~ N(θ_hist, τ²) • τ² (heterogeneity) estimated from data: large τ → little borrowing; small τ → strong borrowing • Preferred by FDA for adaptive borrowing when prior-data conflict possible

3. Meta-analytic predictive (MAP) prior: • Synthesize k available historical studies: μ_1..μ_k → predictive distribution for new study θ_new • Normal-normal hierarchical model: μ_i | μ, τ² ~ N(μ, τ²) • Robust MAP: mixture of MAP + vague component — protects against prior-data conflict • EMA "reflection paper" (2018): recommends MAP for rare diseases, pediatric extrapolation

4. Current study results integration: Prior (from RWE): HR ~ LogNormal(log(0.77), 0.09²) derived from observational analysis Likelihood (from RCT): log(HR_new) | log(HR_true) ~ N(log(HR_new), SE²) Power prior (α=0.3): π(HR) ∝ L_RWE(HR)^0.3

Posterior HR: 0.79 (2.5th–97.5th percentile: 0.71–0.87) Bayes factor (BF): • BF₁₀ = P(data | H₁: HR<1) / P(data | H₀: HR=1) = 12.4 • Interpretation: data 12.4× more probable under H₁ than H₀ → "strong evidence" • Jeffreys scale: BF>10 = strong; BF>30 = very strong; BF>100 = decisive

5. FDA regulatory use of RWE (2024 landscape): • Ibrance (palbociclib): RWE supported label expansion to male breast cancer (N=27 in RCT, N=13,000 in RWE) • Keytruda (pembrolizumab): synthetic control arm from RWD for rare tumor indications • CDRH medical devices: 67 device submissions used RWE 2015–2020 per FDA report • Accelerated approval confirmatory trials: RWE allowed as one source of evidence for conversion

6. Regulatory considerations and limitations: • FDA RWE framework (2018): fit-for-purpose RWD; reliability of study design; relevance to regulatory question • EMA RWE methodology guidelines: Appendix 1 (methodological standards); DARWIN EU network • Key limitation: unmeasured confounding cannot be fully eliminated — E-value analysis required • Replication: FDA expects RWE finding replicated in ≥2 independent databases before regulatory acceptance

The first FDA approval supported by a real-world synthetic control arm (instead of a randomized placebo group) occurred in 2019 for pembrolizumab in TMB-high tumors. The RWE synthetic control used 85,000 advanced solid tumor patients from the Flatiron Health database to substitute for a randomized control arm — allowing approval with N=158 single-arm trial patients. This precedent has opened the door for RWE-supported approvals in rare diseases where randomized trials are impractical.