🚨 Disproportionality Signal Detection (PRR/ROR)

From Yellow Cards to Structured Data — Building the Global ICSR Corpus

Post-marketing drug safety surveillance runs almost entirely on spontaneous reporting: patients, clinicians, and pharmaceutical companies voluntarily submit Individual Case Safety Reports (ICSRs) describing a suspected adverse drug reaction. These reports are noisy, incomplete, and wildly unrepresentative of true incidence — yet aggregated across tens of millions of cases in FAERS, EudraVigilance, and WHO VigiBase, they contain the earliest computable trace of rare and unexpected harms that no clinical trial was ever powered to detect.

• ~2.4M: FAERS reports (annual) (FDA Adverse Event Reporting System)
• ~30M+: EudraVigilance cases (cumulative ICSRs, EU/EEA)
• >40M: VigiBase cases (Uppsala Monitoring Centre, 150+ countries)
• ~26k PTs: MedDRA terms (Preferred Term hierarchy)

ICSR structure and the ICH E2B(R3) data standard

Every spontaneous report entering a pharmacovigilance database is structured according to ICH E2B(R3), the international electronic transmission standard for individual case safety reports:

Core ICSR elements: • Patient data: age, sex, weight, relevant medical history, concomitant medications • Drug data: suspect drug(s) vs. concomitant drug(s), dose, route, indication, start/stop dates, dechallenge/rechallenge outcome • Reaction data: verbatim reported term(s), coded to MedDRA Preferred Term(s), onset date, seriousness criteria (death, life-threatening, hospitalization, disability, congenital anomaly, other medically important) • Report source: healthcare professional, consumer, literature, clinical trial, regulatory authority, or manufacturer • Case narrative: free-text clinical summary

Reporting channels feeding FAERS: FDA MedWatch (voluntary, healthcare professionals/consumers) and mandatory manufacturer 15-day expedited reports for serious+unexpected events, plus periodic aggregate submissions. EudraVigilance receives ICSRs from EU/EEA national competent authorities and marketing authorization holders under the EU pharmacovigilance legislation (Regulation (EU) No 726/2004, Directive 2010/84/EU).

MedDRA coding: the Medical Dictionary for Regulatory Activities organizes ~26,000 Preferred Terms (PTs) into System Organ Classes (SOCs), High-Level Group Terms, and High-Level Terms. A single clinical concept (e.g., "rhabdomyolysis") maps to exactly one PT, but analysts often aggregate related PTs into Standardised MedDRA Queries (SMQs) — e.g., the SMQ "Torsade de pointes/QT prolongation" bundles ~50 individual PTs — to avoid signal dilution across synonymous terms.

Underreporting is the central limitation of spontaneous surveillance: the widely cited "rule of ten" (originating with the 1985 Rawlins estimate and reaffirmed by subsequent FAERS gap analyses) suggests only ~1–10% of serious adverse drug reactions are ever reported. Disproportionality methods are explicitly designed to be robust to this — and to reporting-rate differences across drugs — by comparing relative, not absolute, reporting frequencies.

Duplicate detection, case triage, and data quality gates

Before any statistic is computed, raw ICSRs pass through data-quality pipelines:

• Duplicate detection: probabilistic matching (age, sex, event date, drug, reporter) flags likely duplicate submissions of the same case from multiple sources (patient + physician + manufacturer) — FDA estimates duplicate rates of 10–15% in raw FAERS extracts before deduplication • Case validity: reports missing a minimum data set (identifiable patient, identifiable reporter, suspect drug, adverse event) are excluded per ICH E2D minimum criteria • Follow-up merging: initial and follow-up reports for the same case are consolidated into a single case record with the most complete data • Seriousness classification: automatic flagging of death, hospitalization, life-threatening, disability, congenital anomaly, or "other medically important condition" drives expedited (15-day) vs. periodic reporting timelines

Only after this curation does a case enter the denominator/numerator pools used for disproportionality analysis — and even then, every downstream PRR/ROR/EBGM number inherits whatever residual noise, reporting stimulated by media attention ("notoriety bias"), and confounding by indication remain in the cleaned dataset.

The 2×2 Table — Every Disproportionality Statistic Reduces to Four Numbers

However sophisticated the downstream statistic, the entire computational engine of disproportionality analysis reduces to a single 2×2 contingency table built for each drug–event pair against the rest of the database. Getting the comparator group right — what counts as "b," "c," and "d" — matters as much as the arithmetic that follows.

• 4: Table cells per pair (a, b, c, d)
• ~10–50M: Typical FAERS drug–event pairs (sparse matrix, most cells near 0)
• a ≥ 3: Minimum case count convention (MHRA/Evans "rule of three")
• 2: Comparator scope options (whole database vs. matched class)

Defining cells a, b, c, and d

For a target drug D and target adverse event E, the database is partitioned into four counts:

Event E present Event E absent Drug D a b Other drugs c d

• a = reports of drug D with event E (the cell of interest) • b = reports of drug D with any other event • c = reports of any other drug with event E • d = reports of any other drug with any other event

All four counts are derived from the same underlying spontaneous report database — there is no external "true incidence" denominator. This is the defining feature (and limitation) of disproportionality analysis: it measures whether an event is reported unusually often for a drug relative to everything else reported for that drug and relative to everything else reported for that event, not whether the drug actually causes the event at an elevated rate in the population.

Comparator scope choices materially change results: • Whole-database comparator: c and d drawn from the entire spontaneous report corpus (default in FAERS/EudraVigilance public dashboards) • Class-restricted comparator: c and d drawn only from pharmacologically related drugs (e.g., comparing one SSRI against other SSRIs) — controls for confounding by indication and channeling bias, at the cost of smaller, noisier comparator cells • Time-restricted comparator: cases restricted to a fixed calendar window to control for reporting-stimulated trends (e.g., post-black-box-warning reporting surges)

Sparsity, the masking problem, and stratification

Real pharmacovigilance databases are extremely sparse: for a database with ~20,000 drugs and ~26,000 MedDRA PTs, the vast majority of the ~500 million possible drug–event cells have a=0. Two structural distortions follow directly from this sparsity:

• Masking (competition bias): if drug D is very frequently reported for event E1, cell "b" for D inflates, artificially deflating the PRR/ROR for a genuinely associated but less-reported event E2 on the same drug — because b, not just a, sits in the denominator • Cloaking: the inverse effect — an unrelated but very commonly co-reported event can make an unrelated drug appear disproportionately associated with it purely through database-wide reporting volume

Stratification mitigates both: computing the 2×2 table separately within age bands, sex, reporting year, or region, then pooling with a Mantel-Haenszel-type estimator, removes confounding that a single crude table would otherwise absorb into the PRR. EMA's GVP Module IX explicitly recommends stratified analysis by age and sex as a standard sensitivity check before a signal is escalated for clinical review.

Proportional Reporting Ratio and Reporting Odds Ratio — The Frequentist Core

PRR and ROR are the two workhorse disproportionality measures used by regulators worldwide. Both are simple ratio statistics computable in closed form from the 2×2 table, both have known asymptotic confidence intervals, and both were adopted operationally in the late 1990s and early 2000s precisely because they require no specialized software — a spreadsheet and the four cell counts suffice.

• 1998: PRR formula origin (Evans, Waller & Davis, MHRA)
• 2002: ROR formula origin (van Puijenbroek et al., Lareb)
• PRR≥2, χ²≥4, a≥3: Standard signal criteria (MHRA/CIOMS convention)
• log-normal Wald: 95% CI method (on log(PRR) or log(ROR))

PRR: proportion-of-reports comparison

The Proportional Reporting Ratio compares the proportion of a drug's reports that describe the event of interest to that same proportion among all other drugs in the database:

PRR = [a / (a+b)] / [c / (c+d)]

Interpretation: PRR=1 means the event is reported for drug D in the same proportion as for the rest of the database (no disproportionality). PRR=3.4 means the event is reported roughly 3.4× more often, as a share of that drug's total reports, than for everything else combined.

95% confidence interval (Evans et al. 1998), on the log scale:

ln(PRR) ± 1.96 × sqrt(1/a − 1/(a+b) + 1/c − 1/(c+d))

The chi-square statistic (with Yates' continuity correction, as originally specified by the MHRA) is computed on the same 2×2 table and used jointly with PRR as a signal-strength gate: a large PRR built on a=3 cases and a wide confidence interval is treated very differently from the same PRR built on a=300 cases.

MHRA/CIOMS Working Group VIII operational signal threshold: PRR ≥ 2, χ² ≥ 4, and a ≥ 3 cases — the "rule of three" ensures a minimum evidentiary base before any ratio, however extreme, triggers review.

ROR: odds-based comparison and its relationship to PRR

The Reporting Odds Ratio, more common in European practice (Netherlands Lareb, EudraVigilance), uses the odds form instead of the proportion form:

ROR = (a × d) / (b × c) = (a/c) / (b/d)

95% CI: exp[ln(ROR) ± 1.96 × sqrt(1/a + 1/b + 1/c + 1/d)]

For rare events (which almost all adverse-event PTs are, relative to total database size), b≈(a+b) and d≈(c+d), so PRR and ROR converge numerically — the classic rare-disease approximation linking odds ratios to relative risk. They diverge as the event becomes less rare or as cell counts shrink, which is why regulators typically report both alongside case counts rather than relying on either statistic alone.

Neither PRR nor ROR estimates absolute risk, incidence, or even relative risk in the epidemiological sense — both are purely relative reporting-frequency measures within a passive surveillance database with an unknown, non-random sampling process. A PRR of 5 does not mean a patient's risk is five times higher; it means the event is reported five times more disproportionately for that drug than for the reporting background.

Rofecoxib (Vioxx) is the textbook case validating — and cautioning against over-reading — disproportionality analysis: FAERS-style PRR signals for myocardial infarction began exceeding threshold years before the 2004 voluntary market withdrawal, but the same database generated thousands of PRR-positive pairs across the catalog that were never confirmed as causal, illustrating why disproportionality output is treated as hypothesis-generating, never as proof of causation.

MGPS and EBGM — Taming Small-Count Noise with Bayesian Shrinkage

Raw PRR and ROR are unstable exactly where signals matter most: rare events with tiny case counts produce wildly inflated ratios by chance alone. The Multi-item Gamma Poisson Shrinker (MGPS), developed by William DuMouchel at AT&T Bell Labs and adopted by the FDA in the Empirica Signal / FAERS pipeline, applies empirical Bayes shrinkage to pull unstable estimates toward the database-wide average in proportion to how little evidence supports them.

• 1999: MGPS publication (DuMouchel, American Statistician)
• EB05: FDA operating statistic (lower 5th %ile of EBGM posterior)
• FDA, Empirica Signal: Adopted by (used across FAERS surveillance)
• IC / IC025: WHO-UMC analogue (BCPNN, Information Component)

The shrinkage model: gamma-Poisson mixture

MGPS models the observed count a in each drug–event cell as Poisson-distributed around an expected count E (derived from the row and column marginals of the full table, i.e., what count would be expected under independence — the same "no association" baseline PRR=1 assumes):

Relative Reporting Ratio (RR) = a / E

Rather than trusting RR directly, MGPS fits a two-component gamma mixture prior across all drug–event pairs in the database simultaneously — most pairs cluster near RR≈1 (no association), a minority form a heavier-tailed component of genuinely elevated pairs. Bayes' theorem then combines each cell's prior (learned from the whole database's empirical distribution of RR values) with its own observed count to produce a posterior distribution for the "true" relative reporting ratio — the Empirical Bayes Geometric Mean (EBGM).

Key shrinkage behavior: • Small a (e.g., a=3): posterior pulled strongly toward the prior mean (≈1) — EBGM sits well below the raw PRR • Large a (e.g., a=300): posterior dominated by the data — EBGM converges toward the raw PRR/RR • EB05: the lower bound of the 90% credible interval (5th percentile) around EBGM — a conservative, count-aware signal statistic used operationally because it penalizes small-sample noise more explicitly than PRR's Wald confidence interval

FDA convention treats EB05 ≥ 2 as a common (though not universal) automated flag for further clinical triage — deliberately more conservative than the PRR≥2 rule alone, because two independent pairs with identical raw PRR=3.4 can have very different EB05 depending on case count.

The WHO-UMC alternative: BCPNN and the Information Component

The Uppsala Monitoring Centre, custodian of WHO VigiBase, uses a parallel Bayesian approach: the Bayesian Confidence Propagation Neural Network (BCPNN), developed by Bate, Lindquist, and Orre (1998). Its core statistic, the Information Component (IC), is a log2 measure of observed-vs-expected reporting:

IC = log2 [ P(drug, event) / (P(drug) × P(event)) ]

Equivalently, IC compares the observed joint reporting probability to what would be expected if drug and event were reported independently — conceptually the same disproportionality logic as PRR/ROR/EBGM, but expressed as a bit-valued information measure with its own Bayesian shrinkage built into the prior on P(drug,event).

VigiBase's operational threshold, analogous to EB05, is IC025 > 0 — the lower bound of the 95% credible interval on IC exceeding zero indicates the observed co-reporting exceeds chance expectation with statistical confidence. Both EBGM/EB05 (FDA/Empirica ecosystem) and IC/IC025 (WHO-UMC/VigiBase ecosystem) converge on the same underlying idea — shrink noisy small-count ratios toward the population prior — arrived at independently by two regulatory statistics groups in the same two-year window (1998–1999).

DuMouchel's original 1999 MGPS validation against FDA's historical safety-withdrawal cases showed that EBGM-based ranking surfaced known signal drug–event pairs (e.g., terfenadine–torsade de pointes) years earlier in the simulated reporting stream than a fixed-count threshold rule would have, while suppressing a substantial fraction of the small-count false-positive pairs that plague raw PRR screening — the core justification for shrinkage becoming the FDA's default automated screening layer.

From Statistical Flag to Label Change — Human Review, Causality, and Regulatory Consequence

A statistical signal is a hypothesis, not a conclusion. Every PRR/ROR/EBGM/IC flag that clears its threshold enters a structured medical review process — combining WHO-UMC causality assessment, biological plausibility, literature triangulation, and cross-database replication — before it can translate into a regulatory action that changes what is printed on a drug label or communicated to prescribers.

• hundreds–low thousands: Statistical flags reviewed/quarter (per major regulator)
• ~5–15%: Confirmed → label change rate (of triaged statistical signals)
• 6: WHO-UMC causality categories (certain → unclassifiable/unassessable)
• monthly: EMA PRAC signal cycle (EPITT tracking, GVP Module IX)

WHO-UMC causality assessment and the medical review layer

A statistically disproportionate drug–event pair moves to a pharmacovigilance physician/pharmacist review team, which applies structured causality assessment — most widely the WHO-UMC system, which classifies each individual case (not the aggregate signal) into one of six categories: Certain, Probable/Likely, Possible, Unlikely, Conditional/Unclassified, and Unassessable/Unclassifiable, based on time relationship, dechallenge/rechallenge response, alternative explanations, and known pharmacology.

Parallel algorithmic tools (Naranjo Adverse Drug Reaction Probability Scale, a 10-question weighted checklist yielding a Definite/Probable/Possible/Doubtful score) are used for individual case adjudication in tandem with the aggregate statistical signal.

Signal validation checklist typically includes: • Case series review: reading the underlying narrative for a sample of contributing ICSRs — is there a plausible temporal relationship, positive dechallenge, positive rechallenge? • Biological plausibility: does the drug's pharmacology (receptor binding, metabolism, known drug-class effects) support a mechanism? • Replication: does the signal reproduce independently in FAERS, EudraVigilance, and VigiBase, or is it an artifact of one database's reporting pattern (e.g., a single litigation-driven reporting cluster)? • Confounding by indication: is the "event" actually a manifestation of the disease being treated, not the drug (e.g., a signal for a cardiac event in a drug used for heart failure)? • Stratified re-analysis: does the PRR/EBGM persist after age/sex/region stratification?

Regulatory consequences and the governance cycle

Confirmed signals feed into formal regulatory workflows governed by ICH E2C(R2) (Periodic Benefit-Risk Evaluation Report, PBRER), EMA GVP Module IX (Signal Management), and FDA's Sentinel Initiative for active-surveillance confirmation:

• EMA: signals are logged in EPITT (European Pharmacovigilance Issues Tracking Tool) and assessed monthly by PRAC (Pharmacovigilance Risk Assessment Committee); confirmed signals can trigger a Direct Healthcare Professional Communication (DHPC), an Article 31/107i referral, or a Summary of Product Characteristics (SmPC) update • FDA: signals move through the Office of Surveillance and Epidemiology; confirmed associations can prompt a label change, a boxed warning, a Risk Evaluation and Mitigation Strategy (REMS), or in rare cases a market withdrawal; FDA increasingly requires Sentinel System active-surveillance claims-data replication (structured query language across >100M covered lives) before major label actions, since Sentinel can estimate actual incidence rates that FAERS structurally cannot • Periodic reporting: every marketing authorization holder must submit PSURs/PBRERs on a defined cycle, cumulatively re-running disproportionality analysis against the growing database and formally documenting signal status (new, ongoing, closed, refuted)

The full loop — spontaneous report, 2×2 table, PRR/ROR, EBGM/IC shrinkage, causality review, regulatory action — typically spans months to a few years for a genuine signal, compressed to weeks under expedited safety review for life-threatening reactions.

Varenicline (Chantix) illustrates the full cycle in both directions: an initial 2008 FAERS disproportionality signal for neuropsychiatric adverse events (depression, suicidal ideation) drove a 2009 FDA boxed warning; a decade later, the large randomized EAGLES trial (2016) found no significant excess neuropsychiatric risk versus placebo/nicotine replacement, and FDA removed the boxed warning in 2016 — a rare, well-documented case of a spontaneous-report signal being formally reassessed and downgraded once controlled trial data became available, underscoring that disproportionality output always remains provisional pending confirmatory evidence.