Assembling a research-grade patient cohort from federated electronic health records — code-based phenotyping, washout logic, and covariate balancing against an OMOP Common Data Model
Real-world cohort research begins with a data engineering problem long before it becomes a clinical one. Patient records live scattered across incompatible Epic, Cerner, and Allscripts instances, regional health information exchanges, and payer claims warehouses — each with its own schema, code sets, and update cadence. The OHDSI OMOP Common Data Model has become the de facto standard for harmonizing this chaos into a single, queryable, vocabulary-normalized structure.
Every health system generates data in its own dialect. Epic stores clinical facts in Clarity/Caboodle reporting schemas; Cerner exposes HealtheIntent extracts; smaller practices run on Allscripts, athenahealth, or eClinicalWorks with wildly inconsistent field naming. The ETL (extract-transform-load) layer is where cohort-building projects live or die.
OMOP CDM (maintained by OHDSI, the Observational Health Data Sciences and Informatics collaborative) imposes a fixed relational structure: PERSON, CONDITION_OCCURRENCE, DRUG_EXPOSURE, MEASUREMENT, VISIT_OCCURRENCE, and OBSERVATION_PERIOD tables, each row tagged with a standard concept_id drawn from the OHDSI Standardized Vocabularies (itself a merge of SNOMED CT, RxNorm, LOINC, ICD-10-CM as source-to-standard mappings).
Mapping work typically consumes 40-60% of total cohort-study timeline: • Source vocabulary reconciliation — local lab codes mapped to LOINC via string-matching plus manual adjudication • Concept set curation in ATLAS (the OHDSI cohort-definition and vocabulary browser) — building reusable value sets, e.g. "Type 2 diabetes mellitus" resolving to ~180 descendant SNOMED concepts • Domain assignment — deciding whether a fact belongs in CONDITION_OCCURRENCE vs. OBSERVATION • ETL-CDM conformance testing with Achilles / DataQualityDashboard, which flag implausible value distributions (negative ages, drug exposures before birth)
HL7 FHIR R4 resources (Condition, MedicationRequest, Observation) are increasingly used as the intermediate exchange layer, with the USCDI (United States Core Data for Interoperability) mandating a minimum data class set that certified EHRs must expose via API under the 21st Century Cures Act information-blocking rule.
Two architectural philosophies dominate multi-site real-world data research:
Centralized pooling: patient-level data physically copied into one warehouse (e.g., N3C, the National COVID Cohort Collaborative, aggregated de-identified records from >75 clinical sites into a single enclave). Simplifies analysis but requires heavyweight data-use agreements and creates a single point of privacy exposure.
Federated / distributed query networks: analytic code is shipped to each site; only aggregate statistics return. FDA Sentinel Initiative — built to conduct active safety surveillance across >100 million covered lives — never centralizes patient-level data; partners run a common parameterized SAS/R program locally and return summary tables. PCORnet operates similarly across its Clinical Data Research Networks, spanning some 66 million patients.
TriNetX and Komodo Health occupy a hybrid middle ground: site-hosted CDM instances behind a federated query layer, with real-time cohort-count feasibility responses (<60 seconds) but analyst-initiated data pulls requiring separate governance approval.
The FDA Sentinel System, launched in response to the 2007 FDAAA mandate for active drug-safety surveillance, now queries a federated network exceeding 100 million patients without ever centralizing a single medical record — each participating data partner retains physical and legal custody of its own data, running standardized analytic routines locally and returning only aggregate risk estimates.
A cohort definition is the formal, machine-executable encoding of a clinical research question: which patients, over what time window, meeting which diagnosis, medication, procedure, and lab-value criteria, qualify for the study. Ambiguity in this step propagates directly into bias — a criterion written too loosely inflates false positives, one written too tightly starves statistical power.
A cohort entry criterion in ATLAS or a comparable rules engine is built from nested logical blocks:
Primary event: a qualifying CONDITION_OCCURRENCE (e.g. any concept in the "Heart failure" descendant set), constrained to the first occurrence per person, anchored to an index_date.
Inclusion rules (all patients must satisfy): • Continuous observation — ≥365 days of OBSERVATION_PERIOD coverage prior to index (ensures a real look-back window, not a data artifact of a patient simply not appearing in the source yet) • Age at index between protocol-specified bounds • At least 2 qualifying diagnosis codes on separate encounter dates (a "2-code rule" — single ICD-10 codes on outpatient claims have documented positive predictive values as low as 40-60% for chronic conditions, while requiring two occurrences ≥30 days apart raises PPV substantially)
Exclusion rules (any match disqualifies): • Competing/confounding diagnosis in the same domain (e.g., excluding type 1 diabetes when defining a type 2 diabetes cohort) • Prior exposure to the index drug/procedure during the washout window (ensures a genuinely treatment-naive new-user cohort — the "new-user design" of Ray, 2003) • Pregnancy, hospice enrollment, or other protocol-specified contraindications
Each rule block independently reduces the candidate pool. A typical chronic-disease cohort definition applied to a 5-million-life claims database might retain only 8-25% of the initially screened population once every gate is applied — the funnel visualized in this stage.
Diagnosis and procedure code sets are not static. ICD-10-CM receives annual updates (effective October 1 in the U.S. fiscal year cycle); SNOMED CT International publishes biannual releases; RxNorm updates monthly as new drug products are approved. A cohort definition frozen against a 2019 vocabulary snapshot will silently miss patients coded under 2024 additions unless the concept set is actively re-curated.
Best practice, formalized in OHDSI's phenotype library and echoed in ISPE/ISPOR good-practice guidance for real-world evidence studies, requires: • Version-pinning the vocabulary release used for a given study, documented for reproducibility • Descendant-concept expansion performed at run time against the pinned vocabulary, not hard-coded as a static code list • A documented rationale for every included/excluded code, peer-reviewed before execution — the PheKB (Phenotype KnowledgeBase) repository hosts hundreds of community-validated, versioned phenotype algorithms for exactly this purpose
Misspecification here is the single largest source of what pharmacoepidemiologists term outcome misclassification bias, and it degrades statistical power non-linearly: a phenotype with 70% sensitivity and 95% specificity applied to a rare (1% prevalence) outcome can inflate the apparent event rate by more than 4-fold from false positives alone.
Structured billing codes were designed for reimbursement, not research, and systematically under-capture clinical reality: a symptom mentioned in a progress note, a family history noted in an intake form, or a medication documented only in a discharge summary rarely generates a discrete, queryable code. Natural language processing over the free-text note corpus recovers this signal, typically lifting phenotype sensitivity by 15-30 percentage points over code-only algorithms.
Clinical NLP tools — Apache cTAKES, MetaMap, and increasingly transformer-based clinical language models fine-tuned on de-identified note corpora — perform a multi-stage extraction:
1. Sentence and section segmentation: notes are split into semantically meaningful units (History of Present Illness, Assessment/Plan, Family History), since a diagnosis mentioned in "Family History" must not count as a patient-level finding
2. Named entity recognition: candidate clinical concepts (symptoms, diagnoses, medications, anatomical sites) are tagged and mapped to UMLS Concept Unique Identifiers, then cross-walked to SNOMED CT
3. Negation and assertion detection: the NegEx algorithm and its context-aware successor ConText classify each mention as affirmed, negated ("denies chest pain"), hypothetical ("if fever develops"), or historical — a critical step, since 30-50% of raw entity mentions in typical clinical notes are negated or non-current and would otherwise generate massive false-positive rates
4. Temporality resolution: distinguishing past medical history from a diagnosis newly established at the current encounter, using narrative-time and document-time anchors
5. Aggregation logic: a patient-level phenotype flag typically requires ≥2 independent affirmed mentions (structured OR unstructured) across separate encounters, mirroring the "2-code rule" used for structured-only algorithms
Modern deployments increasingly layer a fine-tuned clinical BERT-family model (ClinicalBERT, GatorTron-style architectures) on top of rule-based negation for higher recall on complex syntactic constructions rule-based systems miss.
No computable phenotype is trusted for a regulatory-grade or publication-grade cohort without empirical validation against manual chart review by a clinician abstractor — the accepted gold standard.
Standard validation protocol: • Draw a stratified random sample (typically 100-200 charts) from the algorithm-flagged cohort, plus a comparison sample of algorithm-negative charts • Blinded clinician reviewers independently adjudicate true case status against the full chart • Compute sensitivity, specificity, positive predictive value (PPV), and negative predictive value against reviewer determination • Report inter-rater reliability (Cohen's kappa) between independent reviewers, with κ >0.8 considered strong agreement
Published phenotype algorithms in the PheKB library commonly report PPVs in the 85-97% range for well-characterized conditions (type 2 diabetes, myocardial infarction) but can fall below 70% for phenotypes with vague or highly heterogeneous presentations (e.g., early-stage cognitive impairment, certain autoimmune conditions).
Regulatory context: FDA's 2018 Framework for Real-World Evidence and the accompanying guidance on using EHR/claims data for regulatory decisions explicitly require sponsors to report validated phenotype performance characteristics (sensitivity/PPV with confidence intervals) as a precondition for using an RWD-derived cohort in support of an efficacy or safety claim.
Even a perfectly specified phenotype produces a biased cohort if temporal anchoring is sloppy. Assigning the wrong index date, omitting a washout period, or comparing an exposed group to a systematically different comparator introduces confounding that no amount of downstream statistical modeling can fully repair. This stage is where epidemiologic design discipline — new-user design, active comparator selection, propensity-score matching — is operationalized.
The single most replicated design flaw in observational drug-effect studies is prevalent-user bias: comparing patients newly starting a drug to patients who have already survived months or years on it (and are therefore enriched for tolerance and treatment response) systematically underestimates early-onset adverse events. The new-user design (Ray, 2003) forces every cohort member to be drug-naive at index.
Operational implementation: • Washout window: typically 180-365 days of continuous enrollment/observation immediately preceding index, during which the patient must show zero exposure to the index drug (or zero diagnosis, for a disease-incidence cohort) • Index date: the date of first qualifying exposure (drug cohort) or first qualifying diagnosis (disease cohort) — every downstream covariate is measured relative to this single anchor, never to calendar time • Baseline covariate window: typically the 6-12 months immediately preceding index, used to characterize comorbidity burden, concurrent medications, and prior healthcare utilization intensity (a strong proxy for unmeasured frailty) • Outcome window: begins at index (or after a protocol-specified induction period) and extends to the earliest of outcome occurrence, disenrollment, death, or administrative censoring
Longer washout windows reduce misclassification of prevalent users but shrink the eligible cohort — every increment of look-back demands a matching stretch of prior continuous data availability, which most claims and EHR sources cannot guarantee for all patients equally (a phenomenon sometimes called the "healthy-database-user" truncation effect).
Once exposed and candidate-comparator pools are defined, a propensity score — the predicted probability of receiving the exposure, conditional on measured baseline covariates — is estimated via logistic regression or gradient-boosted trees, then used to construct a balanced comparison:
1:1 nearest-neighbor matching (with caliper, commonly 0.2 × the standard deviation of the logit-propensity score) is the most widely reported approach in pharmacoepidemiology, though full matching, stratification, and inverse-probability-of-treatment weighting (IPTW) are increasingly common where retaining full sample size matters.
Covariates typically entering the propensity model: • Demographics — age, sex, region, insurance type • Comorbidity indices — Charlson Comorbidity Index (17 weighted conditions) or the more granular Elixhauser Comorbidity Index (31 categories) • Prior healthcare utilization — hospitalization count, ED visits, distinct medication count in baseline window (a strong frailty proxy) • Concurrent medications and relevant lab values at baseline
Balance is assessed post-match via the standardized mean difference (SMD) for every covariate; an SMD below 0.10 is the conventional threshold for "well balanced." Negative control outcomes (health events with no plausible causal link to the exposure) are frequently run alongside the primary analysis as an empirical calibration check for residual confounding — a technique formalized and popularized by the OHDSI methods community and used routinely in FDA Sentinel active-surveillance protocols.
A widely cited empirical calibration study (Schuemie et al., 2014-2018, across the OHDSI network) tested hundreds of drug-outcome pairs with no expected causal relationship and found that naive observational estimates falsely flagged a statistically significant association roughly 40-50% of the time at nominal p<0.05 — motivating routine use of empirical calibration and negative/positive control sets as a standard cohort-validation step, not an optional add-on.
A locked cohort is not yet a usable dataset. Its phenotype performance must be independently confirmed, patient privacy must be provably protected under HIPAA and, for multinational studies, GDPR, and the final extract must conform to the data standards regulators and downstream analysts actually expect — CDISC SDTM domains for FDA submissions, or an OMOP CDM export for OHDSI-network reproducibility.
The HIPAA Privacy Rule permits use of patient data for research under two distinct de-identification standards, and cohort-export pipelines must commit explicitly to one:
Safe Harbor (§164.514(b)(2)): mechanically strips 18 enumerated identifier categories — names, all geographic subdivisions smaller than a state, all elements of dates (except year) directly related to an individual including birth date and admission/discharge dates, telephone/fax numbers, email, SSN, medical record numbers, and more — plus ages over 89 collapsed into a single "90+" category. Mechanical and auditable, but destructive: precise dates are essential for washout-window and index-date logic, so cohort pipelines typically retain dates internally under a data-use agreement and only strip them from the final shareable extract.
Expert Determination (§164.514(b)(1)): a qualified statistician applies formal disclosure-risk methodology (e.g., k-anonymity, HIPAA Expert Determination re-identification risk modeling) to certify that the risk of re-identification is "very small" — commonly operationalized as below a 0.04% probability threshold — even when some quasi-identifying detail (partial dates, finer geography) is retained. This path is far more common in cohort research, since analysis frequently requires the very date-level granularity Safe Harbor removes.
For multinational real-world evidence programs, GDPR pseudonymization (Article 4(5)) imposes an additional, non-equivalent standard: re-identification keys must be held separately and access-controlled, and data subjects retain rights (access, erasure) that HIPAA de-identified data does not carry at all.
The final cohort deliverable depends entirely on its downstream consumer:
For FDA/EMA regulatory submissions (e.g., an external control arm derived from real-world data supporting a single-arm oncology trial, or a post-marketing safety study under a PMR/PMC commitment): the cohort is mapped into CDISC SDTM (Study Data Tabulation Model) domains — DM (demographics), AE (adverse events), CM (concomitant medications), LB (laboratory), VS (vital signs) — each governed by the CDISC Controlled Terminology and validated against FDA's Study Data Technical Conformance Guide before submission through the FDA electronic gateway. Adverse-event-adjacent findings destined for pharmacovigilance systems additionally map to ICH E2B(R3) structured case elements for entry into FAERS (FDA) or EudraVigilance (EMA), coded in MedDRA.
For network-reproducible epidemiology (OHDSI-style multi-site studies, PCORnet distributed queries): the cohort is exported as an OMOP CDM cohort table (subject_id, cohort_start_date, cohort_end_date) referencing the same standardized vocabulary used at ingestion, allowing the identical analytic package to execute unmodified across any conformant network site — the entire premise of OHDSI's "study-once, replicate-everywhere" methodology.
Either path concludes with a locked, versioned data dictionary and an audit trail linking every inclusion/exclusion decision back to its source rule — required under ICH E6(R2) Good Clinical Practice data-integrity expectations whenever the cohort supports a regulatory claim, and increasingly expected by peer reviewers even for purely academic real-world evidence publications.
The National COVID Cohort Collaborative (N3C) assembled a harmonized OMOP-format research cohort from more than 20 million patients across 75+ U.S. clinical sites in under a year — validating phenotype algorithms against site-level chart review, applying Expert Determination de-identification inside a secure enclave, and enabling over 3,000 authorized external researchers to run analyses without ever exporting patient-level data off the platform, a template now widely cited as a model for rapid-response real-world cohort infrastructure.