Mining missed-diagnosis malpractice claims to surface recurring specialty and condition clusters behind diagnostic error
Every closed malpractice claim is a forensic record of a diagnostic process that failed a real patient. Aggregating tens of thousands of these records — de-identified and pooled across insurers, hospital systems, and the National Practitioner Data Bank — turns isolated tragedies into a dataset large enough to reveal statistical patterns invisible in any single case review.
Diagnostic error is notoriously hard to measure directly — it rarely triggers an incident report the way a wrong-site surgery or medication overdose does, because the harm often only becomes visible months or years later, once the correct diagnosis is finally made elsewhere. Malpractice claims fill this gap: each one comes bundled with an independent expert-reviewer determination of whether the standard of care was breached, a structured allegation code, and a documented outcome (severity, indemnity paid, litigation result).
The 2015 Institute of Medicine report "Improving Diagnosis in Health Care" identified diagnostic error as an underappreciated blind spot in patient safety, estimating that most people will experience at least one diagnostic error in their lifetime, sometimes with devastating consequences. Malpractice claim databases — closed-claim registries maintained by liability insurers such as CRICO Strategies (the Harvard-affiliated malpractice insurer) — became one of the primary empirical sources cited by the IOM committee precisely because they are large, standardized, and outcome-linked.
This is not a perfect proxy: claims under-represent errors that caused harm but were never litigated, and over-represent cases with severe, visible outcomes (claims require identifiable damages). But at scale, the patterns that repeat across thousands of independently reviewed claims are far too consistent to be litigation noise — they are the fingerprints of systemic diagnostic failure points.
A widely cited CRICO Strategies "Malpractice Risks in the Diagnostic Process" study of over 55,000 diagnosis-related claims found diagnostic error was the single leading allegation category in malpractice litigation nationally — ahead of surgical and medication errors combined, in several claim-year cohorts (illustrative figures representative of the published literature).
Before any analysis, each intake record is a heterogeneous bundle of structured and unstructured data:
• Structured fields: treating specialty, care setting (ED, outpatient, inpatient), claim status, indemnity paid, defense cost, date of alleged error, date of claim filing • Coded fields: ICD-10 diagnosis codes (both the missed/delayed diagnosis and any eventual correct diagnosis), procedure codes for relevant tests ordered or not ordered • Unstructured narrative: expert panel review summaries, plaintiff allegation text, defense rebuttal, chart-review notes describing the sequence of clinical decisions • Severity scoring: National Association of Insurance Commissioners (NAIC) severity scale, typically 1 (emotional only) to 9 (death), used to weight the clinical significance of each miss
At the intake stage, these 12,400 pooled records exist only as unlabeled points — no specialty tag, no condition category, no cluster membership. The analytic value only emerges once each claim is systematically coded against a shared taxonomy, which is the work of Stage 2.
Because claim narratives contain sensitive clinical and legal detail, pattern-analysis pipelines built on this data operate under strict de-identification and aggregation rules before any cross-institution pooling: patient identifiers, provider names, and institution names are stripped or tokenized, and results are reported only at the level of specialty/condition/setting aggregates with minimum cell-size suppression (typically no published cluster below n=10) to prevent re-identification of any single claim or clinician.
This is the same governance model used by patient-safety organizations (PSOs) under the Patient Safety and Quality Improvement Act, which allows hospitals to share adverse-event and near-miss data for aggregate learning without the shared data being discoverable in litigation against the contributing institution — a legal protection specifically designed to encourage this kind of systemic pattern analysis rather than case-by-case blame.
A closed claim file is mostly free text: dense narrative summaries written by clinical reviewers and attorneys. To find patterns across thousands of claims, each record must first be mapped onto a shared, structured taxonomy of treating specialty and missed/delayed condition — a task that combines rule-based ICD-10 mapping with NLP text classification over the narrative fields.
Two orthogonal taxonomies are applied to every claim:
Specialty axis — which clinical service was responsible for the diagnostic process at the point of alleged failure: emergency medicine, primary/internal medicine, radiology, pathology, general surgery, obstetrics, and roughly a dozen other categories. A single claim can touch multiple specialties (e.g., a mass missed on a radiology read that a primary-care physician also failed to follow up on) and is tagged with a primary and contributing specialty.
Condition axis — the macro-category of the missed or delayed diagnosis: oncology (cancer of any type), cardiovascular (myocardial infarction, aortic dissection, pulmonary embolism), infection (sepsis, meningitis, necrotizing fasciitis), neurological (stroke, spinal cord compression), and a residual "other" bucket. Within each macro-category, 40+ micro-condition tags capture the specific diagnosis (e.g., "breast cancer," "lung cancer," "colorectal cancer" all roll up under oncology).
This two-axis tagging is exactly what allows the same underlying failure mode — for instance, "abnormal finding acknowledged but not acted upon" — to be recognized whether it happened in a radiology report about a lung nodule or a lab report about a rising white count.
The tagging pipeline layers three extraction methods:
1. Rule-based ICD-10 mapping: where a structured diagnosis code exists (the eventual correct diagnosis, once made), it is deterministically mapped to a macro-condition category via a maintained crosswalk table.
2. Named-entity recognition over narrative text: a clinical NLP model (trained on de-identified claim narratives and validated against manually double-coded samples) extracts candidate diagnosis mentions, anatomic sites, and test/procedure references from free-text reviewer summaries — necessary because many narratives describe the missed condition in prose before any code was ever assigned in the medical record.
3. Specialty attribution logic: cross-references the narrative's described care setting and provider role language ("the ED physician discharged the patient," "the radiologist's report stated") against the specialty taxonomy, resolving ambiguous or multi-specialty claims using the sequence of the described care episodes.
Human expert reviewers audit a stratified sample (typically 10%) of tagged claims to measure precision against manual coding; pipelines below roughly 85% agreement on the specialty or condition axis are flagged for taxonomy or model refinement before results are used downstream.
Once tagged, this cohort's average indemnity payout for diagnosis-related claims ($362K, illustrative) already exceeds the reported average for malpractice claims overall — consistent with published findings that diagnostic-error claims tend to involve more severe patient harm (permanent disability or death) than claims arising from other error types.
With every claim now tagged along specialty and condition axes, unsupervised clustering (k-means over engineered categorical embeddings, or density-based HDBSCAN for irregular cluster shapes) groups claims that share not just a diagnosis category but a similar breakdown mechanism — the same point in the diagnostic process where the process broke down.
Clustering does not simply group claims by condition label (that grouping already exists from Stage 2) — it groups them by the underlying process failure, using features such as: was the finding present but misinterpreted, was it present but not communicated, was it never tested for, was follow-up recommended but not completed, and how long the delay was between the missable moment and eventual correct diagnosis.
This distinction matters clinically. Two "missed lung cancer" claims can belong to entirely different clusters: one where a nodule was correctly identified on imaging but the report's recommendation for follow-up CT was never communicated to the ordering physician (a communication-breakdown cluster), and another where the nodule was present on the film but never identified by the interpreting radiologist at all (a perceptual-error cluster). Grouping by process mechanism rather than just diagnosis label is what allows the analysis to point toward a specific, fixable intervention rather than a vague "be more careful" recommendation.
Cluster validity is checked with silhouette scores and manual review of a sample from each cluster by clinical patient-safety reviewers, confirming that claims grouped together genuinely share a recognizable failure narrative.
Across multiple independent malpractice claims analyses (CRICO Strategies, and research led by David Newman-Toker's group at Johns Hopkins using both closed-claims and national epidemiological modeling), three condition categories consistently dominate the highest-severity, highest-frequency missed-diagnosis clusters — informally referred to in the diagnostic-safety literature as the "Big Three": cancer, cardiovascular events (particularly myocardial infarction, aortic dissection, and venous thromboembolism), and infections (particularly sepsis and meningitis).
In this cohort's clustering output, oncology claims form the single largest cluster at 29% of the labeled dataset, driven predominantly by breast, lung, and colorectal cancer sub-clusters where an abnormal finding (mammogram density, chest nodule, positive fecal occult blood test) was present in the record but did not translate into timely diagnostic follow-through. Cardiovascular claims (23%) cluster heavily around emergency-department chest-pain presentations. Infection claims (19%) cluster around sepsis recognition delays in both ED and inpatient settings.
Newman-Toker and colleagues' 2023 analysis estimated that just fifteen diseases — dominated by vascular events, infections, and cancers — account for roughly half of all serious harms from diagnostic error nationally, a striking concentration that mirrors exactly the cluster dominance seen in claims-based pattern analysis like this one.
Because clustering is sensitive to which specialties and settings are included in the analysis, the "specialty filter breadth" control determines how many condition clusters remain visible: narrowing the filter to a single specialty (e.g., emergency medicine only) collapses the multi-specialty oncology follow-up-failure clusters (which typically span radiology, primary care, and specialist referral) while sharpening acute-setting clusters like ED chest pain and sepsis triage — because those failure modes are contained within a single care episode and specialty.
This is analytically important: a hospital system reviewing only its emergency department's claims will see a very different top pattern (cardiovascular and infection-dominant) than a system reviewing its full ambulatory and diagnostic-imaging chain (oncology follow-up-dominant) — both are real, but they point to different intervention targets.
A cluster is a statistical grouping; a root pattern is the specific, actionable clinical narrative that explains why it recurs. This stage drills into the dominant clusters from Stage 3 and extracts the recurring causal chain — the exact point in the diagnostic process where independent clinicians, across independent patients and independent institutions, made the same kind of misstep.
The single most common root pattern within the largest (oncology) cluster is not a perceptual failure — the abnormal finding was correctly identified by a radiologist, pathologist, or lab — but a communication and follow-up failure: the report recommending further workup was filed, but no reliable system existed to ensure the ordering clinician read it, acted on it, and that the patient returned for the recommended follow-up imaging or biopsy.
This "closed-loop communication" failure is one of the best-documented root causes in the diagnostic-error literature. It typically involves multiple simultaneous small breakdowns rather than one dramatic error: a report routed to the wrong inbox, a patient who did not receive a callback, a scheduling system with no tracking mechanism for incidental-finding follow-up, and no safety-net reminder at 3 or 6 months when the recommended repeat imaging never happened.
The Joint Commission has repeatedly flagged failure to communicate critical and non-critical significant test results as a National Patient Safety Goal area precisely because closed-claim analyses like this one keep surfacing it as the dominant recurring root pattern behind missed-cancer litigation.
Within the cardiovascular cluster, the dominant recurring pattern is anchoring: atypical-presentation myocardial infarction (particularly in women, diabetics, and older adults, who more often present without classic crushing substernal chest pain) is misattributed to gastrointestinal reflux, musculoskeletal strain, or anxiety — and the patient is discharged from the emergency department without further cardiac workup.
A structurally similar pattern recurs for aortic dissection (misattributed to musculoskeletal back pain) and pulmonary embolism (misattributed to anxiety or costochondritis). In all three, the shared root mechanism is the same cognitive bias — premature closure on the first plausible, benign explanation, without a structured decision rule (e.g., HEART score for chest pain, Wells score for PE, or a standardized aortic-dissection risk checklist) forcing consideration of the higher-acuity alternative.
Within the infection cluster, the recurring pattern is not failure to measure vital signs — the data (rising heart rate, falling blood pressure, elevated lactate, new confusion) was almost always present and charted — but failure to escalate: nursing or covering-physician recognition of an early warning score trend did not reliably trigger a rapid-response evaluation or sepsis-bundle activation before the patient deteriorated into septic shock.
This is functionally identical to the oncology closed-loop pattern in structure, even though the clinical content is completely different: information existed in the system, but no reliable process guaranteed it reached a decision-maker with authority to escalate care in time. Naming this shared structural root — "detection without escalation" — across three clinically unrelated condition clusters is exactly the kind of insight that only becomes visible once claims are pooled and clustered at scale rather than reviewed one at a time.
The entire purpose of malpractice pattern analysis is not academic — it is to translate recurring root patterns into specific, testable system-level interventions, then track whether those interventions measurably reduce the recurrence of the same failure mode in future claims. This final stage converts each identified root pattern into a targeted prevention recommendation.
Directly targeting the dominant oncology root pattern, health systems implement automated test-result tracking software that flags any report containing a follow-up recommendation, assigns it a tracking status, and escalates to a designated safety-net coordinator if the recommended follow-up (repeat imaging, biopsy, specialist referral) has not occurred within the clinically appropriate window.
This converts an informal, person-dependent process (trusting that the ordering clinician remembers and the patient returns) into a system-level guarantee with an audit trail — directly closing the specific gap identified in Stage 4's Pattern 1.
Directly targeting the cardiovascular anchoring pattern, emergency departments adopt structured, validated risk-stratification tools — the HEART score for chest pain, Wells or PERC criteria for pulmonary embolism, and standardized aortic-dissection detection risk checklists — that force explicit consideration and documentation of the dangerous alternative diagnosis before a benign one is accepted, rather than relying on unstructured clinical gestalt alone.
These tools do not replace clinical judgment; they structure it, ensuring the "what am I missing" step happens reliably rather than only when a clinician happens to think of it.
Directly targeting the infection detection-without-escalation pattern, inpatient and ED units implement automated early-warning scoring (NEWS2, qSOFA, or institution-specific composite scores) wired directly into the electronic health record so that a qualifying trend automatically pages a rapid-response team or triggers a sepsis-bundle order set — removing the dependency on an individual clinician noticing and choosing to escalate.
Across all three interventions, the shared design principle is the same one exposed by the clustering analysis itself: replace single-point-of-failure human vigilance with a system-level, auditable, closed-loop process. Each intervention is then itself tracked against future claims data — the same pattern-analysis pipeline run again in subsequent years should show a measurable decline in the specific cluster it targeted, providing an outcome metric for whether the intervention actually worked.
This is the complete loop the diagnostic-error-reduction literature calls for: individual tragic cases feed a pooled claims database, clustering surfaces recurring systemic patterns invisible at the single-case level, root-cause analysis names the specific mechanism, and targeted system interventions are deployed and then re-measured against the next cohort of claims — turning malpractice litigation data from a purely retrospective, punitive record into a forward-looking patient-safety instrument.