⚠️ Fact-Checking Layer for Clinical LLM Output Simulator

The Primary LLM Generates Its Clinical Output First — Unchecked

Generation happens before any safety pass runs.

• 1–3 s: Typical clinical query latency (raw generation only)
• ~5–15%: Unverified hallucination rate (reported across clinical LLM studies)
• Free text: Output form (no structured claim boundaries yet)
• None: Safety check applied so far (fact-check layer runs after this)

Why post-hoc checking starts after the fact

Generation is left unconstrained here. Checking is deliberately deferred to a separate pass.

Extracting Discrete, Checkable Claims From Free-Text Output

The output is decomposed into atomic factual statements.

• 3–15: Claims per clinical response (typical extracted count)
• LLM-based decomposition: Extraction method (splits text into atomic statements)
• Excluded: Non-factual spans (opinions, hedges skipped)
• ~200–500 ms: Extraction latency added (per response)

Turning prose into a checklist of claims

Each sentence is split into single, checkable assertions. Vague or subjective phrasing is dropped.

Verifying Each Claim Against a Trusted Reference Source

Every claim is checked, not just skimmed.

• 3: Reference source types (drug DB, guideline, literature)
• Separate from generator: Verifier model (independent judge pass)
• ~50–150 ms: Per-claim check time (scales with strictness)
• 70–90%: Retrieval hit rate (claims with a matching source)

One claim, one lookup, one verdict

Each claim is queried against a trusted source independently. No source match leaves a claim unverified.

Scoring Claims as Verified, Unverified, or Contradicted

Strictness controls how aggressively claims get flagged.

• 3: Score classes (verified, unverified, contradicted)
• Higher: Strict-mode flag rate (more claims sent for review)
• Lower: Lenient-mode flag rate (only clear contradictions flagged)
• Highest priority: Contradiction severity (always blocks release)

Strictness is a dial, not a switch

Stricter settings flag more borderline claims. Contradicted claims are never released as-is.

Releasing a Filtered, Safety-Checked Output to the Clinician

Only verified claims pass through untouched.

• Released as-is: Verified claims (pass through unchanged)
• Removed or annotated: Flagged claims (never silently released)
• ~300–900 ms: Added end-to-end latency (extraction + verification combined)
• Safety over speed: Net effect (checkpoint trades latency for trust)

A checkpoint, not a rewrite

The layer subtracts risk rather than generating new text. Latency is the price of the safety margin.

Post-hoc checking catches errors after generation, trading added latency for a safety checkpoint the clinician can trust.