This is the Fellegi–Sunter model of probabilistic record linkage — the mathematics behind how governments and hospitals deduplicate citizen records that live in separate databases with no shared key.
Every candidate pair (one record from Agency A, one from Agency B) is compared field by field. For each field i we know two probabilities from historical audits:
m_i = P(fields agree | the two records are the same person)
u_i = P(fields agree | the two records are different people, by chance)
Each field contributes a log-likelihood weight to a total match score R:
agree on field i: + log2( m_i / u_i )
disagree on field i: + log2( (1-m_i) / (1-u_i) )
R = sum of all four field weights
A highly discriminative field (rare to agree by chance, e.g. date of birth or an ID number) swings R hard when it agrees. A weak field (common to share by chance, e.g. a shared street address) barely moves R.
| Field | m (agree if same) | u (agree by chance) |
|---|
| Name | 0.90 | ~0.003 |
| Date of birth | 0.97 | ~0.00005 |
| Street address | 0.75 | ~0.083 |
| 4-digit ID suffix | 0.90 | 0.0001 |
Declare a match when R ≥ threshold T. Raise T and you demand stronger evidence — fewer false links but more true pairs missed (lower recall). Lower T and you catch more true pairs but let in coincidental look-alikes (lower precision). Raising the field error rate simulates messier government data entry (typos, address changes between agency updates) — it pushes true pairs' scores down, since fewer fields agree even for the same citizen.
The top panel plots Agency A records on the left, Agency B on the right (deliberately shuffled — in real siloed databases there is no shared ordering to exploit). Drag to pan, scroll/pinch to zoom. Green links are correct matches at the current threshold; red links are false matches. The bottom panel is the actual score distribution: every true pair (same citizen) vs. every false pair (different citizens), histogrammed by their Fellegi-Sunter score R, with the threshold drawn as a moving vertical line — this is the real precision/recall trade-off, not just its outcome.