Home▸AI & Machine Learning▸3D Inter-Annotator Agreement: Cohen's Kappa, Chance Correction, the Prevalence Paradox and Confidence Intervals

🏷 3D Inter-Annotator Agreement: Cohen's Kappa, Chance Correction, the Prevalence Paradox and Confidence Intervals

Two robot annotators label a stream of items on a Blender-built floor while a 3D confusion-matrix plinth grows: observed agreement, chance agreement from the marginals, Cohen's kappa, its ceiling and a 95% confidence interval, with the prevalence paradox, checked against a textbook table, exact expectations and a coverage simulation.

AI & Machine Learning3DModerate60 FPS📱 Mobile-adapted🟠 Blender⇄ 2D version
3D Inter-Annotator Agreement: Cohen's Kappa, Chance Correction, the Prevalence Paradox and Confidence Intervals ↗ Open standalone
⚙ Under the hood

Two robot annotators label a stream of items on a Blender-built floor while a 3D confusion-matrix plinth grows: observed agreement, chance agreement from the marginals, Cohen's kappa, its ceiling and a 95% confidence interval, with the prevalence paradox, checked against a textbook table, exact expectations and a coverage simulation.

Cohen's KappaInter-Annotator AgreementData LabelingDataset QualityChance AgreementPrevalence ParadoxConfidence IntervalStatistics for ML

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)