Home▸AI & Machine Learning▸2D Inter-Annotator Agreement: Cohen's Kappa, Chance Correction, the Prevalence Paradox and Confidence Intervals

🏷 2D Inter-Annotator Agreement: Cohen's Kappa, Chance Correction, the Prevalence Paradox and Confidence Intervals

Top view of two annotators labelling items: live confusion matrix, observed and chance agreement, Cohen's kappa, the kappa ceiling and a Fleiss standard error, verified by a textbook table, two code paths, exact expectation, the 95% / 5% prevalence paradox and a 600-trial coverage test.

AI & Machine Learning2DModerate60 FPS📱 Mobile-adapted🟠 Blender⇄ 3D version
2D Inter-Annotator Agreement: Cohen's Kappa, Chance Correction, the Prevalence Paradox and Confidence Intervals ↗ Open standalone
⚙ Under the hood

Top view of two annotators labelling items: live confusion matrix, observed and chance agreement, Cohen's kappa, the kappa ceiling and a Fleiss standard error, verified by a textbook table, two code paths, exact expectation, the 95% / 5% prevalence paradox and a 600-trial coverage test.

Cohen's KappaInter-Annotator AgreementData LabelingDataset QualityChance AgreementPrevalence ParadoxConfidence IntervalStatistics for ML

2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)