🏷 2D Inter-Annotator Agreement: Cohen's Kappa, Chance Correction, the Prevalence Paradox and Confidence Intervals
Top view of two annotators labelling items: live confusion matrix, observed and chance agreement, Cohen's kappa, the kappa ceiling and a Fleiss standard error, verified by a textbook table, two code paths, exact expectation, the 95% / 5% prevalence paradox and a 600-trial coverage test.
⚙ Under the hood
Top view of two annotators labelling items: live confusion matrix, observed and chance agreement, Cohen's kappa, the kappa ceiling and a Fleiss standard error, verified by a textbook table, two code paths, exact expectation, the 95% / 5% prevalence paradox and a 600-trial coverage test.
Cohen's KappaInter-Annotator AgreementData LabelingDataset QualityChance AgreementPrevalence ParadoxConfidence IntervalStatistics for ML
2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install