Home▸Algorithms & AI▸LLM Evaluation & Benchmarks

📊 LLM Evaluation & Benchmarks — Simulating How Leaderboards Work

Interactive LLM evaluation simulation: watch benchmark scores (MMLU, HumanEval, BIG-Bench, Chatbot Arena) fluctuate with sample size and noise, and see how leaderboards shift with benchmark weighting.

Algorithms & AI3DModerate60 FPS

This simulation turns LLM benchmarking into something you can watch happen: six model bars rise and fall across four benchmark columns — MMLU, HumanEval, BIG-Bench and Chatbot Arena — as sampling noise and benchmark weighting change the leaderboard in real time.

🔬 What It Demonstrates

Each bar's observed score is the model's underlying skill plus random noise drawn from a normal distribution with standard error SE = σ/√n. Small benchmarks like HumanEval (only 164 problems) have a much noisier score than large ones like MMLU (about 14,000 questions) — the white whisker on top of each bar shows the resulting 95% confidence interval, and it visibly shrinks as you raise the sample size.

🎮 How to Use

Drag Sample size n to see how more test items narrow the confidence interval, Task noise σ to make individual questions harder to grade consistently, and Weight: code ↔ knowledge to shift the composite leaderboard between knowledge/reasoning benchmarks and code/human-preference benchmarks. Press Reroll sample to redraw a fresh evaluation run, or Reset to return to the defaults.

💡 Did You Know?

Real leaderboards are sensitive to exactly this kind of weighting: a model can rank #1 on a code-heavy composite and drop several places on a knowledge-heavy one, which is why serious evaluation suites like HELM report many benchmarks side by side instead of collapsing everything into one score.

⚙ Under the hood

Simulate how LLM benchmark scores (MMLU, HumanEval, BIG-Bench, Chatbot Arena) fluctuate with sample size and measurement noise: adjust the number of test items and reroll the sample to watch confidence intervals shrink and the composite leaderboard reorder.

Three.jsAIBenchmarksStatisticsInstancedMesh

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install