Order Effects & Counterbalancing Lab

Design

Participants 40

True treatment effect (B − A) 0.60

Order effect (practice/fatigue) 0.80

Trial noise σ 0.50

Live Estimates

True effect (ground truth)

0.60

Naive estimate (mean B − mean A)

—

Bias (estimate − truth)

—

Observed order gap (pos2 − pos1)

—

How it works

Each virtual participant completes both conditions in a within-subjects design — one score at test position 1, one at position 2. Participant i's score at position p is generated as:

score = baseline_i
      + trueEffect · [condition == B]
      + orderEffect · [position == 2]
      + noise ~ N(0, σ²)

Fixed order — every participant does A first, then B. Now "condition == B" and "position == 2" are perfectly confounded: every B score also carries the full order effect. The naive estimator, mean(B) − mean(A), equals trueEffect + orderEffect, so it is biased by exactly the order-effect slider's value.

Counterbalanced — half the participants run A→B, the other half B→A (an ABBA-style scheme). Condition and test position are now decorrelated: each condition appears equally often at position 1 and position 2, so the order effect cancels out of the group averages and mean(B) − mean(A) converges back to the true treatment effect as sample size grows.

This is the textbook rationale for counterbalancing repeated-measures designs: it is not about making each individual trial "fair," it is about breaking the correlation between the nuisance variable (order/practice/fatigue) and the variable of interest (treatment), so the group-level contrast stops absorbing that nuisance effect.

Participants — sample size; larger N shrinks the naive estimate's sampling noise around its (biased or unbiased) expectation.
True treatment effect — the real causal effect of condition B vs A, built into the simulation as ground truth for comparison.
Order effect — practice (positive) or fatigue (negative) added to every score taken at the second position, regardless of which condition it happens to be.
Trial noise σ — per-trial measurement noise; higher σ widens the spread of bar heights without changing the expected bias.

Condition A Condition B Position 1 (faded) → Position 2

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.