Constitutional AI (Bai et al., Anthropic 2022) aligns a language model without relying on humans to label every harmful example. Instead, the model critiques and revises its own draft responses against a written constitution — a short list of principles — then a second model is trained on those AI-generated preference pairs (RLAIF: reinforcement learning from AI feedback) instead of pure RLHF.
Each candidate response is a point in a 3-axis score space:
x = harm score (policy/safety violation risk)
y = hallucination (unsupported / fabricated claims)
z = unhelpfulness (refusals, evasiveness, low content)
This 2D edition plots x against y directly (the top scatter panel) and encodes z as each dot's radius — so the full 3-axis mechanic from the original 3D model is preserved exactly, only the camera changed. Every Critique + Revise step moves each candidate along the weighted sum of the active principles' pull vectors, plus random critique noise (the AI critic is not perfect either):
p(t+1) = p(t) + η · Σ wᵢ·dᵢ + σ·noise
wᵢ = 1 if principle i is active, else 0
dᵢ = that principle's fixed pull direction (x,y,z)
- Harmlessness pulls harm down hardest, with a small side-effect of more refusals (higher unhelpfulness) — the classic helpfulness/harmlessness tension.
- Honesty (no fabrication) pulls hallucination down — closest to what RAG grounding and fact-checking passes target at inference time.
- Privacy-respecting gives a moderate harm reduction for a narrower failure mode (PII leakage).
- Helpfulness balance pulls unhelpfulness down but nudges harm and hallucination back up slightly — omitting it makes the model safe but useless; over-weighting it alone is exactly how untuned "maximally helpful" models hallucinate and comply with unsafe requests.
The dashed green circle on the scatter marks the aligned region's harm/hallucination cross-section. Aligned fraction — like in the 3D version — tests the true 3D distance from every candidate to the target point, so a candidate can sit inside the drawn circle and still count as unaligned if its unhelpfulness (dot size) is off. The trend panel (bottom-left) plots all four readouts across revision steps so convergence — or a badly unbalanced constitution failing to converge — is visible over time, and the bar (bottom-right) gauges mean unhelpfulness live.