🛡️ AI Alignment and Red-Teaming

About this simulation

Before a language model ships, safety teams "red-team" it: they throw thousands of adversarial prompts at it — jailbreaks, disguised harmful requests, prompt injections — to see what gets through. This scene models a simplified safety pipeline as three sequential gates a prompt must clear before it reaches the model's output.

How it works

• Each prompt (a small glowing packet) carries a hidden true risk score. Benign prompts score low; adversarial ones score high.
• Three gates check it in sequence — a content filter, an intent classifier, and a final refusal policy — each comparing a perceived risk to the sensitivity threshold.
• A gate flags a prompt when perceived risk ≥ threshold; flagged packets fall into that gate's quarantine tray and turn red.
• Jailbreak sophistication lets adversarial prompts disguise themselves, lowering the risk the content filter and intent classifier perceive — but the refusal policy is built to resist most of that disguise, so defense-in-depth still catches many.
• A prompt that clears all three gates reaches the output terminal: benign ones are delivered (green), but a harmful one that slips through is a leak / breach (red flash) — exactly what red-teamers are hunting for.

Controls

• Filter sensitivity — raises or lowers every gate's threshold. Higher catches more harmful prompts, but also more false positives on benign ones.
• Adversarial traffic mix — the share of fired prompts that are adversarial/harmful rather than benign.
• Jailbreak sophistication — how well adversarial prompts disguise their intent from the filters.
• Auto-fire prompts — continuously streams prompts; uncheck and use "Fire prompt" to send one at a time.

Did you know?

Real red-teaming exercises, like those run before GPT-4 and Claude releases, combine human adversarial testers with automated attack generators — and defense-in-depth (multiple independent checks) consistently catches more jailbreaks than any single filter alone, which is exactly what the sophistication slider demonstrates here.