Guardrail Cascade: Latency, Caching & Early Exit

Every request entering this pipeline first checks a small cache, then walks a cascade of three classifiers of rising cost and accuracy — a cheap keyword filter, a mid-cost model and an expensive final classifier — committing to Allow / Transform / Block the moment one stage is confident enough, or escalating onward when it isn't. Tune the early-exit confidence margin, the repeat-prompt cache-hit rate, incoming traffic and the share of genuinely harmful requests, and watch average latency, cache hit rate and decision accuracy respond live — the exact trade-off real LLM guardrail stacks navigate between speed and safety.