LLM Observability, Monitoring & Drift Detection | ML Knowledge Hub

LLM Observability, Monitoring & Drift Detection

Trace prompts, context, and outputs; score quality continuously; detect drift in models, embeddings, and data; respond with playbooks.

Observability for LLM systems requires full-fidelity traces (prompt, context chunks, tool calls, outputs), quality scoring, safety signals, and drift detection. Treat prompts and models like code: version, test, roll back, and monitor.

Correlate a request ID across retrieval, rerank, generation, and post-

Log prompt, system instructions, context chunk IDs, tool calls, latency per step.

Redact secrets and PII; store hashed user identifiers for privacy.

Check recent deploys (prompt/model/index)

Re-run golden set; compare before/after traces

Roll back to last good version; freeze deployments

Frequently asked questions