Guardrails overview

Guardrails are real-time checks that run inline with your LLM calls to catch unsafe inputs and outputs before they reach a user. Unlike offline testing, guardrails operate in production on live traffic, acting as a safety layer around your application.

What Opik offers

• Five guard types — PII detection, allowed and restricted topics, prompt injection and jailbreak detection, an LLM-as-a-judge check written in plain language, and your own trained classifier.
• Stored policies — define your checks once in the Opik UI and reference them by name from your application. Workspace-wide policies can also be enforced on every application, whether or not it asks for them.
• Model fine-tuning — for a check the built-in guards do not cover, fine-tune a language model on your own labeled examples. Opik hosts the training, reports how accurate the result is, and serves the model inline. No training infrastructure of your own, and no deployment step.
• Full observability — every guardrail run is logged as a span on the trace it protects, so you can inspect what failed and track failure rates over time.

Concepts

Three words come up throughout these pages, and it helps to separate them up front.

In short: a policy is configuration, a guardrail is what executes. One guardrail can combine several policies, and one policy can be used by any number of applications.

Why guardrails matter

LLMs are non-deterministic. The same prompt can produce different responses, and no amount of pre-production testing can guarantee that every response in production will be safe. Guardrails close that gap by enforcing your requirements on every request, as it happens.

Common risks guardrails help you contain:

• Sensitive data leakage — personally identifiable information (PII) such as names, addresses, or credit card numbers appearing in inputs or outputs.
• Off-topic responses — the model answering questions outside the scope of your application.
• Harmful or non-compliant content — outputs that violate your policies or regulatory requirements.
• Business-specific risks — competitor mentions, disallowed claims, or any rule unique to your domain.

Because a guardrail runs before the response is returned, it lets you stop a bad response, retry, or fall back to a safe default rather than exposing the problem to your users.

Guardrails vs. online evaluation

Opik gives you two complementary ways to check production traffic. They solve different problems, and most teams use both.

The key difference is blocking versus non-blocking: guardrails run inline and can prevent a response from being returned, while online evaluation runs asynchronously on traces that have already been logged.

Use guardrails when a failing check must change what the user sees — for example, blocking a response that leaks PII or drifts off topic. The check is worth the added latency because the cost of letting the response through is high.

Use online evaluation when you want to understand how your application is behaving across production traffic without adding latency. It runs LLM-as-a-Judge metrics on logged traces, samples at whatever rate you choose, and surfaces trends so you can identify issues over time.

Next steps