> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://www.comet.com/docs/opik/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://www.comet.com/docs/opik/_mcp/server.

# Guardrails server

Guardrails are an enterprise feature. Reach out to the Opik team at
[sales@comet.com](mailto:sales@comet.com) to enable them for your account.

The guardrails server is a standalone backend that runs the machine learning models behind Opik's built-in guardrails. The [Python SDK](/guardrails/guardrails) sends text to this service over HTTP, and the service returns whether each check passed along with detailed scores.

It powers these validation types:

* **Topic** — a zero-shot classifier (`facebook/bart-large-mnli`) that checks whether text is on or off topic.
* **PII** — named entity recognition (Microsoft Presidio with the spaCy `en_core_web_lg` model) that detects sensitive information such as names, emails, and credit card numbers.
* **Prompt injection** — a fine-tuned classifier (a Qwen LoRA adapter) that detects prompt injection and jailbreak attempts. Its model lives in a private Hugging Face repository, so the server needs a Hugging Face token (see [Configuration](#configuration)).
* **Custom models** — models [fine-tuned on your own labeled examples](/guardrails/custom-guardrails). Training runs on this server, and it loads the results by name from its adapters directory.

The server is optional and disabled by default. You need it to run the built-in Topic, PII, or Prompt injection guardrails, or any custom model. The [LLM judge guardrail](/guardrails/guardrails#llm-judge) does not use this server: it calls the LLM provider configured in your workspace.

## Installation and startup

### Run alongside Opik (recommended)

If you self-host Opik with Docker Compose, start the guardrails backend by passing the `--guardrails` flag:

```bash
./opik.sh --guardrails
```

This enables the `guardrails` profile, starts the `guardrails-backend` container, and configures the Opik frontend to proxy requests to it. When Opik is started this way, the Python SDK reaches the server automatically with no extra configuration.

### Run as a standalone container

You can also run the guardrails backend on its own, which is useful when you want to run it on a dedicated GPU node.

The image is not published to a registry, so build it locally from `apps/opik-guardrails-backend` (only needed once):

```bash
cd apps/opik-guardrails-backend
docker build -t opik-guardrails-backend:latest .
```

Run it with GPU acceleration:

```bash
docker run -p 5000:5000 --gpus all opik-guardrails-backend:latest
```

Or run it CPU-only:

```bash
docker run -p 5000:5000 opik-guardrails-backend:latest
```

The server listens on port `5000`. Confirm it is up with the health check:

```bash
curl http://localhost:5000/healthcheck
# OK
```

To use GPU acceleration you need the
[NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
installed on the host. The models are baked into the image at build time, so no model download happens at startup.

## Configuration

The server reads the following environment variables:

| Variable                                      | Default     | Description                                                                                                                                           |
| --------------------------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `OPIK_GUARDRAILS_DEVICE`                      | `cuda:0`    | Device used for model inference. Falls back to `cpu` when no GPU is available.                                                                        |
| `CUDA_VISIBLE_DEVICES`                        | all         | Controls which GPUs are visible to the container.                                                                                                     |
| `HF_TOKEN`                                    | —           | Hugging Face token used to download the private prompt injection model. Required for the prompt injection guardrail; other guardrails do not need it. |
| `OPIK_GUARDRAILS_PROMPT_INJECTION_MODEL`      | —           | Hugging Face repository of the prompt injection LoRA adapter. Required for the prompt injection guardrail.                                            |
| `OPIK_GUARDRAILS_PROMPT_INJECTION_BASE_MODEL` | —           | Base model the adapter is applied to. Required for the prompt injection guardrail.                                                                    |
| `OPIK_GUARDRAILS_ADAPTERS_DIR`                | `/adapters` | Directory the server loads [custom models](/guardrails/custom-guardrails) from. Must be the same directory training writes to.                        |

The prompt injection model is distributed privately by Comet. To use the prompt injection guardrail, reach out to
Comet to get access: they will provide the model repository, its base model, and a Hugging Face token to set as
`HF_TOKEN`.

The model is downloaded lazily the first time the prompt injection guardrail is used, so the server starts without these variables set; only the prompt injection guardrail fails (closed) until they are provided. Pass them to the container with `-e HF_TOKEN=<token> -e OPIK_GUARDRAILS_PROMPT_INJECTION_MODEL=<repo> -e OPIK_GUARDRAILS_PROMPT_INJECTION_BASE_MODEL=<base-model>`.

For example, to pin inference to the second GPU:

```bash
docker run -p 5000:5000 --gpus all -e OPIK_GUARDRAILS_DEVICE=cuda:1 opik-guardrails-backend:latest
```

The service detects at startup whether CUDA is available. If no GPU is present, or the NVIDIA Container Toolkit is not configured, it automatically falls back to CPU mode without any configuration changes.

If you run the server at a custom address (for example, on a separate node), point the Python SDK at it with the `OPIK_GUARDRAILS_URL_OVERRIDE` environment variable:

```bash
export OPIK_GUARDRAILS_URL_OVERRIDE=http://my-guardrails-host:5000
```

The Topic guardrail accepts a maximum input of 1024 tokens, and both the Topic and PII guardrails support English
only. Running on CPU works but is significantly slower; a GPU node is strongly recommended for production.

## Calling the server

Most users never call the server directly — the [Python SDK](/guardrails/guardrails) handles the request, response parsing, and trace logging for you. Call the API directly only when integrating from a language or environment without the SDK.

Send a `POST` request to `/api/v1/guardrails/validations` with the text to check and a list of validations to run:

```bash
curl -X POST http://localhost:5000/api/v1/guardrails/validations \
  -H "Content-Type: application/json" \
  -d '{
    "text": "My name is John Doe and my email is john.doe@example.com.",
    "validations": [
      {
        "type": "TOPIC",
        "config": {
          "topics": ["politics", "religion", "artificial intelligence"],
          "threshold": 0.5,
          "mode": "restrict"
        }
      },
      {
        "type": "PII",
        "config": {
          "entities": ["PERSON", "EMAIL_ADDRESS"],
          "language": "en",
          "threshold": 0.5
        }
      }
    ]
  }'
```

The response reports an overall `validation_passed` flag plus per-validation results with the scores that drove each decision.

#### Request and response reference

**Request fields:**

* `text` (required) — the text to validate.
* `validations` (required) — a list of validations to run. Each has a `type` (`TOPIC` or `PII`) and a `config`:
  * **`TOPIC` config:**
    * `topics` (required) — the list of topics to check against.
    * `threshold` (default `0.5`) — confidence threshold for topic detection.
    * `mode` (required) — `restrict` passes when none of the topics match (content filtering); `allow` passes when at least one matches (content classification).
  * **`PII` config:**
    * `entities` (optional) — entity types to detect. Defaults to `IP_ADDRESS`, `PHONE_NUMBER`, `PERSON`, `MEDICAL_LICENSE`, `URL`, `EMAIL_ADDRESS`, `IBAN_CODE`. See the [Presidio supported entities](https://microsoft.github.io/presidio/supported_entities/) for the full list.
    * `language` (default `en`) — language of the text.
    * `threshold` (default `0.5`) — confidence threshold for PII detection.

**Example response:**

```json
{
  "validation_passed": false,
  "validations": [
    {
      "validation_passed": false,
      "type": "PII",
      "validation_config": {
        "entities": ["PERSON", "EMAIL_ADDRESS"],
        "language": "en",
        "threshold": 0.5
      },
      "validation_details": {
        "detected_entities": {
          "PERSON": [{ "start": 11, "end": 19, "score": 0.85, "text": "John Doe" }],
          "EMAIL_ADDRESS": [{ "start": 36, "end": 56, "score": 1.0, "text": "john.doe@example.com" }]
        }
      }
    }
  ]
}
```

## Next steps

#### [Guards](/guardrails/guardrails)

The five guard types and how to configure each one.

#### [Guardrails overview](/guardrails/overview)

Understand when to use guardrails versus online evaluation.