LLM Model Registry Configuration

The list of LLM models that appear in Opik’s dropdowns (Playground, LLM-as-Judge, Automation Rules, Optimization Studio) is served by the backend from a YAML registry. The registry is composed from up to three sources, merged in this order:

1. Classpath defaults — llm-models-default.yaml shipped inside the backend JAR. Always loaded. This is the source every deployment sees out of the box.
2. Remote CDN YAML — opt-in. When enabled, the backend fetches a YAML from a URL you configure and refreshes it on a schedule. Self-hosted deployments are not required to use this; it’s primarily for operators who want to pick up new models between Opik releases without redeploying.
3. Local override YAML — optional. A YAML file you mount into the backend container; its entries override or extend the defaults and the remote content.

This page describes how to configure these sources for self-hosted deployments.

Environment variables

All configuration is via env vars on the opik-backend container.

YAML schema

1
openai:
2
  - id: "gpt-4o"
3
    label: "GPT 4o"
4
    structuredOutput: true
5
    reasoning: false
6
anthropic:
7
  - id: "claude-opus-4-7"
8
    label: "Claude Opus 4.7"
9
    reasoning: false
10
vertex-ai:
11
  - id: "gemini-2.5-pro"
12
    qualifiedName: "vertex_ai/gemini-2.5-pro"
13
    label: "Gemini 2.5 Pro"
14
    structuredOutput: true

Fields:

• id (required) — the model identifier used at inference time.
• qualifiedName (optional) — disambiguates models that exist under multiple providers (e.g. Gemini via Vertex AI vs. the Gemini API directly). Used as the routing key when set.
• label (optional) — the human-readable name shown in dropdowns. Falls back to id when omitted.
• structuredOutput (optional, default false) — whether the model supports JSON schema / tool-calling structured output mode.
• reasoning (optional, default false) — whether the model is a reasoning model (enforces temperature = 1.0 and unlocks reasoning-effort parameters in the UI).

Merge behavior

Models are keyed by id across every provider. qualifiedName is used for routing lookups (to disambiguate gemini-2.5-pro under the Gemini direct API vs. Vertex AI), but override deduplication always uses id. When a merge happens:

• Add: an id not present in lower layers is appended to that provider’s list.
• Override: an id that matches a lower layer replaces the full definition. Partial overrides are not supported — supply all fields you want on the final model.
• Remove: not currently supported. Contact support if you need to hide a default model entirely.

Configuration scenarios

Default behaviour

Leave the defaults in place. The backend serves the classpath llm-models-default.yaml shipped with your Opik release — no outbound traffic, no extra configuration. Upgrade Opik to pick up new models.

Enable the remote CDN fetch (optional)

If you want new models to reach your running deployment between Opik releases — e.g. if you run long-lived stacks on an extended upgrade cadence and want provider-side additions to land automatically — point the backend at a remote YAML:

LLM_MODEL_REGISTRY_REMOTE_ENABLED=true
LLM_MODEL_REGISTRY_REMOTE_URL=https://your-cdn.example.com/opik/llm-models-default.yaml
LLM_MODEL_REGISTRY_REFRESH_INTERVAL_SECONDS=3600

Comet SaaS uses https://cdn.comet.ml/opik/llm-models-default.yaml, regenerated daily by the Opik sync workflow — you can either mirror that content on your own CDN or point directly at it if your policies allow.

Remote fetch failures are logged but non-fatal: the backend keeps serving the last successful registry (or the classpath defaults if the first fetch fails), so enabling the remote tier never risks losing model routing.

Add a private fine-tuned model (Docker Compose)

Create /etc/opik/my-models-override.yaml on the host:

1
openai:
2
  - id: "ft:gpt-4o-2024-08-06:my-org::abc123"
3
    label: "Our Fine-Tuned GPT-4o"
4
    structuredOutput: true

Mount it into the backend container and set the path:

1
# docker-compose.override.yaml
2
services:
3
  backend:
4
    volumes:
5
      - /etc/opik/my-models-override.yaml:/opt/opik/models-override.yaml:ro
6
    environment:
7
      LLM_MODEL_REGISTRY_LOCAL_OVERRIDE_PATH: /opt/opik/models-override.yaml

Add a private fine-tuned model (Kubernetes / Helm)

Create a ConfigMap with your override YAML:

$
kubectl create configmap opik-llm-models-override \
>
  --from-file=models-override.yaml=/path/to/models-override.yaml

Mount it in the backend Deployment by extending your Helm values:

1
# values.yaml overrides
2
component:
3
  backend:
4
    env:
5
      LLM_MODEL_REGISTRY_LOCAL_OVERRIDE_PATH: "/etc/opik/models-override.yaml"
6
    volumes:
7
      - name: llm-models-override
8
        configMap:
9
          name: opik-llm-models-override
10
    volumeMounts:
11
      - name: llm-models-override
12
        mountPath: /etc/opik/models-override.yaml
13
        subPath: models-override.yaml
14
        readOnly: true

Verification

After restart, check that your model appears:

$
curl -s https://your-opik/api/v1/private/llm/models | jq '.openai[] | select(.id | contains("ft:"))'

The same list appears in the UI dropdowns within seconds of a browser refresh.

Failure modes