LLM Model Registry Configuration

The list of LLM models that appear in Opik’s dropdowns (Playground, LLM-as-Judge, Automation Rules, Optimization Studio) is served by the backend from a YAML registry. The registry is composed from up to three sources, merged in this order:

1. Classpath defaults — llm-models-default.yaml shipped inside the backend JAR. Always loaded. This is the source every deployment sees out of the box.
2. Remote CDN YAML — opt-in. When enabled, the backend fetches a YAML from a URL you configure and refreshes it on a schedule. Self-hosted deployments are not required to use this; it’s primarily for operators who want to pick up new models between Opik releases without redeploying.
3. Local override YAML — optional. A YAML file you mount into the backend container; its entries override or extend the defaults and the remote content.

This page describes how to configure these sources for self-hosted deployments.

Environment variables

All configuration is via env vars on the opik-backend container.

YAML schema

openai:
  - id: "gpt-4o"
    label: "GPT 4o"
    structuredOutput: true
    reasoning: false
anthropic:
  - id: "claude-opus-4-7"
    label: "Claude Opus 4.7"
    reasoning: false
vertex-ai:
  - id: "gemini-2.5-pro"
    qualifiedName: "vertex_ai/gemini-2.5-pro"
    label: "Gemini 2.5 Pro"
    structuredOutput: true

Fields:

• id (required) — the model identifier used at inference time.
• qualifiedName (optional) — disambiguates models that exist under multiple providers (e.g. Gemini via Vertex AI vs. the Gemini API directly). Used as the routing key when set. Vertex AI entries need it, in the form vertex_ai/<id>.
• label (optional) — the human-readable name shown in dropdowns. Falls back to id when omitted.
• structuredOutput (optional, default false) — whether the model supports JSON schema / tool-calling structured output mode.
• reasoning (optional, default false) — whether the model is a reasoning model (enforces temperature = 1.0 and unlocks reasoning-effort parameters in the UI).

An optional field that an override leaves out keeps the lower layer’s value (see below). The defaults apply only when no layer sets the field.

Merge behavior

Models are keyed by id across every provider. qualifiedName is used for routing lookups (to disambiguate gemini-2.5-pro under the Gemini direct API vs. Vertex AI), but override deduplication always uses id. When a merge happens:

• Add: an id not present in lower layers is appended to that provider’s list.
• Override: an id that matches a lower layer updates that model. Fields the overriding entry sets replace the lower layer’s values. Fields it leaves out (qualifiedName, label, structuredOutput, reasoning) keep them. So an override can change a single field, and an older remote copy can’t turn off a flag your release sets. To turn a flag off, set it to false explicitly.
• Remove: not currently supported. Contact support if you need to hide a default model entirely.
• Order: providers keep the order of the classpath file. A provider that only an upper layer adds comes last.

Configuration scenarios

Default behaviour

Leave the defaults in place. The backend serves the classpath llm-models-default.yaml shipped with your Opik release — no outbound traffic, no extra configuration. Upgrade Opik to pick up new models.

Enable the remote CDN fetch (optional)

If you want new models to reach your running deployment between Opik releases — e.g. if you run long-lived stacks on an extended upgrade cadence and want provider-side additions to land automatically — point the backend at a remote YAML:

LLM_MODEL_REGISTRY_REMOTE_ENABLED=true
LLM_MODEL_REGISTRY_REMOTE_URL=https://your-cdn.example.com/opik/llm-models-default.yaml
LLM_MODEL_REGISTRY_REFRESH_INTERVAL_SECONDS=3600

Comet SaaS uses https://cdn.comet.ml/opik/llm-models-default.yaml — you can either mirror that content on your own CDN or point directly at it if your policies allow. A daily job checks the providers for new models and opens a pull request in the Opik repository. The CDN copy changes only when that reviewed update merges.

Remote fetch failures are logged but non-fatal: the backend keeps using the last remote copy it fetched successfully (or the classpath defaults if no fetch has succeeded since startup), so enabling the remote tier never risks losing model routing. Your local override is still applied on top.

Add a private fine-tuned model (Docker Compose)

Create /etc/opik/my-models-override.yaml on the host:

openai:
  - id: "ft:gpt-4o-2024-08-06:my-org::abc123"
    label: "Our Fine-Tuned GPT-4o"
    structuredOutput: true

Mount it into the backend container and set the path:

# docker-compose.override.yaml
services:
  backend:
    volumes:
      - /etc/opik/my-models-override.yaml:/opt/opik/models-override.yaml:ro
    environment:
      LLM_MODEL_REGISTRY_LOCAL_OVERRIDE_PATH: /opt/opik/models-override.yaml

Add a private fine-tuned model (Kubernetes / Helm)

Create a ConfigMap with your override YAML:

kubectl create configmap opik-llm-models-override \
  --from-file=models-override.yaml=/path/to/models-override.yaml

Mount it in the backend Deployment by extending your Helm values:

# values.yaml overrides
component:
  backend:
    env:
      LLM_MODEL_REGISTRY_LOCAL_OVERRIDE_PATH: "/etc/opik/models-override.yaml"
    volumes:
      - name: llm-models-override
        configMap:
          name: opik-llm-models-override
    volumeMounts:
      - name: llm-models-override
        mountPath: /etc/opik/models-override.yaml
        subPath: models-override.yaml
        readOnly: true

Verification

After restart, check that your model appears:

curl -s https://your-opik/api/v1/private/llm/models | jq '.openai[] | select(.id | contains("ft:"))'

The same list appears in the UI dropdowns within seconds of a browser refresh.

Failure modes