Changelog

Error, duration, and cost cells across the Traces, Spans, and Threads tables now have an “Explain” button (the Ollie owl icon) that streams a plain-language explanation of that specific value without leaving the table — no need to open the trace panel and dig through raw payloads to understand why a call errored, took as long as it did, or cost what it did.

LLM-as-a-judge scoring runs (on traces, spans, and threads) are now recorded as monitoring traces automatically, with a prepare_evaluation span and one span per scoring call, including token usage and cost. They’re hidden from the main Logs view by default, and each rule on the Online Evaluation page now has a Go to traces action that opens a scoped, filtered view of that rule’s evaluation activity — useful for confirming a rule is running, checking what it cost, or debugging why it errored. This is on by default for all workspaces.

• Diagnostics now explains why a run failed — Previously, a Diagnostics run that crashed or never started (for example, from exhausted LLM credits) just spun until it timed out, with no indication of what went wrong. Failed runs now show a specific reason (out of credits, rate limited, provider error, or never started) with a “Try again” action, and the failure timeout was cut from 12 minutes to 5.

• Playground and online scoring: Sonnet 5 and Fable 5 no longer error out — These models were missing from the capability list, so the Playground kept showing temperature/Top P sliders and sending those parameters, which the models reject. The sliders are now hidden and the parameters are no longer sent, matching the behavior already in place for Opus 4.7/4.8.

• Azure-hosted OpenAI model prices now load correctly — Model prices for the azure/* model family (gpt-4o, gpt-5, codex-mini, and others) were silently failing to load because of a provider-mapping gap, causing cost to show as unavailable for these models.

• Experiment and Playground traces now show up in scoped “Go to logs” views — Traces created by experiment runs and Playground runs were sometimes missing from their respective “Go to logs” tables even though the traces existed, due to a visibility-filter mismatch. These views now correctly scope by the experiment or run itself.

• Playground: dataset and metric selection redesigned — The “Run experiment” dialog is replaced by inline dataset and metrics dropdowns in the Playground header, so you can change either independently without reopening a modal. Metrics can also be created and edited in place.

• Permission gating extended to more pages — Users with the Annotator workspace role no longer see Agent Playground, Online Evaluation, or Alerts in the sidebar. Prompt Library view/edit access is now governed by the same permission system, at both the UI and API level.

• opik import --to-workspace for cross-workspace imports — The opik import CLI command accepts a --to-workspace option to import exported data into a different workspace than the one it was exported from, without needing to restructure the export directory as a workaround.

• opik migrate dataset --exclude-experiments — This flag skips migrating a dataset’s experiments and optimizations, migrating only the dataset and its version history. Useful for large datasets where the experiment/optimization cascade isn’t needed.

• Fixed intermittent errors loading prompt and dataset versions under load — A query pattern that caused MySQL to materialize large temporary tables could fail outright under load. Prompt and dataset version lookups are now rewritten to avoid the issue.

• Faster trace ingestion — Removed a redundant ClickHouse lookup that ran on every trace ingestion just to check the trace’s last-updated timestamp, reducing ingestion latency and ClickHouse load.

• SDK: shared connection resources across Opik() clients — Code that creates multiple Opik() clients (for example, one per task or request) now shares the underlying connection pool and background threads across clients with matching configuration instead of creating a full new stack each time, reducing thread and connection overhead.

• Self-hosted: ClickHouse FINAL reads no longer over-read on skip-indexed tables — Disabled the new ClickHouse 25.x default for use_skip_indexes_if_final_exact_mode, which was causing skip-indexed queries to read far more data than necessary; Opik’s queries already prune by primary key and project ID, so the exact mode isn’t needed.

Online LLM-as-judge scoring rules can now reason over files attached to traces — images, PDFs, documents, and other binary content — in addition to the trace’s text fields. Previously, attachment-aware evaluation was only available for thread (conversation) scoring; single-trace scoring rules now share the same capability.

When a trace carries attachments, the scorer automatically routes to an agentic tool-calling loop. The judge model receives a ReadTool that lists the attached files and a GetAttachmentTool that fetches each file as multimodal content. Up to eight attachments are injected per evaluation. When the configured provider does not support tool calls, a warning is surfaced rather than silently failing.

The opik export and opik import CLI commands now require a project argument, matching how Opik v2 organises data (every dataset, prompt, and experiment lives inside a project).

New command shape:

$
opik export WORKSPACE PROJECT ITEM [NAME] [OPTIONS]

Where ITEM is one of all, dataset, traces, experiment, or prompt. On-disk output is written to PATH/WORKSPACE/projects/<id>/ with a project.json name index, so the same --path value round-trips between export and import.

• MCP server: opik mcp status command + auto-detection of hosted server — opik configure and opik mcp configure now probe for a Comet-hosted MCP server (HTTP + OAuth) and fall back to the local uvx opik-mcp server only when the hosted one is unavailable. A new opik mcp status command (and the equivalent opik configure status) reports the active Opik configuration and which AI assistants have the MCP server registered, flagging any drift from ~/.opik.config. Use --local-server to force the local path.

• Vercel AI SDK v7 and Vercel eve agent support — OpikExporter now captures traces from AI SDK v4 through v7 and from the Vercel eve agent framework. Multi-turn eve conversations are automatically grouped into a single thread via the session ID, and cached-token usage is captured.

• track_openai: custom provider argument — An optional provider parameter (str or opik.LLMProvider) can now be passed to track_openai to override the auto-detected provider on every span. Useful for OpenAI-compatible APIs where the hostname-derived default is not descriptive.

  1
  client = track_openai(openai.OpenAI(base_url="..."), provider="my-provider")

• Configurable local runner poll interval — The opik connect / opik endpoint runner polls for jobs every 0.5 s by default (~120 requests/min). Setting OPIK_RUNNER_POLL_INTERVAL (seconds) reduces this for environments where firewalls or proxies throttle sustained polling. Sustained failures now escalate to an actionable warning with a firewall/proxy note rather than going silent.

• Assertion scoring errors now shown in the UI — When an LLM-as-judge assertion scorer fails (for example, because the configured model is unavailable), an orange Scoring error badge with an actionable tooltip is shown instead of a spinning indicator or a generic grey Skipped tag.

• Test suite items no longer lost after editing an assertion — A ClickHouse parameter-escaping bug caused the \n sequences inside LLM-judge prompts to be unescaped to literal newlines, producing invalid stored JSON and silently dropping the row on read. Items now survive edit round-trips correctly.

• ChrF metric: char_order and ignore_whitespace now forwarded to NLTK — These parameters were stored in the ChrF object but never passed to nltk.translate.chrf_score.sentence_chrf. char_order is now forwarded as max_len and ignore_whitespace is forwarded as-is.

• OpenAI audio models: audio input tokens billed at the correct rate — For models such as gpt-4o-audio-preview and gpt-4o-realtime, audio input tokens are now charged at the input_cost_per_audio_token rate (up to 16× the standard text rate). Previously they were silently billed at the standard text input rate.

• opik export span fetch performance — When exporting a filtered subset of traces, spans are now fetched with a trace_id range bound derived from the matched trace IDs rather than scanning the entire project. Large filtered exports that previously stalled now complete promptly.

• Optimization studio renamed to Optimization runs — The v2 UI section previously called “Optimization studio” is now labelled Optimization runs everywhere.

• Attachment base64 detection is now O(n) — The regex that detects whether an attachment payload is already base64-encoded was rewritten with a possessive quantifier, making it linear in the payload size and avoiding catastrophic backtracking on large binary attachments.

• Self-hosted: ClickHouse upgraded to 25.8 LTS — The bundled ClickHouse instance (Helm chart, docker-compose, and testcontainers) is now pinned to 25.8 LTS and the operator to 0.27.1. This release requires no manual migration; existing data is read correctly by the new version.

Traces from PydanticAI agents ingested over OpenTelemetry (via Logfire) now show complete, correctly-typed data and accurate cost. Several gaps in the OTel → Opik span mapping were fixed:

• Tool call I/O is now captured — tool span inputs and outputs now appear in the correct Input/Output fields. Previously, logfire’s tool_response attribute fell into the Input bucket and the Output field was empty.
• Errors surface at the trace level — when a span carries an OTel exception event or STATUS_CODE_ERROR status, the error is translated into error_info and propagated up to the parent trace so failed agent runs are flagged at the top level.
• Gemini/Google cost now calculated — PydanticAI reports the generic google provider name, which previously matched no pricing row and showed $0 cost. The backend now resolves it to google_vertexai or google_ai based on the server hostname.
• Agent-run spans correctly typed — spans from invoke_agent operations are now typed general instead of llm, so the trace tree accurately reflects the agent structure.

Setting up Claude Desktop (or any MCP client) to connect to Opik no longer requires manually editing JSON config files. Running opik configure now offers an opt-in MCP setup step at the end of the flow. The standalone opik mcp configure command runs the same wizard on its own. Both automatically write the Opik MCP server block to your Claude Desktop configuration.

• OpenClaw added to onboarding integrations — The Get Started page now includes OpenClaw with its four-step CLI setup guide (install plugin → configure → status → run), rendered with bash syntax highlighting. The integration details dialog also supports multi-step and non-pip integrations going forward.

• Logs view: search bar repositioned inline with filters — The search bar now sits alongside active filter chips in a single wrapping row. Toolbar controls (Columns, Row height, Refresh) moved to the tab row, reducing visual clutter in the main search area. Chart bars changed to violet and table tags to green.

• Annotation queue: scores scoped per queue — Feedback scores and comments now carry a source_queue_id so each queue’s metrics (average scores, reviewer counts) are computed independently. A score submitted through one queue no longer counts toward another queue’s statistics when the same trace appears in both.

• Annotation queue: false “another annotator is reviewing” message resolved — After submitting a feedback score, the lock heartbeat now correctly recognizes existing lock holders, so annotators no longer see a false “another annotator is reviewing this item” message immediately after scoring.

• Thread-scoped annotation queues no longer crash — Opening a thread annotation queue without additional queue filters no longer throws an Unknown expression or function identifier error and crashes the queue view.

• Experiments page: charts stay fixed during horizontal table scroll — The feedback-score charts now remain pinned to the top while the experiments table scrolls horizontally underneath.

• Playground: immediate failure on exhausted OpenAI quota — When an OpenAI API key has insufficient quota (HTTP 429 insufficient_quota), the Playground now returns an error immediately instead of retrying the request multiple times and delaying the response.

Long-running evaluation jobs that get cut short — by Ctrl-C, an OOM error, a failed scoring metric, or a network blip — can now be continued from where they stopped instead of restarting from scratch. opik.evaluate_resume(experiment_id, task, scoring_metrics=[...]) replays only the trials that did not complete, merges them with the ones that did, and returns a single EvaluationResult covering the whole experiment.

The Playground and LLM-as-a-Judge now support OpenAI’s /v1/responses API, making it possible to use o-series reasoning models (o1, o3, o3-mini, o4-mini) and other deployments that are only available on the newer API path. Previously, sending these models through the Chat Completions path returned “This is not a chat model and thus not supported in the v1/chat/completions endpoint.”

• Annotation queues: claim mechanism for parallel annotation — Multiple annotators working the same queue simultaneously now see each item locked while another reviewer is looking at it, preventing duplicate work. Items show an “In review” indicator (orange) when all annotator slots are occupied by a combination of active locks and existing scores. Locks are kept alive by a heartbeat and expire via TTL when the reviewer navigates away. The sidebar also gains “To review” / “Processed” filter tabs, and each annotator sees items in a distinct shuffled order to reduce contention.

• Collapsible JSON/YAML in trace and span detail view — JSON objects, arrays, and YAML blocks in the trace and span detail view can now be folded and unfolded with an inline chevron at the end of each foldable line. Collapsed blocks render as a clickable gray placeholder. This makes it easier to navigate large payloads without scrolling past content you don’t need.

• Redesigned dataset and test suite creation flow — The creation dialog now presents two explicit paths: Upload a file (CSV or JSON dropzone with auto-naming and optional evaluation criteria for test suites) and Use SDK (name + code snippet). Both options are accessible from the header button and from the list empty state. On success the panel closes and a “Go to …” toast appears.

• Evaluate experiment traces directly from the UI — The Compare Experiments page has a new Evaluate button (brain icon) in the action bar. It opens the online evaluation dialog scoped to all traces in the current experiment, so you can score an experiment’s output without leaving the page.

• Span filtering by created_at and last_updated_at — The span search API now accepts created_at and last_updated_at as filter fields with all comparison operators (=, !=, >, >=, <, <=). These fields were already supported on traces; span support was missing.

• OpenTelemetry: in-process spans now linked to the active @opik.track trace — When an OTel-instrumented library (such as logfire or PydanticAI) emits spans from inside an @opik.track-decorated function, those spans are now nested under the active tracked trace rather than starting a separate trace. Distributed flows where parent spans or W3C baggage carry Opik IDs continue to take precedence over the in-process context.

• Optimization: best trial configuration now shows the optimized prompt — The Best Trial Configuration panel was displaying the baseline prompt instead of the prompt produced by the optimizer. It now shows the correct optimized result. The Trials table also gains a Prompt column with per-message formatting and a diff-vs-baseline popover.

• Experiment views: prompt version labels instead of commit hashes — The Experiments table, the single-experiment Configuration tab, and the Dashboard Experiments leaderboard now display prompts as “name (v3)” instead of raw commit hashes, consistent with the display already used in the Prompt Library.

• AI Spend dashboard: total tokens KPI card and onboarding empty state — The placeholder “Budget remaining” card is replaced by a Total tokens KPI showing the sum of all token tiers across models, with a period-over-period trend indicator. The dashboard also shows an onboarding empty state with setup instructions and a ready-to-copy configuration snippet when no trace data has been received yet.

• Cost calculation: tiered pricing above 200k tokens now applied — For models such as gemini-2.5-pro and vertex_ai/claude-sonnet-4-5 that carry above_200k_tokens rate tiers, requests exceeding the 200k-token threshold were being billed at the base input rate. Opik now applies the tier rate when the threshold is crossed (the entire request is billed at the tier price, mirroring LiteLLM’s semantics).

• Cost calculation: Claude on Vertex AI cached tokens now discounted — Claude models on Vertex AI (vertex_ai/claude-haiku-4-5, vertex_ai/claude-sonnet-4-5, vertex_ai/claude-opus-4-1) were having cache-read tokens billed at the full input rate. They now use the Anthropic cache calculator, correctly applying the discount on cached tokens.

• Vertex AI: model selection preserved across provider switches — Switching away from Vertex AI and back in the Playground no longer resets the previously selected model.

• Span timestamp filters use ClickHouse skip indexes — created_at and last_updated_at on the spans and traces tables now have minmax skip indexes. Range filters on these columns prune granules instead of scanning the full project partition, significantly reducing query time and ClickHouse CPU load on large tables.

The Prompt Library is now part of the Opik 2.0 UI, accessible from the project sidebar under Prompt library. Alongside that, prompt versions have gained first-class environment support — you can tag a version as production or staging and retrieve it by name from the SDK, without tracking version numbers in application code.

What’s new:

The Traces, Spans, and Threads tabs now have a redesigned filter bar that makes it faster to narrow down what you’re looking at. Filters appear as chips directly in the toolbar — pick a field, set a value, and the table updates instantly. Frequently-used filters can be pinned to the bar so they’re always one click away, and filter state is preserved in the URL so you can share an exact filtered view with a teammate.

• Test suite assertions: sub-span inspection — the evaluator LLM can now issue get_trace_spans and read tool calls to inspect intermediate spans during evaluation, enabling correctness checks about tool usage, model selection, and per-span errors inside complex agents
• Google ADK integration: images render in trace attachments — URL-safe base64 image data sent by ADK is automatically normalized to standard base64; PNG, JPEG, GIF, and WebP attachments all render correctly
• Optimization trials page: all constituent experiments shown — experiments belonging to multi-project optimizations are now visible from the trials page regardless of which project the user is currently viewing the optimization from
• Error rate KPI: now shows a percentage — the error rate dashboard card was displaying a raw event count; it now shows the rate as a percentage
• Annotation queue: trace logs shown inline — trace log entries are rendered inline on the annotation queue page instead of requiring navigation away
• Online evaluation rules: ClassCastException resolved — thread-level rules that include filters no longer throw a ClassCastException under certain configurations
• Attachments: data URI prefix handled — base64 attachment payloads that include a data:<type>;base64, prefix are now stripped correctly in both the SDK and the frontend
• SDK: built-in environment colors preserved — workspace environments with reserved names retain their designated color after updates or syncs
• opik migrate: skipped items reported clearly — the migration command now reports each skipped item with its reason, count, and sample source IDs, and exits with code 1 so CI pipelines detect incomplete migrations
• Qianfan integration documentation — the Qianfan LLM provider integration now has a dedicated documentation page

Alert rules now support structured condition grouping: conditions within a group are evaluated with AND, while groups themselves are combined with OR. This makes it possible to express logic such as “flag a trace if (hallucination score > 0.8 AND relevance score < 0.3) OR (toxicity score > 0.5)”.

Existing single-condition alerts continue to work exactly as before — each legacy condition is automatically treated as its own group, so no migration is needed.

• Prompt masks (Python & TypeScript SDKs) — prompt_mask_context(masks) / promptMaskContext(masks) lets you run agent code with specific prompt IDs silently redirected to a different version ID, non-destructively. The agent calls get_prompt() as usual and receives the overridden template without any permanent change to the prompt library. Designed for A/B testing and optimizer sweep scenarios.
• Experiments: dataset version shown inline — the dataset version is now displayed as a pill alongside the item source in both the experiments table and the experiment detail header. The standalone “Test suite version” column has been removed; the same information is now visible in context.
• Dataset items: conflicting key names no longer cause errors — iterating a dataset whose items contain a key that matches a DatasetItem field (e.g. id, as in HotpotQA) previously raised TypeError: multiple values for keyword argument. The SDK now strips conflicting keys and emits a one-time warning so iteration completes.
• Harbor integration: supports harbor <0.8 and >=0.8 — track_harbor() now patches whichever method name the installed version of harbor exposes (_setup_environment or _setup_agent_environment), so tracing works regardless of which version is installed.
• New Playground models — Gemini 3.5 Flash and qwen/qwen3.7-max are now available in the model picker.