Dashboards

Dashboards chart your traces, threads, spans, and experiment scores so you can see how an agent behaves over time. You can use them in three places:

• Project dashboards: open a project and select Dashboards under Observability in the sidebar. Every project includes the built-in Project overview dashboard, and you can add your own views.
• Workspace dashboards: select Dashboards in the workspace sidebar to build dashboards that span several projects or compare experiments.
• Experiment insights: the Insights tab of the experiment comparison page charts the scores of the selected experiments.

Project dashboards

The Dashboards page of a project opens on Project overview, a read-only dashboard with three sections:

• At a glance: trace count, error count, p50 and p99 latency, and total estimated cost.
• Volume, latency & cost: trace volume, trace duration, and estimated cost over time.
• Quality & conversations: trace feedback scores, thread volume, thread duration, and thread feedback scores.

Custom views

To build your own view of a project:

1. Open the view selector at the top of the page (it shows Project overview by default).
2. Click Add new.
3. Enter a name and click Create view.

Widgets in a project view always read data from the current project, so they have no project selector. To start from the built-in charts, hover Project overview in the view selector and click Duplicate. The copy is fully editable.

Hover a custom view in the selector to Edit its name, Duplicate it, or Delete it.

Workspace dashboards

Workspace dashboards are not tied to a project. Each widget picks its own data, so one dashboard can follow several agents side by side.

Every workspace dashboard has a type, which sets the widgets you can add:

Creating a workspace dashboard

1. Select Dashboards in the workspace sidebar.
2. Click Create dashboard.
3. Enter a Name and pick a Type: Multi-project dashboard or Experiments dashboard.
4. Optionally, expand Description and add one.
5. Click Create dashboard.

The new dashboard opens with an empty Overview section, ready for its first widget.

The Dashboards page lists every workspace dashboard with its type and when it was last updated. Use the row menu to Edit the name and description, Clone the dashboard, or Delete it, and filter the list by Type.

Experiment insights

On the experiment comparison page, the Insights tab shows Experiments overview, a read-only dashboard for the experiments you selected. It contains a radar chart and a bar chart of their feedback scores. The tab isn’t available for test suite experiments.

To track experiments over time or build your own charts, create an Experiments workspace dashboard.

Adding widgets

1. Click Add widget in the header of a section, or Ask Ollie to describe a custom one (see Custom widgets with Ollie).
2. Pick a Widget type. The list depends on the dashboard type.
3. Fill in the settings for that widget type. The Widget preview on the right updates as you go.
4. Optionally, click the title or Click to add a description in the preview to rename the widget or describe it.
5. Click Add widget.

Widget types

Time series

Charts a project metric over time as a Line chart or a Bar chart.

Settings:

• Project (workspace dashboards only): one or more projects, or All projects in the workspace. With more than one project, the only metric available is Span token usage.

• Metric type:

  • Traces: Trace metrics (feedback scores), Number of traces, Trace duration, Token usage, Estimated cost, Failed guardrails
  • Threads: Number of threads, Thread duration, Thread estimated cost, Thread metrics
  • Spans: Number of spans, Span duration, Span estimated cost, Span metrics, Span token usage

• Metrics, Duration metrics, or Usage metrics: pick the scores, percentiles (P50, P90, P99), or token counts to plot. Leave the field empty to plot all of them.

• Filters: limit the data to matching traces, threads, or spans, for example by tag, metadata, or feedback score.

• Group by: split the metric into one series per value of a field:

  • Trace metrics: Tags, Metadata (with a key), Name, Has error, Error type
  • Failed guardrails: the trace fields plus Guardrail name
  • Span metrics: the trace fields plus Model, Provider, and Span type
  • Thread metrics: Tags
  • Thread estimated cost can’t be grouped

  To group a score, duration, or token metric, select exactly one score, percentile, or token count first. Set Aggregation mode to Time-based for one value per time bucket, or to Total for one value per group over the whole date range.

  Cost by tag uses overlapping groups. Each tag receives the full cost of its matching spans. For example, a $1 span tagged `prod` and `agent` contributes$1 to each group. Adding the groups (including in a stacked chart) can therefore exceed actual spend. Use Provider, Model, or Span type for span cost groups that add up to total spend. Untagged spans appear as Unknown. When there are more than ten groups, Others sums the remaining groups’ costs once per time bucket; tag overlap still applies.

Click a series name in the legend to open the project Logs filtered to that series.

Single metric

Shows one number, such as a trace count, a latency percentile, or a total cost.

Settings:

• Project (workspace dashboards only): one project, several projects, or All projects in the workspace. With more than one project, the metric is Total span token usage and you pick the Usage metric.
• Source: Traces or Spans.
• Metric type:
  • Traces only: Total trace count, Total thread count, Average LLM span count, Average span count, Average estimated cost per trace, Total guardrails failed count
  • Spans only: Total span count, Average estimated cost per span
  • Both: P50 duration, P90 duration, P99 duration, Total input count, Total output count, Total metadata count, Average number of tags, Total estimated cost sum, Avg input tokens, Avg output tokens, Avg total tokens, Total error count, and an Average entry for each feedback score
• Filters: limit the data to matching traces or spans.

Metrics

Compares experiment feedback scores as a Bar chart (default), Line chart, or Radar chart. Available on Experiments dashboards.

Settings:

• Metrics: the scores to show. Leave it empty to show all of them.
• Filter experiments: target experiments by Experiments, Item source (the dataset or test suite), Tags, or Configuration (experiment configuration keys such as model). Leave it empty to include all experiments.
• Group by: aggregate scores by Item source, Tags, or Configuration, up to five levels. Grouping isn’t available when you filter by specific experiments.
• Max experiments: how many experiments to load, up to 100. The default is 10.

Leaderboard

Ranks experiments in a table. Available on Experiments dashboards.

Settings:

• Filter experiments and Max experiments: the same as the Metrics widget.
• Columns: pick and reorder columns, such as Item source, Avg duration, Total estimated cost, Pass rate, experiment configuration keys, and feedback scores.
• Ranking metric: a feedback score to rank by, or No ranking.
• Ranking order: High first or Low first.

Markdown

Adds text, links, and notes to a dashboard, for example a description of what the dashboard tracks or an on-call checklist. Available on all dashboards.

Ollie chart

A chart or table that Ollie built from a description you typed, backed by a query over your workspace data. It isn’t in the Pick a chart list: you create one by asking Ollie, as described in Custom widgets with Ollie. Available on all dashboards on Opik Cloud.

Custom widgets with Ollie

When no built-in widget shows what you need, describe the chart to Ollie. Ollie writes a read-only query over your workspace’s traces, spans, threads, feedback scores, experiments, and datasets, draws the chart in the chat, and adds it to a dashboard as an Ollie chart widget. The widget keeps the query and re-runs it whenever the dashboard loads, so the chart stays live.

Ask Ollie for a widget

How an Ollie chart widget works

• The title carries an owl badge, Created with Ollie. The header shows when the chart was last computed, with a refresh icon next to it.
• Each time the dashboard loads, the widget re-runs its query. On project dashboards and Multi-project dashboards it uses the dashboard’s date range, so changing the range changes the chart. Experiments dashboards have no date range, so the widget shows whatever period the query covers.
• The widget’s … menu offers Edit in Ollie, Refresh, Duplicate, and Remove.
• The query is read-only, scoped to your workspace, and only Ollie can create or change it.

Editing an Ollie chart

1. Open the widget’s … menu and click Edit in Ollie.
2. Ollie opens with the current chart pinned at the top of the chat, marked On the dashboard now. Describe what to change, for example “show it as a line chart” or “group by provider instead of model”.
3. Under the redrawn chart, click Save changes to replace the widget in place, keeping its position and size, or Duplicate to add the new version next to the original.

If Ollie isn’t available, the menu shows Edit instead, which opens the chart’s Vega-Lite spec for manual changes.

When a widget shows a message instead of a chart

Organizing a dashboard

Sections group widgets:

• Click Add section at the bottom of the dashboard to add one.
• Hover a section title and click the pencil icon to rename it.
• Click a section header to collapse or expand it. Opik remembers the state.
• Use the section … menu to Insert section above, Insert section below, or Delete section. A dashboard always keeps at least one section.
• Drag the handle on the right of a section header to reorder sections.

Widgets sit on a six-column grid:

• Drag a widget to move it, and drag its edge to resize it.
• Use the widget … menu to Edit widget, Duplicate widget, Move to section, or Delete widget. An Ollie chart’s menu offers Edit in Ollie, Refresh, Duplicate, and Remove instead.

Date range

Project dashboards and Multi-project dashboards have a date range picker in the toolbar. Pick Past 24 hours, Past 3 days, Past 7 days, Past 30 days, Past 60 days, or Custom dates. The default is the past 30 days.

Time series, Single metric, and Ollie chart widgets use the date range. Experiments dashboards and experiment insights have no date range: they always show the selected experiments.

Saving and permissions

Opik saves every change automatically, and the toolbar shows Saving… and then Saved. There is no save button.

Project overview and Experiments overview are read-only. Duplicate Project overview to get an editable copy. Users without edit permission see every dashboard as read-only.

Sharing a dashboard

Click Share to copy a link to the dashboard. The link includes the selected date range, so your teammates see the same view. They need access to the workspace to open it.

Next steps

• Set up online evaluation rules to score production traces and chart the scores on your dashboards.
• Configure alerts to get notified when a metric crosses a threshold.
• Read Production monitoring for more ways to monitor agents in production.
• Meet Ollie, the assistant that builds custom widgets and helps you debug the traces behind them.