Getting StartedMCP Server

Use cases

View as Markdown

Once your assistant is connected, these are the things worth asking it for. Paste any prompt as written — your assistant works out which tools and skills it needs, and asks you for anything it’s missing.

Start with Instrument this project. It exercises the whole loop — write code, run it, read the trace back — so if it works, everything is wired up correctly.

Jump to: Set up tracing · Debug a trace · Find your data · Review a project · Analyse experiments · Build evaluations · Review and score

Set up tracing

Your assistant detects the language and framework, adds the decorators or integration, wires up config, then runs the app and reads the trace back to prove it landed.

Instrument this project, end to end
Add Opik tracing to this project, then run it and show me the trace you created.
Trace one entry point
Add Opik tracing to the request handler in this file and mark it as the entry point.
Fix tracing that isn't working
My traces aren't showing up in Opik. Work out why and fix it.

Debug a specific trace

Your assistant reads the full span tree, the tool calls, the errors and the feedback scores, so you never paste a screenshot into chat. Give it a trace URL from the Opik UI, or let it find the trace itself.

Why did it fail?
Why did this trace fail? <paste the trace URL>
Health check
Is this trace healthy? Flag anything that looks wrong even if it didn't error.
Inspect the tool calls
Analyze the tool calls in this trace and identify if there are any issues.
Make it faster or cheaper
How would you optimize this trace? Show me where the time and tokens go.
Fix it and prove it
Fix the bug you just found, re-run the same input, and show me the new trace.

Find the data you care about

Your assistant can search and filter your traces, spans and threads by content, status, duration, token usage, feedback score or time window — so it answers these in one pass instead of you clicking through the UI.

Search by content
Find all the traces with inputs about refunds and summarise what users were asking.
Cluster what users ask
What are the topics my users ask about the most? Group the last week of traces
by topic and rank them.
Find the outliers
Show me the ten slowest traces this week, and the ten that used the most tokens.
Conversations that went wrong
Find threads where the user had to repeat themselves, and show me what went wrong.

Review a project

Ask for a read on a whole project rather than one trace — useful on a Monday morning, before a release, or on a codebase you didn’t write.

Anything worth attention?
Analyze my traces and threads from the last 2 weeks. Are there items worth
attention? Is there anything we can improve?
Status report
Prepare a status report for project X with ranked action items.
Find and fix
Check my last traces, identify the issues, and fix them if you can.

If you have Diagnostics enabled, your assistant can also work its issue queue directly — “show me the open Diagnostics issues and fix the one hitting the most traces”. If it isn’t enabled, ask your assistant to turn it on for a project and run the first scan. Everything else on this page works without it.

Analyse experiment results

If you already run evaluations, your assistant can read the results and act on them, rather than you reading a table and relaying what it says.

Read the results
Analyze the results of my latest experiment. What didn't work well?
Catch regressions
Are there any regressions compared with the previous experiment?
Ask for a fix
Tell me how to improve the score. Then implement it and run the evaluation again.
Hand it the whole loop
Implement the feature and run the evaluation. Fix any regressions. Iterate until
the evaluation passes.

Build an evaluation pipeline

Real traces make better test cases than invented ones, because they are inputs your app actually received.

Start a suite
Create an Opik test suite for my agent, then run it and show me the scores.
Build it from what went wrong
Get the traces that didn't perform well — bad feedback scores or failing — and
create a test suite from them.
Cover a bug you just fixed
Add the trace that reproduced this bug to my test suite as a new case, then re-run
the suite.
Compare two prompt versions
Save the prompt in this file as a new version in Opik, run the suite against both
versions, and tell me which one wins.

Review and score

Score what you found
Score the three worst traces 0.2 with the reason you just gave me.
Leave notes for the team
Add a comment to each of those traces explaining what to look at.

What a session looks like

A typical investigative loop, using Claude Code:

You: Why did the experiment “gpt-4o-rerank-v3” regress on factuality?

Claude: (reads the failing traces) Three traces failed because the reranker dropped the system message. The remaining 12 traces scored above 0.8…

You: Score the bottom 3 traces 0.2 with reason “dropped system message”.

Claude: (writes the scores) Done — three scores recorded on traces <id-1>, <id-2>, <id-3>.

From then on your assistant can check its own work against real traces every time you change something.