Log traces

LLM applications are complex systems that do more than just call an LLM API, they will often involve retrieval, pre-processing and post-processing steps. Tracing is a tool that helps you understand the flow of your application and identify specific points in your application that may be causing issues.

Opik’s tracing functionality allows you to track not just all the LLM calls made by your application but also any of the other steps involved.

Opik supports agent observability using our Typescript SDK, Python SDK, first class OpenTelemetry support and our REST API.

We won’t be covering how to track chat conversations in this guide, you can learn more about this in the Logging conversations guide.

Enable agent observability

1. Installing the SDK

Before adding observability to your application, you will first need to install and configure the Opik SDK.

$
npm install opik

$
# Set OPIK_API_KEY and OPIK_WORKSPACE_NAME in your .env file
>
OPIK_API_KEY=your_api_key_here
>
OPIK_WORKSPACE=your_workspace_name
>
>
# Optional if you are using Opik Cloud:
>
OPIK_URL_OVERRIDE=https://www.comet.com/opik/api

2. Using an integration

Once you have installed and configured the Opik SDK, you can start using it to track your agent calls:

$
npm install opik-openai

$
export OPIK_API_KEY="<your-api-key>" # Only required if you are using the Opik Cloud version
>
export OPIK_URL_OVERRIDE="https://www.comet.com/opik/api" # Cloud version
>
# export OPIK_URL_OVERRIDE="http://localhost:5173/api" # Self-hosting

1
import OpenAI from "openai";
2
import { trackOpenAI } from "opik-openai";
3
4
// Initialize the original OpenAI client
5
const openai = new OpenAI({
6
  apiKey: process.env.OPENAI_API_KEY,
7
});
8
9
// Wrap the client with Opik tracking
10
const trackedOpenAI = trackOpenAI(openai);
11
12
// Use the tracked client just like the original
13
const completion = await trackedOpenAI.chat.completions.create({
14
  model: "gpt-4",
15
  messages: [{ role: "user", content: "Hello, how can you help me today?" }],
16
});
17
console.log(completion.choices[0].message.content);
18
19
// Ensure all traces are sent before your app terminates
20
await trackedOpenAI.flush();

If you would like more control over the logging process, you can use the low-level SDKs to log your traces and spans.

3. Analyzing your agents

Now that you have observability enabled for your agents, you can start to review and analyze the agent calls in Opik. In the Opik UI, you can review each agent call, see the agent graph and review all the tool calls made by the agent.

As a next step, you can create an offline evaluation to evaluate your agent’s performance on a fixed set of samples.

Advanced usage

Using function decorators

Function decorators are a great way to add Opik logging to your existing application. When you add the @track decorator to a function, Opik will create a span for that function call and log the input parameters and function output for that function. If we detect that a decorated function is being called within another decorated function, we will create a nested span for the inner function.

While decorators are most popular in Python, we also support them in our Typescript SDK:

1
import { track } from "opik";
2
3
class TranslationService {
4
    @track({ type: "llm" })
5
    async generateText() {
6
        // Your LLM call here
7
        return "Generated text";
8
    }
9
10
    @track({ name: "translate" })
11
    async translate(text: string) {
12
        // Your translation logic here
13
        return `Translated: ${text}`;
14
    }
15
16
    @track({ name: "process", projectName: "translation-service" })
17
    async process() {
18
        const text = await this.generateText();
19
        return this.translate(text);
20
    }
21
}

Using the low-level SDKs

If you need full control over the logging process, you can use the low-level SDKs to log your traces and spans:

1
import { Opik } from "opik";
2
3
const client = new Opik({
4
    apiUrl: "https://www.comet.com/opik/api",
5
    apiKey: "your-api-key", // Only required if you are using Opik Cloud
6
    projectName: "your-project-name",
7
    workspaceName: "your-workspace-name", // Only required if you are using Opik Cloud
8
});
9
10
// Log a trace with an LLM span
11
const trace = client.trace({
12
    name: `Trace`,
13
    input: {
14
        prompt: `Hello!`,
15
    },
16
    output: {
17
        response: `Hello, world!`,
18
    },
19
});
20
21
const span = trace.span({
22
    name: `Span`,
23
    type: "llm",
24
    input: {
25
        prompt: `Hello, world!`,
26
    },
27
    output: {
28
        response: `Hello, world!`,
29
    },
30
});
31
32
// Flush the client to send all traces and spans
33
await client.flush();

Logging traces/spans using context managers

If you are using the low-level SDKs, you can use the context managers to log traces and spans. Context managers provide a clean and Pythonic way to manage the lifecycle of traces and spans, ensuring proper cleanup and error handling.

1
import opik
2
3
# Basic trace creation
4
with opik.start_as_current_trace("my-trace", project_name="my-project") as trace:
5
    # Your application logic here
6
    trace.input = {"user_query": "What is the weather?"}
7
    trace.output = {"response": "It's sunny today!"}
8
    trace.tags = ["weather", "api-call"]
9
    trace.metadata = {"model": "gpt-4", "temperature": 0.7}

1
import opik
2
3
# Basic span creation
4
with opik.start_as_current_span("llm-call", type="llm", project_name="my-project") as span:
5
    # Your LLM call here
6
    span.input = {"prompt": "Explain quantum computing"}
7
    span.output = {"response": "Quantum computing is..."}
8
    span.model = "gpt-4"
9
    span.provider = "openai"
10
    span.usage = {
11
        "prompt_tokens": 10,
12
        "completion_tokens": 50,
13
        "total_tokens": 60
14
    }

1
import opik
2
3
with opik.start_as_current_trace("chatbot-conversation", project_name="chatbot") as trace:
4
    trace.input = {"user_message": "Help me with Python"}
5
    
6
    # First span: Process user input
7
    with opik.start_as_current_span("process-input", type="general") as span:
8
        span.input = {"raw_input": "Help me with Python"}
9
        span.output = {"processed_input": "Python programming help request"}
10
    
11
    # Second span: Generate response
12
    with opik.start_as_current_span("generate-response", type="llm") as span:
13
        span.input = {"prompt": "Python programming help request"}
14
        span.output = {"response": "I'd be happy to help with Python!"}
15
        span.model = "gpt-4"
16
        span.provider = "openai"
17
    
18
    trace.output = {"final_response": "I'd be happy to help with Python!"}

1
import opik
2
3
try:
4
    with opik.start_as_current_trace("risky-operation", project_name="my-project") as trace:
5
        trace.input = {"data": "important data"}
6
        # This will raise an exception
7
        result = 1 / 0
8
        trace.output = {"result": result}
9
except ZeroDivisionError:
10
    # The trace is still properly closed and logged
11
    print("Error occurred, but trace was logged")

1
import opik
2
3
# Parameters set outside the context manager
4
with opik.start_as_current_trace(
5
    "dynamic-trace",
6
    input={"initial": "data"},
7
    tags=["initial-tag"],
8
    project_name="my-project"
9
) as trace:
10
    # Override parameters inside the context manager
11
    trace.input = {"updated": "data"}
12
    trace.tags = ["updated-tag", "new-tag"]
13
    trace.metadata = {"custom": "metadata"}
14
    
15
    # The final trace will use the updated values

1
import opik
2
3
# Immediate flush
4
with opik.start_as_current_trace("immediate-trace", flush=True) as trace:
5
    trace.input = {"data": "important"}
6
    # Data is sent immediately when exiting the context
7
8
# Deferred flush (default)
9
with opik.start_as_current_trace("deferred-trace", flush=False) as trace:
10
    trace.input = {"data": "less urgent"}
11
    # Data will be sent asynchronously later or when the program exits

Best Practices

1. Use descriptive names: Choose clear, descriptive names for your traces and spans that explain what they represent.

2. Set appropriate types: Use the correct span types (“llm”, “retrieval”, “general”, etc.) to help with filtering and analysis.

3. Include relevant metadata: Add metadata that will be useful for debugging and analysis, such as model names, parameters, and custom metrics.

4. Handle errors gracefully: Let the context manager handle cleanup, but ensure your application logic handles errors appropriately.

5. Use project organization: Organize your traces by project to keep your Opik dashboard clean and organized.

6. Consider performance: Use flush=True only when immediate data availability is required, as it can slow down your application by triggering a synchronous, immediate data upload.

Logging to a specific project

By default, traces are logged to the Default Project project. You can change the project you want the trace to be logged to in a couple of ways:

1
import { Opik } from "opik";
2
3
const client = new Opik({
4
    projectName: "my_project",
5
    // apiKey: "my_api_key",
6
    // apiUrl: "https://www.comet.com/opik/api",
7
    // workspaceName: "my_workspace",
8
});

Flushing traces and spans

This process is optional and is only needed if you are running a short-lived script or if you are debugging why traces and spans are not being logged to Opik.

1
import { Opik } from "opik";
2
3
const client = new Opik();
4
client.flush();

Disabling the logging process

Next steps

Once you have the observability set up for your agent, you can go one step further and:

• Logging chat conversations
• Logging user feedback
• Setup online evaluation metrics