Observability for OpenAI with Opik

This guide explains how to integrate Opik with the OpenAI Python SDK. By using the track_openai method provided by opik, you can easily track and evaluate your OpenAI API calls within your Opik projects as Opik will automatically log the input prompt, model used, token usage, and response generated.

Account Setup

Comet provides a hosted version of the Opik platform, simply create an account and grab your API Key.

You can also run the Opik platform locally, see the installation guide for more information.

Getting Started

Installation

First, ensure you have both opik and openai packages installed:

$
pip install opik openai

Configuring Opik

Configure the Opik Python SDK for your deployment type. See the Python SDK Configuration guide for detailed instructions on:

• CLI configuration: opik configure
• Code configuration: opik.configure()
• Self-hosted vs Cloud vs Enterprise setup
• Configuration files and environment variables

Configuring OpenAI

In order to configure OpenAI, you will need to have your OpenAI API Key. You can find or create your OpenAI API Key in this page.

You can set it as an environment variable:

$
export OPENAI_API_KEY="YOUR_API_KEY"

Or set it programmatically:

1
import os
2
import getpass
3
4
if "OPENAI_API_KEY" not in os.environ:
5
    os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter your OpenAI API key: ")

Logging LLM calls

In order to log the LLM calls to Opik, you will need to wrap the OpenAI client with track_openai. When making calls with that wrapped client, all calls will be logged to Opik:

1
from opik.integrations.openai import track_openai
2
from openai import OpenAI
3
import os
4
5
os.environ["OPIK_PROJECT_NAME"] = "openai-integration-demo"
6
7
client = OpenAI()
8
openai_client = track_openai(client)
9
10
prompt = """
11
Write a short two sentence story about Opik.
12
"""
13
14
completion = openai_client.chat.completions.create(
15
    model="gpt-3.5-turbo",
16
    messages=[{"role": "user", "content": prompt}]
17
)
18
19
print(completion.choices[0].message.content)

Advanced Usage

Using with the @track decorator

If you have multiple steps in your LLM pipeline, you can use the @track decorator to log the traces for each step. If OpenAI is called within one of these steps, the LLM call will be associated with that corresponding step:

1
from opik import track
2
from opik.integrations.openai import track_openai
3
from openai import OpenAI
4
5
os.environ["OPIK_PROJECT_NAME"] = "openai-integration-demo"
6
7
client = OpenAI()
8
openai_client = track_openai(client)
9
10
@track
11
def generate_story(prompt):
12
    res = openai_client.chat.completions.create(
13
        model="gpt-3.5-turbo", messages=[{"role": "user", "content": prompt}]
14
    )
15
    return res.choices[0].message.content
16
17
@track
18
def generate_topic():
19
    prompt = "Generate a topic for a story about Opik."
20
    res = openai_client.chat.completions.create(
21
        model="gpt-3.5-turbo", messages=[{"role": "user", "content": prompt}]
22
    )
23
    return res.choices[0].message.content
24
25
@track
26
def generate_opik_story():
27
    topic = generate_topic()
28
    story = generate_story(topic)
29
    return story
30
31
# Execute the multi-step pipeline
32
generate_opik_story()

The trace can now be viewed in the UI with hierarchical spans showing the relationship between different steps:

Using Azure OpenAI

The OpenAI integration also supports Azure OpenAI Services. To use Azure OpenAI, initialize your client with Azure configuration and use it with track_openai just like the standard OpenAI client:

1
from opik.integrations.openai import track_openai
2
from openai import AzureOpenAI
3
4
# gets the API Key from environment variable AZURE_OPENAI_API_KEY
5
azure_client = AzureOpenAI(
6
    # https://learn.microsoft.com/azure/ai-services/openai/reference#rest-api-versioning
7
    api_version="2023-07-01-preview",
8
    # https://learn.microsoft.com/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource
9
    azure_endpoint="https://example-endpoint.openai.azure.com",
10
)
11
azure_client = track_openai(azure_client)
12
13
completion = azure_client.chat.completions.create(
14
    model="deployment-name",  # e.g. gpt-35-instant
15
    messages=[
16
        {
17
            "role": "user",
18
            "content": "How do I output all files in a directory using Python?",
19
        },
20
    ],
21
)

Cost Tracking

The track_openai wrapper automatically tracks token usage and cost for all supported OpenAI models.

Cost information is automatically captured and displayed in the Opik UI, including:

• Token usage details
• Cost per request based on OpenAI pricing
• Total trace cost

Grouping traces into conversational threads using thread_id

Threads in Opik are collections of traces that are grouped together using a unique thread_id.

The thread_id can be passed to the OpenAI client as a parameter, which will be used to group all traces into a single thread.

1
import openai
2
3
from opik.integrations.openai import track_openai
4
5
client = openai.OpenAI()
6
wrapped_client = track_openai(
7
    openai_client=client,
8
    project_name="opik_args demo",
9
)
10
messages = [
11
    {"role": "system", "content": "You are a helpful assistant."},
12
    {"role": "user", "content": "What is Opik?"},
13
]
14
15
_ = wrapped_client.responses.create(
16
    model="gpt-4o-mini",
17
    input=messages,
18
    opik_args={"trace": {"thread_id": "f174a"}}
19
)

More information on logging chat conversations can be found in the Log conversations section.

Supported OpenAI methods

The track_openai wrapper supports the following OpenAI methods:

• openai_client.chat.completions.create(), including support for stream=True mode.
• openai_client.beta.chat.completions.parse()
• openai_client.beta.chat.completions.stream()
• openai_client.responses.create()

If you would like to track another OpenAI method, please let us know by opening an issue on GitHub.