Getting started with Agent Configuration

Agents depend on prompts, model settings, and tool definitions that change frequently. Opik lets you manage all of these outside your codebase, update them without redeploying, and keep a full history of every change.

Adding Agent Configuration to your code

$
npx skills add comet-ml/opik-skills

Version my prompts and agent parameters in Opik using the /instrument:agent-configuration command.

Managing Agent Configuration versions

You can update your agent’s behavior through the Opik platform without triggering a new code release. This works through environment labels and version pinning.

By default the SDK fetches the version labeled prod, but you can control which version is fetched using two parameters:

• env — Fetch by environment label:
  • "prod" (default): The version labeled as production
  • "staging", "canary", or any custom label you create in the Opik UI
• version — Fetch by version name:
  • "latest": The most recently created version
  • "v3" (or any version name): A specific pinned version

Pass the parameter when fetching your config:

1
@opik.track(project_name="my-agent")
2
def run_agent(user_input: str):
3
    # Fetch the staging version
4
    cfg = client.get_or_create_config(
5
        fallback=MyConfig(
6
            model="gpt-4o-mini",
7
            temperature=0.7,
8
            system_prompt=opik.Prompt(
9
                name="system_prompt",
10
                prompt="You are a helpful assistant.",
11
            ),
12
        ),
13
        env="staging",
14
    )
15
    # ...

Supported configuration parameter types

You can version prompts, model settings, tool definitions, and any other parameter your agent relies on. Here are the supported types:

Prompts

Use opik.Prompt to version text prompts. Prompts support variable substitution using {{variable_name}} syntax, allowing you to define templates that are filled in at runtime.

1
system_prompt=opik.Prompt(
2
  name="system_prompt",
3
  prompt="You are a helpful assistant specializing in {{domain}}."
4
)

Chat Prompts

Use opik.ChatPrompt to version a list of chat messages. This is preferred over opik.Prompt when your agent uses a multi-turn message format with system, user, and assistant roles.

1
chat_prompt=opik.ChatPrompt(
2
  name="chat_prompt",
3
  messages=[
4
    {"role": "system", "content": "You are a helpful assistant."},
5
    {"role": "user", "content": "{{user_query}}"},
6
  ]
7
)

Strings

Use strings to version model names, tool definitions, or any other text parameter.

1
class MyConfig(opik.Config):
2
    model: str = "gpt-4o-mini"
3
    tool_definition: str = "{}"

Floats

Use floats to version numeric parameters like LLM sampling settings or RAG retrieval thresholds.

1
class MyConfig(opik.Config):
2
    temperature: float = 0.7
3
    similarity_threshold: float = 0.8