Prompt management

Opik provides a prompt library that you can use to manage your prompts. Storing prompts in a library allows you to version them, reuse them across projects, and manage them in a central location.

Using a prompt library does not mean you can’t store your prompt in code, we have designed the prompt library to work seamlessly with your existing prompt files while providing the benefits of a central prompt library.

Opik supports two types of prompts:

• Text Prompts: Simple string-based prompts with variable substitution
• Chat Prompts: Structured message-based prompts in OpenAI format for conversational AI applications, supporting multimodal content (text, images, videos)

Text Prompts

Text prompts are simple string-based templates that support variable substitution. They are ideal for single-turn interactions or when you need to generate a single piece of text.

Managing text prompts stored in code

The recommended way to create and manage text prompts is using the Prompt object. This will allow you to continue versioning your prompts in code while also getting the benefit of having prompt versions managed in the Opik platform so you can more easily keep track of your progress.

1
import opik
2
3
# Prompt text stored in a variable
4
PROMPT_TEXT = "Write a summary of the following text: {{text}}"
5
6
# Create a prompt
7
prompt = opik.Prompt(
8
    name="prompt-summary",
9
    prompt=PROMPT_TEXT,
10
    metadata={"environment": "production"}
11
)
12
13
# Print the prompt text
14
print(prompt.prompt)
15
16
# Build the prompt
17
print(prompt.format(text="Hello, world!"))

The prompt will now be stored in the library and versioned:

Using the low level SDK for text prompts

If you would rather keep text prompts in the Opik platform and manually update / download them, you can use the low-level Python SDK to manage your prompts.

Creating text prompts

You can create a new prompt in the library using both the SDK and the UI:

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Create a new prompt
7
prompt = client.create_prompt(name="prompt-summary", prompt="Write a summary of the following text: {{text}}", metadata={"environment": "development"})

Adding prompts to traces and spans

You can associate prompts with your traces and spans using the opik_context module. This is useful when you want to track which prompts were used during the execution of your functions:

1
import opik
2
from opik.opik_context import update_current_trace
3
4
# Create prompts
5
system_prompt = opik.Prompt(
6
    name="system-prompt",
7
    prompt="You are a helpful assistant that provides accurate and concise answers."
8
)
9
10
# Get prompt from the Prompt library
11
client = opik.Opik()
12
user_prompt = client.get_prompt(name="user-prompt")
13
14
@opik.track
15
def process_user_query(question: str) -> str:
16
    # Add prompts to the current trace
17
    update_current_trace(
18
        name="user-query-processing",
19
        prompts=[system_prompt, user_prompt],
20
        metadata={"query_type": "general"}
21
    )
22
    
23
    # Your processing logic here
24
    formatted_prompt = user_prompt.format(question=question)
25
    # ... rest of your function
26
    return "Response to: " + question

You can view the prompts associated with a trace or span in the Opik UI:

Further details on using prompts from the Prompt library are provided in the following sections.

Using prompts in supported integrations

Prompts can be used in all supported third-party integrations by attaching them to traces and spans through the opik_context module.

For instance, you can use prompts with the Google ADK integration, as shown in the example here.

Downloading your text prompts

Once a prompt is created in the library, you can download it in code using the Opik.get_prompt method:

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Get a dataset
7
dataset = client.get_or_create_dataset("test_dataset")
8
9
# Get the prompt
10
prompt = client.get_prompt(name="prompt-summary")
11
12
# Create the prompt message
13
prompt.format(text="Hello, world!")

If you are not using the SDK, you can download a prompt by using the REST API.

Searching prompts

To discover prompts by name substring and/or filters, use search_prompts. Filters are written in Opik Query Language (OQL):

1
import opik
2
3
client = opik.Opik()
4
5
# Search by name substring only
6
latest_versions = client.search_prompts(
7
    filter_string='name contains "summary"'
8
)
9
10
# Search by name substring and tags filter
11
filtered = client.search_prompts(
12
    filter_string='name contains "summary" AND tags contains "alpha" AND tags contains "beta"',
13
)
14
15
# Search for only text prompts
16
text_prompts = client.search_prompts(
17
    filter_string='template_structure = "text"'
18
)
19
20
for prompt in filtered:
21
    print(prompt.name, prompt.commit, prompt.prompt)

You can filter by template_structure to search for only text prompts ("text") or only chat prompts ("chat"). Without the filter, search_prompts returns both types.

The filter_string parameter uses Opik Query Language (OQL) with the format: "<COLUMN> <OPERATOR> <VALUE> [AND <COLUMN> <OPERATOR> <VALUE>]*"

Supported columns for prompts:

Examples:

• tags contains "production" - Filter by tag
• name contains "summary" - Filter by name substring
• created_by = "user@example.com" - Filter by creator
• tags contains "alpha" AND tags contains "beta" - Multiple tag filtering
• template_structure = "text" - Filter for only text prompts
• template_structure = "chat" - Filter for only chat prompts

search_prompts returns the latest version for each matching prompt.

Chat Prompts

Chat prompts are structured message-based templates designed for conversational AI applications. They support multiple message roles (system, user, assistant) and multimodal content including text, images, and videos.

Key Features

• Structured Messages: Organize prompts as a list of messages with roles (system, user, assistant)
• Multimodal Support: Include images, videos, and text in the same prompt
• Variable Substitution: Use Mustache ({{variable}}) or Jinja2 syntax
• Version Control: Automatic versioning when messages change
• Template Validation: Optional validation of template placeholders

Managing chat prompts stored in code

Similar to text prompts, you can create and manage chat prompts using the ChatPrompt class. This allows you to version your chat prompts in code while benefiting from centralized management in the Opik platform.

1
import opik
2
3
# Define chat messages with variables
4
messages = [
5
    {"role": "system", "content": "You are a helpful assistant specializing in {{domain}}."},
6
    {"role": "user", "content": "Explain {{topic}} in simple terms."}
7
]
8
9
# Create a chat prompt
10
chat_prompt = opik.ChatPrompt(
11
    name="educational-assistant",
12
    messages=messages,
13
    metadata={"category": "education"}
14
)
15
16
# Format the prompt with variables
17
formatted_messages = chat_prompt.format(
18
    variables={
19
        "domain": "physics",
20
        "topic": "quantum entanglement"
21
    }
22
)
23
24
# Use formatted messages with your LLM
25
print(formatted_messages)
26
# Output:
27
# [
28
#     {"role": "system", "content": "You are a helpful assistant specializing in physics."},
29
#     {"role": "user", "content": "Explain quantum entanglement in simple terms."}
30
# ]

The chat prompt will now be stored in the library and versioned, just like text prompts.

Once created, you can view and manage your chat prompts in the Opik UI:

Multimodal chat prompts

Chat prompts support multimodal content, allowing you to include images and videos alongside text. This is useful for vision-enabled models.

1
import opik
2
3
# Chat prompt with image content
4
messages = [
5
    {"role": "system", "content": "You analyze images and provide detailed descriptions."},
6
    {
7
        "role": "user",
8
        "content": [
9
            {"type": "text", "text": "What's in this image of {{subject}}?"},
10
            {
11
                "type": "image_url",
12
                "image_url": {
13
                    "url": "{{image_url}}",
14
                    "detail": "high"
15
                }
16
            }
17
        ]
18
    }
19
]
20
21
chat_prompt = opik.ChatPrompt(
22
    name="image-analyzer",
23
    messages=messages
24
)
25
26
# Format with variables
27
formatted = chat_prompt.format(
28
    variables={
29
        "subject": "a sunset",
30
        "image_url": "https://example.com/sunset.jpg"
31
    },
32
    supported_modalities={"vision": True}
33
)

Using the low level SDK for chat prompts

You can also use the low-level Python SDK to create and manage chat prompts directly.

Creating chat prompts

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Create a new chat prompt
7
messages = [
8
    {"role": "system", "content": "You are a helpful assistant."},
9
    {"role": "user", "content": "Hello, {{name}}!"}
10
]
11
12
chat_prompt = client.create_chat_prompt(
13
    name="greeting-prompt",
14
    messages=messages,
15
    metadata={"environment": "development"}
16
)
17
18
# Format and use the prompt
19
formatted = chat_prompt.format(variables={"name": "Alice"})
20
print(formatted)

Downloading chat prompts

Once a chat prompt is created in the library, you can download it in code using the Opik.get_chat_prompt method:

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Get a chat prompt
7
chat_prompt = client.get_chat_prompt(name="greeting-prompt")
8
9
# Format the messages
10
formatted_messages = chat_prompt.format(variables={"name": "Bob"})
11
12
# Use with your LLM
13
# response = llm.chat(messages=formatted_messages)

Searching chat prompts

You can search for chat prompts specifically by using the template_structure filter with Opik.search_prompts:

1
import opik
2
3
client = opik.Opik()
4
5
# Search for only chat prompts
6
chat_prompts = client.search_prompts(
7
    filter_string='template_structure = "chat" AND name contains "assistant"'
8
)
9
10
for prompt in chat_prompts:
11
    print(f"Chat prompt: {prompt.name}")
12
    print(f"Messages: {prompt.template}")

To search for text prompts only, use template_structure = "text". Without the filter, search_prompts returns both text and chat prompts.

The filter_string parameter uses Opik Query Language (OQL) and supports the same columns and operators as text prompts (see Searching prompts above).

Template types for chat prompts

Chat prompts support two template types for variable substitution, specified using PromptType:

1
import opik
2
from opik.api_objects.prompt import PromptType
3
4
messages = [
5
    {"role": "user", "content": "Hello {{name}}, you live in {{city}}."}
6
]
7
8
chat_prompt = opik.ChatPrompt(
9
    name="mustache-example",
10
    messages=messages,
11
    type=PromptType.MUSTACHE  # Default
12
)
13
14
formatted = chat_prompt.format(variables={"name": "Alice", "city": "Paris"})
15
# Result: [{"role": "user", "content": "Hello Alice, you live in Paris."}]

Chat prompt versioning

Chat prompts are automatically versioned when the messages change. Each version has a unique commit hash.

1
import opik
2
3
# Create initial version
4
messages_v1 = [
5
    {"role": "system", "content": "You are helpful."},
6
    {"role": "user", "content": "Hi!"}
7
]
8
9
chat_prompt_v1 = opik.ChatPrompt(
10
    name="assistant-prompt",
11
    messages=messages_v1
12
)
13
14
print(f"Version 1 commit: {chat_prompt_v1.commit}")
15
16
# Create new version with different messages
17
messages_v2 = [
18
    {"role": "system", "content": "You are very helpful."},
19
    {"role": "user", "content": "Hello there!"},
20
    {"role": "assistant", "content": "How can I assist you?"}
21
]
22
23
chat_prompt_v2 = opik.ChatPrompt(
24
    name="assistant-prompt",
25
    messages=messages_v2
26
)
27
28
print(f"Version 2 commit: {chat_prompt_v2.commit}")
29
30
# Get specific version by commit
31
client = opik.Opik()
32
specific_version = client.get_chat_prompt(
33
    name="assistant-prompt",
34
    commit=chat_prompt_v1.commit
35
)
36
37
# Get version history
38
history = client.get_chat_prompt_history(name="assistant-prompt")
39
print(f"Total versions: {len(history)}")

You can use get_chat_prompt to retrieve a specific version by commit hash, and get_chat_prompt_history to get all versions.

Prompt structure immutability

Once you create a prompt with a specific structure (text or chat), that structure cannot be changed. This ensures consistency and prevents accidental mixing of prompt types.

1
import opik
2
3
# Create a chat prompt
4
chat_prompt = opik.ChatPrompt(
5
    name="my-prompt",
6
    messages=[{"role": "user", "content": "Hello"}]
7
)
8
9
# Attempting to create a text prompt with the same name will raise an error
10
try:
11
    text_prompt = opik.Prompt(
12
        name="my-prompt",
13
        prompt="Hello {{name}}"
14
    )
15
except opik.exceptions.PromptTemplateStructureMismatch as e:
16
    print("Error: Cannot change prompt structure from chat to text")

Similarly, if you create a text prompt first, you cannot later create a chat prompt with the same name.

Both Prompt and ChatPrompt classes will raise a PromptTemplateStructureMismatch exception if you attempt to change the structure of an existing prompt.

Working with prompt versions

Viewing prompt history (all versions)

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Get the complete version history for a text prompt
7
prompt_history = client.get_prompt_history(name="prompt-summary")
8
9
# Iterate through all versions
10
for version in prompt_history:
11
    print(f"Name: {version.name}")
12
    print(f"Commit: {version.commit}")
13
    print(f"Prompt text: {version.prompt}")
14
    print(f"Metadata: {version.metadata}")
15
    print(f"Type: {version.type}")
16
    print("-" * 50)

You can use this information to:

• Audit changes to understand how prompts evolved
• Identify the best performing version by linking commit IDs to experiment results
• Document prompt changes for compliance or review purposes
• Retrieve specific versions by commit ID for testing or rollback

Accessing specific prompt versions

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Get a specific version of a text prompt by commit ID
7
prompt = client.get_prompt(name="prompt-summary", commit="abc123def456")
8
9
# Use the prompt in your application
10
formatted_prompt = prompt.format(text="Hello, world!")
11
print(formatted_prompt)

The commit parameter accepts the commit ID (also called commit hash) of the specific prompt version you want to retrieve. You can find commit IDs in the prompt history in the Opik UI or by using the get_prompt_history or get_chat_prompt_history methods (see above).

This is particularly useful when you want to:

• Pin to a specific version in production to ensure consistent behavior
• Test different versions side by side in experiments
• Roll back to a previous version if issues are discovered
• Compare results across different prompt versions

Using prompts in experiments

Linking prompts to experiments

1
import opik
2
from opik.evaluation import evaluate
3
from opik.evaluation.metrics import Hallucination
4
from openai import OpenAI
5
6
opik.configure()
7
opik_client = opik.Opik()
8
openai_client = OpenAI()
9
10
# Get a dataset
11
dataset = opik_client.get_or_create_dataset("test_dataset")
12
13
# Create a text prompt
14
prompt = opik.Prompt(name="My prompt", prompt="Summarize: {{text}}")
15
16
# Create an evaluation task
17
def evaluation_task(dataset_item):
18
    # Use the prompt in your task
19
    formatted_prompt = prompt.format(text=dataset_item["input"])
20
    
21
    # Call OpenAI API with formatted prompt
22
    response = openai_client.chat.completions.create(
23
        model="gpt-4",
24
        messages=[{"role": "user", "content": formatted_prompt}]
25
    )
26
    
27
    return {"output": response.choices[0].message.content}
28
29
# Run the evaluation
30
evaluation = evaluate(
31
    experiment_name="My experiment",
32
    dataset=dataset,
33
    task=evaluation_task,
34
    prompts=[prompt],
35
)

The experiment will now be linked to the prompt, allowing you to view all experiments that use a specific prompt:

Comparing prompt versions in experiments

1
import opik
2
from opik.evaluation import evaluate
3
4
opik.configure()
5
client = opik.Opik()
6
7
# Get the dataset
8
dataset = client.get_or_create_dataset("test_dataset")
9
10
# Get different versions of the same text prompt
11
prompt_v1 = client.get_prompt(name="prompt-summary", commit="abc123")
12
prompt_v2 = client.get_prompt(name="prompt-summary", commit="def456")
13
14
# Define evaluation task that uses the prompt
15
def evaluation_task_v1(dataset_item):
16
    formatted_prompt = prompt_v1.format(text=dataset_item["input"])
17
    # Call your LLM with the formatted prompt
18
    return {"output": "llm_response"}
19
20
def evaluation_task_v2(dataset_item):
21
    formatted_prompt = prompt_v2.format(text=dataset_item["input"])
22
    # Call your LLM with the formatted prompt
23
    return {"output": "llm_response"}
24
25
# Run experiments with different prompt versions
26
experiment_v1 = evaluate(
27
    experiment_name="My experiment - v1",
28
    dataset=dataset,
29
    task=evaluation_task_v1,
30
    prompts=[prompt_v1],
31
)
32
33
experiment_v2 = evaluate(
34
    experiment_name="My experiment - v2",
35
    dataset=dataset,
36
    task=evaluation_task_v2,
37
    prompts=[prompt_v2],
38
)
39
40
# Compare results in the Opik UI

This workflow allows you to systematically test and compare different prompt versions to identify the most effective one for your use case.