Prompt management

Opik provides a prompt library that you can use to manage your prompts. Storing prompts in a library allows you to version them, reuse them across projects, and manage them in a central location.

Using a prompt library does not mean you can’t store your prompt in code, we have designed the prompt library to work seamlessly with your existing prompt files while providing the benefits of a central prompt library.

Managing prompts stored in code

The recommended way to create and manage prompts is using the Prompt object. This will allow you to continue versioning your prompts in code while also getting the benefit of having prompt versions managed in the Opik platform so you can more easily keep track of your progress.

1
import opik
2
3
# Prompt text stored in a variable
4
PROMPT_TEXT = "Write a summary of the following text: {{text}}"
5
6
# Create a prompt
7
prompt = opik.Prompt(
8
    name="prompt-summary",
9
    prompt=PROMPT_TEXT,
10
    metadata={"environment": "production"}
11
)
12
13
# Print the prompt text
14
print(prompt.prompt)
15
16
# Build the prompt
17
print(prompt.format(text="Hello, world!"))

The prompt will now be stored in the library and versioned:

Using the low level SDK

If you would rather keep prompts in the Opik platform and manually update / download them, you can use the low-level Python SDK to manage your prompts.

Creating prompts

You can create a new prompt in the library using both the SDK and the UI:

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Create a new prompt
7
prompt = client.create_prompt(name="prompt-summary", prompt="Write a summary of the following text: {{text}}", metadata={"environment": "development"})

Adding prompts to traces and spans

You can associate prompts with your traces and spans using the opik_context module. This is useful when you want to track which prompts were used during the execution of your functions:

1
import opik
2
from opik.opik_context import update_current_trace
3
4
# Create prompts
5
system_prompt = opik.Prompt(
6
    name="system-prompt",
7
    prompt="You are a helpful assistant that provides accurate and concise answers."
8
)
9
10
# Get prompt from the Prompt library
11
client = opik.Opik()
12
user_prompt = client.get_prompt(name="user-prompt")
13
14
@opik.track
15
def process_user_query(question: str) -> str:
16
    # Add prompts to the current trace
17
    update_current_trace(
18
        name="user-query-processing",
19
        prompts=[system_prompt, user_prompt],
20
        metadata={"query_type": "general"}
21
    )
22
    
23
    # Your processing logic here
24
    formatted_prompt = user_prompt.format(question=question)
25
    # ... rest of your function
26
    return "Response to: " + question

You can view the prompts associated with a trace or span in the Opik UI:

Further details on using prompts from the Prompt library are provided in the following sections.

Downloading your prompts

Once a prompt is created in the library, you can download it in code using the Opik.get_prompt method:

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Get a dataset
7
dataset = client.get_or_create_dataset("test_dataset")
8
9
# Get the prompt
10
prompt = client.get_prompt(name="prompt-summary")
11
12
# Create the prompt message
13
prompt.format(text="Hello, world!")

If you are not using the SDK, you can download a prompt by using the REST API.

Searching prompts

To discover prompts by name substring and/or filters, use search_prompts. Filters are written in Opik Query Language (OQL):

1
import opik
2
3
client = opik.Opik()
4
5
# Search by name substring only
6
latest_versions = client.search_prompts(
7
    filter_string='name contains "summary"'
8
)
9
10
# Search by name substring and tags filter
11
filtered = client.search_prompts(
12
    filter_string='name contains "summary" AND tags contains "alpha" AND tags contains "beta"',
13
)
14
15
for prompt in filtered:
16
    print(prompt.name, prompt.commit, prompt.prompt)

The filter_string parameter uses Opik Query Language (OQL) with the format: "<COLUMN> <OPERATOR> <VALUE> [AND <COLUMN> <OPERATOR> <VALUE>]*"

Supported columns for prompts:

Examples:

• tags contains "production" - Filter by tag
• name contains "summary" - Filter by name substring
• created_by = "user@example.com" - Filter by creator
• tags contains "alpha" AND tags contains "beta" - Multiple tag filtering

search_prompts returns the latest version for each matching prompt.

Working with prompt versions

Viewing prompt history (all versions)

You can fetch the complete version history for a prompt by its exact name using get_prompt_history:

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Get the complete version history for a prompt
7
prompt_history = client.get_prompt_history(name="prompt-summary")
8
9
# Iterate through all versions
10
for version in prompt_history:
11
    print(f"Commit: {version.commit}")
12
    print(f"Created at: {version.created_at}")
13
    print(f"Prompt text: {version.prompt}")
14
    print(f"Metadata: {version.metadata}")
15
    print("-" * 50)

This returns a list of Prompt objects (each representing a specific version) for the given prompt name.

You can use this information to:

• Audit changes to understand how prompts evolved
• Identify the best performing version by linking commit IDs to experiment results
• Document prompt changes for compliance or review purposes
• Retrieve specific versions by commit ID for testing or rollback

Accessing specific prompt versions

When working with prompts, you may want to retrieve a specific version of a prompt rather than the latest version. You can do this by passing the commit parameter to the get_prompt method:

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Get a specific version of a prompt by commit ID
7
prompt = client.get_prompt(name="prompt-summary", commit="abc123def456")
8
9
# Use the prompt in your application
10
formatted_prompt = prompt.format(text="Hello, world!")
11
print(formatted_prompt)

The commit parameter accepts the commit ID (also called commit hash) of the specific prompt version you want to retrieve. You can find commit IDs in the prompt history in the Opik UI or by using the get_prompt_history method (see above).

This is particularly useful when you want to:

• Pin to a specific version in production to ensure consistent behavior
• Test different versions side by side in experiments
• Roll back to a previous version if issues are discovered
• Compare results across different prompt versions

Using prompts in experiments

Linking prompts to experiments

Experiments allow you to evaluate the performance of your LLM application on a set of examples. When evaluating different prompts, it can be useful to link the evaluation to a specific prompt version. This can be achieved by passing the prompts parameter when creating an Experiment:

1
import opik
2
from opik.evaluation import evaluate
3
from opik.evaluation.metrics import Hallucination
4
5
opik.configure()
6
client = opik.Opik()
7
8
# Get a dataset
9
dataset = client.get_or_create_dataset("test_dataset")
10
11
# Create a prompt
12
prompt = opik.Prompt(name="My prompt", prompt="...")
13
14
# Create an evaluation task
15
def evaluation_task(dataset_item):
16
    return {"output": "llm_response"}
17
18
# Run the evaluation
19
evaluation = evaluate(
20
    experiment_name="My experiment",
21
    dataset=dataset,
22
    task=evaluation_task,
23
    prompts=[prompt],
24
)

The experiment will now be linked to the prompt allowing you to view all experiments that use a specific prompt:

Comparing prompt versions in experiments

You can run experiments with different prompt versions to determine which performs best:

1
import opik
2
from opik.evaluation import evaluate
3
4
opik.configure()
5
client = opik.Opik()
6
7
# Get the dataset
8
dataset = client.get_or_create_dataset("test_dataset")
9
10
# Get different versions of the same prompt
11
prompt_v1 = client.get_prompt(name="prompt-summary", commit="abc123")
12
prompt_v2 = client.get_prompt(name="prompt-summary", commit="def456")
13
14
# Define evaluation task
15
def evaluation_task(dataset_item):
16
    return {"output": "llm_response"}
17
18
# Run experiments with different prompt versions
19
experiment_v1 = evaluate(
20
    experiment_name="My experiment - v1",
21
    dataset=dataset,
22
    task=evaluation_task,
23
    prompts=[prompt_v1],
24
)
25
26
experiment_v2 = evaluate(
27
    experiment_name="My experiment - v2",
28
    dataset=dataset,
29
    task=evaluation_task,
30
    prompts=[prompt_v2],
31
)
32
33
# Compare results in the Opik UI

This workflow allows you to systematically test and compare different prompt versions to identify the most effective one for your use case.