Prompt management

Opik provides a prompt library that you can use to manage your prompts. Storing prompts in a library allows you to version them, reuse them across projects, and manage them in a central location.

Using a prompt library does not mean you can’t store your prompt in code, we have designed the prompt library to be work seamlessly with your existing prompt files while providing the benefits of a central prompt library.

Managing prompts stored in code

The recommend way to create and manage prompts is using The Prompt object. This will allow you to continue versioning your prompts in code while also getting the benefit of having prompt versions managed in the Opik platform so you can more easily keep track of your progress.

1
import opik
2
3
# Prompt text stored in a variable
4
PROMPT_TEXT = "Write a summary of the following text: {{text}}"
5
6
# Create a prompt
7
prompt = opik.Prompt(
8
    name="prompt-summary",
9
    prompt=PROMPT_TEXT,
10
    metadata={"environment": "production"}
11
)
12
13
# Print the prompt text
14
print(prompt.prompt)
15
16
# Build the prompt
17
print(prompt.format(text="Hello, world!"))

The prompt will now be stored in the library and versioned:

Using the low level SDK

If you would rather keep prompts in the Opik platform and manually update / download them, you can use the low-level Python SDK to manage you prompts.

Creating prompts

You can create a new prompt in the library using both the SDK and the UI:

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Create a new prompt
7
prompt = client.create_prompt(name="prompt-summary", prompt="Write a summary of the following text: {{text}}", metadata={"environment": "development"})

Downloading your prompts

Once a prompt is created in the library, you can download it in code using the Opik.get_prompt method:

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Get a dataset
7
dataset = client.get_or_create_dataset("test_dataset")
8
9
# Get the prompt
10
prompt = client.get_prompt(name="prompt-summary")
11
12
# Create the prompt message
13
prompt.format(text="Hello, world!")

If you are not using the SDK, you can download a prompt by using the REST API.

Viewing prompt history (all versions)

You can fetch the complete version history for a prompt by its exact name using get_prompt_history:

1
import opik
2
3
client = opik.Opik()
4
5
history = client.get_prompt_history(name="prompt-summary")
6
for version in history:
7
    print(version.commit, version.prompt)

This returns a list of Prompt objects (each representing a specific version) for the given prompt name.

Searching prompts

To discover prompts by name substring and/or filters, use search_prompts. Filters are written in Opik Query Language (OQL):

1
import opik
2
3
client = opik.Opik()
4
5
# Search by name substring only
6
latest_versions = client.search_prompts(
7
    filter_string='name contains "summary"'
8
)
9
10
# Search by name substring and tags filter
11
filtered = client.search_prompts(
12
    filter_string='name contains "summary" AND tags contains "alpha" AND tags contains "beta"',
13
)
14
15
for prompt in filtered:
16
    print(prompt.name, prompt.commit, prompt.prompt)

The filter_string parameter uses Opik Query Language (OQL) with the format: "<COLUMN> <OPERATOR> <VALUE> [AND <COLUMN> <OPERATOR> <VALUE>]*"

Supported columns for prompts:

Examples:

• tags contains "production" - Filter by tag
• name contains "summary" - Filter by name substring
• created_by = "user@example.com" - Filter by creator
• tags contains "alpha" AND tags contains "beta" - Multiple tag filtering

search_prompts returns the latest version for each matching prompt.

Linking prompts to Experiments

Experiments allow you to evaluate the performance of your LLM application on a set of examples. When evaluating different prompts, it can be useful to link the evaluation to a specific prompt version. This can be achieved by passing the prompt parameter when creating an Experiment:

1
import opik
2
from opik.evaluation import evaluate
3
from opik.evaluation.metrics import Hallucination
4
5
opik.configure()
6
client = opik.Opik()
7
8
# Get a dataset
9
dataset = client.get_or_create_dataset("test_dataset")
10
11
# Create a prompt
12
prompt = opik.Prompt(name="My prompt", prompt="...")
13
14
# Create an evaluation task
15
def evaluation_task(dataset_item):
16
    return {"output": "llm_response"}
17
18
# Run the evaluation
19
evaluation = evaluate(
20
    experiment_name="My experiment",
21
    dataset=dataset,
22
    task=evaluation_task,
23
    prompt=prompt,
24
)

The experiment will now be linked to the prompt allowing you to view all experiments that use a specific prompt: