Custom Optimizer Prompts | Opik Documentation

The Opik Optimizer uses a PromptLibrary system that lets you customize the internal prompts used by each optimizer. This is useful when you need to:

Add domain-specific constraints (legal, medical, coding standards)
Inject safety or compliance requirements
Adjust output formatting or style
Experiment with different reasoning approaches

Quick Start

Every optimizer accepts a prompt_overrides parameter:

1 from opik_optimizer import MetaPromptOptimizer
2 
3 # Simple dict override
4 optimizer = MetaPromptOptimizer(
5     model="gpt-4o",
6     prompt_overrides={"reasoning_system": "Be concise. Focus on clarity."}
7 )

How It Works

Each optimizer defines its own DEFAULT_PROMPTS dictionary with keys specific to that algorithm. The PromptLibrary:

Stores the default prompts
Applies your overrides (dict or callable)
Validates that override keys exist (catches typos early)
Provides get_prompt() for runtime access

Override Methods

Dict Override (Simple Replacement)

Best when you know exactly which prompt to replace with a static string:

1 from opik_optimizer import EvolutionaryOptimizer
2 
3 optimizer = EvolutionaryOptimizer(
4     model="gpt-4o",
5     prompt_overrides={
6         "synonyms_system_prompt": "Return exactly ONE synonym. No explanation.",
7         "infer_style_system_prompt": "Analyze the writing style briefly.",
8     }
9 )

Callable Override (Dynamic Modification)

Best when you need to modify existing prompts, apply conditional logic, or update multiple prompts:

1 from opik_optimizer import MetaPromptOptimizer
2 from opik_optimizer.utils.prompt_library import PromptLibrary
3 
4 def customize_prompts(prompts: PromptLibrary) -> None:
5     # List available keys
6     print("Available keys:", prompts.keys())
7 
8     # Prepend a constraint to the reasoning prompt
9     original = prompts.get("reasoning_system")
10     prompts.set("reasoning_system", "Always respond in English.\n\n" + original)
11 
12     # Append format instructions to another prompt
13     if "candidate_generation" in prompts.keys():
14         prompts.set(
15             "candidate_generation",
16             prompts.get("candidate_generation") + "\n\nUse markdown formatting."
17         )
18 
19 optimizer = MetaPromptOptimizer(
20     model="gpt-4o",
21     prompt_overrides=customize_prompts
22 )

Discovering Available Keys

Each optimizer has different prompt keys. Use list_prompts() to discover them:

1 from opik_optimizer import MetaPromptOptimizer
2 
3 optimizer = MetaPromptOptimizer(model="gpt-4o")
4 print("Available prompt keys:")
5 for key in optimizer.list_prompts():
6     print(f"  - {key}")

Common Keys by Optimizer

Optimizer	Key Examples
MetaPromptOptimizer	`reasoning_system`, `candidate_generation`, `synthesis`, `pattern_extraction_system`
EvolutionaryOptimizer	`infer_style_system_prompt`, `synonyms_system_prompt`, `semantic_mutation_system_prompt_template`
FewShotBayesianOptimizer	`example_placeholder`, `system_prompt_template`
HierarchicalReflectiveOptimizer	`batch_analysis_prompt`, `synthesis_prompt`, `improve_prompt_template`

Reading Prompts at Runtime

After creating an optimizer, you can inspect the current prompts:

1 optimizer = MetaPromptOptimizer(
2     model="gpt-4o",
3     prompt_overrides={"reasoning_system": "Custom prompt here..."}
4 )
5 
6 # Get the current (possibly overridden) prompt
7 current = optimizer.get_prompt("reasoning_system")
8 print(current)
9 
10 # Get the original default (before any overrides)
11 default = optimizer.prompts.get_default("reasoning_system")
12 print(default)

Template Variables

Some prompts contain placeholders that get filled at runtime using Python’s {variable} format. When overriding prompts with placeholders, keep the same placeholders:

1 # Original: "Generate {num_prompts} variations of the prompt."
2 # Your override should keep {num_prompts}:
3 prompt_overrides = {
4     "candidate_generation": "Be creative. Generate {num_prompts} diverse variations."
5 }

Use Cases

Adding Domain Constraints

1 def add_legal_constraints(prompts: PromptLibrary) -> None:
2     for key in prompts.keys():
3         original = prompts.get(key)
4         prompts.set(key,
5             "LEGAL CONTEXT: Do not reference specific case law.\n\n" + original
6         )
7 
8 optimizer = MetaPromptOptimizer(
9     model="gpt-4o",
10     prompt_overrides=add_legal_constraints
11 )

Enforcing Output Format

1 optimizer = EvolutionaryOptimizer(
2     model="gpt-4o",
3     prompt_overrides={
4         "infer_style_system_prompt": """
5 Analyze writing style. Return a JSON object with:
6 {
7   "tone": "formal|casual|technical",
8   "complexity": "simple|moderate|complex",
9   "key_patterns": ["list", "of", "patterns"]
10 }
11 """
12     }
13 )

Adding Safety Guardrails

1 def add_safety_layer(prompts: PromptLibrary) -> None:
2     safety_prefix = """
3 SAFETY REQUIREMENTS:
4 - Never generate harmful or offensive content
5 - Avoid personal identifiable information
6 - Flag uncertain responses
7 
8 """
9     for key in prompts.keys():
10         if "system" in key.lower():
11             prompts.set(key, safety_prefix + prompts.get(key))
12 
13 optimizer = MetaPromptOptimizer(
14     model="gpt-4o",
15     prompt_overrides=add_safety_layer
16 )

Error Handling

The PromptLibrary validates keys to catch typos early:

1 # This will raise KeyError - "reasoing_system" is misspelled
2 optimizer = MetaPromptOptimizer(
3     model="gpt-4o",
4     prompt_overrides={"reasoing_system": "Oops, typo!"}  # KeyError!
5 )
6 
7 # Error message shows available keys:
8 # KeyError: "Unknown prompt keys: ['reasoing_system'].
9 #            Available: ['candidate_generation', 'reasoning_system', ...]"

Best Practices

Tips for effective prompt customization:

List keys first – Always use list_prompts() to see available keys before overriding
Keep placeholders – If a prompt has {variables}, keep them in your override
Test incrementally – Override one prompt at a time to isolate effects
Use callable for complex logic – Dict is simpler, but callable is more powerful
Don’t break JSON – Some prompts expect JSON output; maintain that structure

Full Example

1 from opik_optimizer import MetaPromptOptimizer
2 from opik_optimizer.utils.prompt_library import PromptLibrary
3 
4 def my_customizations(prompts: PromptLibrary) -> None:
5     """Customize prompts for a code generation task."""
6 
7     # 1. Add coding focus to reasoning
8     prompts.set(
9         "reasoning_system",
10         "You are an expert code prompt engineer.\n\n" + prompts.get("reasoning_system")
11     )
12 
13     # 2. Enforce Python-specific patterns
14     prompts.set(
15         "candidate_generation",
16         prompts.get("candidate_generation") + """
17 
18 ADDITIONAL REQUIREMENTS:
19 - Prompts should encourage well-documented code
20 - Prefer type hints and docstrings
21 - Emphasize error handling and edge cases
22 """
23     )
24 
25 # Create optimizer with customizations
26 optimizer = MetaPromptOptimizer(
27     model="gpt-4o",
28     prompt_overrides=my_customizations
29 )
30 
31 # Verify customizations applied
32 print("Customized reasoning prompt:")
33 print(optimizer.get_prompt("reasoning_system")[:200] + "...")

API Reference – Full parameter documentation
Custom metrics – Build specialized evaluation metrics
Extending optimizers – Create custom optimizer subclasses