Evolutionary Optimizer: Genetic Algorithms

The EvolutionaryOptimizer uses genetic algorithms to refine and discover effective prompts. It iteratively evolves a population of prompts, applying selection, crossover, and mutation operations to find prompts that maximize a given evaluation metric. This optimizer can also perform multi-objective optimization (e.g., maximizing score while minimizing prompt length) and leverage LLMs for more sophisticated genetic operations.

How It Works

The EvolutionaryOptimizer is built upon the DEAP library for evolutionary computation. The core concept behind the optimizer is that we evolve a population of prompts over multiple generations to find the best one.

We utilize different techniques to evolve the population of prompts:

• Selection: We select the best prompts from the population to be the parents of the next generation.
• Crossover: We crossover the parents to create the children of the next generation.
• Mutation: We mutate the children to create the new population of prompts.

We repeat this process for a number of generations until we find the best prompt.

Quickstart

You can use the EvolutionaryOptimizer to optimize a prompt:

1
from opik_optimizer import EvolutionaryOptimizer
2
from opik.evaluation.metrics import LevenshteinRatio # or any other suitable metric
3
from opik_optimizer import datasets, ChatPrompt
4
5
# 1. Define your evaluation dataset
6
dataset = datasets.tiny_test() # Replace with your actual dataset
7
8
# 2. Configure the evaluation metric
9
def levenshtein_ratio(dataset_item, llm_output):
10
    return LevenshteinRatio().score(reference=dataset_item["label"], output=llm_output)
11
12
# 3. Define your base prompt and task configuration
13
initial_prompt = ChatPrompt(
14
    messages=[
15
        {"role": "system", "content": "You are a helpful assistant."},
16
        {"role": "user", "content": "{text}"}
17
    ]
18
)
19
20
# 4. Initialize the EvolutionaryOptimizer
21
optimizer = EvolutionaryOptimizer(
22
    model="openai/gpt-4o-mini",
23
    model_parameters={"temperature": 0.4},
24
    population_size=20,
25
    num_generations=10,
26
)
27
28
# 5. Run the optimization
29
optimization_result = optimizer.optimize_prompt(
30
    prompt=initial_prompt,
31
    dataset=dataset,
32
    metric=levenshtein_ratio,
33
    n_samples=5
34
)
35
36
# 6. View the results
37
optimization_result.display()

Configuration Options

Optimizer parameters

The optimizer has the following parameters:

optimize_prompt parameters

The optimize_prompt method has the following parameters:

Model Support

There are two models to consider when using the EvolutionaryOptimizer:

• EvolutionaryOptimizer.model: The model used for the evolution of the population of prompts.
• ChatPrompt.model: The model used to evaluate the prompt.

The model parameter accepts any LiteLLM-supported model string (e.g., "gpt-4o", "azure/gpt-4", "anthropic/claude-3-opus", "gemini/gemini-1.5-pro"). You can also pass in extra model parameters using the model_parameters parameter:

1
optimizer = EvolutionaryOptimizer(
2
    model="anthropic/claude-3-opus-20240229",
3
    model_parameters={
4
        "temperature": 0.7,
5
        "max_tokens": 4096
6
    }
7
)

Next Steps

1. Explore specific Optimizers for algorithm details.
2. Refer to the FAQ for common questions and troubleshooting.
3. Refer to the API Reference for detailed configuration options.