> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://www.comet.com/docs/opik/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://www.comet.com/docs/opik/_mcp/server.

# Tool Optimization (MCP & Function Calling)

> Learn how to optimize prompts that leverage MCP (Model Context Protocol) tools and function calling capabilities across supported optimizers.

Tool optimization is a specialized feature that allows you to optimize prompts that use external tools and the Model Context Protocol (MCP). This capability is supported by all optimizers **except** `FewShotBayesianOptimizer`, `ParameterOptimizer`, and `GepaOptimizer`.

## What is Tool Optimization?

Tool optimization extends traditional prompt optimization to handle prompts that include:

* **MCP tools** - Model Context Protocol tools for external integrations
* **Tool schemas** - Structured tool definitions and parameters
* **Multi-step workflows** - Complex agent workflows involving multiple tools

Tool optimization **does not** change tool names or schemas. It updates:

* Tool **descriptions**
* Tool **parameter descriptions**

**Important Distinction:** Many optimizers (including GEPA, MetaPrompt, etc.) can optimize **agents that use tools** -
this means optimizing prompts for agents that have access to function calling or external tools. However, **true tool
optimization** (optimizing the tool descriptions and parameter descriptions) is a separate capability you can enable
with `optimize_tools=True` across supported optimizers.

**Why Tool Optimization Matters:**

Traditional prompt optimization focuses on text-based prompts, but modern AI applications often
require:

* Integration with external APIs and services
* Structured data processing through function calls
* Complex multi-step reasoning with tool usage
* Dynamic tool selection based on context Tool optimization ensures these sophisticated prompts
  can be improved just like simple text prompts.

## Supported Tool Types

### 1. Agent Function Calling (Not True Tool Optimization)

Many optimizers can optimize **agents that use function calling**, but this is different from true tool optimization. Here's an example from the GEPA optimizer:

```python
from opik_optimizer import GepaOptimizer, ChatPrompt

# GEPA example: optimizing an agent with function calling
prompt = ChatPrompt(
    system="You are a helpful assistant. Use the search_wikipedia tool when needed.",
    user="{question}",
    tools=[
        {
            "type": "function",
            "function": {
                "name": "search_wikipedia",
                "description": "This function searches Wikipedia abstracts.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "query": {
                            "type": "string",
                            "description": "The term or phrase to search for."
                        }
                    },
                    "required": ["query"]
                }
            }
        }
    ],
    function_map={
        "search_wikipedia": lambda query: search_wikipedia(query, use_api=True)
    }
)

# GEPA optimizes the agent's prompt, not the tools themselves
optimizer = GepaOptimizer(model="gpt-5-nano")
result = optimizer.optimize_prompt(prompt=prompt, dataset=dataset, metric=metric)
```

This is **agent optimization** (optimizing prompts for agents that use tools), not **tool optimization** (optimizing
the tools themselves).

### 2. MCP (Model Context Protocol) Tools

**True tool optimization** is available for MCP tools and function-calling tools across supported optimizers. MCP tools provide standardized interfaces for external integrations:

```python
# MCP tool optimization example
# See scripts/litellm_metaprompt_context7_remote_example.py for a working example

from opik_optimizer import MetaPromptOptimizer

# MCP tools are configured as OpenAI-style entries (local or remote)
optimizer = MetaPromptOptimizer(model="gpt-5-nano")

# Any supported optimizer works here (e.g., HRPO, Evolutionary, MetaPrompt).
```

MCP Tool Optimization:

* Supported by all optimizers **except** FewShotBayesianOptimizer, ParameterOptimizer, and GepaOptimizer
* Working examples available in `scripts/litellm_metaprompt_context7_remote_example.py`

## How Tool Optimization Works

Supported optimizers handle tool-enabled prompts through a specialized optimization process:

### 1. Tool-Aware Analysis

The optimizer analyzes:

* **Tool schemas** - Understanding available functions and their parameters
* **Tool usage patterns** - How tools are typically invoked in the prompt
* **Tool dependencies** - Relationships between different tools
* **Context requirements** - What information tools need to function effectively

### 2. Prompt-Tool Integration Optimization

The optimizer can improve:

* **Tool selection logic** - Better instructions for when to use which tools
* **Parameter formatting** - Clearer guidance on how to structure tool inputs
* **Error handling** - Instructions for handling tool failures or edge cases
* **Tool chaining** - Optimizing multi-step tool workflows

### 3. Context Enhancement

Tool optimization also improves:

* **Input validation** - Better prompts for validating tool inputs
* **Output processing** - Instructions for handling tool outputs
* **Fallback strategies** - Alternative approaches when tools are unavailable

## Example: Optimizing a Research Assistant

Let's see how tool optimization works with a research assistant that uses multiple tools:

```python
from opik_optimizer import MetaPromptOptimizer, ChatPrompt
from opik.evaluation.metrics import LevenshteinRatio

# Define a research assistant prompt with tools
research_prompt = ChatPrompt(
    messages=[
        {
            "role": "system",
            "content": """You are a research assistant. When given a research question:
1. Search for relevant information using the search tool
2. Analyze the results using the analysis tool
3. Provide a comprehensive answer based on your findings

Always cite your sources and be thorough in your research."""
        },
        {
            "role": "user",
            "content": "{research_question}"
        }
    ],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "search_academic_database",
                "description": "Search academic papers and research",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "query": {"type": "string"},
                        "year_range": {"type": "string"},
                        "max_results": {"type": "integer"}
                    }
                }
            }
        },
        {
            "type": "function",
            "function": {
                "name": "analyze_text",
                "description": "Analyze and summarize text content",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "text": {"type": "string"},
                        "analysis_type": {"type": "string"}
                    }
                }
            }
        }
    ]
)

# Initialize the optimizer
optimizer = MetaPromptOptimizer(
    model="openai/gpt-5-nano"
)

# Define evaluation metric
def research_quality_metric(dataset_item, llm_output):
    return LevenshteinRatio().score(
        reference=dataset_item['expected_answer'],
        output=llm_output
    )

# Run optimization
result = optimizer.optimize_prompt(
    prompt=research_prompt,
    dataset=research_dataset,
    metric=research_quality_metric,
    n_samples=100,
    max_trials=5
)

print("Optimized prompt with tools:")
print(result.prompt)
```

## Best Practices for Tool Optimization

### 1. Tool Schema Design

* **Clear descriptions** - Provide detailed descriptions for each tool
* **Comprehensive parameters** - Include all necessary parameters with types
* **Example usage** - Add examples in tool descriptions when helpful
* **Error handling** - Define expected error conditions and responses

### 2. Prompt Structure

* **Tool introduction** - Clearly explain available tools to the model
* **Usage guidelines** - Provide specific instructions on when and how to use tools
* **Output formatting** - Specify how tool outputs should be processed
* **Fallback instructions** - Define what to do when tools fail

### 3. Evaluation Considerations

* **Tool usage metrics** - Measure not just final output quality but tool usage effectiveness
* **Multi-step evaluation** - Evaluate each step in tool-based workflows
* **Error rate tracking** - Monitor tool failure rates and recovery strategies
* **Context preservation** - Ensure important context is maintained across tool calls

## Limitations and Considerations

### Current Limitations

* **Optimizer coverage** - Tool optimization is not available in `FewShotBayesianOptimizer` or `ParameterOptimizer`
* **Tool Complexity** - Very complex tool workflows may require manual optimization
* **Tool Availability** - Optimization assumes tools are available during evaluation
* **Schema Changes** - Tool schema modifications may require re-optimization

### Performance Considerations

* **Evaluation Cost** - Tool-enabled prompts require more LLM calls for evaluation
* **Tool Latency** - External tool calls can slow down optimization
* **Resource Usage** - Complex tool workflows may require significant computational resources

## Future Roadmap

Tool optimization is an active area of development. Planned improvements include:

* **Tool-specific metrics** - Specialized evaluation metrics for tool usage
* **Automated tool discovery** - Automatic detection and optimization of tool patterns
* **Tool performance optimization** - Optimizing not just prompts but tool usage efficiency

## Getting Started

To start optimizing tool-enabled prompts:

1. **Choose a supported optimizer** - Any optimizer except FewShotBayesianOptimizer and ParameterOptimizer
2. **Define your tools** - Create clear tool schemas with comprehensive descriptions
3. **Structure your prompt** - Include clear instructions for tool usage
4. **Prepare evaluation data** - Ensure your dataset includes tool usage scenarios
5. **Run optimization** - Use the standard optimization process with tool-enabled prompts

**Need Help?**

For questions about tool optimization, please reach out on
[GitHub](https://github.com/comet-ml/opik/issues) or check the
[MetaPrompt Optimizer documentation](/development/optimization-runs/algorithms/metaprompt_optimizer) for detailed configuration options.

## References

* [MetaPrompt Optimizer Documentation](/development/optimization-runs/algorithms/metaprompt_optimizer)
* [Optimize tools (MCP)](/development/optimization-runs/optimization/optimize_tools)
* [Model Context Protocol Specification](https://modelcontextprotocol.io/)