Optimization

Optimize tools (MCP)

Use this guide when you want to optimize tool signatures (descriptions + parameter descriptions) separately from prompt text or agent logic.

What this covers

  • Optimize MCP tool descriptions without touching prompt content.
  • Keep system/assistant/user messages fixed while improving tool calling behavior.
  • Target only specific tools when a server exposes many tools.
Separate from prompt or agent optimization

Tool optimization updates tool signatures. Prompt optimization updates system/user/assistant content. Agent optimization focuses on multi-step agent logic. Keep these runs separate to isolate changes.

Quickstart: tool-only optimization

1from opik_optimizer import ChatPrompt, MetaPromptOptimizer
2
3prompt = ChatPrompt(
4    system="Use tools when needed.",
5    user="{user_query}",
6    tools=[
7        {
8            "type": "mcp",
9            "server_label": "context7",
10            "server_url": "https://mcp.context7.com/mcp",
11            "allowed_tools": ["resolve-library-id", "query-docs"],
12        }
13    ],
14)
15
16optimizer = MetaPromptOptimizer(model="openai/gpt-4o-mini")
17result = optimizer.optimize_prompt(
18    prompt=prompt,
19    dataset=my_dataset,
20    metric=answer_quality,
21    optimize_prompts=False,  # keep prompt text fixed
22    optimize_tools=True,     # optimize tool + parameter descriptions
23)

Tool optimization is supported by all optimizers except FewShotBayesianOptimizer, ParameterOptimizer, and GepaOptimizer (for now).

Tool calling + MCP formats

You can define tools in one of the supported formats. Tool optimization will normalize them to function-calling tools while preserving the original MCP config for reproducibility.

Supported formats

FormatWhen to useExample
OpenAI MCP tool entry (local)Run MCP tool servers locally{"type": "mcp", "server_label": "...", "command": "...", "args": [...]}
OpenAI MCP tool entry (remote)Call remote MCP servers{"type": "mcp", "server_label": "...", "server_url": "...", "headers": {...}}
Cursor MCP configConvert from a Cursor mcpServers JSONcursor_mcp_config_to_tools(cursor_config)
Function toolsNon-MCP tools in OpenAI function format{"type": "function", "function": {...}}

OpenAI MCP tool entries (local/remote)

1prompt = ChatPrompt(
2    system="Use tools when needed.",
3    user="{user_query}",
4    tools=[
5        {
6            "type": "mcp",
7            "server_label": "local_docs",
8            "command": "npx",
9            "args": ["-y", "@upstash/context7-mcp"],
10            "env": {},
11            "allowed_tools": ["resolve-library-id", "query-docs"],
12        },
13        {
14            "type": "mcp",
15            "server_label": "remote_docs",
16            "server_url": "https://mcp.context7.com/mcp",
17            "headers": {"CONTEXT7_API_KEY": "YOUR_API_KEY"},
18            "allowed_tools": ["query-docs"],
19        },
20    ],
21)

Cursor MCP config (JSON) → tools

1from opik_optimizer.utils.toolcalling import cursor_mcp_config_to_tools
2
3cursor_config = {
4    "mcpServers": {
5        "context7": {
6            "url": "https://mcp.context7.com/mcp",
7            "headers": {"CONTEXT7_API_KEY": "YOUR_API_KEY"},
8        }
9    }
10}
11
12prompt = ChatPrompt(
13    system="Use tools when needed.",
14    user="{user_query}",
15    tools=cursor_mcp_config_to_tools(cursor_config),
16)

Cursor vs OpenAI ChatPrompt styles

ChatPrompt.tools accepts OpenAI-style MCP entries directly. Cursor configs use a different shape (mcpServers) and must be converted first:

1# OpenAI-style MCP tool entry (directly supported by ChatPrompt.tools)
2openai_tools = [
3    {
4        "type": "mcp",
5        "server_label": "context7",
6        "server_url": "https://mcp.context7.com/mcp",
7        "headers": {"CONTEXT7_API_KEY": "YOUR_API_KEY"},
8        "allowed_tools": ["resolve-library-id", "query-docs"],
9    }
10]
11
12# Cursor-style config (convert before assigning to ChatPrompt.tools)
13cursor_config = {
14    "mcpServers": {
15        "context7": {
16            "url": "https://mcp.context7.com/mcp",
17            "headers": {"CONTEXT7_API_KEY": "YOUR_API_KEY"},
18        }
19    }
20}

After normalization, both styles execute the same way during evaluation and optimization.

Mixed function + MCP tools

1prompt = ChatPrompt(
2    system="Use tools when needed.",
3    user="{user_query}",
4    tools=[
5        {
6            "type": "function",
7            "function": {
8                "name": "search_wikipedia",
9                "description": "Search Wikipedia abstracts.",
10                "parameters": {
11                    "type": "object",
12                    "properties": {"query": {"type": "string"}},
13                    "required": ["query"],
14                },
15            },
16        },
17        {
18            "type": "mcp",
19            "server_label": "context7",
20            "server_url": "https://mcp.context7.com/mcp",
21            "allowed_tools": ["query-docs"],
22        },
23    ],
24)

Target only specific tools

When a server exposes many tools, pass a dict to select the subset you want:

1result = optimizer.optimize_prompt(
2    prompt=prompt,
3    dataset=my_dataset,
4    metric=answer_quality,
5    optimize_prompts=False,
6    optimize_tools={
7        "context7.resolve-library-id": True,
8        "context7.query-docs": False,
9    },
10)

Tool optimization limits

To keep runs manageable, optimize_tools=True is limited by DEFAULT_TOOL_CALL_MAX_TOOLS_TO_OPTIMIZE (default: 3). If you need more, pass a dict to select a smaller subset of tools.

Disable tool use while optimizing tools

If you want to optimize tool descriptions without executing tools during evaluation, set allow_tool_use=False:

1result = optimizer.optimize_prompt(
2    prompt=prompt,
3    dataset=my_dataset,
4    metric=answer_quality,
5    optimize_prompts=False,
6    optimize_tools=True,
7    allow_tool_use=False,  # do not execute tools during evaluation
8)

What changes

  • Tool descriptions are updated.
  • Tool parameter descriptions are updated.
  • Prompt text stays unchanged when optimize_prompts=False.

The optimized tools are returned in result.prompt.tools. History entries include both the resolved tools and the original MCP config for reproducibility.

When to use tool optimization

ScenarioUse tool optimization?Why
MCP client injects assistant instructionsYesKeep assistant text fixed and improve tool usage.
Prompt wording needs improvementNoUse optimize_prompts instead.
Multi-agent workflowsMaybeOptimize tools separately before agent-level changes.

Limitations

  • Tool optimization only supports single prompts.
  • Not supported in FewShotBayesianOptimizer, ParameterOptimizer, or GepaOptimizer.

Troubleshooting

See tool-calling debug logs

Set the optimizer log level to surface tool calls:

$export OPIK_OPTIMIZER_LOG_LEVEL=DEBUG

You’ll see tool call lines like:

tool: event=tool_call call=query-docs({...})

I get “MCP remote server missing url”

Make sure your tool entry includes server_url (OpenAI MCP format) or url (Cursor config).

Tools are not showing up

  • Verify the MCP server is reachable.
  • If using allowed_tools, confirm the names match the server’s tool list.
  • For remote servers, confirm auth headers are correct.

Tool calls are slow or failing during evaluation

  • Set allow_tool_use=False to optimize descriptions without running tools.
  • Reduce dataset size while iterating on tool descriptions.

Next steps