Someone on your team defaulted to the latest and greatest model available, which is also the most expensive model. Maybe it was an agent calling the flagship model for every tool call or a coding agent set to its most capable model out of the box. It worked, so nobody revisited it. Then the bill showed up, bigger than the task required or the anticipated cost savings.
The core problem with LLM model selection is that it usually isn’t a decision at all. It’s a default setting that nobody chose on purpose. Fixing this issue means finding the cheapest model that can still do the job and knowing where that threshold sits.
Model selection has a huge impact in two places:
- If you’re building an agent, you need to know which model each step actually requires.
- If you’re managing a team running Claude Code or Codex, you need to know when the coding agent reached for an expensive model on a task a cheaper one could have handled.
LLM Model Selection at a Glance
- LLM model selection means matching the model to a task by the minimum required capability, not maximum available capability.
- Every major foundation model provider ships a tiered lineup instead of one model.
- The goal isn’t the “best” model in any lineup. Aim for the cheapest model that clears your performance threshold for a specific task.
- Model evaluation finds that threshold by running the same prompt, context and task through two or more models and comparing side-by-side results.
- LLM observability tools like Opik let teams building agents evaluate multiple models’ performance on the same task, rather than relying on vibes. Test Suites make traditional LLM evaluation workflows easier with plain-text assertions and pass/fail results.
- Cost Intelligence shows teams running coding agents which sessions called a premium model unnecessarily, so they can fix the default instead of guessing.
What Is LLM Model Selection?
LLM model selection is the decision of which model handles a given task, made deliberately, not by default. Every provider ships a range of models at different capability and price points, from lightweight models built for speed to frontier models built for reasoning-heavy work.
The mistake most teams make isn’t picking a bad model. It’s never picking at all.
The flagship model becomes the default for every call, including the simple ones, because nobody tested whether a cheaper option would hold up.
When the cheapest model beats the best model available
A more capable model rarely makes a simple task better. Classifying a support ticket or reading a small file doesn’t need the same model as multi-step reasoning over a large codebase. Once a model clears the quality bar, paying more for a model adds cost without adding value. Finding that bar is what model evaluation does.
Model Selection for Building an Agent
If you’re building an agent, model selection isn’t a one-time choice. A routing step that decides which tool to call needs far less capability than a step that synthesizes a final answer from retrieved context.
Comparing Models Across Providers
To compare two tiers from one provider or test Claude Fable 5 against GPT-5.6 Sol and Gemini 3.1 Pro, run the same prompt, context and harness against each model. Score the outputs the same way. The hard part is evaluating and comparing it consistently instead of eyeballing a handful of outputs.
This is where dataset-and-metric evaluation and assertion-based testing apply. Score outputs against a metric like faithfulness or relevance, or write a plain-English assertion, like “the response cites a source when quoting pricing.” Run every model against that defined metric to see which ones pass so you can choose the cheapest model that passes for the task.
Running the Comparison in Opik
Opik’s experiment dashboard is built for this comparison.
- Run the dataset against multiple models.
- See performance scores and cost data side by side.
- Find your threshold for the cheapest model that passes the same assertions at the same rate as a frontier model.
Use case: Cutting $60,000 a year without losing quality
Pattern, an e-commerce technology company, ran into this expensive problem. Its Content Brief tool analyzes product listings across marketplaces. The tool relied on large models for every request, but the team needed to know whether a smaller model could hold up.
Cayden Blake, an AI engineer at Pattern, said, “Our hope was to investigate this process and use Opik to uncover a more cost-effective model without sacrificing performance.” The team pulled production data into datasets, defined evaluation metrics and ran side-by-side comparisons of multiple models, measuring quality and cost in the same view.
They found a smaller model that matched their baseline’s quality. Jeremy Mumford, Pattern’s lead AI engineer, said, “Benchmark evaluations gave us a scientific way to confirm that a smaller, more efficient model produced consistent results across a wide range of data.”
The team estimated $60,000 in savings each year, without a drop in output quality, proving testing the threshold beats assuming it.
Model Selection for Managing Coding Agent Spend
If you’re leading an engineering team, model selection shows up differently. Instead of choosing a model for each agent you build, you’re managing the defaults baked into Claude Code, Codex and other coding agents your developers use every day.
Why the Default Model Matters
Claude Code and Codex both default to a specific model unless a developer or workspace policy changes it. Those default models were often set once in the implementation process, and then never revisited as pricing and model options changed.
The tiering pattern now shows up across every major foundry. The spread is wide when comparing the price per million input/output tokens:
- Anthropic: Claude Sonnet 5 runs $3/$15. Claude Fable 5, the new top tier, runs more than three times Sonnet’s rate at $10/$50.
- OpenAI: GPT-5.6 spans Luna at $1/$6, Terra at $2.50/$15, up to flagship Sol at $5/$30.
- Google: Gemini 3.5 Flash runs $1.50/$9, while Gemini 3.1 Pro runs $2/$12.
If your coding agent’s default sits at the top of any of these lineups, you pay that premium on every file read and tool call, whether or not the task needed it.
Spotting Overpayment on Model Choice
Model choice is often invisible in a billing invoice. /usage in Claude Code or /status in Codex only shows a single developer’s current session, not a pattern across your team. You need to see which tasks called an expensive model, across every developer and session, and whether a cheaper one would have finished the job just as well.
That’s what Cost Intelligence is built to surface. It breaks Claude Code and Codex spend down by developer, project, task type and model. The tool flags cost savings when a premium model is running on tasks that don’t need it.
Fixing the default beats restricting the team
The fix isn’t asking developers to think harder about model choice on every request. It’s setting better defaults at the workspace level, by task type, so the right model gets called automatically.
Find Your Performance Threshold
Whether you’re building an agent or managing a coding agent rollout, the process is the same:
- Pick a task worth testing. The job should be something that runs often or costs the most.
- Hold the task constant. Use the same prompt, context and harness across every model.
- Define what “good enough” means. This can be a metric, a pass/fail assertion or both.
- Run the task across tiers and providers. Compare quality and cost in the same view.
- Set the default to the cheapest model that passes. Revisit when pricing or the task changes.
Frequently Asked Questions About LLM Model Selection
What is LLM model selection?
LLM model selection is the process of matching an AI model to a task based on the minimum capability it requires, rather than defaulting to the most capable or most expensive model available.
What’s the default model in Claude Code?
Claude Code ships with a default model based on the customer plan. Codex starts with their 5.6 Sol model, and falls back to a recommended model if one is unspecified. A developer or workspace policy can change the model default for a team, project, or individual. That default often gets set once and left alone, so it’s worth periodically checking whether it still fits the tasks your team runs most.
Do Anthropic, OpenAI and Google price their models the same?
Not identically, but the pattern is the same: a cheap, fast tier for routine tasks and an expensive flagship tier for hard ones. Claude Sonnet 5 is priced at $3 per million input tokens; Claude Fable 5 is priced at $10 per million, more than three times the Sonnet rate. OpenAI’s GPT-5.6 family runs from Luna ($1/$6 per million tokens) up to Sol ($5/$30). Google’s Gemini lineup runs from Gemini 3.5 Flash ($1.50/$9) up to Gemini 3.1 Pro ($2/$12). The premium price of a flagship model applies to every request, so a mismatched default compounds quickly.
How do I know if my team is overpaying by defaulting to an expensive model?
Look at task-level spend broken down by model, not just a total bill. Cost Intelligence shows which sessions called a premium model and what those sessions actually did, so you can tell whether the extra cost matched the task.
Get a demo of Opik to see where your model choices are driving unnecessary spend or explore benchmarking models across providers.
