Skip to navigation

Unused skills

View as Markdown

How unused skills are found, per developer

Cost Intelligence sees every request a developer’s coding agent sends. For each developer and each installed skill, over the last 30 days, it measures two things:

  • What the skill cost to carry. Every installed skill adds its name and description to the skill list sent with each request. Cost Intelligence prices that entry at the model and cache rates the developer’s requests actually paid.
  • How often the skill ran.

A skill that was carried but ran zero times in those 30 days is unused for that developer. The same skill can be unused for one developer and essential for another, so the decision is never made fleet-wide. The recommendation’s dollar figure is the carrying cost summed over every developer and skill pair that qualifies.

Unlike most recommendations, this one does not change the organization baseline. Applying it writes a user policy for each affected developer, listing only the skills that developer left unused; everyone else is untouched. Policies reach devices as described in Roll out cost policies. On the device, an unused skill keeps its name in the skill list and loses its description, which is where almost all of its weight is.

Why this saves money

The skill list sits near the start of every request, so every request in a conversation carries it again. Caching makes a repeated list cheaper, but cached tokens are still billed each time they are read.

Ten turns of one conversation as horizontal bars. The system prompt and unused skills are always present, the same in every request; the conversation grows with each turn, and each turn sends the whole conversation again.

Keeping skills available when they are needed

A name alone is often not enough for the agent to know a skill fits. review-pr explains itself; a team-specific name such as payments-runbook-v2 says little about when to use it. The skill gate makes sure a hidden skill is still found when a prompt needs it. It runs on the developer’s laptop.

Before Claude reads a prompt, the gate sends it to a small decision model together with the hidden skills, and gets back a yes or a no.

When a skill fits, the model answers yes and names it. The gate injects that skill’s description into the prompt, and Claude sees the skill as if it had always been listed.

When a skill fits: the skill gate sends the prompt and the hidden skills to a decision model, which answers yes: docs. The docs skill description is injected into the prompt, and Claude uses the skill.

When nothing fits, the model answers no. Nothing is injected, and the prompt reaches Claude unchanged.

When no skill fits: the decision model answers no. Nothing is injected, and the prompt reaches Claude unchanged.

Each skill is added at most once per conversation, and again after the conversation is compacted. The check costs a small fraction of the request it saves on.

The question the gate asks

One call to the decision model asks two questions about the prompt and the hidden skills: does any skill fit, and which fits best.

Request
{
"state": "USER MESSAGE:\nTurn these meeting notes into a doc the team can comment on\n\nSKILLS:\n- docs: Editable docs people share and comment on…\n…",
"questions": {
"any": { "type": "noul", "instructions": "Would any of the listed skills help answer the user's message?" },
"skill0": {
"type": "choice",
"instructions": "Which listed skill would best help answer the user's message?",
"criteria": { "docs": null, "skill-creator": null, "…": null }
}
}
}

The decision model’s answer

any is how likely it is that some skill helps. skill0 shares the probability across skills, so the gate adds every skill above a small floor, up to three.

Response
{
"answers": {
"any": { "type": "noul", "noul": 0.78 },
"skill0": {
"type": "choice",
"probabilities": { "docs": 0.71, "skill-creator": 0.22, "…": 0.07 }
}
}
}

How much it saves

Across organizations using Cost Intelligence, unused skill descriptions make up about 5% of coding agent spend. Your own share depends on how much your organization uses skills: more installed skills, and more requests per developer, mean a bigger saving.

  • Most of what skill descriptions cost goes to skills nobody runs. In our own organization, close to 9 in every 10 dollars spent carrying skill descriptions went to skills that developer never used in 30 days. A typical developer carried around 150 of them.
  • The check is far cheaper than what it removes. The decision model reads each prompt together with that developer’s hidden skills, once per prompt. At typical decision model pricing, checking every prompt costs under a hundredth of what the hidden descriptions cost.
Share of monthly coding agent spend
Unused skill descriptions, carried on every requestAbout 5%, depending on skill use
Decision model, checking every promptUnder 0.05%

When a hidden skill is needed, its description comes back for that conversation, which adds a little of the cost back.

Next steps