Title: Rename A Tool, Lose The Entire Cache

Anthropic’s new cache pricing means a single character tweak—renaming a tool or adding a timestamp to a system prompt—can turn a cheap cache hit into a full-price miss, inflating bills by dozens of times.

The shift comes from Anthropic’s two-tier cache discount. For some models a cached read costs only 0.025 × the regular input price; for others the discount is 0.1 ×. The discount applies only when the request matches a previously cached entry exactly. Misses are billed at the base rate, so the gap between hit and miss widens dramatically. In practice, a modest prompt change can push a cost up to forty times the expected amount.

Why the change matters

Anthropic added the cache tier to encourage reuse of identical prompts, a common pattern in agents that repeatedly call the same toolset. The idea is simple: store the result of a prompt-tool combination once, then retrieve it cheaply on later calls. The new multipliers make the “retrieval” side much cheaper, but they also make the penalty for a “miss” far steeper.

Developers who built agents around stable system prompts now see any drift—intentional or accidental—break the cache chain. The result is a hidden tax on the service: a lower cache-hit ratio translates directly into higher operating costs.

What breaks the cache

Anthropic’s documentation describes a hierarchy where changes at higher levels invalidate everything beneath them. The practical upshot is that seemingly harmless edits can cascade into a full-price charge.

  • Tool definitions – Adding, removing, renaming a tool, or altering its description wipes the cache for tools, the system prompt, and all message history.
  • Web-search toggle – Flipping the boolean that enables a web-search tool clears the system prompt and message cache.
  • Tool-choice parameter – Tweaking the parameter that selects which tool to run only invalidates the message cache.
  • Image payloads – Adding or removing images affects only the message cache.

Common patterns developers inadvertently trigger:

  1. Reordering tools – Some codebases sort tool dictionaries on each deployment. The new order creates a different cache key, forcing a miss every time.
  2. Timestamped prompts – Embedding a “generated at HH:MM:SS” string in the system prompt makes every request unique, guaranteeing a miss.
  3. Rotating prompt fragments – Swapping out a greeting or a version banner changes the prompt hash and busts the cache.

Spotting the hidden cost

Anthropic’s usage logs expose the cache dynamic through three fields:

  • cache_read_input_tokens – Tokens read from a cache entry.
  • cache_creation_input_tokens – Tokens that caused a new cache entry to be stored.
  • input_tokens – Tokens billed at the regular rate (the remainder after cached reads).

A sudden rise in input_tokens alongside a dip in cache_read_input_tokens signals that something in the prompt stack has shifted. Monitoring these metrics lets teams react before the bill balloons.

The developer’s response

Facing the new pricing reality, many teams now treat prompt stability as a first-class performance metric. Common strategies include:

  • Static system prompts – Store the prompt in a version-controlled file and inject it without runtime modifications.
  • Deterministic tool ordering – Define tool lists directly in code rather than relying on dictionary ordering or external generators.
  • Timestamp removal – Move logging or timing information to a separate metadata channel that does not affect the prompt string.
  • Cache-aware testing – Add unit tests that verify the hash of the full prompt (system + tools + messages) remains constant across builds.

These practices add a small engineering overhead but protect against the “hidden tax” that a cache miss now represents.

Anthropic’s perspective

Anthropic argues that the deeper discount incentivizes reuse, which can reduce overall compute load on its servers. By making cached reads dramatically cheaper, they hope developers will design agents that call the same toolset repeatedly rather than constantly reshaping prompts. The trade-off is a higher penalty for non-reusable calls, which the company says nudges developers toward better prompt hygiene.

Critics note that many real-world agents need to adapt prompts on the fly—adding context, timestamps, or dynamic tool selections is often essential. For those workloads, the new pricing could make Anthropic less attractive compared with providers that charge a flat rate regardless of cache hits.

What to watch next

  • Pricing revisions – Anthropic may fine-tune the multipliers if community feedback shows the hit-miss gap is too wide.
  • Cache-control features – Future API updates could let developers specify which parts of a prompt should be excluded from the cache key, offering a middle ground.
  • Competitor responses – Other LLM providers might adjust their own cache models to stay competitive, either by offering flatter pricing or by exposing more granular cache controls.

Takeaway

With Anthropic’s new cache pricing, the cost of a prompt miss is no longer a marginal inconvenience; it’s a financial lever that can swing a project’s budget dramatically. Keeping system prompts, tool definitions, and related metadata immutable is now as important as writing efficient code. Teams that treat prompt determinism as a measurable metric will avoid surprise bills and stay in control of their AI-agent expenses.