Title: Rename A Tool, Lose The Entire Cache

Anthropic’s new cache pricing means a single character tweak—renaming a tool or adding a timestamp to a system prompt—can turn a cheap cache hit into a full-price miss, inflating bills by dozens of times.

The shift comes from Anthropic’s two-tier cache discount. For some models a cached read costs only 0.025 × the regular input price; for others the discount is 0.1 ×. The discount applies only when the request matches a previously cached entry exactly. Misses are billed at the base rate, so the gap between hit and miss widens dramatically. In practice, a modest prompt change can push a cost up to forty times the expected amount.

Why the change matters

Anthropic added the cache tier to encourage reuse of identical prompts, a common pattern in agents that repeatedly call the same toolset. The idea is simple: store the result of a prompt-tool combination once, then retrieve it cheaply on later calls. The new multipliers make the “retrieval” side much cheaper, but they also make the penalty for a “miss” far steeper.

Developers who built agents around stable system prompts now see any drift—intentional or accidental—break the cache chain. The result is a hidden tax on the service: a lower cache-hit ratio translates directly into higher operating costs.

What breaks the cache

Anthropic’s documentation describes a hierarchy where changes at higher levels invalidate everything beneath them. The practical upshot is that seemingly harmless edits can cascade into a full-price charge.

  • Tool definitions – Adding, removing, renaming a tool, or altering its description wipes the cache for tools, the system prompt, and all message history.
  • Web-search toggle – Flipping the boolean that enables a web-search tool clears the system prompt and message cache.
  • Tool-choice parameter – Tweaking the parameter that selects which tool to run only invalidates the message cache.
  • Image payloads – Adding or removing images affects only the message cache.

Common patterns developers inadvertently trigger:

  1. Reordering tools – Some codebases sort tool dictionaries on each deployment. The new order creates a different cache key, forcing a miss every time.
  2. Timestamped prompts – Embedding a “generated at HH:MM:SS” string in the system prompt makes every request unique, guaranteeing a miss.
  3. Rotating prompt fragments – Swapping out a greeting or a version banner changes the prompt hash and busts the cache.

Spotting the hidden cost

Anthropic’s usage logs expose the cache dynamic through three fields:

  • cache_read_input_tokens – Tokens read from a cache entry.
  • cache_creation_input_tokens – Tokens that caused a new cache entry to be stored.
  • input_tokens – Tokens billed at the regular rate (the remainder after cached reads).

A sudden rise in input_tokens alongside a dip in cache_read_input_tokens signals that something in the prompt stack has shifted. Monitoring these metrics lets teams react before the bill balloons.

The developer’s response

Facing the new pricing reality, many teams now treat prompt stability as a first-class performance metric. Common strategies include:

  • Static system prompts – Store the prompt in a version-controlled file and inject it without runtime modifications.
  • Deterministic tool ordering – Define tool lists directly in code rather than relying on dictionary ordering or external generators.
  • Timestamp removal – Move logging or timing information to a separate metadata channel that does not affect the prompt string.
  • Cache-aware testing – Add unit tests that verify the hash of the full prompt (system + tools + messages) remains constant across builds.

These practices add a small engineering overhead but protect against the “hidden tax” that a cache miss now represents.

Anthropic’s perspective

Anthropic argues that the deeper discount incentivizes reuse, which can reduce overall compute load on its servers. By making cached reads dramatically cheaper, they hope developers will design agents that call the same toolset repeatedly rather than constantly reshaping prompts. The trade-off is a higher penalty for non-reusable calls, which the company says nudges developers toward better prompt hygiene.

Los críticos señalan que muchos agentes del mundo real necesitan adaptar los prompts sobre la marcha; añadir contexto, marcas de tiempo o selecciones dinámicas de herramientas suele ser esencial. Para esas cargas de trabajo, el nuevo esquema de precios podría hacer que Anthropic sea menos atractivo en comparación con proveedores que cobran una tarifa plana independientemente de los aciertos de caché.

Qué observar a continuación

  • Revisiones de precios – Anthropic podría ajustar los multiplicadores si los comentarios de la comunidad muestran que la brecha entre aciertos y fallos es demasiado amplia.
  • Funciones de control de caché – Las futuras actualizaciones de la API podrían permitir a los desarrolladores especificar qué partes de un prompt deben excluirse de la clave de caché, ofreciendo un punto medio.
  • Respuestas de la competencia – Otros proveedores de LLM podrían ajustar sus propios modelos de caché para seguir siendo competitivos, ya sea ofreciendo precios más uniformes o exponiendo controles de caché más granulares.

Conclusión

Con el nuevo esquema de precios de caché de Anthropic, el coste de un fallo de prompt ya no es un inconveniente marginal; es una palanca financiera que puede alterar drásticamente el presupuesto de un proyecto. Mantener los prompts del sistema, las definiciones de herramientas y los metadatos relacionados como inmutables es ahora tan importante como escribir código eficiente. Los equipos que traten el determinismo de los prompts como una métrica medible evitarán facturas sorpresa y mantendrán el control de sus gastos en agentes de IA.