The memory log of a production LLM agent swelled from a 2 KB file to 29,446 bytes, pushing the token count the agent reads from about 500 to 7,360 per run — a hidden expense that appeared without any alert on the monitoring dashboard. The developer behind the agent says the silent rise in read cost is a concrete instance of “agent cost drift,” a problem that can erode budgets even when the system appears to function normally.
What is agent cost drift?
Agent cost drift describes the gradual increase in the computational cost of an AI agent’s routine operations, caused by the agent’s own growing state. In the example, the agent maintains a work-log file that records why it rejected previous article topics. Each new entry adds a paragraph of reasoning, and the file is read in full before the agent generates new content. Because the log expands quadratically—each entry not only adds its own length but also references earlier entries—the amount of text the agent must ingest accelerates over time.
The drift is not a spike or a failure; it is a linear tax that compounds. The agent still produces two articles a day, and the dashboard shows no errors, yet each run now consumes 7,360 tokens, up from about 500 a month ago. Since most LLM providers charge per token, the increase in tokens per run translates directly into higher operating costs.
Why it matters
- Budget impact – Token-based pricing means every extra token read is money spent. The token count increased from 500 to 7,360.
The hidden mechanics
The file’s growth pattern is key. A line-by-line log that merely appends new entries would increase linearly, but because each entry explains the reasoning behind earlier rejections, the text length compounds. The result is a quadratic growth curve: doubling the number of entries more than doubles the file size, and the token count needed to process it grows faster still.
Developers rarely spot the rising cost because each entry “makes sense on its own.” The agent’s output remains correct, and the log continues to serve its purpose, masking the inefficiency.
Mitigation strategy
The developer proposes a three-part restructuring of the log:
- Recent-entry file – Keep a small, actively read file that contains only the latest few entries needed to avoid immediate repeats.
- Compact index – Store a one-line summary for older entries. The index can be scanned quickly to check for topic duplication without pulling in full paragraphs.
- Archived full text – Move the complete historical reasoning to a separate archive file that the agent does not read during normal operation.
This approach preserves the agent’s ability to avoid topic repetition while dramatically cutting the token load per run.
How to detect cost drift in your agents
- Instrument token reads – Record the number of tokens the agent consumes when loading its memory file each time it runs.
- Track over time – Compare today’s token count with the count from a week or month ago. A steady upward trend signals drift.
Simply verifying that the file still loads without error is insufficient; you must measure the cost of that load.
What to watch next
Agent cost drift is an invisible tax that can silently gnaw at your AI budget. By treating the agent’s memory file as a cost center—measuring token reads, watching growth curves, and restructuring logs—you can keep the tax low and the output sharp.
Join the discussion: https://t.me/GyaanSetuAi