๐ง๐ต๐ฒ ๐ ๐ฒ๐๐๐ฎ๐ด๐ฒ๐ ๐๐ฟ๐ฟ๐ฎ๐ ๐๐ผ๐๐ ๐ง๐ฟ๐ฎ๐ฝ
Part 3 of Building TinyAgent. AI agents have no memory. You create memory with a messages array. You send the full history every time you call the API.
This costs you money. Turn 1: You send 1 message. Turn 10: You send 19 messages. Turn 30: You send 59 messages.
The cost is not a straight line. It curves up. You pay for every old message on every new turn.
The first 10 turns feel free. This is a trap. The bill jumps at turn 20.
Three ways to fix this:
- Full History: Send everything. Good for short chats.
- Sliding Window: Keep the last few turns. The agent forgets the start.
- Summarization: Shrink old turns into a summary. This saves money and keeps context.
Anthropic has a trick called caching. It stores the prompt state. You pay 10% for cached input. This makes long system prompts cheap.
Stop guessing. Measure your turns. Pick the best strategy for your users.
Next: We teach the agent to run functions.
Source: https://dev.to/jasmin/the-messages-array-in-4-gifs-1k1j