𝗪𝗵𝘆 𝗔𝗜 𝗖𝗵𝗮𝗿𝗮𝗰𝘁𝗲𝗿𝘀 𝗙𝗼𝗿𝗴𝗲𝘁 𝗧𝗵𝗲𝗶𝗿 𝗜𝗱𝗲𝗻𝘁𝗶𝘁𝘆
You spend two hours building a character. After thirty turns, the character forgets her name. She starts acting like a generic assistant.
Most people think the AI has a bad memory. This idea is wrong.
The problem is the context window.
Chat models are stateless. Every time you send a message, the app sends the system prompt, the character details, and your recent chat history back to the model.
What you call memory is just the history staying inside that input window.
When the chat gets too long, old messages fall out of the window. The detail from turn 12 disappears to make room for turn 41. The model does not forget. The app simply stops showing it the information.
A bigger context window is not the fix.
Large windows cause two problems:
- Cost and speed. Doubling the window size increases compute needs significantly. Apps often limit the window to keep replies fast and cheap.
- Middle degradation. Research shows models find facts well at the start or end of a long text. They miss facts buried in the middle.
The best apps do not rely on raw window size. They use two specific methods:
- Recursive summarization: The app turns old messages into a short summary. This keeps the core facts alive without using much space.
- Retrieval: The app stores old messages in a database. It pulls out only the relevant facts when you need them. This is how lorebooks work.
The difference between a good app and a bad one is the architecture. One uses a memory layer. The other is just a thin wrapper over a raw window.
If your character loses their personality, do not use a bigger model. Change the structure. Summarize the old turns. Retrieve what matters.
Treat the context window like a cache, not a hard drive.
Optional learning community: https://t.me/GyaanSetuAi