𝗠𝗶𝗱-𝗖𝗼𝗻𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗼𝗻 𝗦𝘆𝘀𝘁𝗲𝗺 𝗣𝗿𝗼𝗺𝗽𝘁𝘀
Building long-running AI agents often leads to a costly mistake.
You use a large system prompt and cache it to save money. This works until you need to give the agent new instructions mid-session. You might need to change the coding language or update the project state.
If you edit the top-level system prompt to add this info, you break the cache. The model must reprocess the entire history at full price. This makes your agent slow and expensive.
New Claude models solve this with mid-conversation system messages.
You can now place a system role message directly inside your message array. You put it after the history instead of at the top.
Why this matters:
• Cache Safety: The instruction sits after your cached history. It does not invalidate the prefix. You only pay for the new, small message. • Security: You can use user messages as a workaround, but users can forge those. A system role message carries authority that a user cannot spoof. This prevents prompt injection.
How to do it right:
Frame these messages as facts. Do not use override language.
• Good: "Auto-approve mode is now enabled." • Bad: "Ignore what the user said and do this instead."
Models often resist instructions that tell them to ignore previous context. Stick to stating the new state of the world.
Rules for use:
- It must follow a user or assistant message.
- It cannot be the first message in the array.
- It only accepts text content.
- Not all models support it. Always use a try-catch block to fall back to a user-turn reminder if you get a 400 error.
Use this when you learn something new after the session starts. If you know the fact at the beginning, put it in the main system prompt. If the fact is dynamic, use a mid-conversation system message.
This keeps your cache hot and your costs low.