Amazon Bedrock now offers prompt-caching for Claude 4.6, a feature that can cut response latency and reduce inference spend for generative-AI applications. The capability works by remembering a static portion of a prompt for up to five minutes, so subsequent calls skip the costly re-processing of that text.

How the cache fits into the request chain

When a Claude 4.6 request arrives, two layers cooperate.

  • Model level – Claude 4.6 maintains a key-value (KV) cache in GPU memory. The first time the model parses a block of instructions, it stores the resulting internal representation. On later calls that reuse the same block, the model can retrieve the representation instead of recomputing it.

  • Bedrock level – Bedrock computes a fingerprint of the static prompt segment. If a new request carries a matching fingerprint, Bedrock routes it straight to the GPU that already holds the cached state, bypassing the “warm-up” stage.

Think of it as loading a saved game rather than starting a new one each time.

Rules that keep the cache alive

  1. Minimum token count – Claude Sonnet 4.6 requires at least 1,024 tokens in the cached segment; Claude Opus 4.6 needs 4,096 tokens. Anything smaller is ignored.

  2. Five-minute lifetime – The cache expires after five minutes of inactivity. Each hit resets the timer, so a steady stream of calls can keep the cache alive indefinitely.

  3. Prompt ordering – Bedrock reads the prompt sequentially. The static instructions must appear first, followed by a cachePoint marker, with all user-generated messages after that marker. Changing even a single character before the marker breaks the fingerprint and forces a cold read.

Putting the feature into practice

The Bedrock Converse API is the entry point. Below is a minimal Python snippet that demonstrates the required structure.

import boto3

bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
MODEL_ID = "anthropic.claude-sonnet-4-6"

# Must be >1,024 tokens for Sonnet
BASE_SYSTEM_PROMPT = "Your long instructions here..."

system_configuration = [
    {"text": BASE_SYSTEM_PROMPT},
    {"cachePoint": {"type": "default"}}
]

conversation_history = []

def run_chat_turn(user_input):
    global conversation_history
    conversation_history.append(
        {"role": "user", "content": [{"text": user_input}]}
    )

    response = bedrock.converse(
        modelId=MODEL_ID,
        system=system_configuration,
        messages=conversation_history,
        inferenceConfig={"maxTokens": 500, "temperature": 0.4}
    )

    assistant_message = response["output"]["message"]
    conversation_history.append(assistant_message)

    metrics = response["usage"]
    print(f"Read from cache: {metrics.get('cacheReadInputTokens', 0)}")

The cachePoint tells Bedrock where the immutable segment ends. After the first call, the cacheReadInputTokens metric should show a non-zero value, confirming that the cache was hit.

Why developers should care

Separating static instructions from dynamic user input shifts work from expensive GPU cycles to a lightweight routing step. For chatbots, retrieval-augmented generation (RAG) pipelines, or any service that repeats the same system prompt, the result is faster replies and lower billable token counts. In high-throughput scenarios, even a modest reduction in compute time can translate into noticeable cost savings.

Limits and trade-offs

The feature only helps when the prompt segment meets the token minimums and stays unchanged. Applications that frequently tweak system instructions, or that rely on short prompts, will see little benefit. The five-minute window also means that bursty traffic with long idle gaps may repeatedly incur cold reads, eroding the latency advantage. Finally, the cache lives in GPU memory; if multiple models share the same hardware, contention could affect performance, though Bedrock does not expose those details.

What to watch next

  • Metric dashboards – Keep an eye on cacheReadInputTokens and overall latency to verify that the cache is being used as intended.
  • Prompt engineering – Designing prompts that satisfy the size thresholds without bloating the request is a new discipline for developers.
  • Future extensions – If Bedrock expands the cache duration or relaxes token limits, the economics of long-running conversations could shift further.

Takeaway: Prompt caching gives Claude 4.6 users a concrete lever to trim both response time and spend, provided they can lock down a sufficiently large, immutable prompt and keep calls within a short window. For any GenAI service that repeats the same system instructions, the feature is worth testing early.