Prompt caching patterns for using OpenAI models on Amazon Bedrock
According to AWS. Learn how prompt caching patterns help AI agents maintain continuity and reduce token costs when using OpenAI models on Amazon Bedrock.
- • AI agents repeatedly process their entire interaction history on every single turn.
- • Prompt caching stores input prompts to reuse them at a fraction of standard input pricing.
- • Amazon Bedrock supports both implicit and explicit caching mechanisms.
- • Combining both modes provides a reusable starting point while managing growing conversations.
The Agentic Loop
Every time an AI agent takes a step, it has to send the model its entire story so far to maintain continuity, including tool definitions, instructions, and previous results. Because the model lacks internal memory between requests, it reads all prior data repeatedly on every turn, incurring repeated token costs for every interaction.
Caching Mechanism
Prompt caching breaks this cycle by storing the input prompt and reusing it on subsequent requests at a fraction of the input price. During the prefill stage, the model processes input tokens and builds its working state, which the caching service then saves to avoid recomputing identical prefixes.
Implicit and Explicit Modes
Implicit caching lets Amazon Bedrock choose what to cache as conversations grow, whereas explicit caching allows developers to mark shared content using prompt_cache_breakpoint. Using both approaches together delivers a reliable shared starting point while the service automatically handles the expanding conversation history.
- 1. aws.amazon.comOpen source