Prompt caching
Cache-hit billing and how to improve the hit rate.
Repeated prefixes, such as conversation history, system prompts, and tool definitions, are billed at the cache-read price when cached. This is usually much lower than the input price; see the catalog for current prices. Long sessions in agent tools such as Claude Code and Codex benefit especially.
Confirm a cache hit
Use response usage:
| Protocol | Field |
|---|---|
| OpenAI | usage.prompt_tokens_details.cached_tokens |
| Anthropic | usage.cache_read_input_tokens |
Cached tokens are billed at the cache-read price; other input tokens at the input price.
How caching works
- Automatic caching: providers of models with context caching, such as DeepSeek, detect repeated prefixes without additional parameters.
- Explicit markers: with Anthropic, add
cache_control: {"type": "ephemeral"}to system or message blocks to mark cache breakpoints. The gateway forwards them when the model supports them.
{
"system": [
{
"type": "text",
"text": "<Long system prompt>",
"cache_control": {"type": "ephemeral"}
}
]
}Improve the hit rate
Caching requires identical prefixes. Place stable content first and changing content last:
- Put tool definitions and system prompts first and keep their bytes stable, including JSON field order.
- Append conversation history without rewriting previous messages.
- Add new content only at the end of each turn.
See model capabilities and prices in the catalog for cache support and cache-read pricing. Check actual charges per request in the console’s request logs.