Prompt Caching

What is Prompt Caching

Prompt caching lets a stable portion of context, most often a long system prompt or a set of skill descriptions, be processed once and reused across later requests instead of being reprocessed from scratch every single time. Since the system prompt is resident on every turn of a session, caching it is often the single biggest lever available for cutting repeated cost.

The mechanism only helps content that actually stays the same between requests. A long, stable set of tool definitions and skill descriptions is a strong candidate. The most recent user message obviously is not, it changes every turn by definition, so caching applies to the parts of context that repeat, not the parts that are genuinely new each time.

Prompt caching does not change what the model actually produces, it is an efficiency mechanism, not a behavioral one. It is also fragile in one specific way: any change to content before the cached portion, editing the system prompt, adding a new skill, invalidates the cache for everything after it, since the cache depends on that earlier content staying exactly the same.