ClaudeDevs posted a short video this week of Jake Dejno and Lucca Siaudzionis, who work on Anthropic’s API. They talk about the features that ship with new Claude models, and how to tell whether an upgrade is actually cheaper. Dejno, on-screen as head of API engineering, and Siaudzionis, platform intelligence lead, put prompt caching first for cutting costs. They spend much of the clip on two knobs you can turn mid-conversation without breaking that cache: effort, and the system prompt.
The video opens with the line “A lot of roads lead to prompt caching.” In an agent run, they say, a later request is often “more like an append” to the one before it, so blocks that have already run can be cached rather than processed again. Caching, Siaudzionis and Dejno say, is “the first recommendation you get.”
New video: @dejno and @luccasiau discuss how we launch new models on our API and the features that ship with them. They also dig into how to rethink your harness and measure cost per task when you upgrade to a new model.
November 12, 2026
Knobs that used to break the cache
“Being able to change effort and system prompt mid-conversation are two things for me that really show the power of being able to have a knob later on,” one of them says in the video. You could always change those settings, they add, “but then you break prompt caching. And that’s, for a user, like a deal breaker.” Their example is an agent that has been running at high effort, then hits a follow-up that is actually easier. Dropping the effort mid-conversation, they say, is what gives you that control.
Anthropic’s docs already describe both moves, and they match that account. A mid-conversation system message is a role: "system" entry you append to messages, instead of editing the top-level system field, which sits near the start of the cached prefix and misses the cache if it changes. Per-message effort, which is in beta, uses the same kind of system message, with empty content and a new output_config.effort value, plus the mid-conversation-output-config-2026-07-01 beta header. You leave the top-level effort setting alone, so the cached prefix still matches. Changing that top-level value between requests can still invalidate the cache.
The docs also set limits the video doesn’t cover. Mid-conversation system messages are listed for current Claude models including Fable 5.2, Opus 5.5, Sonnet 5.5 and Haiku 5.5. Claude Sonnet 5 isn’t on that list. Per-message effort is still in beta, and only some models and platforms support it. On some setups it fails: Sonnet 5.5 with thinking: {"type": "between_tools"}, and Haiku 5.5 with thinking turned off, both return a 400 error if you try to change effort mid-conversation. Prompt caching itself is opt-in. You add a cache_control field to mark the blocks you want cached. A cache hit costs a fraction of a fresh input token. Anthropic publishes those rates on its pricing page.
| Topic | What they say in the video | What the docs add |
|---|---|---|
| Prompt caching | The first recommendation for cutting costs. In an agent run, a later request is often an append, so already-run blocks can be cached. | Caching is opt-in with cache_control. Cache hits need a byte-identical prefix, in the order tools, system, then messages. |
| System prompt mid-conversation | A knob you can turn later without breaking the cache. | Append a role: "system" message instead of editing the top-level system field. Listed for current models including Fable 5.2, Opus 5.5, Sonnet 5.5 and Haiku 5.5; not available on Sonnet 5. |
| Effort mid-conversation | Useful when an agent has been on high effort and the next request is easier. | Per-message effort is in beta. Leave the top-level effort value unchanged and send the mid-conversation-output-config-2026-07-01 header. A top-level change can still miss the cache. |
| Upgrades and cost | Use evals so a new model, or a change to thinking or effort, still hits the same outcome on a more favourable cost curve. | Anthropic’s cost cookbook says to measure cost per completed task, not cost per token, and to read pass rate and cost together. |
How they want you to measure an upgrade
From there, the conversation turns to how you measure a model change. Make sure your harness, the model you’ve chosen, and the knobs and dials are tuned, they say, so that if you upgrade to a new model or change a dial on thinking or otherwise, you still get the same outcome, “on a different cost curve, one that’s favorable to you.”
They’ve also noticed a habit: teams stay on Haiku, “which is great,” they say, and miss that a more capable model would leave them “in a much better place.” That lines up with Anthropic’s other published advice to measure cost per completed task, because a cheaper model that takes more turns can cost more in the end. The company’s model guide, published earlier this week, makes a similar point: start most work on Opus 5.5, and plot score against spend on your own tasks before you standardise on a mix of models.
Discover more from Sulat AI & Tech News
Subscribe to get the latest posts sent to your email.
