OpenAI’s Agents API can now keep an agent’s prompt in its cache for at least 12 hours, up from the default 30 minutes. Michael Luo, who works on product at OpenAI, announced the change alongside two others: async functions, which let an agent keep working while a slow tool runs, and a per-session budget control that’s in beta by request.
The Agents API is OpenAI’s hosted service for running agents. It’s been in public beta since 10 September. Developers bring the tools, and OpenAI handles each session, trims old context when it piles up and recovers runs that fail. Luo said customers running long-lived agents, “especially personal agents”, were getting “a lot of use and cost savings” from the longer cache. He didn’t give numbers, and OpenAI hasn’t published any.
🕛 12 hour prompt cache – now you can keep prompts in cache for 12 hours! We’ve seen customers of long running agents, especially personal agents, finding a lot of use and cost savings with this
— Michael Luo on X
💨 Async functions – Let your agent continue working after it calls a tool, without waiting for that tool’s result. Use it to start slow lookup requests early, answer independent parts of a request, and provide results when your application has them.
💸 Budget control (beta) – Limit, monitor, and control your agent budgets on a per-session level with Agents API. Now you don’t have to worry about a single customer consuming all your tokens! Interested? DM or email me and I can get you in.
The 12-hour cache isn’t new to OpenAI’s platform. The company added it to the Responses API, its main endpoint for model calls, on 20 October, and Luo’s recap of October’s Agents API releases already listed it. What the Agents API reference now shows is a session setting, prompt_cache_options.ttl, that accepts "30m" or "12h" and sets the default cache lifetime for every turn in that session.
You can now choose 12-hour prompt caching in the API.
— OpenAI Developers on X
Persistent agents, like dots, reuse context throughout the day.
You can keep shared instructions, tools, and context cached longer and pay the lower cached-input rate, even when requests are hours apart.
What a prompt cache does
Before a model answers, it has to process everything in the prompt: the instructions, tool descriptions and conversation so far. A prompt cache saves that work for the opening stretch of a prompt, which OpenAI calls the prefix. If the next request starts with exactly the same prefix, the model can pick up from the saved state instead of processing those tokens again.
OpenAI charges less for tokens read from the cache and more for tokens written to it. An agent that comes back to the same long instructions all day can save money, but only if its prefix is still cached and still matches.
What the longer cache costs
On GPT-5.6 and later models, writing a prefix to the cache for the default 30 minutes costs 1.25 times the normal input price. A 12-hour write costs 2.5 times. Reading from the cache costs the same under either lifetime: a tenth of the normal input price on most models, and a twentieth on GPT-6.1 Sol and GPT-6.1 Terra, according to OpenAI’s caching guide. OpenAI says the write prices are totals, so tokens written to the cache aren’t also charged the normal input price.
| Model | Normal input | Cache read | 30-minute write | 12-hour write |
|---|---|---|---|---|
| GPT-6 Astra | $10.00 | $1.00 | $12.50 | $25.00 |
| GPT-6.1 Sol | $2.00 | $0.10 | $2.50 | $5.00 |
| GPT-6.1 Terra | $1.00 | $0.05 | $1.25 | $2.50 |
| GPT-6 Luna | $0.10 | $0.01 | $0.125 | $0.25 |
That makes the 12-hour option a bet on gaps. Under the 30-minute lifetime, each read pushes the expiry back by at least another 30 minutes, so an agent that’s called every few minutes keeps its cache warm without paying for a second write. A 12-hour write costs the same as two 30-minute writes. The longer lifetime comes out cheaper only when the prefix would otherwise go cold and be rewritten at least three times in the same 12 hours.
Here’s how that plays out for an agent whose fixed instructions and tool descriptions take up 50,000 tokens on GPT-6.1 Sol. Sending that prefix without caching costs 10 US cents each time. Writing it costs 12.5 cents for 30 minutes or 25 cents for 12 hours, and each later read costs half a cent.
| Usage pattern | 30-minute cache | 12-hour cache |
|---|---|---|
| Wakes once an hour, 12 times | $1.50 (a fresh write every time) | $0.31 (one write, 11 reads) |
| Called every 20 minutes, 36 times | $0.30 (one write, 35 reads) | $0.43 (one write, 35 reads) |
In the hourly case, the default setting costs more than sending the prefix uncached would, because each rewrite is billed at 1.25 times the normal rate. Luo singled out personal agents, and the Agents API’s scheduled automations, added in October, can wake an agent at set times.
Switching lifetimes can cost extra
The guide also warns about moving a prefix that’s already cached for 30 minutes onto the 12-hour lifetime: OpenAI charges a new 12-hour write for those tokens, with no credit for the earlier write. Marco Kotrotsos, who writes the AI newsletter Gloss, tested this on his own API key with a 7,131-token prefix on GPT-6 Luna. A request with the 12-hour setting, sent six seconds after a 30-minute write of the same prefix, reported zero cached tokens and a full 12-hour write, so he paid 3.75 times the normal input price for that prefix across the two requests. Going the other way cost nothing extra: a 30-minute request against a 12-hour entry came back as a cache read.
Kotrotsos ran his checks on the Responses API, and the guide says Agents API model calls follow the same caching rules. They’re one developer’s results on one model, and they match what the guide says should happen. He also found that Chat Completions, OpenAI’s older endpoint, rejected the 12-hour value, which fits the guide’s description of it as a Responses API option.
A longer cache doesn’t guarantee a hit
“The minimum lifetime governs eligibility for cache-read pricing. It does not guarantee that every request reuses stored KV tensors or has lower latency.”
— OpenAI prompt caching guide
OpenAI keeps the cached state on individual machines, and a request can only use it if it reaches a machine that holds a matching entry. Traffic above 15 requests a minute can be sent to other machines, the guide says. So the 12-hour setting fixes the price for reads within the window, while any speed gain still depends on where each request lands. Cached tokens also still count towards rate limits, and caches aren’t shared between organisations.
The prefix also has to match exactly. Replies to Luo asked whether the cache survives a model swap mid-run and how changed tool output affects later turns (David Hampl, AJ). The guide answers both in general terms: a different model can use different weights and caching behaviour, and changes to tools or reasoning settings can break the match. When a tool result changes, everything after the first changed token misses the cache, while earlier eligible breakpoints, the points where OpenAI saves a prefix, can still be reused. Compaction, where the agent summarises old conversation to save space, rewrites the prefix too. The Agents API has run compaction in the background by default since 30 October.
There has been one change in the agent’s favour. Steve Coffey of OpenAI’s Agents API team said on 30 October that system prompts are now cached automatically across all sessions, so a new session with the same instructions doesn’t have to start cold.
💰 Better prompt-cache awareness: System prompts are now automatically cached across all sessions, saving you money and time
— Steve Coffey on X
On data handling, the guide says the cache stores the model’s intermediate calculations rather than the prompt text, kept as encrypted state for a maximum of 24 hours. Choosing 12 hours sets a minimum lifetime; OpenAI may keep entries longer, and the setting doesn’t change its data-retention policy.
Billing visibility lags behind the price change
The higher write price makes it harder to estimate a bill from Agents API data. In the Responses API, usage reports now split cache writes into 30-minute and 12-hour counts. The Agents API’s usage fields report cached tokens but no separate cache-write count, and OpenAI’s own Agents API documentation says they “cannot determine the exact model charge when that pricing applies.”
One developer raised exactly that under Luo’s post. Luo replied that “we should do this” and asked how the counts would be used. The developer said they want to pass usage costs on to their customers precisely.
The agents API doesn’t currently return counts for cache write or uncached tokens-in. I’d really like to have that so that we can bill customers precisely.
— lost name on X
Third-party tools can lag as well. A pull request on LiteLLM, a popular open-source proxy for routing model calls, says the tool had been pricing every cache write at the 30-minute rate, so “spend logs and budgets showed about half the real cost of a 12h write.” The fix was opened on 21 October and hadn’t been merged when we checked.
Async functions
Function tools are how an agent calls a developer’s own code, such as a customer lookup. Normally the agent pauses until the result comes back. The session reference now lists an async option on function tools, described as “whether the agent continues without waiting for the function result.” It defaults to off.
Luo suggested using it to start slow lookups early and answer the independent parts of a request in the meantime. The Responses API gained a similar async tool-calling feature in September. The Agents API’s function tools guide still describes only the waiting flow, in which the application returns each result using the turn and call IDs the session supplies, so it doesn’t yet say how async results should be handled once the agent has moved on.
Budget control is in beta, by request
The third item limits and tracks spending for each session. Luo said it would stop “a single customer consuming all your tokens” and asked developers to message or email him for access. The public session reference doesn’t list a budget setting, and OpenAI hasn’t published how limits are counted or what happens when one is reached.
OpenAI already offers monthly spend limits for whole organisations and projects, which return an error once tracked spending reaches the cap. Per-session limits would work at a much finer level, inside a single customer’s conversation. Replies to Luo asked for warnings before a hard stop, so an agent can save its progress (NewYorkboy, Mangesh Raut), and whether a budget would count async calls that are still running (Véu). Luo hadn’t answered those publicly.
Luo ended his post by asking for feature requests to prioritise “for next week”. Until the usage data changes, OpenAI’s documentation says Agents API developers can see how many tokens came from the cache but not how many were written to it.
Discover more from Sulat AI & Tech News
Subscribe to get the latest posts sent to your email.
