What Strands Decider is, and how Amazon engineers say it cut ticket-labelling costs by 98%

Strands Decider is a small open-source decision model: it does not write prose, code or summaries. You give it some state, a question, and a fixed list of options. In one forward pass it returns a probability for each option, plus a confidence score for the choice. The Strands team, part of the open-source Strands Agents project, launched the first model on 1 October and has since published larger variants.

That design is the opposite of a general-purpose large language model. An LLM can generate any text, including a label in JSON and a chain of reasoning. A decision model can only pick from the options you offered, or give a score. In exchange, the authors say it is faster at a given size, always answers from the offered set, and can attach a reliability score that frontier APIs usually do not return. They also say it is a poor fit for hard reasoning, and unsuited to coding, chatbots or document summarisation.

How it works

The first release, Strands Decider 2B, starts from a pre-trained Qwen3.5-2B “torso.” The language-modelling head that would have produced the next token is removed, so the model can no longer generate text. It is replaced with a small pointer head of just over a million parameters. That head scores the hidden state at each option against the hidden state at a special marker position. The torso is fine-tuned with a rank-16 LoRA adapter. The GitHub repository and introductory post by Marc Brooker, Mike Chambers and Fabio Nonato de Paula describe this as a second architecture: an earlier slot head, they say, performed significantly worse.

Step	What happens
1. You supply a task	State (for example a ticket or a user message), a question, and a closed list of options
2. One forward pass	The torso reads the text once. The pointer head scores each option. There is no decoding loop
3. You get a choice	A probability for every option, and a confidence score for the selected answer

The authors measure the 2B model at a median of about 115 milliseconds on an Nvidia RTX 3090, and about 153 milliseconds for small tasks on an M3 MacBook. Accuracy and calibration are reported on JevBench, a third-party benchmark for this class of model. The introductory post said the 2B model was third of 33 in its class on that public set, or first of 30 if slightly larger “just-over-2B” models are excluded. Those figures are the team’s own reported scores, not an independent re-run.

The family has grown since that first post. The repository now lists a later 2B checkpoint, strands-decider-2B-qwen3.5-v1-2610, and Gemma 4 builds in several sizes, including strands-decider-E4B-gemma4-v1-2610. Weights are on Hugging Face, and the repo includes training data and scripts for the published recipes.

Amazon’s ticket-labelling swap

In a 9 November post on the same site, two Amazon engineers, Carr Onstott and Mehul Jaiswal, writing with the Strands Agents team, describe replacing Claude Sonnet 4.6 with a Strands Decider model on an internal ticketing system. They say the system is one of the world’s largest, with tickets that can be customer escalations, capacity requests, alarms, security findings or other work, and that each ticket had to be placed in one of 20 category types. This account comes from that Strands blog, not from a separate Amazon or AWS press release.

The previous path ran Sonnet 4.6 with a long system prompt on every ticket, and asked it to generate JSON and reasoning against a fixed label set. A decision model cannot generate that text, but the job is a closed choice: the ticket is the state, the catalogue is the options. The engineers started with the original 2B model, then moved to the E4B Gemma 4 build after it did better on a test set, and tuned a confidence threshold before applying a label. They say E4B then beat production Sonnet on a separate set of their tickets.

They served E4B on SageMaker ml.g5.xlarge instances (one A10G GPU with 24 GB of memory). The model used about 19 GiB, and one request took about 640 milliseconds on a single instance. Ticket events landed in an SQS queue, a Lambda function called the endpoint, AppConfig held a three-way switch, and CloudWatch carried logs and alarms. The switch had three modes: Sonnet only, both models in parallel with Sonnet’s result applied and Decider’s compared, and Decider only. They say judging disagreements on live traffic told them more than an offline evaluation, and that going back would have been a 45-second configuration change.

The cost and speed figures they published

On the measurements they published, average time per call fell from 4,215 milliseconds with Sonnet 4.6 to 897 milliseconds with Strands Decider E4B, a 78% drop. Tokens per ticket fell from 6,724 to 2,010, partly because Decider returns a single choice rather than a JSON answer, and partly because the prompt was shortened. Production label descriptions alone were longer than Decider’s 4,096-token window, so each of the 20 labels was rewritten as a one- or two-sentence criterion, with a “none” option. On long tickets, older comments were dropped to fit. They also say average Lambda duration fell 74% on the day Sonnet was turned off.

	Claude Sonnet 4.6 (Bedrock)	Strands Decider E4B (SageMaker ml.g5.xlarge)
Published list price, us-east-1 on-demand	$3.00 per million input tokens, $15.00 per million output tokens	$1.408 per instance-hour
Cost they calculate per ticket	$0.0222	about $0.0005
Cost they calculate per million tickets	about $22,200	about $500

The $0.0005 figure is not a raw token price. At about 640 milliseconds per request, they say one instance can handle about 5,600 tickets an hour, or $250 per million tickets at full use. They keep headroom for bursts, “and in practice it comes to about $500 per million.” That is the 44-times, or 98%, reduction they report. These are list prices, and they leave out the Lambda, which now also runs for less time. The same post says the change “will save millions of dollars of token costs in a year.” That is a projection from these unit costs. The engineers do not publish how many tickets the system handles in a year, so the annual figure cannot be checked from the post alone.

They also found that JSON key order changed the model’s answers, moving output probabilities by up to 0.19 on the same content, and now send every request with sorted keys. They say the pattern worked because the task is a closed choice from a known catalogue, the volume is high, and the old path’s latency mattered. If labels need explanation text, the catalogue changes weekly, or the useful context will not fit in a few thousand tokens, they would not expect a decision model to replace the LLM.