チケット分類AIの刷新により、100万件あたりのコストを22,200ドルから500ドルへ削減できた理由

Amazon is reported to have reduced the AI cost of its large-scale internal ticket classification system by approximately 98%. It did not simply switch to a cheaper large language model. Instead, it changed the way it had the AI process the same task: from writing a 'long explanation (composition)' to selecting only a 'correct answer (one word).'

By changing the output format, the average processing time dropped from 4 seconds to under 1 second, and the cost per million cases fell from approximately $22,200 to$500. We examine, through the case disclosed by Amazon's internal ticket system team (Carr/Mehul), why dramatic improvements in cost and speed are possible solely by reducing the output. ¹, ²

Why were 'long analytical texts' necessary for ticket classification?

Amazon's internal ticket system processes ticket classification tasks on a large scale, assigning customer escalations, server capacity requests, automated alerts, and security findings to one of 20 fixed categories. ²

Previously, a large language model (LLM) named 'Claude Sonnet 4.6' handled this task. The AI analyzed ticket content and generated complex JSON-formatted text, including reasons for selecting the respective labels. The amount of text (tokens) the AI output per ticket reached an average of 170. ²

However, when breaking down the essence of the classification task, it boils down to 'selecting one from a given list of 20 items.' Excessive 'composition' was being performed for a task that did not require long-form writing. Large language models typically charge fees proportional to the number of tokens used for both input and output. The longer the AI unnecessarily generated explanations, the more costs escalated. ²

'Decision-making models' that cut off text generation devices

The internal Amazon engineering team, which disclosed the system structure through the strandsagents.com blog, replaced the system with a decision-making model called 'Strands Decider.' This model differs from general generative AI in its internal structure. ², ³

General AI possesses a text generation device called an 'LM head' that selects the next character to build sentences. In contrast, Strands Decider cut off this LM head from the pre-trained language model's torso. ³

Instead, a small structure called a 'pointer head' that scores each candidate option was attached. Rather than commanding the AI to write the answer directly, it was configured to calculate only probabilities and confidence scores by asking, "Which is most suitable among A through T, and what is the probability?" Since it cannot generate text, it cannot produce nonsensical answers outside the list, and the process of writing character by character disappears, significantly speeding up processing. ³

The Amazon team initially used a 2B-scale model but switched to operating a 'E4B' model based on Gemma 4 after it showed better results in tests. ² For reference, the E4B model recorded an accuracy of 81.0% in an evaluation of 231 public tasks on JevBench v1. ⁴

Compressing input data and controlling variables based on order

It was not just the output format that was changed. Input data was also efficiently compressed to fit the model's processing limits.

The detailed description documents for the 20 labels in the existing operating environment already exceeded the decision-making model's maximum processing capacity of 4,096 tokens. Therefore, these were rewritten into short 'judgment criteria' of one or two sentences. Additionally, a 'none' option was added to handle tickets that did not exactly fit any list. For tickets with lengthy content, the system trimmed the volume by cutting from the oldest comments. ²

A phenomenon was also discovered where the order of data notation changed the results. Even with ticket data that was identical in content, if just the sorting order of JSON keys read by the AI changed, the probability designated as the answer fluctuated by up to 0.19. To solve this, the Amazon team adopted a method of pre-sorting key values of all request data in alphabetical order or similar before transmission. ²

Cost calculation showing a 98% reduction by shifting from metered to fixed cost structure

Through these adjustments, the cost incurred per ticket classification decreased significantly. The core reason for the sharp drop in costs lies in the 'change in billing structure.'

Comparison item	Existing: Claude Sonnet 4.6	Replaced: Strands Decider E4B
Average call time	4,215 ms	897 ms
Total tokens used	6,724	2,010
Output format	JSON averaging 170 tokens	1 option (1 token)
Cost per case	$0.0222	Approx. $0.0005
Cost per million cases	Approx. $22,200	Approx. $500

Table source: Amazon internal ticket classification system comparison data ²

Large language model APIs like Claude Sonnet 4.6 operate on a 'metered pricing' basis. $3 is charged per million input tokens, and$15 per million output tokens. Therefore, it is natural that costs decrease when output volume is reduced and input is compressed. ²

On the other hand, running Strands Decider E4B follows a 'fixed cost' structure where cloud server instances (ml.g5.xlarge) are directly operated at an hourly fee. The hourly cost of this server instance is $1.408. ¹, ²

Since the processing time per case is only about 640ms (0.64 seconds), one server can process approximately 5,600 tickets per hour when fully utilized. In this state, the unit price per million cases drops to the level of $250. However, in actual operation, reserve capacity must be maintained to handle traffic surges, so the actual cost per million cases, including all factors, was calculated at approximately$500. ²

Moving away from 'character-count proportional billing' calculated based on how long the AI writes, to a 'fixed cost' system proportional to server uptime is expected to lead to large-scale cost reductions estimated to reach millions of dollars annually. ¹, ²

Dual verification process under actual traffic

Even if performance indicators are good, applying a new model suddenly to an operating system can cause unexpected errors. The Amazon team introduced a dual structure called 'DUAL mode' to safely transition the system. ²

In DUAL mode, actual customer data is delivered simultaneously to the existing Sonnet model and the new Decider model. However, the classification result ultimately applied to the system remained that of the existing Sonnet. Only when the results of the two models differed were they recorded separately for precise comparison and analysis of classification differences. The Amazon engineering team reported gaining more operational information through this process than offline evaluations. They also established a safety measure allowing immediate rollback to the existing system in 45 seconds solely by changing settings if issues arose with the new model. ¹, ²

Meanwhile, after adjusting the confidence threshold and evaluating both models on a separate set of actual tickets, the Decider model showed better classification results than the operating Sonnet. ¹, ²

For what tasks can this approach be applied?

Replacement with decision-making models brings clear efficiency under specific conditions. According to materials disclosed by the Amazon team, this approach was particularly effective in environments that were 'closed tasks requiring selection from a predefined list,' had 'vast processing volumes,' and where 'latency (delay) is important.' ¹, ²

Conversely, due to the nature of models that do not generate text, it is unsuitable for generating detailed analytical sentences. It may be difficult to replace large language models for tasks where long explanatory documents are essential for classification labels, lists change every week, or useful context required for classification exceeds processing limits of thousands of tokens. ¹, ²

If it is a closed task requiring only selection from a predefined list without instructing the AI on complex reasoning or long text composition, and if processing volumes are vast and latency in existing paths is critical, alleviating the burden of long sentence generation creates significant advantages in both cost and speed. ¹, ²

参考文献

1. Amazon, Sonnet 4.6을 Strands Decider로 대체해 수백만 달러 절감 | GeekNews · news.hada.io
   ↩
2. Amazon saves millions by replacing Sonnet 4.6 with Strands Decider · strandsagents.com
   ↩
3. Introducing Strands Decider 2B: a small, open source, decision model · strandsagents.com
   ↩
4. Strands Decider adds four Gemma 4 models · strandsagents.com
   ↩