Amazon engineers replaced Claude Sonnet 4.6 with Strands Decider E4B to classify tickets in a large internal system, where each ticket must be assigned to one of 20 categories. They argue that this task did not require a text-generating language model: it could be framed as choosing among fixed options, with the ticket as input and the category descriptions as choices. The team tested an initial 2-billion-parameter Decider, then moved to the E4B Gemma 4 build after it performed better. On a separate ticket set, E4B also outperformed the production Sonnet system after the engineers tuned the confidence threshold for applying labels.
In production comparisons, Decider cut average processing time from 4,215 to 897 milliseconds and reduced tokens per ticket from 6,724 to 2,010, a 70% drop. The team shortened the descriptions of all 20 labels into concise criteria and added a “none” choice, while Decider’s output was just a label rather than generated JSON. They ran the model on SageMaker instances and used an AppConfig setting to control a cautious rollout: Sonnet alone, both models in parallel for comparison, or Decider alone. The switch to Decider is projected to save millions of dollars annually, with model costs reported as 98% lower. The setup uses standard AWS services, including SQS, Lambda, SageMaker, AppConfig, CloudWatch, and CDK.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
Researchers tested 16 LLMs on all 55 TIS-100 puzzles, giving each one attempt and time to refine its code. The models found 133 new performance tradeoffs across 18 puzzles and set three outright cycle records, though human players improved on most solutions within days. (tis-100.wincon.dev)
Australia is considering a new criminal offence for AI companies that fail to take reasonable steps to prevent harm caused by their models. The government is also weighing an AI safety watchdog with power to impose significant fines, following a hack of a Medicare statistics portal involving OpenAI. (abc.net.au)
AI is increasingly becoming a gateway through which people find information. Audiences still resist having AI produce their news. (niemanlab.org)
Nvidia holds a central position in the AI industry, making disruption to the company potentially catastrophic for the wider ecosystem. The race for advanced AI is also intensifying economic and geopolitical conflict, while concentrated corporate power and infrastructure face growing political resistance. (france24.com)
LLM inference can require reading about 26 GB of model weights to generate each token from a 13-billion-parameter model. The explanation shows how model size drives memory bandwidth demands during text generation. (youtube.com)
Deletion Tuning matches full fine-tuning across three LLM post-training methods by zeroing fewer than 0.05% of model weights. Its compact masks also preserve prior-task accuracy better than full fine-tuning, LoRA, and random sparse tuning. (arxiv.org)
Today's best Hacker News stories, summarized and screenshotted, one email a day.