Trustle tested Jev, a fast, low-cost decision model, for choosing least-privilege IAM roles and assessing access risk. Unlike a conventional language model that generates text, Jev answers typed questions from a fixed set of options and supplies probabilities that can support confidence thresholds. In a secret-access example, it selected the three-permission role needed to read a secret, with 0.99 confidence, while Trustle’s existing search ranking favored a role with 2,563 permissions. Risk-scoring experiments showed that broad questions produced uninformative results, while focused questions combined in code worked better. After refining the inputs and fitting weights against an independent reference, agreement rose from 0.75 to 0.86 on held-out roles.
In tests of 66 access requests, Jev alone reached 91.4% accuracy, compared with Gemini’s 98.0%; a hybrid that used Jev when confident and sent uncertain cases to Gemini reached 98.5%. Jev selected an over-broad role in 1.5% of cases, responded in about 0.1 seconds, and cost roughly $0.10 per 1,000 decisions. The hybrid cut Gemini requests by more than half while preserving accuracy; a 0.9 confidence threshold also scored 97.9% on held-out cases. Jev struggled with indirect privilege escalation, counting permissions, multi-role requests, and tasks better handled by rules. Its effectiveness depended on precise, narrow questions and well-chosen response options.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
Researchers tested 16 LLMs on all 55 TIS-100 puzzles, giving each one attempt and time to refine its code. The models found 133 new performance tradeoffs across 18 puzzles and set three outright cycle records, though human players improved on most solutions within days. (tis-100.wincon.dev)
Australia is considering a new criminal offence for AI companies that fail to take reasonable steps to prevent harm caused by their models. The government is also weighing an AI safety watchdog with power to impose significant fines, following a hack of a Medicare statistics portal involving OpenAI. (abc.net.au)
AI is increasingly becoming a gateway through which people find information. Audiences still resist having AI produce their news. (niemanlab.org)
Nvidia holds a central position in the AI industry, making disruption to the company potentially catastrophic for the wider ecosystem. The race for advanced AI is also intensifying economic and geopolitical conflict, while concentrated corporate power and infrastructure face growing political resistance. (france24.com)
LLM inference can require reading about 26 GB of model weights to generate each token from a 13-billion-parameter model. The explanation shows how model size drives memory bandwidth demands during text generation. (youtube.com)
Deletion Tuning matches full fine-tuning across three LLM post-training methods by zeroing fewer than 0.05% of model weights. Its compact masks also preserve prior-task accuracy better than full fine-tuning, LoRA, and random sparse tuning. (arxiv.org)
Today's best Hacker News stories, summarized and screenshotted, one email a day.