hn.today

Inkling 2 Mini, Cutting-Edge Performance for Your Local Setup

huggingface.co15 points1 comments
Screenshot of Inkling 2 Mini, Cutting-Edge Performance for Your Local Setup

Inkling 2 Mini is a multimodal mixture-of-experts model designed to deliver strong performance on local AI setups. It has about 26 billion physical parameters but activates roughly 3 billion per token, and accepts text and images with a context window of up to one million tokens. Its defining feature is a looped transformer: the middle 12 of its 24 physical decoder layers run three times, with learned messages passed between runs to refine their computations. This produces 48 layer visits per token while reusing weights, reducing the number of distinct layers stored. The model routes each token to eight of 128 experts. A research variant allows fewer loop passes for faster inference, potentially at some cost to output quality.

Benchmark results place Inkling 2 Mini ahead of Gemma 4 31B, Muse Glimmer, Qwen3.8-27B, and Nemotron 3.5 Super VL on 32 of 48 evaluated tasks spanning coding, tool use, vision, reasoning, and safety. It leads FrontierCode, ties Qwen on Terminal-Bench, and trails it on AutomationBench; informal tests suggest performance comparable to Qwen3.8-27B. The release includes mixed-precision expert weights and an optional drafter for faster generation. Users can run it through Transformers, llama.cpp and Llama App, or MLX, while SGLang provides production inference support. Quantized GGUF versions are available in 4-bit and 8-bit formats, and Transformers supports image inputs, adjustable reasoning effort, and tool calling.

Read on huggingface.co1 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.