hn.today

Show HN: Lumabri – Run large AI models across your computers and servers.Pure C

github.com5 points0 comments
Screenshot of Show HN: Lumabri – Run large AI models across your computers and servers.Pure C

Lumabri is a lightweight C-based inference engine that shards large models across multiple personal computers and servers so models that do not fit on one machine can run by distributing layers and state. It integrates with the Colibri runtime and provides a terminal TUI plus a bundled browser chat and local API. Workers do not need full model checkpoints; each donor loads contiguous layer ranges and retains the state for those layers while the requesting host keeps the conversation text. Models and resident allocations are controlled by explicit memory budgets, per-donor approvals, and saved plans; up to four model allocations and 1-8 conversation slots can coexist on a donor within its budget.

Operation emphasizes safety and measured capacity rather than automatic speed scaling: donors must accept requests (defaults to decline), optional calibration runs measure short inference speed, and exhausted capacity returns BUSY rather than preempting running work. Network discovery and household membership use fixed TCP/UDP ports and a shared household key; an advertise override supports multi-interface setups. Sharing respects local RAM reserves and runtime capabilities (OpenMP affects threading), replica policies can optionally restore managed-capacity after failures, and resident weights persist across TUI sessions. The project builds against a compatible Colibri checkout and documents firewall, service lifecycle, and local-API authentication.

Read on github.com0 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.