Janus is a single Go binary that runs local .gguf LLMs (via llama.cpp) with Vulkan GPU acceleration on AMD/Intel/NVIDIA or a CPU fallback, exposing an OpenAI-compatible API and a built-in web UI. It routes prompts, runs inference, and can execute toolchains so models drive behavior while Go handles execution and routing. Key features include /v1/chat/completions and /v1/models endpoints (streaming supported), hot-swappable models in the UI, an extensible toolset (file I/O, shell commands, math, docx/PDF export, OCR via Tesseract), optional Ollama proxying, and admin/auth/safe-mode controls. It targets local workflows with no Python/Docker required and supports integration with standard OpenAI clients.
Practical specifics: Windows is the primary target but Linux and macOS are supported; Go 1.22+ is required and a Vulkan-capable GPU is recommended for GPU inference. Typical model files are 2-8 GB and Janus uses ~50 MB for the binary plus llama.dll; initial model load can take 10-60 seconds. Quick start covers cloning, building (prebuilt llama.cpp Vulkan DLLs provided for Windows), placing a .gguf in ./models, editing .env (INFERENCE_BACKEND, JANUS_MODEL_PATH, VRAM and token limits, JANUS_AUTH), and running on http://127.0.0.1:8990. Common pitfalls and fixes (DLLs, port conflicts, wrong paths, missing Tesseract) are documented in the repo.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
AstaBrief is an open-source model designed for fast scientific report generation, capable of turning research questions and literature into cited reports. It reduces generation time by nearly three and a half times compared to proprietary models, making it a useful tool for researchers. (allenai.org)
Shrivu Shankar argues that SaaS companies are evolving into 'harnesses,' where core tasks are performed by AI agents rather than humans. This shift transforms organizational structure, with humans overseeing and reviewing AI-driven processes within the company. (blog.sshh.io)
Three AI agents - Meta’s Muse, Anthropic’s Claude, and OpenAI’s GPT - were tested on multilingual research tasks involving U.S. and Iranian data. The experiment focused on how language and context influence agent performance across reasoning, sourcing, and artifact creation. (royapakzad.substack.com)
An open-source project named LDRAW-NOVA allows users to generate Lego models using AI. The repository contains tools, examples, and documentation for creating Lego designs with artificial intelligence. (github.com)
DwarfStar 4 (ds4) is a local inference engine that runs large language and vision models on high-memory Mac, CUDA, and ROCm machines. It supports models like DeepSeek V4 and Qwen3.8, offering CLI, APIs, and persistent cache features for efficient local deployment. (dwarfstar.sh)
Rai is a CPU-only large language model inference engine written entirely in Rust. It aims to provide efficient and pure Rust solutions for LLM deployment without relying on GPUs. (github.com)
Today's best Hacker News stories, summarized and screenshotted, one email a day.