DeepInfer is an open-source, high-performance inference engine released by DeepSeek for serving large-scale DeepSeek models, with immediate support for DeepSeek-V4.1-Flash. It bundles optimized kernels (FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect, custom TileLang), parallelism modes (attention/data/mixture-of-experts variants), and runtime features such as DSpark with confidence-scheduled verification, SWA bounded replay to reduce prefill work, an asynchronous zero-overhead batch scheduler, batch-invariant inference, deterministic sampling, and structured-output constraints (regex/JSON/EBNF). The release targets NVIDIA SM100 and Huawei Ascend 950 hardware, includes performance-focused configuration knobs (e.g., DEEPINFER_ENABLE_MTP_VARLEN, DEEPINFER_USE_NEXTN_DECODE, DEEPINFER_YOCO_PREFILL_* and DEEPINFER_BATCH_INVARIANT) and documents required but excluded components for a full service (KV cache, VE/Prefill/Decode disaggregation, and cross-replica load balancing). It is licensed under Apache-2.0.
Build and run instructions require Linux, GCC ≥13, Python 3.12 and CUDA 13 or CANN 9.2; a simple build.sh drives compilation after setting DEEPINFER_ACCELERATOR and optional DEEPINFER_HF_MODEL_DIR for local model checkpoints (default downloads deepseek-ai/DeepSeek-V4.1-Flash-1017). Examples include offline_inference.py and benchmark scripts for prefill/decode (measured on SM100 clusters with reproducer run_sweep.sh). The codebase credits vLLM, SGLang and TensorRT-LLM for influence, and the repo shows community traction (≈3.1k stars, ~220 forks).
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
A researcher created a website with 50,000 fake statistics to test how AI models like ChatGPT incorporate fabricated data. The fake stats were cited in over 10,000 AI sessions, academic papers, and by major companies, highlighting vulnerabilities in AI reliance on unverified sources. (peec.ai)
Reticle is an AI tool that enables coding agents to verify their own work, improving accuracy and reliability. It integrates with web and desktop applications to enhance AI-generated code testing and validation. (github.com)
ChatGPT offers superior insights compared to Codex but has limited tool calling capabilities, according to a user. They want to combine both models' strengths locally on their laptop for better AI agent performance. (news.ycombinator.com)
Parallel Language Models (PLMs) generate multiple tokens simultaneously using latent auxiliary variables, reducing inference latency compared to traditional autoregressive models. Empirical results show that PLMs achieve faster inference speeds with comparable or better performance across various benchmarks, especially at larger scales. (arxiv.org)
Yana is a local-first, human-governed AI runtime and IDE built in Rust that enables users to run, connect, and govern AI with deterministic control. It provides a cross-platform system for managing AI actions, access, and execution under human oversight. (github.com)
Lights-on software factory boilerplate for Claude Code provides a ready-made setup for AI code development. It is based on a framework originally developed for Votito and is available as an open-source project on GitHub. (github.com)
Today's best Hacker News stories, summarized and screenshotted, one email a day.