The author decided to move AI inference for USA Arms onto hardware he owns after encountering model refusals and unpredictable behavior from hosted models. Having transitioned from decades of outsourced human data work to AI-driven processes, he rebuilt his inference harness across multiple iterations and favored Qwen 3.8 27B for its reliability on local, smaller models. Price cuts on hosted models like Luna initially pushed heavy use of cloud inference, but repeated refusals on routine tasks (FFL identification, ammo-dealer pages, range policies) accelerated the shift to internal compute to maintain control, privacy, and consistent output for a controversial vertical.
He built three Linux-based rack servers, each with four ASRock Radeon AI PRO R9700 (32 GB) GPUs, an AMD EPYC 7452 CPU, 128 GB ECC RAM, 1 TB NVMe, and robust cooling/power, spending about $10,734 per machine ($32,202 total). On his bench, twelve R9700 GPUs processed ~175 validated jobs/min versus 12 jobs/min on six Mac Studios (~15x throughput) and also run CI/CD, saving ~$300-$450/month in rented compute. Macs remain in the stack for large-model experiments and auditing. He published a 93-page build guide and video documenting parts, BIOS, wiring, tuning, and costs so others can replicate the setup.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
Pg_vectorsearch is a high-performance vector search index designed for PostgreSQL, outperforming pgvector's IVFFlat and HNSW methods. It offers faster search capabilities for applications requiring efficient vector similarity queries. (github.com)
QuEra has developed an AI-driven system to automatically relock and tune lasers used in quantum computers, reducing the need for human intervention. This innovation enhances the stability and uptime of their quantum processors, enabling more scalable and reliable quantum computing operations. (quera.com)
A solo entrepreneur used Grok Bot, an AI-powered team of agents, to automate tasks, fix bugs, and manage marketing for a week. The experiment demonstrated increased productivity, creative freedom, and the potential for small startups to compete with larger companies. (daily.tinyprojects.dev)
Fish Audio Drama 3 is an open-weight, directable text-to-speech model available on Hugging Face. It allows for customizable speech synthesis with adjustable weights and directions. (huggingface.co)
People often claim personal credit by saying 'I built it,' even when AI models like Claude perform the actual work. This reflects a tendency to identify with the outcome rather than the tool used. (news.ycombinator.com)
LLMs are advancing rapidly, raising questions about when human involvement in software design will become obsolete. Experts speculate that AI could soon handle architecture and coding, reducing the need for traditional programming skills. (news.ycombinator.com)
Today's best Hacker News stories, summarized and screenshotted, one email a day.