An engineer with little FPGA experience used coding agents to build an LLM inference system on an AWS F2 FPGA, testing whether agents could handle low-level C++ and Verilog as well as performance tuning and correctness checks. The target was a ternary-weight model from DeepGrove, suited to limited memory. Over several days, an autonomous experiment loop took the full model from a bit-exact baseline of 6.43 tokens per second to 213 tokens per second, a 33-fold increase. It ran hundreds of experiments, producing about three million lines of experiment history and processing 30 million lines of output, at a reported $10,000 in model-token costs. Optimizations included parallelizing model operations, improving memory access, overlapping attention work, and adding speculative decoding. A soak test exposed intermittent faults at a higher clock rate, so the final design ran slower for stability.
The account argues that coding agents now make specialized hardware development accessible even to people without prior FPGA skills, though the result came from sustained iteration rather than a single clever prompt. Human direction was mostly encouragement and occasional technical suggestions. Guidance from DeepGrove’s founders largely confirmed ideas already in the agent’s plan; the agent’s retrospective says it adopted some suggestions but rejected five numerical shortcuts. The emphasis on matching reference outputs mattered: the system was checked for bit-exact full-model behavior, not merely plausible generation. The experiment demonstrates striking progress in agent-led hardware optimization, while offering a specific benchmark and design story rather than evidence that every FPGA workload - or every agent-built design - will achieve similar performance.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.