Make this forward pass as fast as possible

AI agents are increasingly being asked to optimize GPU kernels directly, prompting debate over whether machine-learning compilers remain necessary. The key challenge is not simply generating faster code: optimization is an iterative search, and every experiment consumes time and tool calls. TVM’s approach argues for a shared, stable compiler foundation and intermediate representation that agents can use to analyze and verify programs, including checks for memory safety and synchronization. Together’s compiler work finds that conventional intermediate representations can constrain performance: agents given access to full source code can discover optimizations those representations cannot express, despite spending time tracing code paths. Its system instead combines rapid kernel swapping, tiered performance measurements, and analytical cost models to narrow the configurations worth testing.

The emerging shift is less about eliminating compilers than changing their job: helping agents search efficiently and verify results, rather than relying on a fixed pipeline of automatic optimizations. Heuristics can make performance unpredictable, as when Triton’s automatic fusion of multiply-add operations changed numerical behavior; making such transformations explicit can produce a smoother search landscape. TVM’s TIRx, OpenAI’s Gluon, and Meta’s TLX represent different attempts to create more extensible, agent-friendly tools, while Together emphasizes practical infrastructure over a single preferred toolchain. In Together’s experiments, a knowledge hub and expert-provided strategies did not improve optimization, and human-designed research tools sometimes diverted agents from kernel work. The best practices change quickly with agent capabilities, so no approach has clearly won.