ExecuTorch WebGPU: PyTorch Models on the GPU, in the Browser

ExecuTorch’s new WebGPU backend lets PyTorch models run directly on a browser’s GPU without converting them to formats such as ONNX, GGUF, or TFLite. Developers export a model once with torch.export into an ExecuTorch .pte file; the WebAssembly runtime then uses WGSL compute shaders and the browser’s WebGPU API to execute it. The same model artifact is designed to work across WebGPU browsers and native platforms, with WGSL translated for Metal, Vulkan, or D3D12 as needed. Dynamic shapes are supported through maximum-sized buffers and runtime selection of specialized kernels. For users, inference needs no native installation or server, and prompts, images, and audio remain on the device, enabling private and potentially offline use while reducing network delay and server costs.

Testing covers language models including Llama, Qwen, and Gemma; image-question answering, segmentation, depth estimation, detection, and super-resolution; speech recognition and synthesis; and experimental on-device fine-tuning. In Chrome on an Apple M1 Pro, quantized Llama 3.2 1B and Qwen3-0.6B ran 4-7 times faster on WebGPU for prompt prefill and 2-10 times faster during token generation than ExecuTorch’s multithreaded WebAssembly/XNNPACK CPU backend. These results use specific models, hardware, and quantization settings, rather than establishing performance for every device. The backend remains early-stage, with work planned on speed, stability, operator support, data types, and model coverage.