Show HN: GPUWorks – A 40 MB GPU cluster manager for vLLM/SGLang, no K8s needed

GPUWorks is a lightweight, Kubernetes-free platform for managing GPU clusters and serving models through a shared Model-as-a-Service layer. Its control plane and node agent are static binaries, with the web interface embedded and state stored in SQLite; nodes can connect outbound over WebSocket, avoiding inbound ports. The system combines privately owned servers, cloud VMs and rented instances, and aims to schedule across NVIDIA, AMD, Huawei Ascend, T-Head, Hygon and Apple Silicon hardware. A built-in catalog offers 1,235 launch configurations for 392 models, with tested settings for supported model-and-accelerator combinations. Before deployment, it estimates memory for weights, cache and engine overhead, then explains placement choices.

GPUWorks supports vLLM, SGLang and platform-specific engines, with an OpenAI-compatible gateway for chat, embeddings, reranking, audio, images and video. Features include replicas across nodes, multi-node parallelism, prefill/decode separation, GPU memory quotas for sharing cards, and sleep/wake scheduling on supported hardware. Monitoring includes GPU and engine metrics, logs, request traces, host access and benchmarking; an AI assistant and read-only MCP endpoint are also included. The project reports a test scaling to 1,003 rented GPU machines across seven regions, placing over 500 replicas in four minutes and handling 2,000 concurrent requests. Support varies by accelerator and feature: some combinations are experimental, and the repository documents compatibility limits.