Microsoft open-sources DACS: stalled data loads fell from 14% to 0.1%

Microsoft has open-sourced DACS, a Kubernetes-native caching system designed to speed up AI training and inference by placing frequently read data on local NVMe or SSD storage in GPU nodes. Azure Blob Storage remains the durable source of truth: when a client requests a blob chunk, DACS serves it from the cluster cache if available; otherwise it fetches the chunk from Blob Storage and caches it for later use. The design targets large, immutable or versioned artifacts such as model weights, datasets, and checkpoints, reducing repeated reads without replacing persistent storage.

Microsoft reports that, in a large-scale audio-training evaluation, warm-cache reads reduced the share of dataset-loading operations stalled for at least five seconds from 14% to 0.1%, helping an unstable workload complete training. For inference, sharing cached model weights can reduce duplicate downloads, network congestion, throttling, and delays that leave accelerators idle before serving begins. The initial release includes a cache server, C/C++ client, Python and Rust wrappers, Helm charts, a Kubernetes controller, Ethernet and InfiniBand transport, health probes, and Prometheus metrics. It currently integrates with Azure Blob Storage and does not isolate mutually untrusted tenants. Further storage backends and integrations are possible future contributions, not current capabilities.