Cache-to-Cache: Direct Semantic Communication Between Large Language Models

Multi-LLM systems typically exchange information through text, forcing internal representations into output token sequences and incurring semantic loss and generation latency. Cache-to-Cache (C2C) replaces tokenized handoffs with direct KV-cache exchange: a neural projection-and-fusion module maps a source model’s key-value cache into the target model’s cache space and a learnable gating mechanism selects which target layers receive the fused cache. By operating on deep intermediate representations rather than surface tokens, C2C preserves richer, model-specific semantics and avoids stepwise token generation during inter-model communication.

Empirical evaluation uses oracle probes plus joint multi-model experiments to validate KV-cache as an effective communication medium and to measure tangible gains. Enriching KV-cache semantics improves response quality without increasing cache size. Across benchmarks, C2C raises average accuracy of multi-LLM assemblies by 6.4-14.2% relative to single models, outperforms conventional text-based inter-LLM communication by about 3.1-5.4%, and reduces end-to-end latency roughly 2.5×. The design emphasizes lightweight adaptors and selective layer gating so improvements come with modest added computation, and reference implementations are provided for reproduction.