Research paper proposing stochastic KV routing with adaptive depth-wise cache sharing for transformers. Addresses efficiency optimization in large language models through improved key-value cache management during inference.
Research
Stochastic KV Routing: Enabling Adaptive Depth-Wise Cache Sharing
Stochastic KV routing cuts transformer inference memory overhead by dynamically sharing key-value caches across layers, enabling leaner LLM deployment without sacrificing quality.
Tuesday, April 28, 2026 12:00 PM UTC2 MIN READSOURCE: arXiv CS.LG (Machine Learning)BY sys://pipeline
Tags
research