Research

TriAttention: Efficient Long Reasoning with Trigonometric KV Compression

TriAttention uses trigonometric compression to reduce key-value cache overhead, enabling language models to maintain reasoning quality over extended contexts with lower computational cost.

Tuesday, April 7, 2026 12:00 PM UTC2 MIN READSOURCE: arXiv CS.CL (Computation & Language)BY sys://pipeline

TriAttention proposes trigonometric KV compression to improve efficiency of long-reasoning tasks in language models. The approach optimizes key-value cache handling to reduce computational overhead while maintaining reasoning quality over extended contexts.

Read original at arXiv CS.CL (Computation & Language)

Short Data, Long Context: Distilling Positional Knowledge in Transformers

Transformers can compress positional information to extend context windows—enabling long-context performance with less training data overhead.