Models

Gated-SwinRMT: Unifying Swin Windowed Attention with Retentive Manhattan Decay via Input-Dependent Gating

Input-dependent gating unifies Swin's windowed spatial attention with sequential retention mechanisms, bridging two transformer paradigms for improved efficiency and expressiveness.

Wednesday, April 8, 2026 12:00 PM UTC2 MIN READSOURCE: arXiv CS.LG (Machine Learning)BY sys://pipeline

Proposes Gated-SwinRMT, a new attention mechanism combining Swin's windowed attention with retentive components and input-dependent gating. Unifies spatial locality and sequential recurrence paradigms to improve transformer efficiency and expressiveness.

Read original at arXiv CS.LG (Machine Learning)

BookStack Moves from GitHub to Codeberg

Self-hosted documentation platform BookStack migrated to privacy-focused Codeberg forge in July 2024 over GitHub's code-scraping-for-AI practices and Microsoft's shift toward an "AI-powered developer platform."

PolicyApr 22

AI failure could trigger the next financial crisis, warns Elizabeth Warren

Senator Warren warns that unsustainable AI spending financed through opaque private credit channels mirrors pre-2008 conditions, with systemic risk concentrated across interconnected banks, insurers, and pension funds.