Δ-Mem: Compact Online Memory State Boosts LLM Long-Term Recall
Original: Δ-Mem: Efficient Online Memory for Large Language Models View original →
The Problem
LLMs struggle to accumulate and reuse historical information across long conversations or multi-step agent tasks. Expanding the context window is expensive and doesn't guarantee that the model properly utilizes distant context — it just makes the window bigger.
The Δ-Mem Approach
Δ-Mem adds a fixed-size state matrix to a frozen LLM backbone. This matrix is updated via delta-rule learning and generates low-rank corrections to the attention computation during generation. The result is effective long-term memory without full model fine-tuning or architectural replacement.
Performance Results
Despite using just an 8×8 online memory state, the gains are meaningful: 1.10× improvement over the frozen baseline, 1.15× over non-Δ-Mem baselines on general benchmarks, 1.31× on MemoryAgentBench, and 1.20× on LoCoMo — all while maintaining general capabilities. The efficiency is striking for such a compact mechanism.
Significance
Δ-Mem demonstrates that effective memory can be realized through a compact online state directly coupled with attention, without requiring full model retraining or separate memory modules. This makes it potentially applicable to existing deployed models as an efficient memory augmentation for long-horizon tasks.
Related Articles
The March 20, 2026 HN discussion around Attention Residuals focused on a simple claim with large implications: replace fixed residual addition with learned depth-wise attention and recover performance with modest overhead.
The HN discussion focused less on the fame of the list and more on how beginners can actually read it.
Inception Labs has released Mercury 2, the first production-ready diffusion language model for reasoning. Running at over 1,000 tokens per second on Blackwell GPUs, it is dramatically faster and cheaper than leading autoregressive competitors.