Δ-Mem: Compact Online Memory State Boosts LLM Long-Term Recall

The Problem

LLMs struggle to accumulate and reuse historical information across long conversations or multi-step agent tasks. Expanding the context window is expensive and doesn't guarantee that the model properly utilizes distant context — it just makes the window bigger.

The Δ-Mem Approach

Δ-Mem adds a fixed-size state matrix to a frozen LLM backbone. This matrix is updated via delta-rule learning and generates low-rank corrections to the attention computation during generation. The result is effective long-term memory without full model fine-tuning or architectural replacement.

Performance Results

Despite using just an 8×8 online memory state, the gains are meaningful: 1.10× improvement over the frozen baseline, 1.15× over non-Δ-Mem baselines on general benchmarks, 1.31× on MemoryAgentBench, and 1.20× on LoCoMo — all while maintaining general capabilities. The efficiency is striking for such a compact mechanism.

Significance

Δ-Mem demonstrates that effective memory can be realized through a compact online state directly coupled with attention, without requiring full model retraining or separate memory modules. This makes it potentially applicable to existing deployed models as an efficient memory augmentation for long-horizon tasks.