Continual Learning via Sparse Memory Finetuning

Key Takeaways

• Memory layers replace FFN with key-value slots
• Fine-tuning updates only a few memory slots
• TF‑IDF selects task‑specific slots, masks general ones
• Achieves similar QA accuracy to full finetuning
• Reduces forgetting from 89% to 11% on held‑out data

Summary

Continual learning for large language models (LLMs) is hampered by catastrophic forgetting when traditional fine‑tuning updates all parameters. A new approach replaces transformer feed‑forward layers with sparse memory layers, updating only a handful of key‑value slots identified via TF‑IDF. Experiments on question‑answering tasks show performance comparable to full fine‑tuning and LoRA while dramatically reducing forgetting. The method isolates task‑specific parameters from general knowledge, offering a scalable solution for production‑grade model updates.

Pulse Analysis

Continual learning has become a strategic priority for organisations deploying large language models (LLMs) in dynamic environments. Traditional fine‑tuning updates every weight in the transformer, which quickly erodes the knowledge acquired during pre‑training—a phenomenon known as catastrophic forgetting. As models are exposed to fresh data streams such as breaking news, regulatory updates, or personalised user feedback, the interference between new and old tasks can degrade performance on previously mastered domains. Solving this interference without retraining from scratch is essential for maintaining both relevance and reliability in production AI systems.

The paper “Continual Learning via Sparse Memory Finetuning” proposes a hardware‑agnostic solution that swaps the conventional feed‑forward network (FFN) layers for lightweight memory layers composed of key‑value slots. During adaptation, only a tiny fraction of these slots—identified through TF‑IDF ranking of the new corpus—are updated, while the remaining slots that encode general pre‑training knowledge are masked out. This isolation turns the parameter‑interference problem into a sparse‑update problem, allowing the model to acquire task‑specific facts without perturbing the broader linguistic capabilities that underpin its performance across all tasks.

Empirical results on several question‑answering benchmarks show that sparse memory finetuning matches the accuracy of full‑parameter fine‑tuning and low‑rank adapters such as LoRA, while slashing forgetting rates—from an 89 % drop with conventional fine‑tuning to just 11 % on held‑out data. This dramatic reduction translates into lower maintenance costs, as models no longer require costly retraining cycles to restore lost capabilities. For enterprises, the approach offers a scalable path to keep LLMs up‑to‑date with domain‑specific knowledge, regulatory changes, or user‑generated corrections, all while preserving the core competencies that drive downstream applications.