RC RANDOM CHAOS

Sleep Framework Lets Deployed LLMs Keep Learning

· via Hacker News

Original source

Sleep framework: A new way for LLMs to retain knowledge in continual learning

Hacker News →

Deployed language models hit a fundamental wall: they operate with fixed knowledge cutoffs and cannot continuously incorporate new information. Updating parameters with new facts typically triggers catastrophic forgetting, where learning new material degrades performance on previously learned knowledge. In-context learning avoids this degradation by storing facts temporarily, but that information disappears when sessions end. Google Research proposes a “sleep framework” inspired by how human brains consolidate memories during rest. The approach gives models a two-phase lifecycle: active phases where they process inputs normally, and sleep phases where they consolidate fragile memories into permanent parameters and self-improve through synthetic data.

The mechanism operates through memory consolidation and a “dreaming” stage. Consolidation dynamically expands model capacity using Mixture of Experts architecture, adding low-rank expert modules that absorb new knowledge without disrupting existing parameters. Short-term context stored in higher-frequency layers gets distilled into these new permanent modules using reinforcement learning-based imitation. During dreaming, the model generates synthetic training data by mixing knowledge across different expert modules, filters samples based on gradient signal strength, and fine-tunes on high-impact examples. This process diversifies the learning landscape and creates more balanced optimization for sustained improvement.

Empirical testing shows substantial practical benefits. On translation tasks in previously unseen languages, the sleep framework maintained nearly zero performance degradation across sequential learning stages, while standard continual learning baselines suffered severe catastrophic forgetting. In long-context understanding benchmarks, it scaled to 10 million tokens—matching or exceeding GPT-4 and other strong baselines—by converting long-context information into permanent parameters during sleep phases.

Read the full article

Continue reading at Hacker News →

This is an AI-generated summary. Read the original for the full story.