Self-Adapting Language Models
This paper introduces Self-Adapting Large Language Models (SEAL), a novel framework that enables LLMs to autonomously improve by generating their own training data and finetuning instructions, termed "self-edits." This adaptation process is driven by a reinforcement learning (RL) loop that rewards the model for generating self-edits that subsequently improve its performance on downstream tasks, contrasting with static models that learn from data "as-is." The authors demonstrate SEAL's effectiveness in two key domains: knowledge incorporation, where it generates synthetic data to efficiently integrate new facts, and few-shot learning, where it autonomously configures optimal data augmentations and training hyperparameters. Although promising, the work notes limitations regarding computational overhead and susceptibility to catastrophic forgetting during continuous adaptation.