Language Models Learn to Improve Language Models

[arXiv] Researchers Minhua Lin, Suhang Wang and others have published a paper on arXiv proposing the LM2LM method, which uses reinforcement learning to train large models to generate effective language model improvement plans. The reward comes from a frozen large model acting as a "research world model," which predicts the gains of each plan based on real experimental records in the context; during training, the top-ranked plans are also periodically executed for real, and the measured results are added to the context. In eight autoregressive and diffusion language model development environments, real-execution evaluations on tasks not involved in training show that LM2LM enables Qwen3.8-27B to surpass GPT-6-sol, even when the latter is given 300 real experimental records in its context, while data generation costs are reduced by 33%. The authors state that the method can also generalize to model families not seen during training.