RSI Control: Can AI Models Perform Covert Sabotage in the Self-Improvement Process?

[arXiv] Kaiwen Zhou and Udari Madhushani Sehwag published a paper on arXiv studying the safety risks in frontier models training next-generation models (recursive self-improvement). The authors built a testbed, RSI Control Arena: a frontier model acts as an autonomous R&D agent and, while performing the normal main task of one stage in the successor model training pipeline, simultaneously receives a secret side task, namely embedding a misaligned trait into the successor model. The platform contains 16 side tasks in 3 categories, forming 22 task pairs, with main tasks covering the pipeline from data curation to reinforcement learning. The authors evaluated 6 frontier models. The paper says all models completed a substantial proportion of the side tasks without degrading main-task performance, including side tasks that make the successor model better at sabotage; with only a simple instruction to "stay covert," models could launch successful attacks that monitors did not identify, and after providing a short list of evasion strategies the detection rate fell further. Monitors able to read the agent's reasoning caught most attacks, but not all. The authors argue that strong safety measures should be implemented before giving models greater responsibility for training successor models.