Guided-RLVR adds a lightweight controller to widen reasoning exploration in RLVR

[arXiv] Bo Yang, Zishuai Zhang, Weihua Li, Qinnan Zhang, Sijia Wen and Hainan Zhang propose Guided-RLVR, which augments reinforcement learning with verifiable rewards (RLVR) with a lightweight controller. According to the authors, RLVR has shown strong empirical gains in improving the reasoning ability of large language models on tasks such as mathematics and programming, but its rollouts often collapse onto a narrow set of reasoning trajectories, limiting the explored reasoning space. The controller injects semantically meaningful steering tokens into the sampling distribution, encouraging the model to approach problems from multiple reasoning perspectives and thereby expanding the exploration space, the authors said. They said experiments on mathematical reasoning and code generation benchmarks show Guided-RLVR consistently outperforms existing RLVR methods while producing more diverse reasoning trajectories. The work is a preprint and has not been peer reviewed.