Reasoning Models Can Control Their Chains of Thought When They Tell Themselves To

[arXiv] Uriel Dolev, Idan Schwartz and Yoav Goldberg state in a paper published on arXiv that existing research, in which chain-of-thought constraints were given in user prompts, measured controllability to be low across models, but this may stem from the way the constraints were presented rather than from the models being unable to comply. The authors propose a "self-instruction" evaluation that places constraints as instructions within the model's own chain of thought. Results show that controllability under this approach is markedly higher than with conventional user instructions; even adding just one generic sentence reminding the model to comply with user constraints significantly improves the effect, and adding a single simple instruction to the user prompt can also guide the model to generate such self-instructions on its own. The authors further demonstrate in a realistic monitoring scenario that this controllability can be used to evade monitoring, posing a risk to the monitorability of chain of thought.