TARGRS test-time detox framework cuts LLM toxicity by up to 69.38%, authors say
[arXiv] Aishan Liu, Yisong Xiao, Xianglong Liu and co-authors propose Transition-Aware Reward-Guided Representation Sampling (TARGRS), a test-time detoxification framework that learns reward guidance over toxicity transitions in latent representation space, according to the paper "LLM Detoxification via Transition-Aware Reward-Guided Representation Sampling." During exploration, progressive steering along the toxicity direction builds trajectories from non-toxic to toxic states, and toxicity scores along these trajectories turn sparse annotations into dense, listwise supervision for a transition-aware token-level reward model. At inference, TARGRS uses that model to guide coarse-to-fine sampling along the opposite, non-toxic direction. The authors report that experiments on 10 LLMs across RealToxicityPrompts, BOLD and AttaQ show TARGRS consistently outperforms state-of-the-art baselines, reducing toxicity by up to 69.38% on RealToxicityPrompts, while running nearly as fast as the original LLM and preserving core capabilities with minimal impact. By intervening in the language decoder space, the method also extends to large vision-language models, reducing toxicity by up to 64.23% across 3 representative LVLMs, according to the authors. The paper is a preprint and its findings have not been peer-reviewed.