Paper Proposes Next-Concept Modeling for Fast Parallel Coarse-to-Fine Text Generation, Up to 31x Speedup

[HF Papers] A preprint titled "Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation" by Hermann Kumbong, Ishan S. Khare, Iddah Mlauzi, Andrew Woen, Charles Ding and E. Kelly Buchanan introduces NCM, a two-stage coarse-to-fine framework in which a multiscale residual VQ-VAE encodes text into a hierarchy of discrete codes and a transformer generates one level at a time, predicting all codes within a level in parallel. According to the authors, at matched parameter scales NCM achieves better generative perplexity than the strongest diffusion language model baseline on LM1B while running 21 times faster; on OpenWebText it is competitive with the strongest diffusion model while running 10 times faster; on XSum it matches that model's ROUGE-L while being 31 times faster; and on Sudoku it outperforms autoregressive and diffusion baselines across all difficulty levels with fewer function evaluations. The work is a preprint and has not been peer-reviewed.