Youyang Yin, Huanhuan Liu, YY and other authors propose Unlimited OCR in a technical report, using DeepSeek OCR as the baseline and replacing all attention layers in the decoder with the proposed Reference Sliding Window Attention (R-SWA). The paper states that this design, while reducing attention computation overhead, keeps the KV cache constant throughout the decoding process, thereby alleviating the problems of rising memory usage and slowing generation under long output sequences. The authors state that, by combining the high compression rate of the DeepSeek OCR encoder with the constant KV cache design, Unlimited OCR can transcribe dozens of document pages in a single forward pass at a standard maximum length of 32K. The study also states that R-SWA is a general-purpose parsing attention mechanism and, beyond OCR, is also applicable to tasks such as speech recognition and translation. The code and model weights have been made publicly available on GitHub.
Unlimited OCR Works
HF Papers
Sourcesarxiv.org