SmolDocling: 256M-parameter vision-language model targets end-to-end document conversion

[HF Papers] A paper titled SmolDocling, by Ahmed Nassar, Andres Marafioti, Matteo Omenetti, Maksym Lysak, Nikolaos Livathinos and Christoph Auer, introduces an ultra-compact vision-language model for end-to-end document conversion. The model processes entire pages by generating DocTags, a new universal markup format that captures all page elements in full context with location. According to the authors, unlike approaches relying on large foundation models or ensemble solutions built on handcrafted pipelines of specialized models, SmolDocling offers end-to-end conversion in a 256M-parameter vision-language model, capturing content, structure and spatial location of document elements. The paper says the model shows robust performance in reproducing features such as code listings, tables, equations, charts and lists across business documents, academic papers, technical reports, patents and forms. The authors also contribute new publicly sourced datasets for charts, tables, equations and code recognition, and say experiments show SmolDocling competes with vision-language models up to 27 times larger while substantially reducing computational requirements. The model is currently available and the datasets will be publicly available soon.