Baidu Unlimited-OCR reads 40-page documents with only 500M active parameters

Baidu’s Unlimited-OCR is getting renewed attention because it targets a stubborn document-AI problem: reading long documents without cutting them into pages and losing context. The model is described as having 3 billion total parameters but only 500 million active during inference, with a 32K context window for processing 40-page documents in one pass.

Vaibhav Sisinty highlighted the key claim: “3 billion parameters but only 500 million active during inference,” plus 93% benchmark accuracy and a six-point gain over the baseline.

The post came from an AI-focused creator account and was amplified by Yann LeCun, which pushed it into a broader research and open-source discussion. The underlying work is the arXiv report “Unlimited OCR Works,” and Baidu has made code and model weights available through GitHub and Hugging Face. The paper’s central idea is Reference Sliding Window Attention, a parsing attention mechanism designed to keep KV cache constant as output length grows.

That matters because OCR for real documents is not just text extraction. Tables can cross page boundaries, formulas can sit beside prose, and reading order often determines whether the output is usable. Traditional OCR pipelines frequently split documents page by page, then try to stitch the structure back together. Unlimited-OCR instead aims to preserve text, tables, formulas, and reading order as structured Markdown.

The next test is practical reproducibility. Open weights and demos lower the barrier, but scanned PDFs, multilingual contracts, invoices, and low-quality images stress OCR systems differently from curated benchmarks. If the reported 93% accuracy and low long-document error rate hold across messy enterprise documents, one-pass local OCR could become a serious alternative to per-page cloud APIs. The source tweet is here.