mPLUG-DocOwl is a family of multimodal large language models from Alibaba for document understanding without a separate OCR stage. The models take document images, tables, charts, web pages, and scientific diagrams as input and answer questions about them directly. The current flagship, mPLUG-DocOwl2, is an 8B model for multi-page document understanding that encodes each document page image into 324 visual tokens through a high-resolution compression module, so multi-page inputs stay within budget.
Earlier and sibling models in the same repository cover related tasks: DocOwl 1.5 introduced unified structure learning for single-image document understanding (DocVQA 82.2, InfoVQA 50.7, ChartQA 70.2, TextVQA 68.6); TinyChart is a 3B chart-understanding model with visual token merging and program-of-thought reasoning (ChartQA 83.6); PaperOwl targets scientific diagram analysis; and UReader and the original mPLUG-DocOwl are the earlier generations. Training data, inference and evaluation code, and model weights are published on Hugging Face and ModelScope, with fine-tuning supported through ms-swift for DocOwl2 and DeepSpeed for DocOwl 1.5.
| Model | Task focus | Venue |
|---|---|---|
| mPLUG-DocOwl2 | multi-page OCR-free document understanding | ACL 2025 |
| mPLUG-DocOwl 1.5 | unified structure learning for document understanding | EMNLP 2024 |
| TinyChart | 3B chart understanding with program-of-thoughts | EMNLP 2024 |
| mPLUG-PaperOwl | scientific diagram analysis | ACM MM 2024 |
| UReader | universal OCR-free visually-situated language understanding | EMNLP 2023 |
| mPLUG-DocOwl | modularized multimodal LLM for documents | arXiv 2023 |
Features
- OCR-free understanding: models read text and structure straight from images without an OCR pipeline
- Multi-page documents: DocOwl2 handles multi-page inputs at 324 visual tokens per page
- Chart understanding: TinyChart answers chart questions with program-of-thought reasoning
- Scientific diagrams: PaperOwl and the M-Paper dataset (447k diagram images with paragraph analysis) target paper figures
- Released datasets: DocStruct4M, DocDownstream-1.0, DocReason25K, and M-Paper
- Fine-tuning support: training code via ms-swift (DocOwl2) and DeepSpeed (DocOwl 1.5)
- Hosted demos: DocOwl 1.5 and TinyChart-3B Spaces on Hugging Face and ModelScope
