PaddleOCR-VL-1.6 — LiteRT-LM (on-device document-parsing / OCR VLM)

PaddlePaddle/PaddleOCR-VL-1.6 converted to the LiteRT-LM (.litertlm) format for on-device document AI with Google's LiteRT-LM runtime — the first document-parsing / OCR-specialist VLM in this format.

PaddleOCR-VL is the SOTA open document-parsing model (OmniDocBench v1.6 96.33%, ahead of both open and closed alternatives at any size) in a phone-sized package: a NaViT-style dynamic-resolution SigLIP vision encoder + the tiny ERNIE-4.5-0.3B decoder, ~0.9B parameters total, supporting 109 languages. It is a task-prompted model — you select what it does with the text prompt:

Prompt	Task
OCR:	text recognition (page / paragraph / line)
Table Recognition:	table → structured cells (<fcel>/<nl> format)
Formula Recognition:	formula → LaTeX
Chart Recognition:	chart parsing
Spotting:	text + `<
Seal Recognition:	seal/stamp text

File	PaddleOCR-VL-1.6.litertlm (~1.39 GB)
Vision	SigLIP-so400m-class NaViT encoder (27L, hidden 1152) made static 560×560 → 1600 patches → 2×2 merge → 400 image tokens, int8 weights
Adapter	LN → 2×2 spatial merge → MLP (4608→1024), int8
Decoder	ERNIE-4.5-0.3B (18L, hidden 1024, GQA kv2, head_dim 128), fp16 weights (+ fp16 externalized embedder)
Context (KV cache)	4096
Image input	resized by the runtime to 560×560 (mean/std-0.5 normalization baked into the encoder)
Base model	PaddlePaddle/PaddleOCR-VL-1.6 (Apache-2.0)

How to use

1. Install the runtime

pip install litert-lm

2. Run it in one command — this downloads the bundle, encodes your page and transcribes it:

litert-lm run --from-huggingface-repo litert-community/PaddleOCR-VL-1.6 PaddleOCR-VL-1.6.litertlm \
  --attachment page.png \
  --prompt "OCR:"

This is a task-prompted model: the prompt selects the task (OCR:, Table Recognition:, and the others listed above), so a generic instruction such as "describe this image" returns nothing. On a 900×320 report page rendered with four lines of text, OCR: prints:

Quarterly Report
Revenue increased by 18.4% to $2,315 million. Contact: finance@example.com
Phone: +81 3-5555-0142

litert-lm serve exposes the same bundle as a local OpenAI-compatible API. The same file runs on macOS, Linux and Windows.

Performance

litert-lm benchmark (litert-lm 0.15.0) on an Apple M4 Max, -p 256 -d 256 --runs 3 (the tool averages three iterations), max-num-tokens 4096, warm-up run discarded, otherwise idle machine. These figures cover the text path; the vision encoder runs once per image and is not included.

Device	Backend	Prefill (256)	Decode	TTFT
Apple M4 Max (macOS)	CPU	653 tok/s	53.2 tok/s	0.50 s
Apple M4 Max (macOS)	GPU (Metal)	4391 tok/s	208.3 tok/s	0.07 s

Every desktop backend listed above was checked by actually generating a caption on it, not just by reading the benchmark tool's output.

Accuracy note

Device-verified on a Pixel 8a (Google AI Edge Gallery 1.0.15, CPU backend, this exact bundle):

• Page OCR (OCR: on a synthetic report page): perfect transcription in 22 s — every figure, the e-mail address and the phone number come out exactly ("Revenue increased by 18.4% to $2,315 million …").
• Table recognition (Table Recognition:): every cell correct including the Total row, in 17 s (Product / Units / Revenue … Total 24,510 $1,207,000).

Also validated on the desktop LiteRT-LM runtime (macOS CPU): table output is byte-identical to the full-precision eager reference, including the structured <fcel>/<nl> cell tokens (the bundle's SentencePiece vocabulary carries all 1 019 added tokens at their exact ids, so they detokenize correctly on-device).

On-device demo (Pixel 8a, CPU)

Page OCR — OCR: (22 s)	Table Recognition — Table Recognition: (17 s)

Captured in Google AI Edge Gallery on a Pixel 8a (CPU backend). The Gallery UI renders the model's structured table markup (<fcel>/<nl>) as plain text.

• Vision tower: tflite vs the reference implementation corr 1.0 (fp32) / 0.9975 (shipped int8), zero FLEX/CUSTOM ops (GPU-clean).
• Decoder: extracted as a standalone Llama-layout model — bit-exact fp32 logits vs the original; shipped fp16 weights are teacher-forced-parity corr 1.0000, top-1 10/10. This 0.36 B decoder is unusually quantization-sensitive (int4 and integer-compute int8 measurably corrupt transcription), so the bundle spends ~460 MB extra on fp16 exactness — for OCR, exactness wins.

M-RoPE note. The base decoder uses Qwen2-VL-style 3-D M-RoPE. The LiteRT-LM fast_vlm contract supplies plain sequential positions — for text tokens this is mathematically identical, and for image tokens an A/B eager test (true M-RoPE vs 1-D) showed no quality loss on OCR/table tasks (raster reading order survives 1-D positions). This is what makes the ride possible.

Scope. This is the recognition component of the PaddleOCR 3.x pipeline. The official server pipeline puts a layout-detection model (PP-DocLayout) in front for full-page multi-element parsing; standalone, this bundle handles pages, paragraphs, text lines, tables, formulas and charts directly — best on single elements or simple pages, exactly like the upstream transformers usage.

Aspect ratio. The runtime resizes input to a fixed square, so extreme aspect ratios (very wide single lines) get distorted. For best results feed roughly page/paragraph-shaped crops; the deployed 560² contract transcribed a 16:9 test page perfectly.

One image per chat. Like the other fast_vlm bundles, send each document in a fresh conversation — a second image in the same chat degrades (context bleed from the first turn was observed on CPU).

Galaxy S26 — GPU backend

The published bundle runs on the Android GPU backend and generates.

file	GPU backend	delegation	peak
PaddleOCR-VL-1.6.litertlm	runs	3601 / 3601 ops across 4 subgraphs on LiteRT GPU	1786 MB

Measured on a Samsung Galaxy S26 (SM-S942Q / SM8850, Android 16) with litert_lm_advanced_main from litert-lm 0.16.0, --backend=gpu --sampler_backend=cpu, prompt What is the capital of France?. Peak is the process high-water mark (VmHWM) sampled during that same run. Gated 2026-08-25.

The op counts above are the LiteRT GPU partitions. XNNPACK additionally takes 1 of the 5 nodes in main; the runtime accepts that split.

The gate prompt carries no image, so this covers engine creation and the text path. The vision path on the GPU is not measured here.

No speed rows, on purpose. On this handset the GPU backend wins prefill and does not win decode, so a GPU throughput figure only means something beside a CPU row from the same handset, and no S26 CPU row exists for this model yet.

GPU wiring, including the Gallery import toggle: GPU guide.

Run on iPhone / macOS

Use the LiteRT-LM Swift runtime (swift-litert-lm). Load PaddleOCR-VL-1.6.litertlm with the vision tower enabled (Modality.textImage), attach a document photo, and send one of the task prompts above (e.g. OCR:).

Vision-only bundle (no audio tower): bring the engine up with the vision modality only — requesting .all fails at session creation on bundles without an audio section.

Run on Android — Google AI Edge Gallery

Install a recent Google AI Edge Gallery, download PaddleOCR-VL-1.6.litertlm, import it (tap +), attach a document image and prompt OCR:. The bundle carries the tokenizer, template and both towers.

Conversion notes

• LiteRT-LM fast_vlm bundle: VISION_ENCODER ([1,560,560,3]→[1,1600,1152]) + VISION_ADAPTER ([1,1600,1152]→[1,400,1024]) + single-token EMBEDDER + PREFILL_DECODE (embeddings-input).
• Static NaViT rewrite: PaddleOCR-VL's encoder is dynamic-resolution (packed patches, interpolated position embeddings, 2-D rotary over h/w ids) and does not torch.export. The LM calls it with full attention (window_size=-1), so the static graph is: whole-image patch Conv2d (the processor packs patches in pure raster order — no gather needed), precomputed bilinear-interpolated position embedding, precomputed 2-D rope cos/sin for the fixed 40×40 grid.
• The projector's 2×2 spatial merge is done GPU-safe with 4 strided slices + concat (all tensors ≤4D) instead of the literal 6-D rearrange.
• Decoder: the ERNIE-4.5-0.3B inside the VLM is layout-identical to Llama — re-hosted as a standalone LlamaForCausalLM (state-dict 1:1, bit-exact logits) and exported with the standard litert-torch path, cache 4096.
• Tokenizer: the base SP model lacks the 1 019 added tokens (<|IMAGE_START|>, <|LOC_0|>…<|LOC_1000|>, <fcel>/<nl> table tokens…). They are appended as USER_DEFINED SentencePiece pieces at their exact ids (padded to vocab 103 424) so table/spotting output detokenizes correctly on device.
• Prompt template (baked into the bundle): <|begin_of_sentence|>User: <image>PROMPT\nAssistant:\n, stop token </s>.

2026-08-30 — tokenizer section replaced (weights unchanged)

The tokenizer in PaddleOCR-VL-1.6.litertlm is the vendor's SentencePiece model, whose special tokens are typed as control symbols; SentencePiece never matches a control symbol in text, so the </s> the chat template writes after each history turn reached the model as several tokens instead of the single </s> id. Those symbols are now user-defined pieces with the same ids, so a multi-turn prompt tokenizes as the upstream tokenizer does. Single-turn prompts were unaffected.

Tokenizer-only change: every section of the bundle except the tokenizer is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the chat template are unchanged and the speed and memory numbers on this card still describe this file per token — only the file's own sha256 differs. Verified on the LiteRT-LM runtime: the default turn, 7 probe strings, the 223 standalone characters U+00A1–U+017F and every special token now tokenize identically to the upstream tokenizer, and the four ASCII-only test questions answer byte-identically to the previous file (same ids in, same tokens out). Prompts containing accented letters, symbols or emoji reach the model differently from before, so individual answers to such prompts can change. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file, and any on-device rows were measured on the previous file too — the on-device gate has not been re-run on this one (the runtime's tokenizer code is the same on macOS and on device; the weights and graph are byte-identical). If you downloaded before 2026-08-30, re-download.

License

Apache-2.0, inherited from the base model PaddlePaddle/PaddleOCR-VL-1.6.