Reza2kn/Qianfan-OCR-GGUF

Qianfan-OCR — GGUF Quantizations

GGUF quantizations of baidu/Qianfan-OCR.

Original model: InternVL Chat architecture with Qwen3 LLM backbone (~4.7B params). Quantized by Reza2kn using llama.cpp.

Files

Filename	Quant	Size	Quality
Qianfan-OCR-f16.gguf	F16	~9.4 GB	Lossless (half precision)
Qianfan-OCR-q8_0.gguf	Q8_0	~5.0 GB	Near-lossless
Qianfan-OCR-q6_k.gguf	Q6_K	~3.8 GB	Excellent
Qianfan-OCR-q5_k_m.gguf	Q5_K_M	~3.3 GB	Very good
Qianfan-OCR-q5_k_s.gguf	Q5_K_S	~3.2 GB	Very good
Qianfan-OCR-q4_k_m.gguf	Q4_K_M	~2.8 GB	Good (recommended)
Qianfan-OCR-q4_k_s.gguf	Q4_K_S	~2.7 GB	Good
Qianfan-OCR-q4_0.gguf	Q4_0	~2.6 GB	Legacy 4-bit
Qianfan-OCR-q3_k_m.gguf	Q3_K_M	~2.2 GB	Moderate
Qianfan-OCR-q3_k_s.gguf	Q3_K_S	~2.1 GB	Moderate
Qianfan-OCR-q2_k.gguf	Q2_K	~1.7 GB	Low quality

Usage (llama.cpp)

llama-cli -m Qianfan-OCR-q4_k_m.gguf --mmproj Qianfan-OCR-mmproj.gguf \
  --image document.jpg -p "Please OCR this document."

Original Model

See baidu/Qianfan-OCR for full documentation, benchmarks (OmniDocBench 93.12, OCRBench 880), and usage examples.