Qwen3.6-35B-A3B-REAM-160-ru-agent-GGUF

GGUF quantizations of WaveCut/Qwen3.6-35B-A3B-REAM-160-ru-agent.

All release GGUF files are produced and smoke-tested with clean upstream ggml-org/llama.cpp at commit 8452824611be321246f33339727f60a90c02c277 (b9739-845282461). Runtime forks are not required.

Source

• Source checkpoint dtype: BF16
• Full precision GGUF baseline: BF16 only
• Source parameters: 23,562,176,368 BF16 parameters
• Main MoE experts: 160
• MTP/NextN MoE experts: 160

Release Files

Release candidates that passed upstream llama-cli smoke:

• Standard: Q1_0, IQ1_S, IQ1_M, Q2_K, Q2_K_S, IQ2_XXS, IQ2_XS, IQ2_S, IQ2_M, Q3_K_S, Q3_K_M, Q3_K_L, IQ3_XXS, IQ3_XS, IQ3_S, IQ3_M, Q4_0, Q4_1, Q4_K_S, Q4_K_M, IQ4_NL, IQ4_XS, MXFP4_MOE, Q5_0, Q5_1, Q5_K_S, Q5_K_M, Q6_K, Q8_0.
• Mixed precision: WC-Q3_K_XL, WC-Q4_K_XL, WC-Q5_K_XL, WC-Q6_K_XL.

Excluded: TQ1_0 and TQ2_0 were generated but failed upstream smoke and are not release files.

Per-file size and SHA256 are recorded in manifests/gguf-*.json.

Mixed Precision Recipe

WC-Q*_K_XL files are upstream-compatible GGUF files produced with llama-quantize --tensor-type-file masks/wc-xl.txt.

The mask keeps sensitive tensors at higher precision:

• token_embd, output, ffn_gate_inp: q8_0
• attn_q, attn_k, attn_v, ffn_down: q6_K
• MTP/NextN tensors and blk.40.*: q8_0

The base quant type remains standard upstream (Q3_K_M, Q4_K_M, Q5_K_M, or Q6_K), so no custom runtime is required.

Quality

Primary metric: KL divergence against the BF16 GGUF baseline using llama-perplexity --kl-divergence. The raw llama.cpp logs report values very close to zero for this short held-out run, so the table below shows the more interpretable same_top percentage from the same logs. Full logs and parsed rows are in quality/ and stats/quality_summary.json.

quant	RU same_top	agent same_top	code same_top	math same_top	mixed same_top
Q3_K_M	86.561	87.194	96.250	95.711	90.809
WC-Q3_K_XL	87.632	88.039	96.275	96.066	91.275
Q4_K_M	92.323	92.267	97.451	97.365	94.338
WC-Q4_K_XL	92.881	93.309	97.880	97.684	94.926
MXFP4_MOE	91.041	92.463	97.500	97.537	94.203
Q5_K_M	93.801	93.958	98.100	97.929	95.539
WC-Q5_K_XL	94.284	94.154	98.113	98.407	96.054
Q6_K	95.566	95.429	98.517	98.407	96.507
WC-Q6_K_XL	95.566	95.527	98.689	98.358	96.728
Q8_0	96.259	96.311	98.799	98.738	97.353

Practical starting points:

• Lowest size: Q3_K_M or WC-Q3_K_XL.
• Balanced RU/agent quality: WC-Q4_K_XL or WC-Q5_K_XL.
• Conservative high quality: Q6_K, WC-Q6_K_XL, or Q8_0.

MTP / NextN Status

MTP/NextN tensors are structurally preserved in the combined GGUF files, and manifests/gguf-*.json confirms the Qwen35MoE/NextN metadata.

Runtime MTP/speculative support is not claimed for this release. Upstream llama-speculative smoke failed for both split draft and combined self-draft forms:

• split trunk/draft exit code: 139
• combined self-draft exit code: 139

The exact result is recorded in stats/mtp_smoke.json.

Reproducibility

The build used:

• llama.cpp commit 8452824611be321246f33339727f60a90c02c277
• BF16 combined GGUF with MTP preserved
• agent/RU/code/math calibration and imatrix
• masks/mtp-q8.txt for standard quantization
• masks/wc-xl.txt for mixed precision candidates

See RECIPE.md, manifests/, stats/, quality/, and logs/ for the full audit trail.