forge: regenerate the model card

README.md

@@ -1,20 +1,21 @@
 ---
 license: apache-2.0
+license_link: https://www.apache.org/licenses/LICENSE-2.0
+thumbnail: https://huggingface.co/AtomicChat/Inkling-GGUF/resolve/main/hero.png
 base_model:
-  - thinkingmachines/Inkling
+- thinkingmachines/Inkling
 base_model_relation: quantized
 quantized_by: AtomicChat
 pipeline_tag: text-generation
 library_name: gguf
 tags:
-  - atomic-chat
-  - inkling
-  - thinking-machines
-  - moe
-  - gguf
-  - imatrix
-  - quantized
-  - llama.cpp
+- atomic-chat
+- inkling
+- thinkingmachines
+- gguf
+- llama.cpp
+- imatrix
+- quantized
 ---
 
 <center>
@@ -27,144 +28,84 @@ tags:
 
 <br/>
 
-<img src="https://huggingface.co/AtomicChat/Inkling-GGUF/resolve/main/hero.png" alt="Inkling" style="width:380px; max-width:100%; height:auto; margin-bottom:0.6em;"/>
+<img src="https://huggingface.co/AtomicChat/Inkling-GGUF/resolve/main/hero.png" alt="Inkling" style="width:100%; max-width:100%; height:auto; margin-bottom:0.6em;"/>
 
 <div style="display:flex; justify-content:center; gap:0.5em;">
 <a href="https://huggingface.co/thinkingmachines/Inkling"><strong>Base model: thinkingmachines/Inkling</strong></a>
 </div>
 </center>
 
-**Inkling** (Thinking Machines Lab), self-quantized to GGUF by
-[Atomic Chat](https://atomic.chat). Built straight from Thinking Machines'
-original BF16 weights with a per-tensor importance matrix. Runs fully offline,
-including a 1-bit build that brings this 975B model down to 226 GB.
+**Inkling**, self-quantized to GGUF by [Atomic Chat](https://atomic.chat). Built straight from Thinking Machines Lab's original weights with a per-tensor importance matrix, so this is not a repack of somebody else's files. Runs fully offline.
 
 ## Highlights
 
-- **975B-parameter MoE with 41B active**: each token is routed to 6 of 256
-  experts, plus 2 shared experts active on every token, across a 66-layer
-  decoder.
-- **Context up to 1M tokens** with a hybrid of local and global attention
-  layers.
-- **Natively multimodal base**: the original model reasons over text, images and
-  audio in a shared hidden space. This repo ships the text path.
-- **Strong reasoning scores** (Thinking Machines-reported, effort=0.99): AIME
-  2026 97.1, GPQA Diamond 87.2, SWEBench Verified 77.6.
-- **Built to be fine-tuned**: Thinking Machines positions Inkling as a base for
-  domain adaptation, released under Apache 2.0.
-- **Full imatrix quantization over a code corpus**, including a 1-bit `IQ1_M`
-  and an `MXFP4` build with Q8 attention and routing.
-
-> [!NOTE] These GGUFs are **self-quantized from the original weights**, not a
-> repack. The importance matrix keeps low-bit quants closer to the
-> full-precision model.
-
-> [!IMPORTANT] Always pass `--jinja` so the **Inkling chat template**
-> (interleaved thinking and tool calls) is applied. Without it the model can
-> emit malformed turns.
-
-> [!IMPORTANT] The `inkling` architecture is not yet in a mainline llama.cpp
-> release. Until [PR #25731](https://github.com/ggml-org/llama.cpp/pull/25731)
-> is merged, build llama.cpp from that PR (instructions below). Standard Ollama
-> / LM Studio flows will work once support lands upstream.
-
-> [!NOTE] This repo ships the **text path only**: no vision or audio projector
-> (`mmproj`) is included. For multimodal use, run the original weights.
+- **952.4B parameters**: the weights this repo quantizes.
+- **66 layers**: Mixture-of-Experts.
+- **Modalities**: the base model handles Text, Image, Audio; this repo ships text-only quants, it carries no vision projector.
+- **Full imatrix ladder**: every quant is calibrated with an importance matrix, published here alongside the quants.
 
-## Model Overview
+> [!NOTE]
+> These GGUFs are **self-quantized from the original weights**, not a repack. The importance matrix keeps low-bit quants closer to the full-precision model.
 
-| Property                  | Value                                                                                                                 |
-| ------------------------- | --------------------------------------------------------------------------------------------------------------------- |
-| Base model                | `thinkingmachines/Inkling`                                                                                            |
-| Total / active parameters | 975B total / 41B active                                                                                               |
-| Layers                    | 66                                                                                                                    |
-| Experts                   | 256 routed (top-6) + 2 shared, active on every token                                                                  |
-| Context length            | up to 1M tokens                                                                                                       |
-| Architecture              | Decoder-only Mixture-of-Experts, hybrid local/global attention, natively multimodal (text, image, audio in; text out) |
-| This repo                 | GGUF quants (imatrix), text path: `Q8_0` reference, `MXFP4` with Q8 attention/routing, and a 1-bit `IQ1_M` (226 GB)   |
+> [!IMPORTANT]
+> Always pass `--jinja` so the **Inkling chat template** is applied. Without it the model can emit malformed turns.
 
-<img src="https://huggingface.co/AtomicChat/Inkling-GGUF/resolve/main/benchmark.png" alt="Inkling benchmark scores" style="width:100%; max-width:900px;"/>
+## Model Overview
 
-Scores are Thinking Machines' published results for the base
-`thinkingmachines/Inkling`, reported at thinking effort 0.99. Quantization
-preserves the large majority of this; low-bit builds trade some quality for
-size.
+| Property | Value |
+|---|---|
+| Base model | `thinkingmachines/Inkling` |
+| Parameters | 952.4B |
+| Layers | 66 |
+| Experts | 256 routed (top-6) |
+| Context length | not stated |
+| Vocabulary | 201,024 |
+| Modalities | Text, Image, Audio in the base model; text only in this repo, it ships no vision projector |
+| Architecture | Mixture-of-Experts, 256 experts (top-6), 64 attention heads over 8 KV heads, `InklingForConditionalGeneration` |
+| This repo | GGUF quants (imatrix); the importance matrix is published here as `imatrix/imatrix-code-at_128.gguf` |
 
-## Choosing a quant
+<img src="https://huggingface.co/AtomicChat/Inkling-GGUF/resolve/main/benchmark.png" alt="Inkling benchmark scores" style="width:100%; max-width:900px;"/>
 
-| Quant       | Size    | Notes                                                                                                                                                                       |
-| ----------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| **`IQ1_M`** | 226 GB  | **Smallest. 1-bit imatrix build that makes a 975B model runnable on a single big-RAM server (about 226 GB plus context). Expect quality tradeoffs; reasoning still works.** |
-| **`MXFP4`** | 514 GB  | **Recommended for quality. Expert FFN weights in the 4-bit MXFP4 block format, with attention, expert router and shared experts held at Q8_0.**                             |
-| `Q8_0`      | 1.01 TB | Reference quality, near-lossless. Also the substrate our importance matrix was computed on. For large multi-node or big-RAM setups.                                         |
+Scores are Thinking Machines Lab's published results for the base `thinkingmachines/Inkling`, not our own measurements. Quantization preserves the large majority of this; `Q4_K_M` and up stay close to full precision.
 
 ## Get started
 
-Inkling needs a llama.cpp build with the `inkling` architecture (see **Run in
-llama.cpp** below). Then:
-
-```bash
-./build/bin/llama-server -hf AtomicChat/Inkling-GGUF:IQ1_M --jinja -c 8192
-```
-
-Or download a quant explicitly:
-
-```bash
-hf download AtomicChat/Inkling-GGUF --include "IQ1_M-final/*" --local-dir Inkling-GGUF
-# "MXFP4/*" for 4-bit, "Q8_0/*" for the reference build
-```
+Run Inkling locally with:
 
-<!-- Atomic Chat: uncomment once the app engine includes inkling support (PR #25731)
 - **[Atomic Chat](https://atomic.chat):** the easiest path. Open the app, search `AtomicChat/Inkling-GGUF`, pick a quant, hit **Use this model**.
--->
+- **llama.cpp:** `llama-server -hf AtomicChat/Inkling-GGUF:None --jinja -c 8192`
+- **Ollama:** `ollama run hf.co/AtomicChat/Inkling-GGUF:None`
+- **LM Studio / Jan:** search the repo id, download any quant.
 
 ## Best practices
 
-Thinking Machines does not publish recommended sampler settings for local
-inference. Two things do matter:
+| Parameter | Value |
+|---|---|
+| sampling defaults | not stated |
 
-- Always pass `--jinja`. The Inkling template carries the model's interleaved
-  thinking and tool-call blocks; without it output breaks.
-- Benchmark numbers above are reported at **thinking effort 0.99**. Inkling's
-  thinking effort is adjustable, so shorter-thinking runs will score below the
-  chart.
+The base model card does not state sampling defaults.
 
 ## Run in llama.cpp
 
-Inkling support lives in
-[PR #25731](https://github.com/ggml-org/llama.cpp/pull/25731) until it is merged
-upstream:
-
 ```bash
 git clone https://github.com/ggml-org/llama.cpp
-cd llama.cpp
-gh pr checkout 25731
-cmake -B build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
-cmake --build build --config Release -j --target llama-cli llama-server
+cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
+cmake --build llama.cpp/build --config Release -j --target llama-cli llama-server
 ```
 
 ```bash
-./build/bin/llama-server \
-    -hf AtomicChat/Inkling-GGUF:MXFP4 \
+./llama.cpp/build/bin/llama-server \
+    -hf AtomicChat/Inkling-GGUF:None \
     --jinja -ngl 99 -c 8192 -fa on
 ```
 
 ## How these were made
 
-1. Download `thinkingmachines/Inkling` (original BF16 weights, about 2 TB).
-2. Convert to GGUF with llama.cpp built from
-   [PR #25731](https://github.com/ggml-org/llama.cpp/pull/25731), which adds the
-   `inkling` architecture.
-3. Produce a `Q8_0` reference and compute an importance matrix over an 18 MB
-   code corpus (Linux, CPython, Rust and llama.cpp sources), 7,040 chunks of 512
-   tokens, with 93-99% expert activation coverage. The imatrix files are
-   published in
-   [`imatrix/`](https://huggingface.co/AtomicChat/Inkling-GGUF/tree/main/imatrix).
-4. Quantize with `--imatrix`: `MXFP4` for expert FFNs with attention, router and
-   shared experts at Q8_0, and `IQ1_M` with the same Q8 overlay for the smallest
-   coherent build.
+1. Download `thinkingmachines/Inkling` (original weights).
+2. Convert to f16 GGUF with [llama.cpp](https://github.com/ggml-org/llama.cpp).
+3. Build an importance matrix over our calibration corpus, published here as `imatrix/imatrix-code-at_128.gguf`.
+4. Quantize the ladder with `--imatrix`.
 
 ## License
 
-Released by Thinking Machines Lab under the Apache 2.0 license. Quantized by
-Atomic Chat.
+Original model by Thinking Machines Lab, released under the Apache 2.0 license. Full terms: [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0). Quantized by Atomic Chat.