Feature Extraction
Transformers
Safetensors
kimi_k3
compressed-tensors
LLM Compressor
vLLM
custom_code
8-bit precision
Instructions to use RedHatAI/Kimi-K3-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use RedHatAI/Kimi-K3-NVFP4 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="RedHatAI/Kimi-K3-NVFP4", trust_remote_code=True)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("RedHatAI/Kimi-K3-NVFP4", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -32,9 +32,9 @@ May require https://github.com/vllm-project/vllm/pull/50500 to run
|
|
| 32 |
This model was compressed using LLM Compressor. For more information, please contact Kyle Sayers via the [vLLM Developers Slack](https://communityinviter.com/apps/vllm-dev/join-vllm-developers-slack) or ksayers@redhat.com.
|
| 33 |
|
| 34 |
## Evaluation ##
|
| 35 |
-
| Benchmark | `
|
| 36 |
-
| - | - | -|
|
| 37 |
-
| GPQA |
|
| 38 |
|
| 39 |
```bash
|
| 40 |
inspect eval hf/Idavidrein/gpqa/diamond \
|
|
|
|
| 32 |
This model was compressed using LLM Compressor. For more information, please contact Kyle Sayers via the [vLLM Developers Slack](https://communityinviter.com/apps/vllm-dev/join-vllm-developers-slack) or ksayers@redhat.com.
|
| 33 |
|
| 34 |
## Evaluation ##
|
| 35 |
+
| Benchmark | `moonshotai/Kimi-K3` | `RedHatAI/Kimi-K3-NVFP4`
|
| 36 |
+
| - | - | -|
|
| 37 |
+
| GPQA | 93.5 | 91.0 |
|
| 38 |
|
| 39 |
```bash
|
| 40 |
inspect eval hf/Idavidrein/gpqa/diamond \
|