Laguna-S-2.1-NVFP4-GGUF model
Can we get a Laguna-S-2.1-NVFP4-GGUF version, please? vLLM is great and all but running the NVFP4 gguf version in llama.cpp will be something that a lot of people will appreciate.
There are already plenty of other NVFP4 models converted to gguf and llama.cpp people are really enjoying it. Please do it for this great model as well.
Hi! We're actually already packaging it up: it just slipped under the radar with the other 16 checkpoints we put out today :) It should be ready tomorrow.
Great news! 🥳 Thank you for the great model and the hard work behind it!
Hi guys, I know you've been working hard on fixing the NVFP4 quant. Any chance to give us an ETA for the NVFP4-GGUF release?
Yes please! I've spent the past hour reading all the docs on how to convert it myself, but coming from upstream would be better :)
Edit: several hours later, I finally managed to convert it from the NVFP4 safetensors to a working GGUF, but it looks like llama.cpp's converter for NVFP4 is not working at the moment - https://github.com/ggml-org/llama.cpp/discussions/23627#discussioncomment-18032147