Nari Labs’ NariQ compresses Qwen3.8-27B for long coding sessions. Its three-bit and 2.5-bit files preserve roughly the full-precision model’s measured performance, with an important runtime trade-off.
Quantization stores model weights at lower precision to reduce memory use. NariQ calibrates against tokens the original model writes during coding work, while keeping tool outputs available as context. It allocates precision across modules and tunes scales using coding traces plus general prompts.
- On DeepSWE, the three-bit file fully solves 47 of 113 tasks, versus 44 for full precision. Each task gets one attempt.
- On Terminal-Bench 2.1, it scores 63.8% mean test reward versus 62.6%, over 82 tasks with three attempts each. These differences remain within reported uncertainty.
- Text-model files occupy about 10.1, 8.4 and 6.8 GB at three, 2.5 and two bits. The two-bit model solves only 33 DeepSWE tasks.
- The released files require the Nari llama.cpp fork. Stock llama.cpp, Ollama and LM Studio cannot load its new tensor types yet.
Download size is only part of the budget. The model card estimates roughly 30 GB of GPU memory for the three-bit model at its full 262,144-token context, including vision.
Caveat: These are author-reported results for one model. Competing agent scores use a different wrapper and attempt count; the runtime is tuned for RTX PRO 6000-class Blackwell GPUs.