Is NVFP4 a game-changer? What is it, and does it matter for me?

Yes — but only on RTX 50-series and Blackwell-class hardware. Older cards stay on Q4_K_M.

NVFP4 (NVIDIA Floating Point 4) is a 4-bit floating-point quantization format introduced with Blackwell. Unlike integer-based Q4 quants (Q4_K_M, AWQ-INT4, GPTQ-INT4), NVFP4 uses an FP4 representation with a per-block exponent. The trade-off:

We deliberately don't quote specific PPL-delta percentages in this table — community NVFP4 benchmarks are still thin and the numbers NVIDIA publishes are vendor-marketing, not independently reproduced. Check the model card you're loading.

The "75% lossless" claim is NVIDIA marketing language for "near-FP16 quality at 4-bit memory footprint, accelerated by FP4 tensor cores on Blackwell." Blackwell has native FP4 matrix-multiply hardware — so NVFP4 isn't just smaller, it's also faster than Q4_K_M on Blackwell cards. The speed-up multiplier vs Q4_K_M depends on model size, batch size, and runtime build; community results vary, and we'd rather you measure on your stack than treat any single number as canonical.

The catch:

1. Hardware lock-in. Pre-Blackwell GPUs (RTX 30/40-series, A100, H100, Apple Silicon) don't have native FP4 tensor cores. They CAN run NVFP4 weights in software but lose the speed advantage and become slower than Q4_K_M.
2. Runtime support is uneven. As of May 2026, TensorRT-LLM and recent vLLM builds have first-class NVFP4 support. llama.cpp + Ollama support is in flight; check the release notes of the build you're using.
3. Model availability is limited. You need NVFP4-quantized weights — these are starting to ship on HuggingFace (search "NVFP4" in the model hub) but most popular models still only have GGUF/AWQ.

Operator decision rule:

• You have an RTX 5090 / RTX 5000 PRO / RTX 6000 PRO Blackwell → NVFP4 is genuinely worth trying. Use it via vLLM or TensorRT-LLM when the weights you need exist.
• You have an RTX 3090 / 4090 / A100 / Apple Silicon → Stay on Q4_K_M or AWQ-INT4. NVFP4 doesn't help you.
• You're picking a new GPU in 2026 → NVFP4 support is a real reason to favor Blackwell over Ada/Ampere if your workload mix includes inference-heavy serving.

The hype check: the 75% claim is "near lossless on the right hardware." It is NOT 75% memory savings vs Q4_K_M (the bit-count is similar). It's NVIDIA's framing for "closer to FP16 quality at the same memory footprint, and faster on Blackwell." Treat vendor numbers as upper bounds until reproduced.