AMD Radeon RX 7900 XTX

24 GB VRAM at materially lower pricing than the RTX 4090, with 960 GB/s memory bandwidth that's competitive on the model class it can fit. On Linux with ROCm 6+, llama.cpp hits 60–75 tok/s on Qwen 3 32B at Q4 — within shouting distance of 4090. The card you buy when "I want to run 32B-class models locally without spending RTX 4090 money."

• Windows ROCm is unreliable — llama.cpp's Vulkan path works but performance lags CUDA by 30–50% on the same model.
• vLLM and ExLlamaV2 are CUDA-first — running them on AMD requires patches or alternative backends.
• Smaller fine-tune ecosystem — every model card and tutorial assumes NVIDIA; you'll spend time translating CUDA-specific commands.

• Sweet spot: Qwen 3 32B / Qwen 2.5 Coder 32B at Q4 on Linux ROCm — 60–75 tok/s, full GPU.
• Stretch: Llama 3.3 70B at Q4 with system-RAM offload — ~18–24 tok/s, slower than 4090 but workable.
• Comfortable: 14B-class models with full 32K context at >100 tok/s.

• Windows-primary users — Vulkan fallback is slower and finickier than CUDA. NVIDIA wins.
• Cutting-edge runners — vLLM, ExLlamaV2 require effort to bring up cleanly on AMD.
• Multi-GPU scaling — AMD's multi-GPU story for inference is weaker than NVIDIA's.

Buy this if you run Linux, need 24 GB VRAM, value $/VRAM, and are comfortable with the rougher software ecosystem. Skip this if you're on Windows, want the easiest setup, or rely on vLLM / ExLlamaV2.

• vs RTX 4090 → 4090 is faster and has the cleaner software stack; 7900 XTX wins on $/VRAM by 30%+ depending on market. Pick 7900 XTX if budget-constrained AND on Linux.
• vs RTX 3090 (used) → 3090 has CUDA + 24 GB at similar used pricing; 7900 XTX is the answer when used 3090 supply is dry.
• vs Apple M3 Max → M3 Max with 64+ GB unified memory runs 70B more comfortably; 7900 XTX is faster on smaller models. Different platforms.
• vs RX 7900 XT (20 GB) → 7900 XT is the price-conscious sibling but 20 GB is awkward for 32B-class — pick XTX for the headroom.