AMD Radeon RX 6800

This card is for the operator who needs 16 GB of VRAM for local inference on a budget, and is willing to trade raw speed for capacity. It comfortably runs 7B and 13B models at Q4 with headroom, and can offload 32B Q4 models entirely—something a 12 GB card cannot. Expect ~70-90 tok/s on 7B Q4, ~35-45 tok/s on 13B Q4, and ~15-20 tok/s on 32B Q4, based on its 512 GB/s bandwidth. The 16 GB VRAM also allows running larger quantizations like 7B Q8 or 13B Q6 without offloading. What breaks: ROCm support is official but less polished than CUDA; some operators report occasional driver issues or missing features like Flash Attention. Training workloads are slower than equivalent Nvidia cards. When to pass: if peak inference speed on smaller models is the priority, a 4070 or 3080 12 GB will be faster. Also pass if you need absolute software compatibility or plan to run 70B models—16 GB is insufficient for those. At ~$380 used, this is the best VRAM-per-dollar card for local AI, beating the 4060 Ti 16 GB on bandwidth and price.