NVIDIA GeForce RTX 2060 Super

This card is for the operator who needs a reliable, budget-friendly entry into local inference with 7B models and occasional 13B experimentation. The 8 GB VRAM is the practical minimum for running 7B Q4 with full context (4K+ tokens) comfortably, and 13B Q4 fits with aggressive offloading to system RAM. On 7B Q4, expect ~50-65 tok/s using ExLlamaV2 or llama.cpp, derived from the 448 GB/s bandwidth. The 2060 Super is a Turing-era card, so it lacks FP8/FP4 tensor core support, meaning no speedups from quantization formats newer than Q4. 13B Q4 runs at a slower ~10-15 tok/s due to offloading overhead, and anything larger (e.g., 30B+) is impractical. Pass on this card if you need to run 13B models entirely in VRAM, or if you plan to work with 30B+ models at any usable speed. At ~$220 used, this is the cheapest 8 GB CUDA option that actually works for local AI without constant VRAM thrashing.