RunLocalAI
nvidia
GPU
48GB VRAM
workstation

NVIDIA L40S

Ada-gen datacenter card. 48GB GDDR6 — popular at cloud GPU rentals as a budget H100 alternative.

Released 2023

Overview

Ada-gen datacenter card. 48GB GDDR6 — popular at cloud GPU rentals as a budget H100 alternative.

Specs

VRAM48 GB
Power draw350 W
Released2023
MSRP$8500
Backends
CUDA

Models that fit

Open-weight models small enough to run on NVIDIA L40S with usable context.

Llama 3.1 8B Instruct
8B · llama
Qwen 3 8B
8B · qwen
Gemma 4 26B MoE
26B · gemma
Qwen 3 14B
14B · qwen
Llama 3.2 3B Instruct
3B · llama
Qwen 2.5 7B Instruct
7B · qwen
Mistral Small 3 24B
24B · mistral
Qwen 2.5 14B Instruct
14B · qwen

Frequently asked

What models can NVIDIA L40S run?

With 48GB VRAM, the NVIDIA L40S runs 70B models in 4-bit quantization, plus everything smaller. See the model list below for tested combinations.

Does NVIDIA L40S support CUDA?

Yes — NVIDIA L40S is an NVIDIA card with full CUDA support, the most mature local-AI backend. llama.cpp, Ollama, vLLM, and ExLlamaV2 all run natively.

Reviewed by RunLocalAI Editorial. See our editorial policy for how we research and verify hardware specifications.