How long does local AI take to set up?

Answer first

Anywhere from 5 minutes to a week, depending on what “set up” means and what hardware you have. The fastest honest path — Ollama on a Mac with a 3B model — really is 5 minutes from a cold start to a working chat. The slowest reasonable path — a multi-GPU homelab serving a 70B model with a real frontend, monitoring, and TLS — is a focused weekend if you know what you're doing and a full week if you're learning as you go. Most people land somewhere in the middle: 30 minutes to first token, then a week of light tinkering as you swap models and add a frontend.

The five scenarios below are calibrated to what actual operators report, not best-case marketing numbers. If you want to skip the whole page and just get a working setup, the fastest path for your situation is in /setup and the beginner learning path is at /paths/beginner-local-ai.

Five honest scenarios with realistic ranges

Time ranges below are wall-clock from “clean machine, no software installed yet” to “chatting with a model.”

Scenario 1 — Ollama on Mac, 5-15 minutes. Download Ollama from the website (about 200 MB), drag to Applications, run ollama pull llama3.2:3b in Terminal (about 2 GB, downloads in 1-5 minutes on most home connections), then ollama run llama3.2:3b. The model loads and you have a chat. Apple Silicon's unified memory removes most of the configuration that goes wrong on other platforms. This is the fastest honest path to a working local model.

Scenario 2 — Ollama on Linux with an NVIDIA GPU, 20-60 minutes. Add 5-15 minutes if NVIDIA drivers are already installed, 30-60 if you have to install them. The Ollama install is one shell command. The risk is the driver version: if your distro's default driver is too old, llama.cpp falls back to CPU and you don't notice until generation feels slow. The fix is to confirm nvidia-smi shows a recent driver before you start. If you're new to Linux GPU setup, plan for an hour the first time and 15 minutes every time after.

Scenario 3 — LM Studio on Windows with consumer GPU, 30-90 minutes. Download LM Studio (about 500 MB), install it, browse the model marketplace inside the app, click download on a 7B-class GGUF (4-8 GB), wait for the download. Total wall time is mostly the model download on a typical home connection. Add 10-30 minutes if your GPU drivers are out of date — the most common Windows-side gotcha.

Scenario 4 — vLLM production setup on a Linux server, half a day. Install Docker or a Python environment, install vLLM with the right CUDA wheel, download a model from Hugging Face, configure the server flags (max-model-len, GPU memory utilization, dtype, paged-attention block size), launch it, hit the OpenAI-compatible endpoint with a test request, debug the first failure. If you've done it once it takes 60-90 minutes; the first time it's 4-8 hours, not because any one step is hard but because there are a lot of steps and at least one will fail in a way that takes Googling.

Scenario 5 — Multi-GPU homelab, weekend to a week. Two or more GPUs, model parallelism, a real frontend like Open WebUI, monitoring with Prometheus or Grafana, an Nginx or Caddy reverse proxy, optional TLS, optional remote access. A confident operator does this in a focused weekend. Someone learning every layer takes a full week of evenings. The time is mostly in the integration: the individual pieces are well-documented; making them work together with the GPU/CPU/RAM/network you actually have is where the hours land.

Where the time actually goes

Across all five scenarios, the time is concentrated in three places.

• Model download. A 7B Q4 model is 4-5 GB; a 14B Q4 is 9-10 GB; a 32B Q4 is 18-20 GB; a 70B Q4 is 40-45 GB. On a typical 100 Mbit home connection these are 7-60 minutes apiece. On gigabit fiber they are 1-7 minutes. This is wall-clock time you cannot compress, but you can do it in the background while you do other steps.
• Driver / runtime mismatch. NVIDIA driver too old for your CUDA version, ROCm not installed properly on AMD Linux, MLX not picking up the right Python — these are the three most common time sinks. Each is fixable in 10-30 minutes once you know the symptom.
• The frontend layer. Choosing between LM Studio, Open WebUI, AnythingLLM, or a homegrown chat UI is a 30-minute research detour for most people, then a 30-60 minute setup. If you know up front which one you want, this collapses to 15 minutes.

What lets you skip steps

If you want the fastest honest path, three shortcuts are durable:

• Pick the runtime that matches your platform's defaults. Ollama on Mac and Linux. LM Studio on Windows. vLLM or SGLang for production Linux. Cross-platform “universal” choices exist but cost you time.
• Start with the smallest model that fits your task. Llama 3.2 3B is enough to confirm your stack works before you commit to downloading 40 GB of 70B weights.
• Don't pick a frontend on day one. The Ollama or LM Studio chat box is fine for the first hour. Add Open WebUI on day two when you know you'll keep using it.

Where most newcomers get stuck

Three specific traps account for most of the “I've been at this for hours” complaints we see in the wild.

Trying to run a model that does not fit in VRAM. The runtime falls back to CPU silently and generation crawls at 0.5-2 tok/s. Diagnosis: check nvidia-smi or Activity Monitor while generation runs; if VRAM is full and the GPU usage is near zero, the model spilled. Fix: use a smaller quantization or a smaller model. Confirm fit before download with /will-it-run/custom.

Wrong-version drivers on Windows or Linux. Symptom: install completes but the runtime cannot see the GPU. Fix: nvidia-smi on Linux/Windows or System Information on Mac to confirm the driver. The error catalog at /errors covers the specific messages you'll see.

Picking the heaviest stack on day one. Trying to install vLLM with multi-GPU model parallelism as your first local-AI experience is a way to spend a Saturday and end with nothing working. Start with Ollama or LM Studio, get one model running, then graduate to heavier setups when you know what you actually need. The full taxonomy of these mistakes is in /guides/common-local-ai-setup-mistakes.

Time-to-confidence vs time-to-first-token

Time-to-first-token is when the model says hello back. Time-to-confidence is when you trust the setup to keep working day after day. Those are different numbers. First-token usually happens in 5-90 minutes depending on the path. Confidence — knowing what tok/s to expect, knowing how to swap models, knowing what the failure modes are — typically takes a week of casual use. Don't conflate them. The honest pattern is “working in an hour, comfortable in a week.”