AI glossary

A Large Language Model is a neural network with billions of parameters trained on massive text corpora to predict the ne

Quantization is the process of reducing a model's numeric precision to shrink its memory footprint with minimal quality

Inference is the act of running a trained model to generate predictions, as opposed to training which produces the model

RAG is the pattern of retrieving relevant documents from a knowledge base and including them in the LLM's prompt so the

Hallucination is when an LLM generates plausible-sounding but factually incorrect information — citing papers that don't

Prompt engineering is the practice of crafting model inputs to elicit better outputs without changing the model itself.

LoRA is a parameter-efficient fine-tuning technique that adapts a large pre-trained model by training small low-rank mat

Fine-tuning is continued training of a pre-trained model on a smaller, task-specific dataset. Pre-training builds genera

An embedding is a fixed-length vector representation of text, image, or other input — typically 384-3072 dimensions — wh

Chain-of-thought prompting is asking a model to show its reasoning step-by-step before giving the final answer. It drama

Latency measures how fast you get a response. Two metrics matter for local LLMs: Time to First Token TTFT — wall-clock

GGUF GGML Unified Format is the file format used by llama.cpp and its ecosystem Ollama, KoboldCPP, LM Studio. A single f

Throughput measures how much work a system completes per unit time — typically tokens-per-second across all concurrent r

QLoRA combines LoRA/glossary/lora fine-tuning with 4-bit quantization of the base model. Introduced by Tim Dettmers in 2

Speculative decoding speeds up LLM inference by using a small fast "draft" model to propose the next several tokens, the

The KV cache stores the key and value tensors from previous attention computations so the model doesn't recompute them a

The context window is the maximum number of tokens a model can attend to at once — both prompt and previously generated

A token is the smallest unit of text a language model processes. Most modern models use subword tokenization, where comm

Tokenization is the process of converting text into the numeric tokens a model can process. Modern systems use subword t

Flash Attention is a memory-efficient implementation of the attention mechanism that reduces memory usage from On² to On

Mixture of Experts is a neural network architecture where multiple specialized sub-networks "experts" exist, but only a

VRAM is the dedicated memory on a GPU. For local AI, VRAM capacity is the single most important spec — it determines whi

CUDA Compute Unified Device Architecture is NVIDIA's parallel-computing platform and the dominant API for GPU-accelerate

An AI agent is software that uses an LLM to decide what to do, takes actions, observes results, and iterates toward a go

Function calling also called tool use is a capability where the model emits structured JSON requesting that specific too

MCP is an open protocol introduced by Anthropic in late 2024 for connecting AI agents to tools and data sources in a sta