Yi

Start with Yi-1.5 9B at Q4_K_M via Ollama — fits on single RTX 3060 12GB at 6 GB VRAM. Yi-1.5 is a strong bilingual Chinese-English model built by 01.AI with Llama-compatible architecture — it scores MMLU ~79% on the 9B variant, competitive with Llama 3.1 8B and Qwen 3 8B. The 9B uses a 64K vocab with RoPE and GQA, scoring particularly well on Chinese benchmarks (C-Eval ~81%) and code generation (HumanEval ~76%). For higher quality, Yi-1.5 34B Q4 (22 GB) fits on RTX 4090 24 GB and matches Llama 3 70B on Chinese tasks. Skip Yi-Lightning and Yi-Large — those are API-only closed-weight variants. Yi-1.5 uses Apache 2.0 license — no commercial restrictions. The family has slowed in development cadence since 01.AI's strategic pivot; for actively maintained Chinese-English models, prefer Qwen.

For single-user local: Ollama + yi:9b Q4_K_M on RTX 3060 12GB or Apple M3 via MLX-LM. Yi uses standard Llama-compatible dense transformer — any Llama-engine works. For multi-user serving: vLLM 0.6.0+ with AWQ 4-bit on L4 24 GB for 9B variant. Yi-1.5 maintains the base Llama chat template format — <|im_start|> system/user/assistant with <|im_end|> delimiters. For bilingual Chinese-English pipelines: pair Yi with BGE-M3 for retrieval using the same tokenizer compatibility. Note: Yi-1.5 development has paused post-2025 — 01.AI has shifted to closed-weight and API-only models. Deploy Yi-1.5 if you need a stable, tested bilingual model with Apache 2.0 licensing but do not expect future open-weight releases from this family.