Mistral Saba 24B

Mistral Saba 24B is Mistral AI's Arabic-specialized 24B dense model. Saba is Mistral's regional language model — optimized for Arabic language understanding, Middle Eastern cultural context, and Arabic+English bilingual tasks. Run at Q4_K_M via Ollama (ollama pull mistral-saba:24b) or llama.cpp with -ngl 999 -fa -c 8192. Q4_K_M file size ~14 GB on disk. Minimum VRAM: 12 GB — RTX 4070 (12GB) at Q4_K_M with KV offload for 4K context. RTX 4090 24GB: Q4_K_M comfortably at 16K+ context. Recommended: RTX 4090 24GB at Q4_K_M. Throughput: ~40-65 tok/s on RTX 4090 at Q4_K_M. Mistral architecture — well-supported. Saba is designed for: Arabic chat, Arabic content generation, Arabic document understanding, Arabic+English code-switching, Middle East context applications. Not for: non-Arabic languages (quality degrades significantly), general-purpose use (use Mistral Small 3.2 24B instead). Context: Mistral's 32K+; practical at Q4 on 24 GB is 16-32K. Mistral Saba is one of the few openly available Arabic-specialized LLMs at this size.

Minimum: RTX 3060 12GB at Q3_K_M with KV offload. Recommended: RTX 4090 24GB at Q4_K_M (16K+ context). VRAM math: 24B dense, Q4_K_M ≈ 14 GB. KV cache at 16K: ~5 GB. Total: ~19 GB at 16K. RTX 4090 24GB: comfortable on-GPU. RTX 3080 10GB: Q3_K_M with KV offload. RTX 4080 16GB: Q4 + 8K context on-GPU. MacBook Pro M4 Pro 24GB+: Q4 at 15-30 tok/s. Cloud: A10 24GB at Q4_K_M. AWQ-INT4 drops to ~12 GB. Arabic text has different tokenization efficiency than English — Arabic may be 1.2-1.5× more token-costly for equivalent semantic content. Budget slightly more tokens for Arabic prompts.

1. Arabic-only specialization. Saba is heavily optimized for Arabic. English is functional but lower quality. Non-Arabic languages (French, Spanish, etc.) degrade significantly. 2. Dialectal Arabic variance. Saba is trained on Modern Standard Arabic (MSA). Dialectal Arabic (Egyptian, Levantine, Gulf) may produce lower-quality results. Test your specific dialect. 3. Cultural context scope. Saba's cultural knowledge is Middle East-focused. North African cultural contexts may have gaps. 4. Smaller community quant coverage. As a regional-specialized model, Saba has fewer pre-converted GGUFs than general-purpose Mistral models. Verify quantization availability before provisioning.

Ollama for quick-start (Saba should be available as a Mistral model). llama.cpp for production. vLLM for serving. Mistral architecture — first-class support everywhere. For Arabic RAG: pair with Arabic-specific embeddings (e.g., camel-bert, Arabic-T5) for document retrieval.

Mistake: Using Mistral Saba for non-Arabic tasks. Fix: Saba is Arabic-specialized. English is functional but lower quality. Use Mistral Small 3.2 24B for general-purpose tasks. Mistake: Expecting Saba to handle all Arabic dialects equally. Fix: Saba is trained on MSA. Test on your specific dialect (Egyptian, Levantine, Gulf, Maghrebi) — quality varies. Mistake: Assuming English tokenization is the same as Arabic. Fix: Arabic may produce 1.2-1.5× more tokens for equivalent semantic content. Adjust context budget accordingly. Mistake: Using Llama chat template with Saba. Fix: Mistral models use Mistral-specific templates. Verify on hf tokenizer_config.json.