RUNLOCALAIv38
→WILL IT RUNBEST GPUCOMPARETROUBLESHOOTSTARTPULSEMODELSHARDWARETOOLSBENCH
RUNLOCALAI

Operator-grade instrument for local-AI hardware intelligence. Hand-written verdicts. Real benchmarks. Reproducible commands.

OP·Fredoline Eruo
DIR
  • Models
  • Hardware
  • Tools
  • Benchmarks
  • Will it run?
GUIDES
  • Best GPU
  • Best laptop
  • Best Mac
  • Best used GPU
  • Best budget GPU
  • Best GPU for Ollama
  • Best GPU for SD
  • AI PC build $2K
  • CUDA vs ROCm
  • 16 vs 24 GB
  • Compare hardware
  • Custom compare
REF
  • Systems
  • Ecosystem maps
  • Pillar guides
  • Methodology
  • Glossary
  • Errors KB
  • Troubleshooting
  • Resources
  • Public API
EDITOR
  • About
  • About the author
  • Changelog
  • Latest
  • Updates
  • Submit benchmark
  • Send feedback
  • Trust
  • Editorial policy
  • How we make money
  • Contact
LEGAL
  • Privacy
  • Terms
  • Sitemap
MAIL · MONTHLY DIGEST
Get monthly local AI changes
Monthly recap. No spam.
DISCLOSURE

Some links on this site are affiliate links (Amazon Associates and other first-class retailers). When you buy through them, we earn a small commission at no extra cost to you. Affiliate links do not influence our verdicts — there are cards we rate highly that we don't have affiliate relationships with, and cards that sell well that we refuse to recommend. Read more →

SYS · ONLINEUPTIME · 100%2026 · operator-owned
RUNLOCALAI · v38
← Home·/apps·SDK / proxy

LiteLLM

Hybrid (offline or cloud)

Drop-in OpenAI-compatible proxy across 100+ providers. Route to local Ollama or cloud, same code.

Editorial verdict: “Best universal LLM proxy. Foundational layer for multi-provider deployments.”

SDK / proxy
Free tier
MIT
★ 4.5 / 5
GitHub ★ 15,000
↗ Homepage↗ GitHub↗ Docs

Compatibility at a glance

Which runtime + OS combos this app works against. Source of truth for "will it run on my setup?"

§ Runtimes supported
ollamallama-cppopenai-compatanthropicgeminiopenai
§ OS / platform
linuxmacoswindows

What it is

LiteLLM is a proxy that exposes OpenAI's API shape but routes to 100+ backends: Anthropic, Gemini, Ollama, Together, Groq, local llama.cpp, etc. Drop it between your app and your LLMs, and your app code stays OpenAI-shaped while you swap providers freely. Excellent for migration and A/B testing.

✓ Strengths

  • +Genuinely universal — 100+ providers
  • +OpenAI-shaped API stays put
  • +Built-in fallback / retry / cost tracking

△ Caveats

  • −Some advanced features paid (LiteLLM Cloud)
  • −Extra moving part in your stack

About the SDK / proxy category

Thin SDK / proxy / compatibility layer.

§ Other sdk / proxy apps
Ollama JS / TS SDK

Foundational primitive for Node + browser apps against Ollama. ESM-native, typed.

Ollama Python SDK

Foundational primitive for Python scripts against Ollama. Official, maintained, typed.

Where to go from here

Stack Builder →

Pre-filled with this app's recommended use case + budget tier. Get the full rig + runtime + model picks.

Back to /apps →

The full directory — filter by category, runtime, OS, privacy posture, or VRAM.

Runtimes (/tools) →

What this app talks to: Ollama, vLLM, llama.cpp, MLX, LM Studio. The upstream layer.

Community benchmarks →

Did this app work for you on a specific rig? Submit the benchmark — it powers the model + hardware pages.