Enterprise Search

1. Install Ollama → ollama pull nomic-embed-text (embeddings) + ollama pull llama3.1:8b (generation).
2. Install a vector DB: docker run -p 6333:6333 qdrant/qdrant (Qdrant) or pip install chromadb.
3. Index your documents:

import chromadb, requests
client = chromadb.Client()
collection = client.create_collection("enterprise")
for i, doc in enumerate(documents):
    resp = requests.post("http://localhost:11434/api/embeddings",
        json={"model": "nomic-embed-text", "prompt": doc["text"]})
    collection.add(documents=[doc["text"]], embeddings=[resp.json()["embedding"]], ids=[f"doc_{i}"])

1. Query: embed the query → search top-20 → rerank with LLM → return top-5 with snippets.
2. For permissions-aware search: add metadata filters (department, classification level) to each document → filter at query time.
3. First search result in <2 seconds for a 10K-document corpus on CPU.

Enterprise search is surprisingly cheap. Nomic Embed Text runs on any CPU — a $300 laptop indexes and searches 100K documents at sub-second query latency. ChromaDB or LanceDB (embedded, no Docker needed) handles the vector storage. The generation step (answer synthesis) needs a GPU only if you want LLM-generated answers from search results. For "search and return snippets" (no generation), any $300 machine works great. For LLM-synthesized answers, add a used GTX 1060 6 GB (~$60) — total ~$360.

Used RTX 3090 24 GB (~$700-900, see /hardware/rtx-3090). Runs the full pipeline: BGE-M3 embeddings (1,000+ docs/second), BGE Reranker V2 M3 for precise ranking, Llama 3.3 70B for answer synthesis — all on one GPU. Qdrant/Weaviate for production vector DB with filtering + permissions. Handles 1M+ documents with sub-500ms query latency. Pair with Ryzen 7 7700X + 64 GB DDR5 + 2TB NVMe. Total: ~$1,800-2,200. For enterprise scale (10M+ docs, 100+ concurrent users): add a second GPU and switch to distributed Qdrant.

The mistake: Using a single-stage keyword search (Elasticsearch BM25) for "enterprise AI search" and calling it done. Why it fails: Keyword search can't handle semantic queries — searching "how to request time off" won't find a document titled "PTO Policy" because the words don't match. Users expect Google-level semantic understanding from enterprise search now. The fix: Use hybrid search: BM25 (keywords) + dense embeddings (semantics) combined. Retrieve top-100 via each method, merge (reciprocal rank fusion), then rerank top-50 with a cross-encoder. This gives both exact-match precision AND semantic recall. The embedding model (Nomic/BGE) + reranker add ~200ms latency and ~1 GB VRAM overhead — trivially cheap for the quality improvement.