Video Understanding

1. Install Ollama → ollama pull qwen2.5-vl:7b (~5 GB — strong video understanding model).
2. Extract keyframes from your video: pip install opencv-python → ffmpeg -i input.mp4 -vf "fps=1" frames/frame_%04d.jpg (one frame per second).
3. Python script to analyze video frames:

import ollama, glob
frames = sorted(glob.glob("frames/*.jpg"))
for i, frame in enumerate(frames):
    with open(frame, "rb") as f:
        img = f.read()
    resp = ollama.chat(model="qwen2.5-vl:7b", messages=[{
        "role": "user",
        "content": f"Frame {i+1}/{len(frames)}: Describe what's happening. Focus on actions, people, and scene changes.",
        "images": [img]
    }])
    print(f"Frame {i+1}: {resp['message']['content']}")

1. For a 1-minute video: 60 frames, ~5-10 minutes total processing on 12 GB GPU.
2. For temporal understanding (action sequences, events): Qwen2.5-VL supports video input natively — feed the video file directly (it auto-samples frames). Use ollama with video file path.
3. Use cases: security footage review, sports analysis, content moderation, video search.

Used RTX 3060 12 GB (~$200-250, see /hardware/rtx-3060-12gb). Runs Qwen2-VL 7B at 5-10 seconds per frame — a 1-minute video (60 frames at 1 fps) takes 5-10 minutes to analyze. For longer videos (10-60 minutes), process overnight. Pair with Ryzen 5 5600 + 32 GB DDR4 + 2TB NVMe (video storage). Total: ~$420-490. For CPU-only: MiniCPM-V via llama.cpp at 30-60 seconds per frame — 30-60 minutes for a 1-minute video, functional for occasional use.

Used RTX 3090 24 GB ($700-900, see /hardware/rtx-3090). Runs Qwen2-VL 72B at 10-20 seconds per frame — highest-quality local video understanding. For production video analysis (surveillance, content moderation), use Qwen2-VL 7B on 2× RTX 3060 in parallel — analyze 2 videos simultaneously. Total: ~$1,500-2,200. For the best quality-to-speed ratio: Qwen2-VL 7B on RTX 4090 ($2,000) at 1-3 seconds per frame. Video understanding scales linearly with frame count — faster GPU = more frames processed per hour.

The mistake: Feeding a 10-minute video directly to a VLM and asking "Summarize this video." Why it fails: VLMs have context limits — Qwen2-VL supports video input but typically samples 16-64 frames total. A 10-minute video at 1 fps has 600 frames. The model either samples sparsely (missing key moments) or exceeds its context window. You get a summary of whatever random frames were sampled. The fix: Always extract keyframes yourself with ffmpeg at a frame rate that captures your use case. For sports: 10 fps (action is fast). For surveillance: 1 fps (changes are slow). For lectures: scene-change detection (only process frames when slides change). Feed the model individual frames with timestamps. Control your sampling rate — don't trust the model's default video loader. The model sees what you show it; if you miss a frame, the model misses the event.