NVIDIA Jetson AI

1. Buy an NVIDIA Jetson Orin Nano Developer Kit ($500, 8 GB) or Jetson Orin NX ($700, 16 GB).
2. Flash JetPack SDK (Ubuntu + CUDA + TensorRT + pre-installed AI libraries) via NVIDIA SDK Manager. Takes ~30 minutes.
3. After flashing, the Jetson is a full Ubuntu desktop with GPU-accelerated AI:

# Object detection
pip install ultralytics
yolo detect predict model=yolo11n.pt source=0  # webcam, real-time

# LLM inference via Ollama (ARM64 build available)
ollama pull llama3.2:3b
ollama run llama3.2:3b "What is the capital of France?"

# Edge deployment
jetson-containers run dustynv/whisper:r36.4.0  # Whisper STT

1. First AI inference in <1 hour from unboxing. The Jetson runs the same CUDA code as desktop GPUs — any PyTorch/TensorFlow model that fits in 8-16 GB works.
2. Power consumption: 7-15W (Nano) to 15-40W (Orin NX). Runs on battery for hours — ideal for drones, robots, mobile deployments.

Jetson Orin Nano Developer Kit (~$500 new, 8 GB, Ampere GPU with 1024 CUDA cores + 32 Tensor cores). Runs YOLO11n at 60+ fps, Llama 3.2 3B at 20-35 tok/s, Whisper tiny at 20× real-time. This is the cheapest entry point for CUDA-on-the-edge. For hobbyist robotics (ROS2 + AI perception on a robot): the Nano handles SLAM, object detection, and voice commands simultaneously at 7-15W. Add a Raspberry Pi Camera Module 3 ($25) + USB microphone ($10). Total: ~$535. The Jetson Orin Nano is the "Raspberry Pi of GPU computing" — accessible, well-documented, huge community.

Jetson Orin AGX 64 GB (~$2,000, industrial/automotive grade). 2048 CUDA cores + 64 Tensor cores + 64 GB unified memory. Runs Qwen 2.5 7B at 15-25 tok/s, YOLO11x at 100+ fps for 4K, multiple concurrent AI pipelines. Designed for autonomous machines: self-driving cars, industrial robots, medical devices. For a research robot platform: Orin AGX handles SLAM + manipulation policy + speech interaction + vision-language reasoning simultaneously. Power: 15-60W. Total robotics build (Orin AGX + sensors + chassis): $5,000-15,000. The Jetson AGX is for when you need datacenter-class AI on a robot/drone/vehicle with a battery budget.

The mistake: Treating the Jetson like a desktop GPU — running FP32 models without TensorRT optimization, then wondering why inference speed is 5× slower than benchmarks show. Why it fails: Jetson benchmarks assume TensorRT-optimized INT8/FP16 models. Raw PyTorch FP32 inference on Jetson uses only CUDA cores (no Tensor cores). TensorRT applies layer fusion, kernel auto-tuning, precision calibration, and memory optimization — this is where the 5-10× speedup comes from. Without TensorRT, your Jetson is an expensive Raspberry Pi. The fix: Always convert models to TensorRT. For PyTorch models: torch2trt or export ONNX → trtexec. For TensorFlow: TF-TRT. For LLMs: use the NVIDIA-optimized builds (jetson-containers has pre-built TensorRT-LLM images for Llama, Whisper, Stable Diffusion). The "just pip install and run" convenience of desktop GPUs doesn't apply to Jetson — optimization is required, not optional. TensorRT is the difference between "this doesn't work for real-time" and "this runs faster than a desktop 3060."