local-ai · hardware for running models at home
Hardware for running AI at home, measured and priced.
Which used GPUs, Macs and server parts to buy for running LLMs locally: the specs that decide speed, our own benchmark runs, sized builds from 24 GB to 96 GB, and sold prices for the parts.
- Benchmarks
- Our runs, settings published
- GPU prices
- Daily, on GPU Sold
- Part prices
- Completed eBay sales
- Sizing
- VRAM calculator
Benchmarks for local LLMs disagree with each other because they are run with different software versions, quantizations and context lengths. We publish our own runs only, with every setting recorded, and leave a card blank until we have measured it. Start with the VRAM calculator to see what fits, then the build guides.
GPUs for local AI
The cards people actually buy for local LLMs, compared on memory, bandwidth and power. Sold prices for each card are tracked daily on GPU Sold.
| Card or part | VRAM | Type | Sold for | Basis |
|---|---|---|---|---|
| NVIDIA RTX 3090 24GB | 24 GB | GPU Sold | elsewhere | |
| NVIDIA RTX 3090 Ti 24GB | 24 GB | GPU Sold | elsewhere | |
| NVIDIA RTX 4090 24GB | 24 GB | GPU Sold | elsewhere | |
| NVIDIA RTX 5090 32GB | 32 GB | GPU Sold | elsewhere | |
| NVIDIA RTX 3060 12GB | 12 GB | GPU Sold | elsewhere | |
| NVIDIA RTX 4060 Ti 16GB | 16 GB | GPU Sold | elsewhere | |
| NVIDIA RTX 5060 Ti 16GB | 16 GB | GPU Sold | elsewhere | |
| NVIDIA RTX A6000 48GB | 48 GB | GPU Sold | elsewhere | |
| NVIDIA RTX A5000 24GB | 24 GB | GPU Sold | elsewhere | |
| NVIDIA RTX Pro 6000 Blackwell 96GB | 96 GB | GPU Sold | elsewhere | |
| NVIDIA Tesla P40 24GB | 24 GB | GPU Sold | elsewhere | |
| NVIDIA Tesla V100 32GB (PCIe) | 32 GB | GPU Sold | elsewhere | |
| AMD Instinct MI50 32GB | 32 GB | GPU Sold | elsewhere | |
| AMD Radeon RX 7900 XTX 24GB | 24 GB | GPU Sold | elsewhere | |
| NVIDIA RTX 2080 Ti 22GB (modded) | 22 GB | GPU Sold | elsewhere |
Multi-GPU build parts
The platform under the GPUs: boards with enough PCIe slots, CPUs with enough lanes, and power supplies big enough to feed it all.
| Card or part | VRAM | Type | Sold for | Basis |
|---|---|---|---|---|
| NVLink bridge for RTX 3090 | Bridge | – | ||
| Supermicro H12SSL-i motherboard | Motherboard | – | ||
| ASUS Pro WS WRX80E-SAGE SE WiFi | Motherboard | – | ||
| AMD EPYC 7532 (32 cores) | CPU | – | ||
| AMD EPYC 7302 (16 cores) | CPU | – | ||
| 64GB DDR4 ECC RDIMM | Memory (single module) | – | ||
| 1600 W power supply (EVGA SuperNOVA 1600) | Power supply | – | ||
| Dell Precision 7920 tower | Workstation | – | ||
| HP Z8 G4 workstation | Workstation | – |
Macs for local AI
Apple Silicon shares one large pool of memory between CPU and GPU, which lets a single quiet machine run very large models.
| Card or part | VRAM | Type | Sold for | Basis |
|---|---|---|---|---|
| Mac Studio M2 Ultra 192GB | Mac | – | ||
| Mac Studio M1 Ultra 128GB | Mac | – | ||
| Mac Studio M3 Ultra | Mac | – |
Builds and methods
- How we benchmark GPUs for local LLMsThe exact method behind every speed on this site: llama.cpp's llama-bench, pinned versions, fixed model files and quantizations, prompt and generation tests, and what we record with each run.
- How much VRAM a model needs: weights, context and overheadWork out the memory an LLM needs from its size, quantization and context length, with worked examples for 8B, 32B and 70B models and the formula behind the VRAM calculator.
- Local AI builds at 24, 48 and 96 GB: what to buy usedThree sized builds for running LLMs at home, from a single used RTX 3090 to a four-GPU EPYC server, with the parts, the power, the trade-offs and the Mac alternative at each size.