Local AI Hardware

local-ai · hardware for running models at home

Hardware for running AI at home, measured and priced.

Which used GPUs, Macs and server parts to buy for running LLMs locally: the specs that decide speed, our own benchmark runs, sized builds from 24 GB to 96 GB, and sold prices for the parts.

Benchmarks
Our runs, settings published
GPU prices
Daily, on GPU Sold
Part prices
Completed eBay sales
Sizing
VRAM calculator

Benchmarks for local LLMs disagree with each other because they are run with different software versions, quantizations and context lengths. We publish our own runs only, with every setting recorded, and leave a card blank until we have measured it. Start with the VRAM calculator to see what fits, then the build guides.

GPUs for local AI

The cards people actually buy for local LLMs, compared on memory, bandwidth and power. Sold prices for each card are tracked daily on GPU Sold.

Card or partVRAMBandwidthPowerTypeSold forBasis
NVIDIA RTX 3090 24GB24 GB936 GB/s350 WGPU Soldelsewhere
NVIDIA RTX 3090 Ti 24GB24 GB1008 GB/s450 WGPU Soldelsewhere
NVIDIA RTX 4090 24GB24 GB1008 GB/s450 WGPU Soldelsewhere
NVIDIA RTX 5090 32GB32 GB1792 GB/s575 WGPU Soldelsewhere
NVIDIA RTX 3060 12GB12 GB360 GB/s170 WGPU Soldelsewhere
NVIDIA RTX 4060 Ti 16GB16 GB288 GB/s165 WGPU Soldelsewhere
NVIDIA RTX 5060 Ti 16GB16 GB448 GB/s180 WGPU Soldelsewhere
NVIDIA RTX A6000 48GB48 GB768 GB/s300 WGPU Soldelsewhere
NVIDIA RTX A5000 24GB24 GB768 GB/s230 WGPU Soldelsewhere
NVIDIA RTX Pro 6000 Blackwell 96GB96 GB1792 GB/s600 WGPU Soldelsewhere
NVIDIA Tesla P40 24GB24 GB347 GB/s250 WGPU Soldelsewhere
NVIDIA Tesla V100 32GB (PCIe)32 GB900 GB/s250 WGPU Soldelsewhere
AMD Instinct MI50 32GB32 GB1024 GB/s300 WGPU Soldelsewhere
AMD Radeon RX 7900 XTX 24GB24 GB960 GB/s355 WGPU Soldelsewhere
NVIDIA RTX 2080 Ti 22GB (modded)22 GB616 GB/s250 WGPU Soldelsewhere

Multi-GPU build parts

The platform under the GPUs: boards with enough PCIe slots, CPUs with enough lanes, and power supplies big enough to feed it all.

Card or partVRAMBandwidthPowerTypeSold forBasis
NVLink bridge for RTX 3090Bridge–
Supermicro H12SSL-i motherboardMotherboard–
ASUS Pro WS WRX80E-SAGE SE WiFiMotherboard–
AMD EPYC 7532 (32 cores)CPU–
AMD EPYC 7302 (16 cores)CPU–
64GB DDR4 ECC RDIMMMemory (single module)–
1600 W power supply (EVGA SuperNOVA 1600)Power supply–
Dell Precision 7920 towerWorkstation–
HP Z8 G4 workstationWorkstation–

Macs for local AI

Apple Silicon shares one large pool of memory between CPU and GPU, which lets a single quiet machine run very large models.

Card or partVRAMBandwidthPowerTypeSold forBasis
Mac Studio M2 Ultra 192GBMac–
Mac Studio M1 Ultra 128GBMac–
Mac Studio M3 UltraMac–