VRAM calculator for local LLMs

Estimate the GPU memory a model needs from its size, quantization and context length, and see which cards and multi-card setups it fits on.

SetupMemoryFits?

Weights = parameters × bits ÷ 8. KV cache = 2 × layers × KV heads × head size × bytes × context. Overhead is set at 1 GB. Multi-card setups lose a little to per-card overhead, so treat a "just fits" result as tight. The formula is explained in the VRAM guide.

The numbers behind it

Model at Q4_K_M8k context32k contextSmallest setup that fits 32k
Llama 3.1 8Babout 6.9 GBabout 10.1 GBOne 12 GB card
Qwen 2.5 32Babout 22.3 GBabout 28.8 GBOne 32 GB card (24 GB fits 8k only)
Llama 3.1 / 3.3 70Babout 45.7 GBabout 53.7 GBThree 24 GB cards, or one 96 GB card

Weights plus a 16-bit KV cache plus 1 GB of overhead. An 8-bit KV cache roughly halves the context part.

GPUs for local AI

Card or partVRAMBandwidthPowerTypical sold priceBasis
NVIDIA RTX 3090 24GB24 GB936 GB/s350 WGPU Soldtracked daily elsewhere
NVIDIA RTX 3090 Ti 24GB24 GB1008 GB/s450 WGPU Soldtracked daily elsewhere
NVIDIA RTX 4090 24GB24 GB1008 GB/s450 WGPU Soldtracked daily elsewhere
NVIDIA RTX 5090 32GB32 GB1792 GB/s575 WGPU Soldtracked daily elsewhere
NVIDIA RTX 3060 12GB12 GB360 GB/s170 WGPU Soldtracked daily elsewhere
NVIDIA RTX 4060 Ti 16GB16 GB288 GB/s165 WGPU Soldtracked daily elsewhere
NVIDIA RTX 5060 Ti 16GB16 GB448 GB/s180 WGPU Soldtracked daily elsewhere
NVIDIA RTX A6000 48GB48 GB768 GB/s300 WGPU Soldtracked daily elsewhere
NVIDIA RTX A5000 24GB24 GB768 GB/s230 WGPU Soldtracked daily elsewhere
NVIDIA RTX Pro 6000 Blackwell 96GB96 GB1792 GB/s600 WGPU Soldtracked daily elsewhere
NVIDIA Tesla P40 24GB24 GB347 GB/s250 WGPU Soldtracked daily elsewhere
NVIDIA Tesla V100 32GB (PCIe)32 GB900 GB/s250 WGPU Soldtracked daily elsewhere
AMD Instinct MI50 32GB32 GB1024 GB/s300 WGPU Soldtracked daily elsewhere
AMD Radeon RX 7900 XTX 24GB24 GB960 GB/s355 WGPU Soldtracked daily elsewhere
NVIDIA RTX 2080 Ti 22GB (modded)22 GB616 GB/s250 WGPU Soldtracked daily elsewhere

Multi-GPU build parts

Card or partTypeTypical sold priceBasis
NVLink bridge for RTX 3090Bridgenot enough sales yetfewer than 6 sales
Supermicro H12SSL-i motherboardMotherboardnot enough sales yetfewer than 6 sales
ASUS Pro WS WRX80E-SAGE SE WiFiMotherboardnot enough sales yetfewer than 6 sales
AMD EPYC 7532 (32 cores)CPUnot enough sales yetfewer than 6 sales
AMD EPYC 7302 (16 cores)CPUnot enough sales yetfewer than 6 sales
64GB DDR4 ECC RDIMMMemory (single module)not enough sales yetfewer than 6 sales
1600 W power supply (EVGA SuperNOVA 1600)Power supplynot enough sales yetfewer than 6 sales
Dell Precision 7920 towerWorkstationnot enough sales yetfewer than 6 sales
HP Z8 G4 workstationWorkstationnot enough sales yetfewer than 6 sales

Macs for local AI

Card or partTypeTypical sold priceBasis
Mac Studio M2 Ultra 192GBMacnot enough sales yetfewer than 6 sales
Mac Studio M1 Ultra 128GBMacnot enough sales yetfewer than 6 sales
Mac Studio M3 UltraMacnot enough sales yetfewer than 6 sales

Prices as of Nov 19, 2026. Each item's page has the basis for its price.