VRAM calculator for local LLMs
Estimate the GPU memory a model needs from its size, quantization and context length, and see which cards and multi-card setups it fits on.
| Setup | Memory | Fits? |
|---|
Weights = parameters × bits ÷ 8. KV cache = 2 × layers × KV heads × head size × bytes × context. Overhead is set at 1 GB. Multi-card setups lose a little to per-card overhead, so treat a "just fits" result as tight. The formula is explained in the VRAM guide.
The numbers behind it
| Model at Q4_K_M | 8k context | 32k context | Smallest setup that fits 32k |
|---|---|---|---|
| Llama 3.1 8B | about 6.9 GB | about 10.1 GB | One 12 GB card |
| Qwen 2.5 32B | about 22.3 GB | about 28.8 GB | One 32 GB card (24 GB fits 8k only) |
| Llama 3.1 / 3.3 70B | about 45.7 GB | about 53.7 GB | Three 24 GB cards, or one 96 GB card |
Weights plus a 16-bit KV cache plus 1 GB of overhead. An 8-bit KV cache roughly halves the context part.
GPUs for local AI
| Card or part | VRAM | Bandwidth | Power | Typical sold price | Basis |
|---|---|---|---|---|---|
| NVIDIA RTX 3090 24GB | 24 GB | 936 GB/s | 350 W | GPU Sold | tracked daily elsewhere |
| NVIDIA RTX 3090 Ti 24GB | 24 GB | 1008 GB/s | 450 W | GPU Sold | tracked daily elsewhere |
| NVIDIA RTX 4090 24GB | 24 GB | 1008 GB/s | 450 W | GPU Sold | tracked daily elsewhere |
| NVIDIA RTX 5090 32GB | 32 GB | 1792 GB/s | 575 W | GPU Sold | tracked daily elsewhere |
| NVIDIA RTX 3060 12GB | 12 GB | 360 GB/s | 170 W | GPU Sold | tracked daily elsewhere |
| NVIDIA RTX 4060 Ti 16GB | 16 GB | 288 GB/s | 165 W | GPU Sold | tracked daily elsewhere |
| NVIDIA RTX 5060 Ti 16GB | 16 GB | 448 GB/s | 180 W | GPU Sold | tracked daily elsewhere |
| NVIDIA RTX A6000 48GB | 48 GB | 768 GB/s | 300 W | GPU Sold | tracked daily elsewhere |
| NVIDIA RTX A5000 24GB | 24 GB | 768 GB/s | 230 W | GPU Sold | tracked daily elsewhere |
| NVIDIA RTX Pro 6000 Blackwell 96GB | 96 GB | 1792 GB/s | 600 W | GPU Sold | tracked daily elsewhere |
| NVIDIA Tesla P40 24GB | 24 GB | 347 GB/s | 250 W | GPU Sold | tracked daily elsewhere |
| NVIDIA Tesla V100 32GB (PCIe) | 32 GB | 900 GB/s | 250 W | GPU Sold | tracked daily elsewhere |
| AMD Instinct MI50 32GB | 32 GB | 1024 GB/s | 300 W | GPU Sold | tracked daily elsewhere |
| AMD Radeon RX 7900 XTX 24GB | 24 GB | 960 GB/s | 355 W | GPU Sold | tracked daily elsewhere |
| NVIDIA RTX 2080 Ti 22GB (modded) | 22 GB | 616 GB/s | 250 W | GPU Sold | tracked daily elsewhere |
Multi-GPU build parts
| Card or part | Type | Typical sold price | Basis |
|---|---|---|---|
| NVLink bridge for RTX 3090 | Bridge | not enough sales yet | fewer than 6 sales |
| Supermicro H12SSL-i motherboard | Motherboard | not enough sales yet | fewer than 6 sales |
| ASUS Pro WS WRX80E-SAGE SE WiFi | Motherboard | not enough sales yet | fewer than 6 sales |
| AMD EPYC 7532 (32 cores) | CPU | not enough sales yet | fewer than 6 sales |
| AMD EPYC 7302 (16 cores) | CPU | not enough sales yet | fewer than 6 sales |
| 64GB DDR4 ECC RDIMM | Memory (single module) | not enough sales yet | fewer than 6 sales |
| 1600 W power supply (EVGA SuperNOVA 1600) | Power supply | not enough sales yet | fewer than 6 sales |
| Dell Precision 7920 tower | Workstation | not enough sales yet | fewer than 6 sales |
| HP Z8 G4 workstation | Workstation | not enough sales yet | fewer than 6 sales |
Macs for local AI
| Card or part | Type | Typical sold price | Basis |
|---|---|---|---|
| Mac Studio M2 Ultra 192GB | Mac | not enough sales yet | fewer than 6 sales |
| Mac Studio M1 Ultra 128GB | Mac | not enough sales yet | fewer than 6 sales |
| Mac Studio M3 Ultra | Mac | not enough sales yet | fewer than 6 sales |
Prices as of Nov 19, 2026. Each item's page has the basis for its price.