Docs

Local AI builds at 24, 48 and 96 GB: what to buy used

Three sized builds for running LLMs at home, from a single used RTX 3090 to a four-GPU EPYC server, with the parts, the power, the trade-offs and the Mac alternative at each size.

Last updated Nov 18, 2026

Builds for local LLMs are sized by memory, because memory decides which models you can run at all. These three sizes cover most people. Prices for every part come from used sales; GPU prices are on GPU Sold and the rest on our build parts page.

24 GB: one card

Runs: models up to about 32B parameters at 4-bit with a context of around 8k tokens (more with an 8-bit KV cache), and any 7B–14B model comfortably.

  • GPU: a used RTX 3090 is the default. An RTX 4090 is much faster at reading prompts but no faster at writing, for several times the price.
  • Platform: any modern desktop with a free x16 slot and room for a three-slot card.
  • Power: an 850–1000 W quality PSU. The 3090's spikes are well above its 350 W rating.

48 GB: two cards or one big one

Runs: 70B models at 4-bit with modest context (see the VRAM guide), and 32B models at 8-bit.

  • Two RTX 3090s: the classic choice. Needs a board with two x16-size slots spaced for three-slot cards (or a riser), and a 1200–1600 W PSU. NVLink is optional for inference.
  • One RTX A6000: 48 GB in two slots at 300 W, blower-cooled, simpler to fit and power, and slower than two 3090s at generation.
  • Two 5060 Ti 16GB or similar get you to 32 GB cheaply, but not to 48.

96 GB: a server, one workstation card, or a Mac

Runs: 70B at 8-bit, 100B-plus models at 4-bit, and long contexts on smaller models.

  • Four RTX 3090s on an EPYC board such as the Supermicro H12SSL-i, with a 16- or 32-core EPYC for the PCIe lanes, risers or an open frame, and two PSUs or one very large one. Power-limiting the cards (many owners run 3090s around 250–280 W) costs little speed and saves a lot of heat.
  • One RTX Pro 6000 Blackwell (96 GB): far simpler and quieter, at a much higher price for the single card.
  • A Mac Studio with 128 GB (M1 Ultra) or 192 GB (M2 Ultra): silent, low power, and able to load very large models, but much slower at prompt processing than NVIDIA cards.

Before you buy for any size

  1. Decide which models you want to run and check the memory they need first.
  2. Count PCIe slots, slot spacing and PSU connectors, not just the number of GPUs.
  3. Check the speeds we have measured on each card's page; we only publish cards we have run.