Local AI Hardware / GPUs for local AI
AMD Instinct MI50 32GB for local AI
Sold prices for the AMD Instinct MI50 32GB are tracked every day on GPU Sold (https://gpusold.com/gpu/instinct-mi50-32).
32 GB of HBM2 for very little money, with real software caveats.
- Memory
- 32 GB
- Memory bandwidth
- 1024 GB/s
- Board power
- 300 W
- Architecture
- Vega 20 (gfx906)
What matters for AI
Recent ROCm releases have dropped official support for gfx906, the MI50's chip. Many people still run it with llama.cpp, but expect to pin older versions or build from source.
Before you buy a used GPU for AI
- Memory first. The model and its context have to fit in VRAM (or it spills to system RAM and slows to a crawl). Use the VRAM calculator.
- Bandwidth second. Token generation speed is mostly limited by memory bandwidth, not compute.
- Check memory temperatures under load on GDDR6X cards (3090, 3090 Ti). Worn thermal pads are common on ex-mining cards.
- Plan power and slot spacing before buying two or more cards. Transient spikes on high-end cards trip undersized power supplies.