Best GPUs for Local LLMs

Graphics cards, VRAM guidance, inference hardware, and local AI build advice for running models at home.

Popular buying paths

Start with the job you need done, then pick the hardware that fits.

Which model sizes and quantizations fit in 8, 12, 16, 24, and 32 GB.

New and used cards compared on memory capacity, bandwidth, power, and software support.

The cheapest cards that run useful models, and the trade-offs that come with them.

Four practical lower-cost GPUs for local AI, compared by VRAM, memory bandwidth, software compatibility, power, and buying risk.

Current inference GPUs compared by VRAM, board power, cooling, software support, physical fit, and used-market risk.

Current GPU recommendations for running LLMs locally, ranked by VRAM, memory bandwidth, power, software support, and buying risk.

Choosing a GPU for Ollama: supported backends by OS, how to check GPU placement, settings that change VRAM use, and cards that fit common models.

How to buy a used GPU for local LLMs: five cards compared by VRAM, bandwidth and risk, plus red flags and a test plan for your return window.

Step-by-step GPU passthrough on Proxmox VE 9 with VFIO: IOMMU setup, driver binding, VM configuration, NVIDIA drivers, and running local LLMs.

How to size GPU memory for local LLMs: weight sizes by quantization, KV-cache math for context length, and what fits on 8GB to 48GB cards.

The M5 Pro Mac mini and NVIDIA CUDA cards compared for local LLMs by memory capacity, bandwidth, software support, model fit, power, and risk.

NVIDIA CUDA and AMD ROCm compared for Ollama, llama.cpp, PyTorch, and home inference, including RDNA 4 support and AMD's 24GB and 32GB cards.

A lifecycle-aware comparison of two 24GB NVIDIA GPUs for used and legacy buyers, with current alternatives for new purchases.

RTX 5060 Ti 16GB versus used RTX 3090 for local inference: current specs, VRAM limits, efficiency, warranty, and used-card risk.