Best GPU for Ollama in 2026

Ollama makes local models easy to launch, but GPU shopping still begins with two questions: does the model fit, and does Ollama support the card on your operating system? This guide covers Ollama’s supported backends, how to confirm a model is running on the GPU, the settings that change memory use, and which cards fit common model sizes.

The used RTX 3090 is the value recommendation when 24GB matters. The RTX 5060 Ti 16GB is our recommendation for a new card when 16GB fits. Those are different risk and capacity choices, not merely a speed ranking.

Which GPUs Ollama supports

Ollama’s GPU documentation lists four hardware paths:

• NVIDIA: cards with compute capability 5.0 or newer and driver 550 or newer. Older cards in the 5.x and 6.x range need driver 570 or newer.
• AMD: Radeon cards through ROCm v7 on Linux, including the RX 9070 XT, RX 9070, RX 9060 XT, and RX 7900 XTX. The Windows list is narrower.
• Apple: Apple silicon through Metal.
• Vulkan: Intel and AMD GPUs on Windows and Linux. Set OLLAMA_VULKAN=0 to turn it off.

To limit Ollama to specific cards, set CUDA_VISIBLE_DEVICES (NVIDIA), ROCR_VISIBLE_DEVICES (AMD), or GGML_VK_VISIBLE_DEVICES (Vulkan).

Check where the model runs

Load a representative model and prompt, then run ollama ps. The processor column reads “100% GPU” when the whole model is on the card. A split such as “48%/52% CPU/GPU” means part of the model is running from system RAM. Partial offload works, but it can slow generation enough that an interactive assistant turns into a batch job.

Settings that change VRAM use

The same model can fit or spill depending on a few Ollama settings, all documented in its FAQ:

• Context length. The default is 4,096 tokens. Raise it with OLLAMA_CONTEXT_LENGTH or the num_ctx parameter, and the KV cache grows in proportion. For Qwen3-32B, 32K tokens of 16-bit cache takes about 8.6GB on top of the weights.
• KV cache type. With OLLAMA_FLASH_ATTENTION enabled, OLLAMA_KV_CACHE_TYPE=q8_0 stores the cache in about half the memory of the default, and q4_0 in about a quarter. The setting applies to every model the server runs.
• Parallel requests. OLLAMA_NUM_PARALLEL (default 1) sets how many requests a model serves at once. Memory scales with parallel requests multiplied by context length, so a household server needs more headroom than a single-user desktop.
• Loaded models. OLLAMA_MAX_LOADED_MODELS limits how many models stay in memory together. Several resident models compete for the same VRAM.

The VRAM planning guide has weight sizes and KV-cache tables for sizing a card before you buy.

Multiple GPUs

If a model fits on one GPU, Ollama loads it there. If it doesn’t, Ollama spreads it across all available GPUs. Two cards can therefore hold a model that neither holds alone, but a single card with enough VRAM is simpler to power, cool, and troubleshoot.

CUDA simplifies software setup

NVIDIA remains the easiest recommendation because CUDA support is broadly documented across local-AI tools. The RTX 5060 Ti 16GB is the current new entry point. The RTX 5070 Ti does not increase VRAM, so buy it only when more throughput inside the same 16GB is worth the extra system cost.

The RTX 3090 gives up efficiency and warranty for 24GB, which keeps 32B models at Q4_K_M entirely on the GPU. Check the PSU, physical clearance, airflow, connector condition, memory temperatures, and seller return policy before treating the card price as the complete cost.

AMD and Intel require stack validation

The Radeon RX 9070 XT is on Ollama’s supported list for Linux through ROCm, and it can also run through Vulkan. Verify your exact operating system and Ollama version before buying.

The Intel Arc B580 runs through Ollama’s Vulkan backend. With 12GB it suits 8B and 14B models. Choose it if you are willing to configure the Vulkan backend before using the server.

Bottom line

Choose capacity first, software support second, and speed third. Buy the used RTX 3090 for a risk-tolerant 24GB CUDA build. Buy the RTX 5060 Ti 16GB for a current, lower-power, warrantied CUDA system. Buy AMD or Intel only after the exact Ollama backend for your card and operating system is confirmed in Ollama’s documentation.