Best Used GPU for Local LLMs: Buying Guide 2026

Used cards are still the cheapest route to 24GB of VRAM for local LLMs. They also carry real risk: worn fans, deteriorated thermal pads, no warranty, and sellers who leave out a card’s history. This guide covers where to buy, what to check, which used cards make sense for local inference, and how to test your purchase before the return window closes.

Used GPU prices move with demand, so we don’t quote them here. Compare completed (sold) listings for the exact board model and condition rather than asking prices or any single listing, including the listings linked from this page.

If you’re still deciding between new and used, start with our best GPU for local LLMs roundup. If you’ve already decided to buy used, keep reading.

Where to Buy Used GPUs

Not all used GPU marketplaces are equal. Your priority is buyer protection: the ability to return a card that doesn’t work as advertised.

eBay

The default recommendation for most buyers. eBay’s Money Back Guarantee covers items that don’t arrive or don’t match the listing, and many sellers also accept returns for other reasons. Choose sellers with a long, positive feedback history, and read each listing’s return terms before you buy.

r/hardwareswap (Reddit)

The enthusiast marketplace. Prices can be lower because sellers avoid platform fees. Buyer protection depends on paying with PayPal Goods and Services (never Friends and Family). The subreddit’s confirmed-trade flair helps, but you’re trusting individuals, not a platform.

Best for: experienced buyers who know exactly which card they want and can evaluate a seller’s history. It is a poor fit for first-time used GPU buyers.

Amazon Renewed

Renewed listings come with Amazon’s Renewed Guarantee and its standard return process. Selection is narrower than on eBay. Check the guarantee terms shown on the listing before you buy.

Best for: buyers who want a simple return path and don’t mind a smaller selection.

Local Marketplaces (Facebook Marketplace, Craigslist, OfferUp)

The advantage is testing before you pay. Ask to see the card running under load in the seller’s system, with a monitoring tool open. The disadvantage is zero buyer protection once cash changes hands.

Best for: buyers in metro areas who can inspect and test in person. Avoid shipped deals on local platforms: you lose the in-person check and gain none of the protection eBay or PayPal offer.

What to Check Before Buying

Evaluate every used GPU listing against these criteria before you commit.

VRAM Capacity and Type

For LLM inference, VRAM is the single most important spec. A model that fits entirely on the GPU runs much faster than one that spills into system RAM.

Know how much VRAM you need before shopping. Our how much VRAM for LLMs guide has the full tables, and the short version is:

• 12 GB: 8B models at up to Q8_0, or 14B at Q4_K_M with a modest context
• 16 GB: 14B models at up to Q6_K
• 24 GB: 32B models at Q4_K_M with about an 8K context

Temperature History

Ask the seller for a screenshot of temperatures under a sustained load. On GDDR6X cards (RTX 3090, RTX 3080 Ti), GPU-Z also shows memory junction temperature. A memory temperature that keeps climbing, or that sits near the point where the card starts throttling, suggests worn thermal pads.

Replacing thermal pads is a common repair, but price it into your offer if the seller can’t show healthy temperatures.

Visual Artifacts

Ask the seller to run a stress test and send a video of the screen. Colored dots, horizontal lines, or flickering during stress testing indicate VRAM or GPU degradation. Walk away: this is not fixable.

Mining History

A card that mined is not automatically a bad buy. Mining often ran cards at fixed, reduced power, and the parts that wear from around-the-clock use are mostly the ones you can inspect:

1. Fan bearings: 24/7 operation wears fans faster than intermittent use.
2. Thermal pads: sustained heat degrades pads over time, reducing cooling effectiveness.
3. Cosmetic condition: open-air mining rigs often leave cards dusty.

A mining card with healthy temperatures and working fans can be a fine inference card. Price in a possible fan or thermal pad replacement.

Warranty and Return Policy

Prefer a return window of at least 30 days, which is long enough to run the full test plan below. Where a platform usually offers no returns, your protection is a not-as-described claim or your own in-person check:

• eBay: Money Back Guarantee for items that don’t arrive or don’t match the listing; other returns depend on the seller
• Amazon Renewed: Amazon’s Renewed Guarantee (check the listing for current terms)
• r/hardwareswap: PayPal Goods and Services, with 180 days to open a claim
• Local: none, so test before paying

Best Used GPUs for Local LLMs

These five cards cover the used options worth considering for local inference, from the most versatile to the most specialized.

1. RTX 3090: Best Overall for 24 GB VRAM

The used RTX 3090 remains the go-to used card when you need 24 GB of VRAM.

• Specs: 24 GB GDDR6X, 936 GB/s, 10,496 CUDA cores, 350W board power; NVIDIA recommends a 750W system power supply
• Fits: 32B models at Q4_K_M with about an 8K context, or 14B at Q8_0 with room for a longer context
• Doesn’t fit: 70B models at Q4_K_M, which need about 43GB for weights

It has the same capacity as the previous-generation RTX 4090 and enough bandwidth for fast generation on models that fit. Unlike the 4090, it also supports NVLink. Look for cards with original backplates, documented temperature history, and sellers who accept returns.

The 350W board rating is the main drawback. At $0.18 per kWh, every hour at full board power costs about 6 cents, so measure idle and load draw at the wall and multiply by your real duty cycle. Budget for a UPS that can handle the load.

For a head-to-head with the previous-generation RTX 4090, see RTX 3090 vs 4090 for LLMs.

2. RTX 3080 Ti: Fast 12 GB for 8B to 14B Models

The used RTX 3080 Ti suits builders who mainly run 8B to 14B models and don’t need room for 32B.

• Specs: 12 GB GDDR6X, 912 GB/s, 10,240 CUDA cores, 350W board power; NVIDIA recommends a 750W system power supply
• Fits: 8B models at up to Q8_0, or 14B at Q4_K_M with a modest context
• Doesn’t fit: 32B models

It has nearly the 3090’s bandwidth (912 vs 936 GB/s) with half the VRAM, which makes it fast on models that fit in 12 GB and a dead end for anything bigger. It also draws as much power as a 3090. If there’s any chance you’ll want 32B models, step up to the 3090’s 24 GB.

3. RTX 3060 12GB: Budget CUDA Entry Point

The used RTX 3060 12GB is the low-power way to get 12 GB of VRAM on a CUDA card.

• Specs: 12 GB GDDR6, 360 GB/s, 3,584 CUDA cores, 170W board power; NVIDIA recommends a 550W system power supply
• Fits: 8B models at up to Q8_0, or 14B at Q4_K_M with a modest context
• Doesn’t fit: 32B models

The 3060 12GB has the same capacity as the 3080 Ti but about 40% of its bandwidth. Generation is usually limited by memory bandwidth, so expect it to be proportionally slower on the same model. The trade is a 170W board rating that suits an always-on inference box. Before buying, compare it with new 16 GB cards such as the RTX 5060 Ti, which add capacity and a warranty.

Check the memory size. NVIDIA also sold an 8 GB RTX 3060 on a narrower 128-bit bus, and the RTX 3060 Ti has 8 GB too. For LLM work, make sure the listing says 12 GB.

4. Tesla P40: 24 GB for Headless Servers

The used Tesla P40 offers 24 GB of VRAM for less than a 3090, but requires additional cooling and power-connector checks.

• Specs: 24 GB GDDR5, 346 GB/s, 3,840 CUDA cores, 250W maximum power, passive dual-slot cooler
• Fits: the same model sizes as a 3090, because capacity decides what loads
• Speed: about 37% of the 3090’s bandwidth, so the same model generates far more slowly

That suits batch processing, background summarization, and other work where you aren’t waiting on every token.

Critical caveats:

• No video output. The P40 is a datacenter card with no display connectors. Use another GPU for display or run headless.
• No fan. The passive heatsink needs server front-to-back airflow, or a duct and blower in a desktop case.
• CPU (EPS) 8-pin power, not PCIe 8-pin. NVIDIA’s product brief lists a CPU-to-PCIe dongle that takes two PCIe 8-pin cables. Use that or a matching server cable, and never force a PCIe plug into the card.
• Above 4G Decoding. The P40 exposes a 32 GB compute BAR, so the host firmware needs Above 4G Decoding enabled.
• Legacy software support. Pascal (compute capability 6.1) has no Tensor Cores. CUDA 13 no longer builds for Pascal, and NVIDIA’s R580 driver branch, supported through mid-2028, is the last to support it. Ollama’s documentation lists compute capability 5.0 and newer, with driver 570 or newer for these older cards. Check every other tool in your stack.

The P40 is best for builders who already have a headless server with the right airflow and power cabling, or who want a second GPU for extra VRAM in a multi-GPU setup.

5. Tesla P100 16GB: Older but Still Useful

The used Tesla P100 16GB is for experimenters who want 16 GB with high bandwidth and can live with datacenter-card limits.

• Specs: 16 GB HBM2, 732 GB/s, 3,584 CUDA cores, 250W, passive dual-slot cooler, one CPU 8-pin power connector
• Fits: 14B models at up to Q6_K
• Doesn’t fit: 32B models

HBM2 gives it more than double the P40’s bandwidth, so it generates faster on models that fit in 16 GB. It shares the P40’s caveats: no video output, passive cooling, a CPU 8-pin power connector, and the same Pascal driver and CUDA limits.

Watch for two variants. NVIDIA also made a 12 GB PCIe P100 with 549 GB/s, and an SXM2 version that needs a special server baseboard. Only buy the 16 GB PCIe card unless you have an SXM2 server.

Red Flags: When to Walk Away

These are deal-breakers regardless of price.

Visual artifacts under stress testing. Colored dots, flickering, horizontal lines, or screen corruption during a stress test or game indicate dying VRAM or a failing GPU. This is hardware failure and is not fixable.

Missing backplate with no explanation. The backplate provides structural support and aids cooling. Cards sold without backplates were often racked in mining rigs. The missing backplate itself isn’t the problem; what it implies about the card’s history and the seller’s transparency is.

Suspiciously low prices. If a listing is far below recent completed sales with no disclosed reason, assume a scam, undisclosed damage, or a bait-and-switch. Check eBay’s sold listings for real price benchmarks.

No returns and no protected payment. A seller who refuses both is telling you something.

BIOS modifications. Some mining cards had their BIOS flashed to alter power limits or fan curves. Ask whether the BIOS is stock. A modified BIOS can cause stability issues and may void RMA eligibility on cards still under manufacturer warranty.

Physical damage to the PCIe connector or power pins. Bent pins, scorch marks, or corrosion on the PCIe edge connector or power connectors indicate electrical damage. The card may work today and die tomorrow.

Testing Procedures After Purchase

Your return window is limited, so run these steps as soon as the card arrives.

Step 1: Visual Inspection (5 Minutes)

Before installing the card, inspect it physically:

• Check the PCIe connector for bent or discolored pins
• Verify all fans spin freely by hand
• Look for bulging or leaking capacitors on the PCB
• Confirm the backplate or shroud is present and properly secured
• Check for signs of liquid damage (white residue, corrosion)

Step 2: Baseline Monitoring (15 Minutes)

Install the card and boot. Record idle temperatures, clocks, and fan behavior with GPU-Z on Windows or nvidia-smi on Linux. GPU-Z also shows memory junction temperature on GDDR6X cards. Listen for clicking or grinding from fan bearings.

Step 3: Stress Test (30 Minutes)

Run a sustained GPU load for at least 30 minutes. During the test:

• Temperatures should level off rather than keep climbing, for both the GPU core and, on GDDR6X cards, the memory junction
• Watch the screen continuously for artifacts; even a single colored dot is a failure
• Listen for coil whine: some is normal, but loud buzzing that changes with load deserves a closer look

Step 4: LLM Inference Check (15 Minutes)

This is what actually matters. Install Ollama, pull a model that fits your card, and run the same prompt with timing output:

ollama run llama3.1:8b --verbose "Write a 500-word essay about GPU computing"

The --verbose flag prints an eval rate after each response. Run the prompt several times, then again after the stress test. A healthy card holds a steady rate. A rate that falls as the card heats up points to thermal throttling, power delivery problems, or failing memory. Run ollama ps too: the processor column should read 100% GPU, because partial CPU offload also lowers the rate. Comparing against someone else’s numbers only makes sense when the model, quantization, context, driver, and settings all match.

Step 5: Extended Burn-In (24-48 Hours)

If the card passes steps 1-4, run continuous inference for 24-48 hours. Use a script that repeatedly prompts the model and logs the eval rate. Watch for:

• Speed that drops over time (usually a thermal issue)
• System crashes or driver resets
• Temperatures that keep rising

If anything fails during the burn-in period, start a return immediately. Don’t troubleshoot a used card while the return window runs out; send it back and buy another.

The Value Ladder: Making Your Decision

If 24 GB fits your budget, buy the RTX 3090. It’s the most versatile card on this list: 24 GB covers 32B models at Q4_K_M, while a 70B model at Q4_K_M needs about 43 GB, which no single card here holds.

If you mostly run 8B to 14B models, the RTX 3080 Ti is fast within its 12 GB, and the RTX 3060 12GB does the same job at much lower power.

The Tesla cards are specialist picks. The P40 is for budget builders who want 24 GB and can handle slower generation, server cooling, and no display output. The P100 is for experimenters who want strong bandwidth in 16 GB.

Whatever you buy, run the tests above within your return window. A used GPU that passes a 48-hour burn-in is likely to last. One that doesn’t should go back immediately.

For recommendations on new GPUs, see our best GPU for local LLMs guide. To size your target models, check how much VRAM for LLMs. For a direct comparison of the 3090 with the previous-generation 4090, read RTX 3090 vs 4090 for LLMs.