The gap between "open-weight model you can download" and "model you can actually run on the GPU in your machine" is the whole game in 2026. There are spectacular open models — DeepSeek V3-class, Llama 405B, Qwen3 235B — that are genuinely competitive with frontier cloud APIs and that you cannot run at home, because they need datacenter memory. This guide is about the other set: the best local LLMs that fit a real consumer or prosumer GPU, ranked honestly by what they're good at and what they cost in VRAM.
The single most useful thing you can do before reading on: check what your card can run. Pick your GPU →and you'll see which of these models run great, which run with trade-offs, and how fast — or, going the other way, find the GPU a specific model needs.
How to read this list
Two numbers decide everything: your VRAMand the model's parameter count. As a rule of thumb, a model quantized to Q4_K_M (the quality/size sweet spot) needs about 0.5–0.6 GB per billion parameters for weights, plus a few GB for the KV cache that holds your context. So an 8B fits comfortably in 8GB, a 14B in 12GB, a 32B in 24GB, and a 70B wants 40GB+ (or a punishing low quant on a single 24GB card). The numbers below are Q4_K_M weight estimates; leave headroom for context.
The best all-rounders (chat, reasoning, general use)
| Model | Params | ~VRAM (Q4) | Best for | Runs on |
|---|---|---|---|---|
| Qwen3 32B | 32B | ~20 GB | Strongest single-GPU all-rounder; reasoning mode | 24GB card |
| Llama 3.3 70B | 70B | ~40 GB | Frontier-feel quality, follows instructions tightly | 2×24GB / 48GB+ |
| Qwen3 14B | 14B | ~9 GB | Best 12–16GB pick; reasoning + multilingual | 12–16GB card |
| Gemma 3 27B | 27B | ~17 GB | Long-context chat, strong writing, multimodal | 24GB card |
| Gemma 3 12B | 12B | ~8 GB | Great mid-size all-rounder, vision-capable | 12GB card |
| Llama 3.1 8B | 8B | ~5 GB | Reliable 8GB workhorse, huge ecosystem | 8GB card |
| Qwen3 8B | 8B | ~5 GB | 8B with a reasoning mode; strong for size | 8GB card |
| Phi-4 14B | 14B | ~9 GB | Punches above its size on reasoning/math | 12–16GB card |
If you have one 24GB card (RTX 3090/4090), Qwen3 32B is the default-best general model — it reasons, it codes acceptably, and it leaves room for a useful context window. On 12–16GB, Qwen3 14B and Gemma 3 12B are the two to try first. On 8GB, Llama 3.1 8B is the safe, well-supported choice and Qwen3 8B is the more capable one if you want a reasoning mode.
The reasoning specialists (math, multi-step, agents)
DeepSeek's R1 line popularized open reasoning models — they "think" in a visible scratchpad before answering, trading latency for accuracy on hard problems. The full R1 (671B) is datacenter-only, but the distilled versions bake much of that behavior into models you can actually run:
| Model | Params | ~VRAM (Q4) | Best for | Runs on |
|---|---|---|---|---|
| DeepSeek R1 Distill Qwen 32B | 32B | ~20 GB | Best local reasoning that fits 24GB | 24GB card |
| DeepSeek R1 Distill Qwen 14B | 14B | ~9 GB | Strong reasoning on mid-range cards | 12–16GB card |
| Qwen3 32B (thinking) | 32B | ~20 GB | Reasoning + general in one model | 24GB card |
| DeepSeek R1 Distill Qwen 7B | 7B | ~5 GB | Reasoning traces on an 8GB budget | 8GB card |
Reasoning models emit a lot of intermediate tokens, so they need a bigger context window and run slower per visible answer. Budget extra VRAM for the KV cache — the context window calculator shows how fast that grows.
The efficient small models (laptops, edge, leave-room-for-context)
| Model | Params | ~VRAM (Q4) | Best for | Runs on |
|---|---|---|---|---|
| Gemma 3 4B | 4B | ~3 GB | Vision-capable, runs on modest laptops | 4–6GB / Apple Silicon |
| Phi-4 Mini 3.8B | 3.8B | ~2.5 GB | Reasoning-dense for its size, very fast | 4–6GB card |
| Llama 3.2 3B | 3B | ~2 GB | Lightweight chat, on-device assistants | low VRAM / CPU |
| Qwen3 4B | 4B | ~3 GB | Tiny model with a reasoning mode | 4–6GB card |
| Mistral Nemo 12B | 12B | ~8 GB | 128K context, great instruction-following | 12GB card |
These shine when you want long context, fast responses, or to run a model alongside everything else on a laptop. A 4B at Q4 leaving most of your VRAM for context often beats a cramped 14B that has no room to think.
The honest ranking
If you want a single decision tree by the only thing that's fixed — your hardware:
| Your GPU | Best general | Best reasoning | Best small/fast |
|---|---|---|---|
| 8 GB | Llama 3.1 8B / Qwen3 8B | R1 Distill Qwen 7B | Phi-4 Mini, Gemma 3 4B |
| 12–16 GB | Qwen3 14B / Gemma 3 12B | R1 Distill Qwen 14B | Mistral Nemo 12B |
| 24 GB | Qwen3 32B | R1 Distill Qwen 32B | Gemma 3 27B |
| 40 GB+ / 2 cards | Llama 3.3 70B | Qwen3 32B + cloud failover | Qwen3 32B |
Coding is its own category — a dedicated coder model beats a general model of the same size at it. If that's your use case, jump to the best local LLMs for coding in 2026.
From "runs on my GPU" to "usable from anywhere"
Picking the model is step one. The moment you want that model from another machine, across more than one GPU, or behind an API key your app can use, you need a layer the runtime doesn't provide. That's the gateway. Wide Area Intelligence runs your chosen model on a node (llama.cpp under the hood) and exposes it through one OpenAI-compatible endpoint with revocable keys, multi-node load balancing, and automatic cloud failover for the requests your local model can't handle.
So the full play is: use the tools to find a model your card runs well, serve it on a node, and point every app and agent at https://wideareaai.com/api/v1. It's free for up to two nodes. Bring your GPU online with Wide Area Intelligence → Not sure where to start? See what an LLM gateway is and why route local + cloud.