Alibaba's Qwen3 family is, in 2026, some of the most capable open-weights you can run on your own hardware — and the Qwen3-Coder models in particular have become a default choice for people running local coding agents. This guide covers the part that matters: picking a Qwen size that fits your GPU (including the clever mixture-of-experts option), getting it running, and wiring it into your editor from anywhere.
Step 1 — Pick a Qwen size (and notice the MoE trick)
Qwen3 ships in an unusually wide range, from sub-1B up to a 235B mixture-of-experts flagship. The ones that matter for local use:
| Model | Params | ~VRAM (Q4_K_M) | Good for |
|---|---|---|---|
| Qwen3 4B | 4B | ~3 GB | Laptops, fast tasks |
| Qwen3 8B | 8B | ~5–6 GB | The general-purpose default |
| Qwen3 14B | 14B | ~10 GB | Better reasoning, 12 GB+ GPUs |
| Qwen3 32B | 32B | ~20 GB | Frontier-ish, 24 GB card |
| Qwen3-30B-A3B (MoE) | 30B / ~3B active | ~18 GB | Big quality, small-model speed |
| Qwen3-Coder-30B | 30B | ~18 GB | Best local coding agent model |
The standout is Qwen3-30B-A3B: 30B total parameters but only ~3B activate per token. Since local speed is bound by memory read per token, it generates roughly as fast as a small model while answering like a much larger one — provided it fits in memory. VRAM math here.
Check your card against a specific size: which GPUs run Qwen3 8B · which GPUs run Qwen3-Coder 30B.
Step 2 — Get it running
The fast path is Ollama. For a desktop GUI, LM Studio or open-source Jan let you search and download any Qwen size and chat immediately.
# Fastest path — Ollama ollama pull qwen3:8b ollama run qwen3:8b # Coding variant for agents: ollama pull qwen3-coder:30b
For full control of the GGUF and flags, use llama.cpp directly. Qwen handles long context well, so bump --ctx-size if your VRAM allows.
# llama.cpp — pick the exact GGUF + quant llama-server \ --hf-repo Qwen/Qwen3-8B-GGUF \ --hf-file Qwen3-8B-Q4_K_M.gguf \ --n-gpu-layers 999 \ --ctx-size 32768 \ --host 0.0.0.0 --port 8080
Step 3 — Wire it into your coding agent
Qwen3-Coder really shines inside an agent. Any of them — Cline, Aider, Continue, Qwen Code — just needs an OpenAI-compatible base URL and a model name. We have step-by-step configs: Qwen Code, Cline, Aider, and Continue.dev.
Step 4 — Make it reachable from anywhere you code
The friction with local Qwen is the same as any local model: the runtime binds to localhostwith no auth, so it's only usable on the machine running it. If your GPU lives in the homelab but you code on a laptop — or you want the same model on your work machine and your teammate's — you need reach.
Wide Area Intelligence is a gateway that adds exactly that, without replacing your runtime:
An endpoint your editor can reach from anywhere
https://wideareaai.com/api/v1) and a revocable wai_sk_… key.One config, many machines
Failover so the agent never hard-stops
Putting it together
Run Qwen3 8B for general use or Qwen3-Coder 30B for agents, pull it with Ollama or a GGUF in llama.cpp, confirm the fit with the GPU-for-model tool, and if you want that GPU reachable from every machine you work on, put it on the network with Wide Area Intelligence → Free for up to two nodes.
Related: How to run Llama 3 locally and How to run DeepSeek locally.