Meta's Llama 3 family is the most widely-run set of open-weights models in the world, and in 2026 getting it onto your own GPU is genuinely easy. The two things people get wrong are picking the right size for their hardware and assuming "local" means "only on this one machine." This guide fixes both: choose the right model, get it chatting in minutes, then make it reachable everywhere you actually work.
Step 1 — Pick the right Llama 3 size
The single biggest decision is which model to run, because it determines whether it fits in your VRAM at all. The whole family at a glance:
| Model | Params | ~VRAM (Q4_K_M) | Good for |
|---|---|---|---|
| Llama 3.2 1B | 1B | ~1 GB | Edge, phones, quick tasks |
| Llama 3.2 3B | 3B | ~2.5 GB | Laptops, fast assistants |
| Llama 3.1 8B | 8B | ~5–6 GB | The default — 8 GB+ GPUs |
| Llama 3.3 70B | 70B | ~40 GB | Frontier quality, big VRAM |
| Llama 4 Scout | MoE | varies | Long context, MoE efficiency |
For most people the answer is Llama 3.1 8B: it's quick, surprisingly capable, and runs on any 8 GB-or-better card with room for a decent context window. Step up to 3.3 70B only if you have the memory for it.
Not sure what your card can handle? See which GPUs run Llama 3.1 8B or start from your GPU and see every Llama size ranked by fit and speed.
Step 2 — Get it running (the fast way)
The quickest on-ramp is Ollama. Install it, pull the model, and you're chatting in under a minute — it picks a sensible quantization and handles GPU offload automatically.
# Fastest path — Ollama, two commands
ollama pull llama3.1:8b
ollama run llama3.1:8b
# It also serves an OpenAI-compatible API on localhost:
curl http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"llama3.1:8b","messages":[{"role":"user","content":"hi"}]}'Prefer a desktop app? LM Studio or open-source Jangive you a chat window, a model browser, and the same underlying engine — search "Llama 3.1 8B", click download, and chat.
Step 3 — Or take full control with llama.cpp
If you want to choose the exact GGUF and tune the flags — context size, GPU layers, parallel slots — go straight to llama.cpp. Pick a quant that fits your VRAM (Q4_K_M is the usual quality/size sweet spot; Q5/Q6 if you have headroom; Q8 for near-lossless).
# Maximum control — llama.cpp with a GGUF you choose llama-server \ --hf-repo bartowski/Meta-Llama-3.1-8B-Instruct-GGUF \ --hf-file Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \ --n-gpu-layers 999 \ # offload all layers to the GPU --ctx-size 16384 \ # context window --host 0.0.0.0 --port 8080
Read GGUF quantization explained if you want to understand exactly what you're trading when you drop from Q8 to Q4, and How much VRAM do you need? for the memory math.
Step 4 — Make it reachable from your other machines
Here's the part the "run it locally" tutorials skip. Ollama, LM Studio, and llama.cpp all bind to localhost with no authentication. That's perfect for the machine in front of you and useless the moment you want Llama 3 from your laptop on another network, your phone, or a teammate's box. Doing it yourself means port forwarding, a reverse proxy, and bolting on auth.
Wide Area Intelligenceis a gateway that does that layer for you — it doesn't replace the runtime, it sits in front of it. Install a small agent on your GPU machine and you get:
Remote access, no port forwarding
A stable OpenAI-compatible endpoint
https://wideareaai.com/api/v1) and revocable wai_sk_… keys. Point any OpenAI-compatible app or coding agent at it.Cloud failover for the big stuff
The model you just got running under Ollama or llama.cpp runs exactly the same — it's simply reachable now.
Putting it together
Pick Llama 3.1 8B if you're unsure, pull it with Ollama (or a GGUF in llama.cpp if you want the knobs), confirm the fit with the GPU-for-model tool, and if you want that GPU available from more than the one desk it sits on, bring it online with Wide Area Intelligence → Free for up to two nodes.
Next up: How to run Qwen locally, How to run DeepSeek locally, and Best Ollama alternatives.