self-hosted + cloud llms, one endpoint
An LLM gateway for the models you host yourself.
Wide Area Intelligence is an OpenAI-compatible LLM gateway — one endpoint in front of every model you run, self-hosted and cloud alike. It serves repeats from an edge cache, routes the rest to self-hosted LLMs on GPUs you own with no per-token fees, and bursts to the cloud only on failover. Point your OpenAI SDK at it and your apps route themselves.
/// definition
What is an LLM gateway?
An LLM gateway is a single OpenAI-compatible API endpoint that sits in front of every large language model an application uses — self-hosted open-weight models and cloud models alike. Instead of wiring each app to a specific model, every call goes through the gateway, which owns model routing, caching, failover, and keys, budgets, and rate limits in one place.
The routing component on its own is often called an LLM router; the gateway is the full endpoint wrapped around it. Wide Area Intelligence is a self-hosted-first LLM gateway: an edge cache answers repeats, your own GPUs answer the rest with no per-token fees, and the cloud is burst-only failover. Your apps see one stable endpoint; the gateway decides where each request runs.
/// the trade-offs
LLM gateway vs. the alternatives
A cloud LLM API bills every token and goes down when the provider does. A self-hosted LLM is free but only reachable on your LAN. A cloud router like OpenRouter spreads risk across providers but still meters every token. A self-hosted-first LLM gateway gives you the reach and failover without renting the baseline.
| Approach | Routing | Baseline cost | Failover | Your data |
|---|---|---|---|---|
| Cloud LLM API only | One hosted model, one key | Every token billed | None — provider outage = your outage | Leaves your network |
| Self-hosted LLM, direct | Your box only, on your LAN | No token fees | None — box down = no answer | Stays local (LAN-only reach) |
| OpenRouter / cloud router | Many cloud LLMs, one key | Every token billed (+ markup) | Across cloud providers only | Leaves your network |
| Wide Area Intelligence | Cache → your LLMs → cloud | No per-token fees on your hardware | Cloud burst on overflow, auto | Yours — leaves only on burst |
/// how the routing works
One LLM gateway, three tiers. Cache the repeats, serve from your GPUs, burst the rest.
EDGE CACHE
An identical request you've served before is returned straight from Cloudflare's edge — the prompt never reaches a GPU. Per-account toggle, your own TTL.
YOUR HARDWARE
A one-line install turns any machine with a GPU into a node. It opens a secure Cloudflare Tunnel — no port forwarding, no static IP — and serves llama.cpp behind your gateway with no per-token fees.
CLOUD FAILOVER
Node busy, offline, or timed out? The same request silently re-routes to the cloud through our managed gateway, billed only against prepaid credits — burst only, never the baseline.
/// the router inside the gateway
The LLM router, built in
An LLM router decides which backend serves each request. In Wide Area Intelligence the routing order is fixed and capability-aware: an exact-match edge cache first, then any of your online nodes that can serve the requested model, then cloud failoveron overflow, timeout, or a node going offline. You don't configure a routing DSL — you register nodes and the gateway routes to your hardware first, automatically.
/// drop-in integration
Point your OpenAI SDK
at your self-hosted LLM.
Anything that already speaks the OpenAI API speaks Wide Area Intelligence — the OpenAI SDK, LangChain, agents, coding tools, curl. Swap the base URL for the gateway and your existing code runs on the LLM under your desk, falling back to the cloud only when it has to.
- Change one line: the base URL.
- Works with the OpenAI SDK, LangChain, agents, curl — anything OpenAI-compatible.
- Bring your own gateway key; routing, caching, and failover are automatic.
from openai import OpenAI client = OpenAI( base_url="https://wideareaai.com/api/v1", api_key="wai_sk_…", ) resp = client.chat.completions.create( model="llama-3.1-8b-instruct", messages=[…], )
/// why this llm gateway
Self-hosted, reachable
A one-line install turns any GPU box into a node and opens a secure Cloudflare Tunnel — no port forwarding, no static IP. Your self-hosted LLM gets a stable OpenAI-compatible URL reachable from anywhere.
No per-token fees on your LLMs
Requests served by your own GPUs cost no metered tokens — only electricity. Repeats are free from the edge cache. The cloud bills only on the overflow it actually catches.
Never a hard down
When a node is asleep, updating, or maxed out, the same request fails over to the cloud instead of erroring. Your LLM endpoint stays up even when your GPU doesn't.
/// go deeper
Plan your self-hosted LLM gateway
New to the concept? Start with what is an LLM gateway, compare options in the best OpenRouter alternatives, or browse the model directory.
/// faq
LLM gateway questions
- What is an LLM gateway?
- An LLM gateway is a single OpenAI-compatible API endpoint that sits in front of every large language model you run — self-hosted open-weight models and cloud models alike. Your applications call the gateway instead of each model directly, and the gateway handles model routing, response caching, failover, key management, and rate limiting. Wide Area Intelligence is an LLM gateway that routes to self-hosted LLMs on GPUs you own first and bursts to the cloud only on overflow.
- What is the difference between an LLM gateway and an LLM router?
- An LLM router is the routing logic — the part that picks which model or backend a given request should go to. An LLM gateway is the full endpoint around that router: it adds an OpenAI-compatible API surface, edge caching, key and budget enforcement, observability, and cloud failover. Wide Area Intelligence is a gateway whose router prefers your own hardware, then falls back to the cloud.
- Can an LLM gateway run self-hosted LLMs?
- Yes — that is the point of Wide Area Intelligence. A one-line install turns any machine with a GPU into a node serving llama.cpp behind the gateway over a secure Cloudflare Tunnel — no port forwarding, no static IP. Your self-hosted LLM gets a stable, reachable OpenAI-compatible URL, and requests it serves carry no per-token fees.
- Does an LLM gateway work with the OpenAI SDK and tools like Cursor or Claude Code?
- Yes. Wide Area Intelligence speaks the OpenAI API, so any OpenAI-compatible client — the OpenAI SDK, LangChain, agent frameworks, coding tools, or curl — works by changing one line: the base URL. Nothing else in your code changes.
- Is a self-hosted LLM gateway cheaper than a cloud API?
- For steady-state traffic, yes. Requests served by your own GPUs through Wide Area Intelligence carry no per-token fees — your only marginal cost is electricity — and repeated requests are answered from an edge cache for free. You pay cloud rates only on the burst overflow, never on the baseline.
/// initialize
Give your self-hosted LLM a real endpoint. Burst to the cloud only when you must.
no credit card · 2 nodes free, forever · openai-compatible