Five live applications now send their AI traffic through Wide Area Intelligence to GPU hardware their operators own, instead of straight to a commercial AI provider. They aren't demos. They serve real users doing real work: transcription, real-time voice, company research, and more.
The case for running AI on your own hardware is usually made on privacy or on price. The harder question — the one that decides whether anyone actually does it — is whether that hardware is fast enough to be worth using. Here is what we measured, and what running this in production has and hasn't solved.
The five applications
OmniCanvas uses it for audio transcription. ContactCenterHQ uses it for real-time voice conversation. SuperpowerResume uses it for company research. And PlanetRoadmap and Calburndown route their AI features through it as well. In the interest of full disclosure: all five are built by InventiveHQ, the same company that builds Wide Area Intelligence. We run our own products on it first.
Each one integrates the same way: point an OpenAI-compatible SDK at the gateway endpoint. No application code knows or cares which machine answered.
How a request is routed
Every request walks the same three tiers, in order:
| tier | what happens | cost |
|---|---|---|
| 1 — edge cache | Identical requests are served from Cloudflare KV. A cache hit returns in roughly 300 milliseconds. | Effectively zero |
| 2 — your hardware | The request goes to a GPU node you own — an office workstation, a lab machine, a rack server on-prem. | Electricity |
| 3 — cloud failover | Only when no node of yours can serve the request, a commercial provider handles it. | Billed by credit |
The routing layer runs on Cloudflare's edge network; inference runs on your hardware. Nodes connect outbound to the gateway over a single WebSocket relay, so there are no inbound ports to open, no port forwarding, and no static IP. If you want the full walk-through of what happens between the SDK call and the first token, see anatomy of an inference request.
The hardware number is the argument
We timed one production workload end to end: transcribing a five-minute recording through the gateway.
| hardware | 300 s of audio took | speed vs realtime |
|---|---|---|
| NVIDIA GTX 1080 Ti (2017) | 10.1 seconds | ~30x realtime |
| Same machine, CPU only | 324.4 seconds | ~0.92x realtime |
A consumer graphics card released in 2017 transcribed five minutes of audio in ten seconds. The marginal cost of that transcription was electricity.
The contrast inside the same machine is the more useful number. On its CPU, the same recording took five and a half minutes — slower than realtime, and unusable for any workflow where someone is waiting. The gap between 30x and 0.92x realtime is the difference between a feature and a progress bar, and it came down entirely to whether a GPU was present. Not how new it was.
For text generation, a local node running Qwen3VL-30B-A3B-Instruct streamed its first content token in approximately 3.4 seconds, as standard OpenAI-format server-sent events.
Not sure whether a card you already own can run the model you need? Check which GPU runs any model →
What routing to your own hardware does not solve
A gateway in front of self-hosted hardware inherits the reliability of that hardware. A node can be busy, offline, or still loading a model into memory. Cloud failover covers that — but the applications built on the gateway don't treat failover as sufficient on its own, and they are right not to.
OmniCanvas keeps an independent path to Cloudflare Workers AI, specifically so that an outage at wideareaai.com cannot take its transcription feature down.
ContactCenterHQstarts a parallel request to Workers AI if the gateway hasn't produced a first token within four seconds, and uses whichever answers first. In a live voice conversation, four seconds of silence is already a failure.
That is the intended pattern, not a workaround. An application that must always answer should keep a route it controls. A gateway that tells you otherwise is overselling — including this one.
The takeaway
Everybody's AI bill is a rental agreement on hardware somebody else owns. The interesting finding isn't that self-hosting is cheaper — everyone assumes that. It's that on the workloads most applications actually run, it isn't slower. Plenty of organizations already have a capable GPU sitting in a workstation or a server room; the missing piece has been a way to put it behind a production endpoint with a safety net.
To see how requests move between tiers, read the routing and failover docs, or see how the same failover logic picks a cloud model that can actually handle the request in capability-aware failover. For batch work that can wait for idle hardware, there's the GPU night shift.