Before you get to sandboxing, egress filtering, or where your agent runs, there is a more basic question, and most architectures answer it by accident: at the moment the agent makes a call, where does the credential live?
If the answer is anywhere the agent can read — its environment, its config file, its context window — you have decided to trust a goal-seeking, stochastic system with a reusable secret. That deserves more scrutiny than it usually gets, because the failure mode is not exotic. Prompt injection, context exfiltration, and plain accidental disclosure sound like three different attacks. They are three delivery mechanisms for the same outcome: the bytes of a credential leave the agent, and they still work.
And leaking is not even the interesting case. An agent can hand over its own credentials without being attacked at all, because it believes doing so solves the problem you gave it.
What March 2026 demonstrated
This stopped being theoretical in March, when the TeamPCP supply-chain campaign compromised LiteLLM — an AI gateway whose entire job is holding API keys for dozens of model providers. The injected credential stealer harvested LLM API keys, SSH keys, cloud credentials, and .env files from every machine running the affected versions. Public estimates put it at 300+ GB of compressed credentials across roughly 500,000 corporate identities.
The lesson is uncomfortable for anyone who operates an AI gateway, ourselves very much included: when AI infrastructure concentrates long-lived credentials into a single process or configuration, compromise of that process yields catastrophic access. “We encrypt at rest” is not an answer, because the compromised process is the thing holding the decryption capability.
The three answers
Kenneth G. Hartman's IETF Internet-Draft Credential Broker for Agents (CB4A), published the same month, sets out the design space. Its core structural idea is to split the component that decides “yes” from the component that hands out the credential, so that compromising one does not yield the other. On top of that split, it defines three ways a broker can mediate access:
| model | what the agent gets | blast radius |
|---|---|---|
| A — proxy gateway | Nothing. It calls a proxy; the real credential is injected on egress | Minimal — there is no secret on the agent's side worth stealing |
| B — token minting | A short-lived, narrowly scoped derivative token | Bounded by the TTL, but a real credential still sits in agent memory |
| C — credential wrapping | The real long-lived credential, revoked afterward | Full, until revocation succeeds — if it succeeds |
The draft recommends Model B as the primary, with Model A as a fallback. We think that is backward for most systems being built today, and so does Christian Posta, who has argued the case at length in the agentgateway project. Model B leans on sender-constrained tokens (DPoP) to make a stolen token useless — but that takes two participants. The issuer has to bind the token to a key, and the receiving service has to verify the proof on every request. GitHub personal access tokens, most SaaS OAuth tokens, and essentially every API key in circulation verify nothing of the kind. Miss either half and you are back to a bearer token, just a shorter-lived one.
Models B and C shrink the window in which a leaked credential is useful. Model A removes the thing that leaks.
What we already do
Wide Area Intelligence is a Model A proxy gateway, and has been since before the draft existed — not because we were following a specification, but because routing inference across hardware you own and clouds you don't makes any other answer untenable. Here is the actual arrangement, layer by layer.
Your application holds a handle, not a secret
wai_sk_… gateway key. We store a SHA-256 hash of it plus a short display prefix — never the key. It cannot be read back out of our database by anyone, ourselves included, and revoking it takes effect on the next request. It is also a handle to us, not to any provider: possessing it does not let you spend against an OpenAI or Google account.The gateway does not hold the provider keys either
We removed the option to give us your keys
Node identity is scoped, rotatable, and expiring
wai_node_… key, again stored only as a hash. Every key carries a scope — the default lets a node act on its own row and nothing else, rather than as an account-wide credential, because a node key sits on every machine in the fleet. The bearer the gateway presents for inference is a separate, independently rotatable secret, deliberately decoupled from the identity credential so that rotating one never bricks the other.Fleet enrollment assumes the credential will leak
wai_enroll_…) exist to be baked into machine images and cloud-init user-data, so we treat them as already-leaked. They can do exactly one thing: exchange themselves for a fresh node key. Expiry is mandatory and cannot be null. Each carries a cap on how many live nodes it may hold at once, so a runaway autoscale — or a stolen token — hits a wall instead of a billing surprise. Every attempt, successful or refused, is written to an audit table with its source IP, so a walked credential shows up as refusals piling up rather than as an inference from node counts.Nodes never receive cloud credentials
Connector tokens are encrypted at rest
What we have not solved
An architecture post that only lists strengths is marketing. Two things here are genuinely unfinished, and both are worth stating plainly.
Our connector envelope is the weak mode. Posta classifies these deployments by who holds the key-encryption key. The strong version keeps it in a KMS and requires a separate broker to authorize each individual unwrap, so the process holding the ciphertext cannot decrypt it using its own authority. Ours does not do that yet: the encryption key is derived inside the same worker that performs the decryption. That is enough to make a database export useless to whoever obtains it, which is what it was built for. It is not enough to contain a compromise of the worker itself. The fix is well understood — a fresh per-row data key on every write, cryptographically bound to the owning user and provider so ciphertext cannot be moved between rows, and the unwrap operation moved behind a separate authority that logs and can refuse. Scoped, and on the list.
The harder question is one the draft does not really cover. CB4A assumes the agent runs on infrastructure the operator controls. Distributed inference breaks that assumption. When the agent's tools execute on a node — a workstation, an on-prem server, and eventually hardware belonging to someone selling spare capacity — then “don't give the agent a credential” becomes a strictly harder problem than the draft's version. You are no longer deciding whether to trust a process. You are deciding whether to ship a bearer credential to a machine you do not own, with no revocation story and no audit of what it did with it.
Our answer, and the design constraint we are holding to as node-side tooling lands: tools that require a credential egress through the gateway with just-in-time injection, and node-local execution is reserved for tools that need no credential at all — local search, retrieval over your own documents, transcription. That costs something real. Routing an authenticated tool call through the gateway means the request leaves your hardware, which is exactly the property some customers chose us for. We would rather name that trade-off than quietly resolve it in the direction that demos better.
The rule underneath all of it
Strip away the specification language and one principle does the work: the authority to use a credential should live as far as possible from the component that can be talked into misusing it. An LLM is, by construction, a system that can be talked into things. It should therefore hold nothing that retains value once it leaves.
Everything else — hashing rather than storing, scoping node keys to their own row, capping and expiring enrollment tokens, keeping provider keys out of our own application, declining to accept yours — is that one rule applied at each boundary in turn.