Any application that uses language models in production ends up in the same place. It starts with one provider and one SDK. Then a cheaper model appears for the bulk summarisation job, so a second provider gets added. Then a reasoning model is needed for one specific path, and that's a third. Now there are three API keys in the secret manager, three billing accounts with three separate prepaid balances, three sets of rate limits to reason about, and three slightly different request shapes in the codebase.

None of that complexity is doing any work for you. It is not making the product better and it is not making the models cheaper. It is integration overhead, and it grows every single time a new model is worth trying.

That is the problem a gateway solves. One endpoint in front of many models, one credential, one bill, one place to see what you spent. The CubePath AI Gateway is our take on it, sitting in front of five providers and using the same API token and the same balance as the rest of your infrastructure.

This post is about what that layer actually buys you, on the engineering side and on the billing side.

The Integration Tax Nobody Budgets For

The cost of adding a second model provider is never the cost of the model. It is everything around it.

Someone has to create a company account and get it approved. Finance has to add another card and another recurring charge nobody can attribute later. A new key goes into the secret manager and into every environment that needs it. Someone writes an adapter, because the request shape is not quite the same. Someone else discovers three weeks later that streaming works differently there. The retry logic gets a second branch. The observability gets a second source of truth. And every one of those pieces has to be maintained by somebody, forever.

Multiply that by three providers and the integration layer starts to look like a small internal product that generates no value on its own.

A gateway collapses it. The application talks to one endpoint in one format. Everything behind that line, the accounts, the credentials, the per-provider differences, becomes somebody else's problem.

Switching Models Stops Being a Project

The most concrete benefit is that the model becomes a string in a config file.

CubePath's gateway speaks the OpenAI chat completions format, which means any client library already pointed at OpenAI works by changing the base URL:

from openai import OpenAI

client = OpenAI(
    base_url="https://ai-gateway.cubepath.com",
    api_key="<your CubePath API token>",
)

response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4-5",
    messages=[{"role": "user", "content": "Summarise this changelog"}],
)

Models are addressed as provider/model_id, so openai/gpt-4.1-mini, anthropic/claude-sonnet-4-5, google/gemini-2.5-pro, xai/grok-4-fast and deepseek/deepseek-chat are all reachable through that one client. Right now that is 27 models across OpenAI, Anthropic, Google, xAI and DeepSeek, and the catalogue is served live: when a model is added it is callable straight away, with no SDK update and nothing to redeploy on your side.

Where the providers genuinely differ, the translation happens inside the gateway. Anthropic and Google do not use the OpenAI format, so requests and responses are converted in both directions, streaming included. Tool calls, multimodal content, temperature, stop sequences and response format all go through the same fields no matter which model ends up serving the request.

The practical effect is on how teams behave, not on how the code looks. When trying a different model means editing one string, someone actually tries it. When it means a vendor account, a procurement conversation and an adapter, the experiment quietly never happens and you keep paying for the model you picked eighteen months ago.

One Bill Instead of Five

The billing side is where a gateway pays for itself in ways that are easy to overlook.

One balance, not five prepaid wallets. Inference is charged against the same organization balance that pays for your servers. There is no separate AI wallet to keep topped up, and no risk of a batch job failing at 3 AM because one provider's balance ran dry while the other four were fine.

No per-vendor minimums or commitments. Trying a fifth provider for one job does not mean opening a fifth account with its own minimum spend and its own payment terms. The cost of evaluating a model drops to the cost of the tokens.

Failed requests cost nothing. If a provider errors, the request is recorded so you can see it happened, and the charge is zero. You are not paying for someone else's outage.

No overdraft, and no surprise invoice. A request that your balance cannot cover is refused before it reaches the provider, with a status code your client can act on. Prepaid means the worst case is a rejected request, not a five-figure bill three weeks after a runaway loop.

No markup. What you pay per million tokens is what the model costs at its provider. The gateway is not a margin play, and the current price of every model is published in the catalogue so you can check that before sending a request.

Costs are tracked to six decimals, so a request that costs $0.000042 shows up as exactly that rather than being rounded into invisibility.

Cost Visibility Is What Teams Underestimate

Ask most teams what they spend on AI and they can answer. Ask them which model, which feature, or which customer is responsible for it and the room goes quiet. With five separate provider dashboards, that question is a reconciliation exercise nobody has time for.

Because everything goes through one place, the answer is just there: spend by model, spend by day, and totals over any period, in the same dashboard as your servers. Every call is also recorded individually with its model, token counts, cost, latency, status and the token and user behind it, and it lands in your organization activity log next to your VPS creations and DNS changes.

That data usually changes a decision within the first week. The pattern is almost always the same: a small number of high-volume calls on an expensive model, doing a job a cheaper one handles fine. You cannot act on that until you can see it, and you cannot see it when the spend is split across five accounts.

One Credential Instead of Five

Every provider key you hold is a key you have to store, distribute, rotate and eventually explain to an auditor. Five providers means five of everything, usually with five different rotation procedures and no consistent way to revoke access when someone leaves.

Through the gateway your application holds one CubePath API token, the same kind you already use for the REST API, the CLI or the MCP server, and it carries the same controls the rest of the platform has: scopes that separate reading the catalogue from spending money, an IP allowlist that refuses a leaked token used from an unexpected address, and one place to revoke it.

The provider keys stay on our side. You never sign up with OpenAI, Anthropic, Google, xAI or DeepSeek, never hold their credentials, and never rotate them. Errors coming back from a provider are scrubbed of anything resembling a credential before you see them, because providers do occasionally echo a key fragment back inside an error message.

There is a governance benefit here that tends to matter more in larger organizations than the engineering one. Access to every model runs through a single credential you control, with a per-request record of who used what. That is a far easier conversation than five shadow accounts opened by five teams.

Failover Between Providers

This is the benefit that only exists once something sits in front of the providers, and it is the one that turns a model integration from a dependency into a component.

Model providers go down. They also degrade, which is worse, because a provider that is slow rather than dead does not trip your error handling and just quietly makes every request in your application take eight seconds. If your code talks to one provider directly, their bad afternoon is your bad afternoon, and there is nothing to do but wait it out and answer questions about it.

When a provider has an outage or its latency degrades, requests are routed automatically to an equivalent model from another provider. Your application keeps answering. Nothing changes on your side, because the endpoint, the credential and the response format are the same regardless of which upstream ends up serving the request.

That is the sort of resilience that is genuinely hard to build yourself. Not the retry loop, which is twenty lines. The hard parts are having accounts and credit with several providers already in place, knowing which model on provider B is a reasonable substitute for the one you asked for on provider A, and detecting degradation quickly enough for the switch to be worth making. Those are exactly the pieces that only make sense to build once, centrally, rather than in every application that calls a model.

The same property protects you commercially. A provider that changes terms, deprecates a model or raises prices is a config change on your side, not a migration. That is the practical definition of not being locked in.

Where It Fits

The case for a gateway is not that it does something you could not do yourself. It is that the thing you would build instead has no upside. Nobody has ever won a customer because their provider abstraction layer was well factored.

What you get back is optionality. Five providers reachable through one endpoint, one credential with the scopes and the IP allowlist you already use elsewhere, one balance drawn down per request, and one place that answers where the money went. Trying a new model becomes a decision you make on a Tuesday afternoon instead of a quarter you plan.

That last part is the one that compounds. This field changes fast enough that the model you should be running in six months probably has not been released yet. Being able to move to it without a migration is worth more than any individual model choice you make today.