--- title: "AI Gateway" description: "One endpoint, one token and one balance for models from OpenAI, Anthropic, Google, xAI and DeepSeek: OpenAI and Anthropic SDKs, Vercel AI SDK, Claude Code and your own provider keys." category: "Deployment" language: en source: https://cubepath.com/docs/products/deployment/ai-gateway updated: 2026-11-17 --- # AI Gateway The CubePath AI Gateway gives you one endpoint, one token and one balance for models from **OpenAI, Anthropic, Google, xAI and DeepSeek**. It speaks the OpenAI API and the native Anthropic API, so the SDKs and tools you already use work by changing a URL. Open the **AI Gateway** page in [my.cubepath.com](https://my.cubepath.com). Base URL: `https://ai-gateway.cubepath.com` ## Get started ```steps ### Create an API token On the AI Gateway page, click **Create API Token**. Use an API token: OAuth tokens are not accepted. A token can do what the role of the member who created it allows: owners, admins and members can use the gateway, viewers can only list the models, and the billing role cannot use it. ### Add funds Click **Add Funds** to top up your balance. Usage is pay as you go, billed per token after each request. ### Point your SDK or tool at the gateway Set the base URL to `https://ai-gateway.cubepath.com` and use your CubePath API token as the key. ``` Models are named `provider/model`, for example `openai/gpt-4o`, `google/gemini-2.5-pro` or `anthropic/claude-sonnet-5`. List them with their prices: ```bash curl https://ai-gateway.cubepath.com/models -H "Authorization: Bearer $CUBEPATH_TOKEN" ``` ## Models Click **View all models** on the AI Gateway page to open the model catalog. It shows what is available and what it costs: - **Price per 1M tokens**, input and output separately. Output is usually several times more expensive than input. `/models` also lists the cache read and cache write prices where the provider has them. - **Context window**: how much you can send in one request. - **Max output**: the ceiling on a single response. - **Capabilities**: streaming, vision, tool use. ![AI model catalog with token prices, capabilities and context size](https://cdn.cubepath.com/img/docs/products/946d16df-ce05-40b1-99a6-d879b80d7529.png) > Tip: Switch providers by changing only the model name. Your token, endpoint and billing stay the same, so you can try a cheaper model against your current one on real traffic. ## OpenAI SDK The base URL works with or without `/v1`. ```python from openai import OpenAI client = OpenAI(base_url="https://ai-gateway.cubepath.com", api_key="") reply = client.chat.completions.create( model="openai/gpt-4o", messages=[{"role": "user", "content": "Hello!"}], ) print(reply.choices[0].message.content) ``` The gateway serves Chat Completions. The OpenAI Responses API (`/v1/responses`) is not available. ## Vercel AI SDK For any provider, use the OpenAI-compatible adapter: ```ts import { createOpenAICompatible } from "@ai-sdk/openai-compatible"; import { generateText } from "ai"; const cubepath = createOpenAICompatible({ name: "cubepath", baseURL: "https://ai-gateway.cubepath.com/v1", apiKey: process.env.CUBEPATH_TOKEN, includeUsage: true, }); const { text } = await generateText({ model: cubepath("google/gemini-2.5-pro"), prompt: "Hello!" }); ``` For Claude, the native adapter gives you thinking, prompt caching and structured output exactly as Anthropic defines them: ```ts import { createAnthropic } from "@ai-sdk/anthropic"; const anthropic = createAnthropic({ baseURL: "https://ai-gateway.cubepath.com/v1", apiKey: process.env.CUBEPATH_TOKEN, }); // anthropic("claude-sonnet-5") ``` If you use `@ai-sdk/openai`, call `.chat("openai/gpt-4.1-mini")`: its default model uses the Responses API, which the gateway does not serve. ## Anthropic SDK The SDK adds `/v1/messages` itself, so the base URL has no `/v1`. ```python import anthropic client = anthropic.Anthropic(base_url="https://ai-gateway.cubepath.com", api_key="") msg = client.messages.create( model="claude-sonnet-5", max_tokens=1024, messages=[{"role": "user", "content": "Hello!"}], ) ``` Both `claude-sonnet-5` and `anthropic/claude-sonnet-5` work, and so do dated ids and `-latest`. `POST /v1/messages/count_tokens` is available and free. Only Anthropic models are served on `/v1/messages`. ## Claude Code ```bash export ANTHROPIC_BASE_URL=https://ai-gateway.cubepath.com export ANTHROPIC_AUTH_TOKEN= export ANTHROPIC_API_KEY= claude ``` `ANTHROPIC_API_KEY` must be empty so Claude Code does not send its own key. Some Anthropic features that carry a separate fee are not available yet and return a 400 that names the feature: server tools (web search, web fetch, code execution, the advisor, the MCP connector), fast mode, and the 1M context beta on models older than Claude 4.6. Your own tools (including bash, text editor, computer and memory) work. Claude Code asks for large outputs, and the gateway reserves their maximum cost on your balance (see Balance reservations, below). If your balance is small, lower the output limit with `export CLAUDE_CODE_MAX_OUTPUT_TOKENS=8192`. ## Bring your own provider key (BYOK) Send your own provider key in the `X-Provider-Api-Key` header and the request goes to the provider with your key. The gateway does not charge you for those tokens (they are still recorded in your usage), and you pay the provider directly. Your CubePath token still goes in the usual place. - Works on `/chat/completions`, `/v1/messages` and `/v1/messages/count_tokens`. - Your organization's cash balance must be $0 or more, not negative (promotional credit does not count). - If the provider rejects your key, you get a 401 or 403 with the header `X-Provider-Key-Rejected: 1`. On `/chat/completions` the message starts with "Your provider key was rejected"; on `/v1/messages` the body is the provider's own error. On a stream that has already started, the error arrives inside the stream, without the header. - `count_tokens` with your own key is limited to 30 requests per minute. - Your key is never logged and is removed from error messages. ```python client = OpenAI(base_url="https://ai-gateway.cubepath.com", api_key="", default_headers={"X-Provider-Api-Key": ""}) ``` Claude Code: `export ANTHROPIC_CUSTOM_HEADERS="X-Provider-Api-Key: "`. Vercel AI SDK: the `headers` option. ## How you are billed You pay per token at the prices from `/models`, deducted from your balance after each request. Promotional credit cannot be spent on the AI Gateway. **Prompt caching.** Cached input has its own prices: tokens read from the cache are cheaper than normal input, and some providers charge a premium for writing to it (Anthropic has two write prices, for a cache that lasts 5 minutes or 1 hour). The response usage shows cached tokens apart. `cache_control` on Claude is passed through. **Reasoning.** Thinking and reasoning tokens are billed as output tokens. Tune them with `reasoning_effort` (`none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`); the gateway maps it to what each model supports. xAI reports its reasoning tokens apart from `completion_tokens`, and both are billed as output. Reasoning models can use many output tokens, so set `max_tokens` with that in mind. **Balance reservations.** Before a request runs, the gateway reserves the most it could cost (your input plus the maximum output) on your balance, and charges the real cost when it ends. Many requests can run at once; the reservation of each one counts against your balance. - `402`: your cash balance is not enough for the request. Top up, or lower `max_tokens`. - `429` with `Retry-After`: you have enough balance, but other requests in progress hold it right now. Retry shortly, or lower `max_tokens`. - `503` with `Retry-After`: the provider failed or did not answer, billing is briefly unavailable, the gateway is restarting, or it is busy with very large requests. Retry. Example: on Claude Opus ($5.25 per million input tokens, $26.25 per million output), a request with 120,000 tokens of context and `max_tokens` 32000 reserves $0.63 + $0.84 = $1.47, even if the real cost ends up in cents. With prompt caching the input is reserved at the cache write price ($6.5625 per million, or $10.50 with a 1 hour cache), so the same request reserves about $1.63; Claude Code uses caching by default, so three of its requests at once need about $4.88 of balance. With `max_tokens` 8192 the reservation is about $1.00. Lowering `max_tokens` lowers the reservation. Reservations are released when the request ends, and they expire on their own if something goes wrong. ## Streaming for long answers Use `stream: true` for long answers, reasoning models and agents: you see the answer as it is written, and while the model thinks, the stream sends keepalive comments that the SDKs ignore, so the connection is not cut for being idle. A call without streaming that takes more than 80 seconds gets its `200` status when those 80 seconds pass, and then a space every 15 seconds until the answer is ready. JSON parsers ignore those leading spaces, so the SDKs read the answer as usual. If the provider fails after that point, the error comes in the body with status `200`: `{"detail": "...", "error": {...}}` on `/chat/completions`, the Anthropic error shape on `/v1/messages`. A call without streaming is cut after 15 minutes, and then you pay the input. What you pay if you disconnect: - Before the provider has your request: nothing. - From a call without streaming, once the provider has it: the whole answer, as if you had waited. The provider keeps working and charges for all of it. - From a stream: what you received, or the input if no text had arrived yet. A provider stream that sends nothing for 300 seconds is cut. ## Tool loops with Claude on /chat/completions Claude models that think return their thinking as `reasoning_details` on `/chat/completions`. In a tool loop, send the assistant message back as you received it. For full control of thinking, caching and tools with Claude, use `/v1/messages` (the Anthropic SDK or the Vercel Anthropic adapter). ## Limits - Rate limit: 60 requests per minute per organization by default. Token counting has its own, higher limit. - Request size: 10 MiB on `/chat/completions`, 32 MiB on `/v1/messages`. - `response_format` (JSON mode and JSON schema) works on OpenAI and xAI models on `/chat/completions`. DeepSeek takes only JSON mode (`json_object`), and the prompt must mention JSON. Claude and Gemini ignore it there; for structured output with Claude use `/v1/messages`. - DeepSeek V4 thinks by default and refuses a forced `tool_choice` while thinking; send `reasoning_effort: "none"` with it. - Some providers charge a higher price above a certain prompt size (long context); the gateway bills those requests at the base price. - Browsers: the gateway allows browser calls only from CubePath sites. Call it from your server. ## Errors `/chat/completions` returns `{"detail": "..."}`. `/v1/messages` returns Anthropic-style errors (`{"type":"error","error":{"type":"...","message":"..."}}`), so the Anthropic SDK handles them. When the provider fails or does not answer, the gateway answers `503` with `Retry-After`, and the SDKs retry it on their own. A `502` or `504` only comes from our network edge when the gateway cannot be reached, as a short plain-text page; treat it like a `503`. ## Watching the cost The dashboard tracks **requests, tokens and cost**, daily and per model, over the last 30 days. Read it per model rather than in total: usually one call in your application, such as a summarizer or a classifier that runs on every request, accounts for most of the bill, and moving just that call to a smaller model changes the number a lot. ## Good practice - **Use the smallest model that passes your evals**, not the biggest one you can afford. The gap in quality is usually smaller than the gap in price. - **Cap the output** with `max_tokens` where a runaway response would be expensive. It also lowers the reservation. - **Reuse long prompts** so the provider can serve them from the cache, and cache results you will need again on your side.