Features
Built for volume, failover, and a readable bill.
The same OpenAI-compatible chat endpoint serves a fixed model or an auto profile. Routing, holds, and settlement stay on the gateway.
Auto router
Set model to auto, auto:cheap, auto:balanced, or auto:best. auto uses the balanced profile. The gateway classifies the task and ranks models that support the features the request needs, such as vision, tools, or a long enough context. The hold covers the short ranked chain. Settlement bills the model that answered.
- auto and auto:balanced aim at a capable mid-price model.
- auto:cheap prefers a lower list price when the model can still do the task.
- auto:best prefers the higher quality score for that task.
- The response names the served model in model and x-routed-model.
Lag detection and automatic switching
Rolling latency and time to first token are recorded per model. After enough healthy samples, a model whose p95 is far above its own baseline is moved to the end of the ranking. For auto requests that are not streamed, a provider that sends no first byte within the hedge window starts the next model. The first finished body is returned, the other call is cancelled, and only the delivered model is billed.
- A fixed model name still retries retryable errors, then walks that model’s fallback list.
- A stream that has already produced tokens stays on that model.
- If every model fails, the hold is released.
Token-saving controls
These controls are off until an organisation turns them on. Amounts on the savings page are charges the gateway did not take. A comparison against a baseline model stays labelled as an estimate.
- Prompt-cache tokens reported by the provider are billed below the full input price.
- An exact or semantic response cache can replay a stored completion without charging it again.
- Context trimming and prompt compression shrink the prompt before the call.
- A batch priority can take the organisation’s batch discount. Lowering max tokens is shown as tokens not reserved, and is not counted as money saved.
Enterprise agent workflows
Members can define a workflow of up to 30 steps: an agent call, a condition, a parallel branch, or an approval. Approval pauses the run until a person approves or rejects it. Templates cover customer-support triage, document summarisation, and data extraction. Each model step uses the same router and billing as chat.
- Roles are owner, admin, member, and viewer.
- Team budgets are an extra ceiling. The member’s wallet still pays.
- An HMAC sign-in hook can provision a user into the organisation. It is not an OIDC login.
Savings and performance dashboard
The signed-in portal shows balance, held funds, and available funds, plus usage by day and by key. The savings page shows measured amounts for that account and can export a monthly CSV. A PDF report is available to organisation owners. Platform-wide savings and uptime stay blank until there is data.
- Routing savings compare the served list price with an estimated baseline.
- The public pricing page uses the same measured averages for its ROI calculator.