Your own provider keys
Add your own key for a provider, choose what happens when the provider rejects it, and see how those requests are billed.
You can bring your own key for a provider. Requests to that provider then go through your account with it: you pay the provider directly, and ai.ml charges only a platform fee.
Open Own keys in the sidebar. The screen is titled Your own provider keys and shows one card per provider that accepts a customer key. That is about thirty providers: those ai.ml also serves from its own pool, others reached only with your own key (Groq, Together, Mistral and more), a LiteLLM proxy you run, and any server that speaks the OpenAI API.
Adding, replacing, validating and deleting keys needs the owner or admin role. Other roles can read the screen.
Add a key
- On the provider's card, press Add key.
- If the card asks for an address, a region or an account, fill it in:
- Base URL for Azure OpenAI, a LiteLLM proxy or a server of your own. It must be https on a public address; an address inside a private network is refused.
- AWS region for Amazon Bedrock.
- The Google Cloud project and region for Vertex AI. The key is the service account's JSON.
- The Cloudflare account id for Workers AI.
- Paste the key into the API key field.
- Optionally enter a Label, for example the account the key belongs to.
- Optionally tick Fall back to our pool if the provider rejects this key. See below. The box is offered only for providers ai.ml also serves from its pool.
- Press Validate and store.
The key is checked against the provider's model list before it is stored. Nothing else is sent during the check. If the provider rejects the key, the form shows the provider's own message and nothing is stored. A provider that has no model list to ask (Z.ai) stores the key without a check; the first request tells you whether it works.
Once stored, the key is encrypted and is only ever decrypted to call that provider. Every decryption is written to the audit log. The key itself is never shown again. The card shows a short hint, the last four characters, so you can tell which key is on file.
What a card shows
- The provider's name, the key's label and its hint.
- A status: not set, key required, active or failing.
- Added, Last used and Last validated.
- Platform fee: your fee for that provider as a percentage of list price.
- API docs: the provider's own page about its keys.
- For a key with an address, a region or an account, those too.
A provider with no key reads "no key on file" and its requests use the ai.ml pool at list price. key required marks a provider that can only be reached with your own key.
Use any model the provider serves
With a key on file you can use any model the provider serves, not only the ones ai.ml lists. Name it as the provider's id, a slash, and the provider's own model name, exactly as the provider spells it; the provider's id is the one shown on the card. The screen gives an example.
Such a request is served only through your key. Your key's model lists, budgets, rate limits and data policy apply as to any request. A provider ai.ml has not reviewed is treated as one that may keep and train on what you send, so a data policy that forbids training or keeps requests in one region refuses it.
How the platform fee is priced
The fee is a percentage of what the request would cost at list price. For a model ai.ml lists, that is its list price. For a model it does not list, ai.ml uses a fallback price it sets for the provider, or one it sets for every provider. If there is neither, the request is refused and says so; contact support and we will price the model.
The percentage is 5 % unless ai.ml has agreed another with your organisation, for one provider or for all. The card shows the one that applies.
How requests are routed and billed
While you have a key for a provider, your requests to that provider use your key and no other.
- Through your key. You pay the provider for the usage. ai.ml charges the platform fee: a percentage of what the request would have cost at list price. The introduction on the screen states your percentage with an example, and each card the percentage for that provider. The default is 5 %.
- Through the ai.ml pool. The request is billed at list price, like any request without your own key.
The Spend split, last 30 days table at the bottom shows the number of requests and the amount charged on each route: Your own keys (platform fee) and Our pool (list price). It appears once you have usage to show.
Fall back to the pool
The checkbox Fall back to our pool when the provider rejects this key decides what happens when the provider refuses your key.
- On. The request is served through the ai.ml pool and billed at list price.
- Off. The request fails. With fallback off, a request for that key is also not moved to any other provider.
You can change it at any time from the card. The change is saved as soon as you tick or clear the box.
When a key is failing
When the provider rejects your key, the card turns to failing and says since when, with the provider's error. ai.ml also sends a byok.failing webhook; see webhooks.
What happens to requests depends on the fallback setting: they fall back to the pool at list price, or they fail until you replace the key.
To fix it, replace the key, or correct the problem at the provider and press Validate.
Validate a key
Validate checks the stored key against the provider again. The card then says that the provider accepted the key and how many models it can see, or shows the provider's message.
Replace a key
Press Replace key and store the new key the same way. The key you have now keeps serving until the new one is stored, then it is retired. A provider holds one active key at a time.
Retired keys are listed under Replaced keys with their hint and the time they were retired.
Delete a key
- Press Delete on the card.
- Type the provider's id, as shown in the confirmation, and press Delete key.
Requests to that provider go back to the ai.ml pool at list price.
Related
- BYOK API reference
- Reselling ai.ml: a reseller's markup does not apply to requests made through a customer's own provider key.