Run a Bolna voice agent on your own model or on an inference provider such as Baseten, Cerebras, Fireworks AI, Groq, Modal, Nebius, Simplismart or Together AI, with your endpoint and your key.
Bring your own model by pointing Bolna at any endpoint that speaks the OpenAI chat completions API. The endpoint can be an inference provider such as Groq or Together AI, or a server you run yourself with vLLM, SGLang, TGI, Ollama or a LiteLLM proxy. Bolna calls it exactly as it calls OpenAI, only at your URL and with your key.Use a custom LLM when:
you want an open-weight or fine-tuned model that Bolna’s built-in providers don’t offer
you want the lowest possible reply latency from fast inference hardware such as Groq or Cerebras
the weights or the traffic have to stay on infrastructure, or in a region, you control
you already pay an inference provider and want the tokens billed there
The Add custom model dialog has a Provider dropdown with presets for the providers below. Each guide covers that provider’s console, its endpoint URL, its model ID format, and the problems most likely to affect a live call.
Inference in a fixed region, for data residency or lower latency to your callers
Nebius dedicated endpoints, Simplismart dedicated deployments, or Modal with routing_region
Network distance adds to every reply. If most of your callers are in India, measure the first reply on a web call before you commit to a provider whose servers are in the US or Europe.
Register the model once, under Settings → Integrations → LLM → Custom LLM or from + Add custom LLM in an agent’s LLM Provider dropdown, then select it on the agent’s Intelligence tab. See Add a custom model in the dashboard for the walkthrough and Add a custom model over the API for the API.
Bolna sends requests with the OpenAI SDK, so the endpoint has to behave like OpenAI’s chat completions API.
Requirement
Detail
POST {base_url}/chat/completions
Give Bolna the base URL that /chat/completions follows, usually ending in the version segment, for example https://your-host/v1. Leave off the trailing slash. Bolna appends the rest of the path.
Streaming responses
OpenAI delta format, terminated by data: [DONE]. Tokens stream straight into speech synthesis, so an endpoint that only replies once the full answer is ready adds audible silence before the agent speaks.
Bearer authentication
Your key is sent as Authorization: Bearer <key>. This is the only auth header Bolna sends, so an endpoint that also needs a custom header, such as an account or deployment ID, can’t be used.
A publicly resolvable host
Private, loopback and link-local addresses are refused, both at registration and on every call.
Streamed tool_calls
Only if your agent uses tools. Support varies between providers, models and self-hosted servers, so test it before relying on it.
GET {base_url}/models (optional)
When the endpoint serves a model list, Bolna uses it to check your key and model name when you save. Without one, mistakes only show up on the first call.
Your model id is passed through unchanged, so it must match whatever your server expects.
One curl with your own URL, key and model shows whether Bolna will be able to call the endpoint. It also shows roughly how long a caller would wait before hearing the first word.
curl -N "$BASE_URL/chat/completions" \ -H "Authorization: Bearer $API_KEY" \ -H "Content-Type: application/json" \ -w "\n\ntime to first byte: %{time_starttransfer}s\n" \ -d '{ "model": "'"$MODEL"'", "stream": true, "max_tokens": 60, "messages": [{"role": "user", "content": "Say hello in one short sentence."}] }'
You should see a series of data: {...} chunks ending in data: [DONE]. A 401 means the key is wrong, a 404 usually means the base URL is missing a path segment such as /v1, /openai/v1 or /inference/v1, and a model error means the model name doesn’t match what the endpoint serves. If the first byte takes more than about a second, callers will notice the pause.
Go to Settings → Integrations in the dashboard and filter by LLM. Find the Custom LLM card and click Add model.You can also add a model without leaving an agent: on the agent’s Intelligence tab, open the LLM Provider dropdown and pick + Add custom LLM at the bottom. The new model is then selected on that agent, and you save the agent as usual.
The Custom LLM card on Settings → Integrations, under the LLM filter
2
Fill in the details
Provider: one of the inference providers, or Other (OpenAI-compatible) for any other server. Picking a provider only changes the placeholder and the hints under the fields. It never fills in the URL for you, because URLs differ by account, deployment and region, and what Bolna stores is the same either way.
Model name: the model id your endpoint serves, for example Qwen/Qwen2.5-3B-Instruct
Endpoint URL: your OpenAI-compatible base URL, usually ending in /v1
API key: the bearer token your endpoint expects
Family: the model family, for example qwen, llama or mistral
The Add custom model dialog, with its Provider list open
3
Save
Bolna checks the endpoint before saving it. Your key is stored encrypted and is never shown again.
Each model you add appears as its own card on the Integrations page, showing its endpoint, with a delete action.
A registered custom model, shown as its own card alongside the built-in providers
Registration is refused when Bolna can prove the configuration is wrong:
The endpoint URL must resolve to a public address. Private, loopback and link-local addresses are rejected.
If the endpoint rejects the API key, you get The endpoint rejected this API key.
If the endpoint publishes a model list that does not include your model name, the error names what it does serve.
A model name must be unique within the scope that owns it, because an agent resolves its custom model by name. A duplicate is refused before the endpoint is contacted.
If the endpoint cannot be reached at all, for example because a serverless deployment has scaled to zero, the model is still saved. Verification is best effort and cannot guarantee that a model will be reachable later.
Open the agent, go to the Intelligence tab, choose custom as the provider, then pick your model. Save the agent. If you added the model from the agent’s + Add custom LLM entry, it is already selected; you only need to save.If your server needs parameters beyond the OpenAI standard, such as turning a model’s thinking off, switch on Extra request params on the same tab and enter them as JSON. They are sent on every request to your endpoint. See Extra request params for examples and the rules.
Calls fail differently on a custom LLM than on a built-in provider: when your endpoint errors, the caller hears silence. Check these before you put an agent on live traffic or start a batch.
Place a web call. Registration can’t catch every mistake, especially on endpoints without a model list. Hold a full conversation, including any tool calls.
Keep capacity warm. A serverless deployment that has scaled to zero either makes the first caller wait for a cold start or rejects the request outright. Keep at least one replica running for live agents.
Check rate limits against your concurrency. Shared, per-token endpoints rate-limit by requests and tokens per minute. A batch that runs many calls at once can hit those limits, and every request that gets a 429 is a turn with no reply. Raise your limits, pace the batch, or move to dedicated capacity.
Turn thinking off. Reasoning models think before every reply, which callers hear as dead air. Use an instruct model, or turn thinking off with Extra request params.
Watch for expiring keys and retired models. Some providers issue keys with an expiry date or retire serverless models on a schedule. Either one breaks every call from that point on. Set a reminder or pick keys that don’t expire.
Many servers accept parameters that are not part of the OpenAI API, such as turning a model’s thinking off. Set them in extra_body and Bolna adds them to every request it sends to your endpoint.
Each key is sent as a top-level field of the request body, exactly as the OpenAI SDK’s extra_body does, so the request above reaches your server as {"model": "Qwen/Qwen3-8B", "messages": [...], "chat_template_kwargs": {"enable_thinking": false}, ...}.Common examples:
It is set per agent, so two agents on the same model can differ. The params go on every request to your endpoint: conversation turns, graph agent routing, and the end-of-call and voicemail checks. Turning thinking off therefore speeds up routing as well as replies.In the dashboard, open the agent’s Intelligence tab with your custom model selected and switch on Extra request params. Type the JSON, or start from one of the example buttons.
Extra request params under Model Parameters, shown when a custom model is selected
Saving the agent is refused when:
extra_body is not a JSON object
the agent’s provider is not custom, since hosted providers never receive extra params
it sets a field Bolna controls: model, messages, stream, stream_options, tools, tool_choice, parallel_tool_calls or response_format
Bolna cannot know which params your server accepts. A misspelled or unsupported key is saved, and your server then either ignores it or rejects every request with a 400, which leaves the agent silent. Test a new extra_body with a web call before putting the agent on live traffic.
Delete a model from its card on the Integrations page. A model that is still selected on an agent cannot be deleted: the error names the agents using it, so you can point them at another model first. This applies to every agent config, including graph agent nodes.Deleting a model also removes the stored API key.
A custom model belongs to the scope it was created in:
A model added on your main account is available to the account and to all of its sub-accounts.
A model added inside a sub-account is available only in that sub-account. It is not visible to the main account or to other sub-accounts, and neither can delete it.
Does Bolna use the OpenAI Responses API for custom models?
No. Custom models always use chat completions, because the Responses API’s server-side conversation chaining is specific to OpenAI and a self-hosted server cannot resolve a previous response id.
How do I turn thinking off on my reasoning model?
Set it in extra_body with the parameter your server uses. On vLLM or SGLang with Qwen3 that is {"chat_template_kwargs": {"enable_thinking": false}}. Thinking adds silence before every reply, so voice agents usually want it off.
Does Bolna send service_tier to my endpoint?
No. service_tier is an OpenAI-only field that strict OpenAI-compatible servers reject, so it is never sent to a custom endpoint.
What happens if my endpoint has scaled to zero?
The first call pays the cold start, which the caller hears as silence before the agent speaks. Keep a warm instance if first-token latency matters. Registration tolerates an unreachable endpoint, so scaling to zero does not block you from adding the model.
Who bills me for the tokens?
Your provider. Bolna records a custom model’s token usage against your own endpoint, so that model’s inference is not billed by Bolna.
Can a sub-account have its own custom model?
Yes. A model added on the account is available to it and to all of its sub-accounts. A model added inside a sub-account stays in that sub-account, invisible to the main account and to sibling sub-accounts.
My agent is silent on calls but the model saved fine. Why?
Bolna saves a model even when it can’t fully check the endpoint, so the first call is often where problems show up. The usual causes are: a deployment that scaled to zero, a 429 rate limit, a misspelled model name on an endpoint without a model list, an expired key, or an extra_body field the server rejects. Run the curl test with the same values to see the error.
Can I switch an agent to a different provider later?
Yes. Register the new endpoint as its own custom model, select it on the agent’s Intelligence tab and save. Delete the old model once no agent uses it.
Does the Provider I pick in the dialog change how Bolna calls my endpoint?
No. The Provider dropdown only changes the hints in the dialog. Bolna stores the same four fields, and calls the endpoint the same way, whichever preset you choose.
Can I point an agent at a model on a private network?
No. The endpoint must resolve to a public address. Expose it through a public hostname, with your key as the bearer token.
