Connect Baseten Model APIs or dedicated deployments to Bolna
Use Baseten Model APIs or your own Truss deployment on dedicated GPUs as the LLM for a Bolna voice agent. Endpoints, model slugs, Fast variants and API keys.
Baseten is a production inference platform with two ways in, and they need different values in Bolna:
Model APIs: a supported set of popular open models that Baseten runs for you, called by model slug at one shared URL. Nothing to deploy.
Dedicated deployments: you deploy any model, including fine-tunes, to GPUs reserved for you with a short config.yaml and the Truss CLI. Each deployment gets its own URL.
Choose Model APIs to get running in minutes. Choose a dedicated deployment when you need a model Model APIs doesn’t offer, your own weights, or capacity nobody else shares.
Create a key on the API keys page of your Baseten workspace settings. Bolna sends it as Authorization: Bearer <key>, which is how Baseten’s OpenAI-compatible endpoints authenticate.
Open Baseten’s Model APIs list and copy the slug, for example zai-org/GLM-5.3. Some models also have a Fast variant: the same weights with dedicated capacity for higher sustained throughput, under a separate slug with its own pricing and rate limits.
2
Choose Baseten in Bolna
Open Settings → Integrations → LLM → Custom LLM → Add model, or choose + Add custom LLM from the LLM Provider dropdown on an agent. Select Baseten.
Baseten selected: Model APIs endpoint as placeholder, hint for dedicated deployments
Baseten’s context and output limits for a Model API can be lower than the model’s advertised maximum. Check the limits in the Model APIs table before writing a very long system prompt.
Follow Baseten’s Deploy your first model guide. The config names the Hugging Face checkpoint (checkpoint_repository), the GPU and the inference engine. With the TensorRT-LLM engine, no Python code is needed.
3
Deploy
truss push
Truss prints a model ID and a deployment ID. Wait until the deployment shows Active in the Baseten dashboard.
4
Register the deployment in Bolna
Choose Baseten in the Add custom model dialog and replace the placeholder with your deployment’s OpenAI-compatible URL:
Use the model name from the deployment’s example request in the Baseten dashboard, and your Baseten API key.
Because the URL points at the production environment, you can promote a new deployment to production in Baseten and your Bolna agents switch to it without a change.
