Use Fireworks AI models in Bolna voice agents

• Serverless: shared, per-token access to popular models. Fastest to start, subject to rate limits.
• On-demand deployments: dedicated GPUs for one model, with no rate limits, fast autoscaling and a wider choice of models. You create them with the firectl CLI.

​

Set up Fireworks AI

1

Create an API key

In the Fireworks dashboard, open API keys, click Create API key and store it safely.

2

Copy the full model ID

Find a chat model in the Fireworks model library and copy its full ID, for example:

accounts/fireworks/models/llama-v3p1-8b-instruct

Note the Fireworks naming: versions are written with p for the decimal point, so Llama 3.1 is llama-v3p1.

3

Choose Fireworks AI in Bolna

Go to Settings → Integrations → LLM → Custom LLM → Add model, or choose + Add custom LLM in an agent’s LLM Provider dropdown. Select Fireworks AI. The hint under the endpoint field reminds you to use the full model ID.

4

Register the model

• Dashboard
• API

• Model name: the full path, for example accounts/fireworks/models/llama-v3p1-8b-instruct
• Endpoint URL: https://api.fireworks.ai/inference/v1. The /inference segment is required.
• API key: your Fireworks key
• Family: for example llama

curl --request POST \
  --url https://api.bolna.ai/user/model/custom \
  --header 'Authorization: Bearer <BOLNA_API_KEY>' \
  --header 'Content-Type: application/json' \
  --data '{
    "custom_model_name": "accounts/fireworks/models/llama-v3p1-8b-instruct",
    "custom_model_url": "https://api.fireworks.ai/inference/v1",
    "custom_model_key": "<FIREWORKS_API_KEY>",
    "custom_model_family": "llama"
  }'

5

Select it on the agent

Choose custom on the Intelligence tab, pick the model and save.

​

Moving to an on-demand deployment

firectl create deployment accounts/fireworks/models/gpt-oss-120b \
  --deployment-shape default \
  --min-replica-count 0 \
  --scale-to-zero-window 5m

accounts/fireworks/models/gpt-oss-120b#accounts/<account-id>/deployments/<deployment-id>

​

Troubleshooting

​

Related

• Custom LLM inference
• Together AI: a large serverless catalog of open models
• Baseten: Model APIs and dedicated deployments