> ## Documentation Index
> Fetch the complete documentation index at: https://www.bolna.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Use Fireworks AI models in Bolna voice agents

> Connect Fireworks AI serverless models or on-demand GPU deployments to a Bolna voice agent. Full model ID paths, API keys, deployments and autoscaling.

[Fireworks AI](https://fireworks.ai) serves open models through an OpenAI-compatible API, with two tiers:

* **Serverless**: shared, per-token access to popular models. Fastest to start, subject to rate limits.
* **On-demand deployments**: dedicated GPUs for one model, with no rate limits, fast autoscaling and a wider choice of models. You create them with the `firectl` CLI.

The one thing that differs from every other provider here is the **model ID**. Fireworks uses a full resource path, `accounts/fireworks/models/<model>`, and the short name on its own won't work.

***

## Set up Fireworks AI

<Steps>
  <Step title="Create an API key">
    In the Fireworks dashboard, open [API keys](https://app.fireworks.ai/settings/users/api-keys), click **Create API key** and store it safely.
  </Step>

  <Step title="Copy the full model ID">
    Find a chat model in the Fireworks model library and copy its full ID, for example:

    ```text theme={"system"}
    accounts/fireworks/models/llama-v3p1-8b-instruct
    ```

    Note the Fireworks naming: versions are written with `p` for the decimal point, so Llama 3.1 is `llama-v3p1`.
  </Step>

  <Step title="Choose Fireworks AI in Bolna">
    Go to **Settings → Integrations → LLM → Custom LLM → Add model**, or choose **+ Add custom LLM** in an agent's **LLM Provider** dropdown. Select **Fireworks AI**. The hint under the endpoint field reminds you to use the full model ID.

    <Frame caption="Fireworks AI selected, with the full model ID hint">
      <img src="https://mintcdn.com/bolna-54a2d4fe/0EXypen329Eb5XE2/images/custom-llm/providers/fireworks-selected.png?fit=max&auto=format&n=0EXypen329Eb5XE2&q=85&s=cda570dc7b5826db64f4d6553494fed1" alt="Bolna Add custom model dialog with Fireworks AI selected, api.fireworks.ai/inference/v1 as the endpoint placeholder and a hint to use accounts/fireworks/models/ model IDs" width="1024" height="1220" data-path="images/custom-llm/providers/fireworks-selected.png" />
    </Frame>
  </Step>

  <Step title="Register the model">
    <Tabs>
      <Tab title="Dashboard">
        * **Model name**: the full path, for example `accounts/fireworks/models/llama-v3p1-8b-instruct`
        * **Endpoint URL**: `https://api.fireworks.ai/inference/v1`. The `/inference` segment is required.
        * **API key**: your Fireworks key
        * **Family**: for example `llama`

        <Frame caption="Fireworks AI serverless model, filled in">
          <img src="https://mintcdn.com/bolna-54a2d4fe/aV72vktown5Rn3f1/images/custom-llm/providers/fireworks-filled.png?fit=max&auto=format&n=aV72vktown5Rn3f1&q=85&s=2bdb2335018a1c9b8b9a7c2f39a7f801" alt="Add custom model dialog with Fireworks AI selected, model accounts/fireworks/models/llama-v3p1-8b-instruct, endpoint api.fireworks.ai/inference/v1, masked key and family llama" width="1024" height="1172" data-path="images/custom-llm/providers/fireworks-filled.png" />
        </Frame>
      </Tab>

      <Tab title="API">
        ```bash theme={"system"}
        curl --request POST \
          --url https://api.bolna.ai/user/model/custom \
          --header 'Authorization: Bearer <BOLNA_API_KEY>' \
          --header 'Content-Type: application/json' \
          --data '{
            "custom_model_name": "accounts/fireworks/models/llama-v3p1-8b-instruct",
            "custom_model_url": "https://api.fireworks.ai/inference/v1",
            "custom_model_key": "<FIREWORKS_API_KEY>",
            "custom_model_family": "llama"
          }'
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Select it on the agent">
    Choose **custom** on the **Intelligence** tab, pick the model and save.
  </Step>
</Steps>

***

## Moving to an on-demand deployment

When a campaign starts hitting serverless rate limits, or you need a model that isn't on serverless, create an on-demand deployment:

```bash theme={"system"}
firectl create deployment accounts/fireworks/models/gpt-oss-120b \
  --deployment-shape default \
  --min-replica-count 0 \
  --scale-to-zero-window 5m
```

`firectl` prints the deployment's `Name`, in the form `accounts/<account-id>/deployments/<deployment-id>`. Fireworks routes a request to your deployment when the model ID ends in `#` plus that name:

```text theme={"system"}
accounts/fireworks/models/gpt-oss-120b#accounts/<account-id>/deployments/<deployment-id>
```

Register that string as the model name in Bolna, with the same endpoint URL and API key.

<Warning>
  `--min-replica-count 0` lets the deployment scale to zero, and the next call then waits for a GPU to start. For live voice traffic, keep at least one replica with `--min-replica-count 1`.
</Warning>

***

## Troubleshooting

<AccordionGroup>
  <Accordion title="Model not found">
    You probably entered a short name such as `llama-v3p1-8b-instruct`. Use the full `accounts/fireworks/models/...` path.
  </Accordion>

  <Accordion title="404 on chat completions">
    The endpoint must include `/inference`: `https://api.fireworks.ai/inference/v1`.
  </Accordion>

  <Accordion title="429 errors on serverless">
    Serverless has rate limits. Move the agent to an on-demand deployment, which has none.
  </Accordion>
</AccordionGroup>

***

## Related

* [Custom LLM inference](/docs/custom-llm-inference)
* [Together AI](/docs/custom-llm-inference/together): a large serverless catalog of open models
* [Baseten](/docs/custom-llm-inference/baseten): Model APIs and dedicated deployments


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.