> ## Documentation Index
> Fetch the complete documentation index at: https://www.bolna.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# A/B test voice agent prompts and versions

> A/B test a Bolna voice agent: run a prompt, voice or model change as an agent version on some calls or a batch, compare results, make the winner live.

To A/B test a Bolna voice agent, save each variant as an [agent version](https://www.bolna.ai/docs/agent-setup/agent-versions.md), send part of your calls to each version, then compare the results by `agent_version_id` and make the winner live. Your live agent keeps answering every other call while the test runs.

A version is a full copy of the agent's setup, so you can test any change: a new prompt or welcome message, a different voice, LLM or transcriber, or other call settings. Each call runs on exactly one version, and records it as the call's `agent_version_id`.

## How to A/B test a voice agent

1. **Save the variants.** Edit the agent and [save each change as a new version](https://www.bolna.ai/docs/agent-setup/agent-versions.md#save-as-a-new-version). The live version is usually your control.
2. **Split the calls.** Pass `agent_version_id` on each [`POST /call`](#split-individual-calls), or [split a batch](#split-a-batch) across 2 to 4 versions by percentage.
3. **Compare the results.** Filter call history and executions by version, and compare answer rate, duration, cost and extracted data. See [Compare the results](#compare-the-results).
4. **Make the winner live.** Every call that doesn't ask for a version then uses it. See [Pick a winner](#pick-a-winner).

There are two ways to split calls across versions:

| | [Individual calls](#split-individual-calls) | [Batches](#split-a-batch) |
| - | - | - |
| **How** | Pass `agent_version_id` on each `POST /call` | Set the split when you upload the batch, in the dashboard or with `version_split` on `POST /batches` |
| **Who decides each call's version** | You, in your own code | Bolna, by the percentages you set |
| **Versions** | Any number | 2 to 4 |
| **Best for** | Calls you trigger from your own system | Outbound campaigns from a CSV |

Calls that don't ask for a version, including all inbound calls, keep using the live version.

> [!NOTE]
>
> Save each variant as its own version first, see [Save as a new version](https://www.bolna.ai/docs/agent-setup/agent-versions.md#save-as-a-new-version). You'll need each version's `id`, from the version switcher or from [List agent versions](https://www.bolna.ai/docs/api-reference/agent/v2/versions/list.md).

***

## Split individual calls

[Make call](https://www.bolna.ai/docs/api-reference/calls/make.md#call-a-specific-agent-version) takes an optional `agent_version_id`. The call runs on that version's prompts, voice, models and settings, and calls without it run on the live version. To A/B test, choose a version for each call in your own code and pass it.

Assign each contact a version once and reuse it, so a contact called again gets the same version. Hashing a stable ID does this without storing anything:

```python Pick a version per contact theme={"system"}
import hashlib
import os

import requests

VERSIONS = [
    ("0c6a5e0e-7b8f-4d1a-9f2e-3a4b5c6d7e8f", 70),  # control, 70%
    ("9e8d7c6b-5a4f-4e3d-8c2b-1a0f9e8d7c6b", 30),  # challenger, 30%
]

def version_for(contact_id: str) -> str:
    bucket = int(hashlib.sha256(contact_id.encode()).hexdigest(), 16) % 100
    for version_id, weight in VERSIONS:
        if bucket < weight:
            return version_id
        bucket -= weight
    return VERSIONS[-1][0]

requests.post(
    "https://api.bolna.ai/call",
    headers={"Authorization": f"Bearer {os.environ['BOLNA_API_KEY']}"},
    json={
        "agent_id": "123e4567-e89b-12d3-a456-426655440000",
        "agent_version_id": version_for("+919876543210"),
        "recipient_phone_number": "+919876543210",
    },
)
```

A version that doesn't exist, belongs to another agent or was deleted returns `404` with `Agent version not found.`

***

## Split a batch

A batch can split its contacts across two to four versions. Each contact is assigned a version when the batch is uploaded, in proportion to the percentages you set, and its retries keep that version.

> [tab] Dashboard
>
> On the **Batches** page, select your agent and click **Upload Batch**. Turn on **A/B test versions**, then pick a version and a traffic share for each variant. See [Test agent versions against each other](https://www.bolna.ai/docs/outbound/batch-calling.md#test-agent-versions-against-each-other).
>
> ![Upload Batch dialog with A/B test versions turned on](https://mintcdn.com/bolna-54a2d4fe/dmTdbxeZGEzAsBts/images/batch-calling/upload-batch-ab-test.png?fit=max\&auto=format\&n=dmTdbxeZGEzAsBts\&q=85\&s=6bd710b43f6d841df2bca61cdb50bf00)
>
> Upload Batch dialog with A/B test versions turned on
>
> Each row is a variant: choose its version and its share of the batch's contacts. The shares must add up to 100%, and **Split evenly** resets them to equal shares.
>
> ![A/B test versions with v1 and v2 at 50% each](https://mintcdn.com/bolna-54a2d4fe/dmTdbxeZGEzAsBts/images/batch-calling/ab-test-versions-split.png?fit=max\&auto=format\&n=dmTdbxeZGEzAsBts\&q=85\&s=e951686b96c9c5f0dff14b8d0c5c5303)
>
> Two variants, the live v1 and v2, each getting 50% of the contacts

> [tab] API
>
> Pass `version_split` on [Create batch](https://www.bolna.ai/docs/api-reference/batches/create.md#split-a-batch-across-agent-versions):
>
> ```bash theme={"system"}
> curl https://api.bolna.ai/batches \
>   -H "Authorization: Bearer $BOLNA_API_KEY" \
>   -F "agent_id=123e4567-e89b-12d3-a456-426655440000" \
>   -F "file=@recipients.csv" \
>   -F 'version_split={"arms":[{"agent_version_id":"0c6a5e0e-7b8f-4d1a-9f2e-3a4b5c6d7e8f","weight":70},{"agent_version_id":"9e8d7c6b-5a4f-4e3d-8c2b-1a0f9e8d7c6b","weight":30}]}'
> ```
>
> Weights are whole numbers from 1 to 99 that add up to 100, and each arm needs a different version.

All versions in the split need the same telephony provider as the live version, and batches with a workflow can't be split.

***

## Compare the results

Every call records the version it ran on, however it was started.

* **Dashboard:** in [call history](https://www.bolna.ai/docs/agent-setup/call-history), select the agent and use the **Version** filter and column. **Download Records** includes an `agent_version_id` column.
* **API:** filter [Get agent executions](https://www.bolna.ai/docs/api-reference/executions/get_executions.md#filter-by-agent-version) with `agent_version_id`. Each execution also returns its `agent_version_id`.

Compare outcomes such as answer rate, call duration, cost and your [extracted data](https://www.bolna.ai/docs/agent-setup/analytics-tab.md) between versions.

***

## Pick a winner

Make the winning version live with **Make live** in the version switcher, or with [Set current version](https://www.bolna.ai/docs/api-reference/agent/v2/versions/set_current.md). From then on, every call that doesn't ask for a version, including inbound calls, uses it.

***

## FAQ

> [accordion] What can I A/B test on a voice agent?
>
> Anything saved in an agent version: the prompts, the welcome message, the voice, the LLM and transcriber models, and the other call settings. Change one thing per version so you know which change made the difference.

> [accordion] Does an A/B test change my live agent?
>
> No. Calls that don't ask for a version, including all inbound calls, keep running on the live version. A version only takes over once you make it live.

> [accordion] How many versions can I test at once?
>
> A batch splits across 2 to 4 versions. With individual calls there's no limit, because your code picks the version for each call.

> [accordion] Can I A/B test inbound calls?
>
> No. Inbound numbers always answer with the live version. To compare versions on inbound traffic, make one version live for a period, then the other, and compare the two periods.

> [accordion] Do retries and repeat calls use the same version?
>
> In a batch, yes: each contact gets its version when the batch is uploaded, and its retries keep it. With individual calls, it's up to your code, so assign each contact one version and reuse it, as in the [example above](#split-individual-calls).

> [accordion] Can I delete a version while it's being tested?
>
> Not while it's in the split of a batch that hasn't finished. [Delete agent version](https://www.bolna.ai/docs/api-reference/agent/v2/versions/delete.md) returns `409` until that batch completes or is stopped. The live version can't be deleted either, so make another version live first. Calls that already ran on a deleted version keep their transcripts and still show it as their version.

> [accordion] Can I A/B test a batch that uses a workflow?
>
> Not yet. Batches with a workflow can't be split across versions.

***

## Next steps

> [card] [Agent versions](https://www.bolna.ai/docs/agent-setup/agent-versions.md)
>
> Save, edit and manage versions

> [card] [Batch calling](https://www.bolna.ai/docs/outbound/batch-calling.md)
>
> Upload and run batches


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.