> ## Documentation Index
> Fetch the complete documentation index at: https://www.bolna.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# A/B test agent versions in workflows

> Split an agent node's calls across saved versions of its agent by weight, see which version each contact got, and compare the versions on the campaign page.

## When to use it

You've saved a new version of an agent, such as a shorter greeting, a Hindi-first opening or a different offer, and want to know whether it does better than the version you run today. An A/B test on an agent node splits that node's calls between two or more [saved versions](https://www.bolna.ai/docs/agent-setup/agent-versions.md) of its agent, by weight. Every call records which version ran, so you can see it on the contact's journey, in the [execution webhook](https://www.bolna.ai/docs/workflows/execution-webhook), and side by side on the campaign page.

An agent node without an A/B test calls the agent's live version.

This page covers workflow agent nodes. For the other ways to A/B test an agent, individual calls and batches, and for picking a winner, see [A/B test voice agent prompts and versions](https://www.bolna.ai/docs/agents/ab-testing.md).

### Compared with a batch split

| | Workflow agent node | [Batch](https://www.bolna.ai/docs/outbound/batch-calling.md#test-agent-versions-against-each-other) |
| - | - | - |
| Versions | 2 or more, with no upper limit | 2 to 4 |
| What each arm runs | The named version, so edits saved into it later reach the test | The exact saved copy chosen when the batch is uploaded |
| Telephony provider | Each version can use its own, and each call uses the provider of the version that runs | Every version needs the live version's provider |
| Assignment | Each call picks a version by weight, so the split is approximate | Exact shares, fixed when the batch is uploaded |
| Retries | The same version while **Keep each contact on one version** is on | The same version |

***

## Before you start

The agent needs at least two saved versions. Create one in the agent editor with **Save agent as a new version** (see [Agent versions](https://www.bolna.ai/docs/agent-setup/agent-versions.md)), or with the [agent versions API](https://www.bolna.ai/docs/api-reference/agent/v2/versions/overview). The versions you test don't have to be live.

***

## Turn on an A/B test

> [tab] Dashboard
>
> **1. Open the agent node**
>
> Select the agent node in the [workflow editor](https://www.bolna.ai/docs/workflows/using-the-editor). The **A/B test** section sits below **Agent** in the Inspector.
>
> **2. Turn on A/B test versions**
>
> Two variants appear: **A** with the live version at 50%, and **B** with an empty **Choose a version** picker at 50%, and the panel shows **Choose a version for B** until you pick one. The picker lists the agent's versions with the live one marked **Live**, and a version already used in another row can't be picked twice. An arm whose version was deleted reads **Version deleted**.
>
> **3. Set the weights**
>
> Type a whole-number percentage for each version; a weight outside 1 to 99 shows **Weights are whole numbers from 1 to 99**. The total beside the split bar, such as **Total 100%**, must reach 100. When it doesn't, the panel shows the gap, such as **10% short** or **10% over**, and offers one fix, such as **Add 10% to A** or **Take 10% from C**. **Split evenly** divides 100 across the rows, with any remainder going to A (34 / 33 / 33 for three versions). **Add variant** adds one until every version of the agent has a variant, and once there are more than two, each one can be removed.
>
> **4. Choose whether contacts keep their version**
>
> **Keep each contact on one version** is on by default, captioned **Retries and repeat visits use the same version.** Turned off, the caption reads **Each call picks a version again.** See [Keep each contact on one version](#keep-each-contact-on-one-version) for what it changes.
>
> **5. Publish**
>
> Validate and publish the workflow. A draft saves even when the weights don't add up to 100, but it can't be published until they do.
>
> Turning **A/B test versions** off removes the A/B test from the node, which then calls the live version. When the agent has only one version the switch is unavailable, with the message **Save the agent as a new version to A/B test it against the live version.** If the versions can't be loaded, the panel shows **Couldn’t load this agent’s versions.** with **Try again**. Choosing a different agent on the node clears the versions. On the canvas, a node with an A/B test shows its split bar and the weights under the agent name, such as **A/B 50 / 30 / 20**.

> [tab] API
>
> Add `version_split` to the agent node's `config`, then [save the draft](https://www.bolna.ai/docs/api-reference/workflows/save-draft.md), [validate](https://www.bolna.ai/docs/api-reference/workflows/validate.md) and [publish](https://www.bolna.ai/docs/api-reference/workflows/publish.md):
>
> ```json theme={"system"}
> {
>   "id": "n_call",
>   "type": "agent",
>   "name": "Qualification call",
>   "config": {
>     "agent_id": "<agent_id>",
>     "variables": {
>       "customer_name": { "value": { "var": "entry.name" } }
>     },
>     "version_split": {
>       "sticky": true,
>       "arms": [
>         { "agent_version_id": "<id of v1>", "weight": 50 },
>         { "agent_version_id": "<id of v2-short-greeting>", "weight": 30 },
>         { "agent_version_id": "<id of v3-hindi-first>", "weight": 20 }
>       ]
>     }
>   },
>   "cases": [
>     { "when": { "cmp": "==", "left": { "var": "call.status" }, "right": { "const": "completed" } },
>       "then": { "to": "n_extract" } }
>   ]
> }
> ```
>
> Take each `agent_version_id` from [List agent versions](https://www.bolna.ai/docs/api-reference/agent/v2/versions/list.md), or the Bolna MCP `list_agent_versions` tool: it is the `id` of the version you want in the test. Leave `version_split` out for a node that calls the live version.

### Settings

| Config | Type | Default | Notes |
| - | - | - | - |
| `version_split.arms` | array | required | The versions to split calls across, at least two. Their order gives them the letters A, B, C and so on |
| `version_split.arms[].agent_version_id` | string | required | The `id` of a version in the agent's [versions list](https://www.bolna.ai/docs/api-reference/agent/v2/versions/list.md) |
| `version_split.arms[].weight` | integer | required | This version's share of calls, in percent: a whole number from 1 to 99 |
| `version_split.sticky` | boolean | `true` | **Keep each contact on one version**, described below |

### Rules

* At least two versions, and a version chosen for every arm.
* Weights are whole numbers from 1 to 99 and add up to exactly 100.
* Each version appears once, even through two different ids of it.
* Every version is one the agent's [versions list](https://www.bolna.ai/docs/api-reference/agent/v2/versions/list.md) shows, for the node's agent (`agent_id`).

Validation reports a broken rule as an error, so the workflow can't be published until it's fixed. Each message names the arm by its position and letter, such as `2 (B)`:

| Problem | Code | Reported at | Message |
| - | - | - | - |
| Fewer than two arms | `invalid_definition` | `config.version_split` | `an A/B test needs at least two arms` |
| An arm with no version | `invalid_definition` | `config.version_split` | `A/B arm 2 (B) needs a version` |
| A weight that is missing, not a whole number, or outside 1 to 99 | `invalid_definition` | `config.version_split` | `A/B arm 3 (C): weight must be a whole number from 1 to 99` |
| The same version id in two arms | `invalid_definition` | `config.version_split` | `each A/B arm must use a different agent version` |
| Weights that don't add up to 100 | `invalid_definition` | `config.version_split` | `A/B arm weights total 90, not 100` |
| A version the agent's versions list doesn't show | `unknown_agent_version` | `config.version_split.arms[2].agent_version_id` | `A/B arm 3 (C): agent version '<id>' not found` |
| Two arms that run the same named version | `invalid_definition` | the later arm, `config.version_split.arms[1].agent_version_id` | `A/B arms 1 (A) and 2 (B) both run version 'v1'` |

The first five checks report only the first problem they find, so after fixing it, validate again to see whether another arm needs attention. The split follows the weights approximately: each call is assigned a version by weight, so over many calls the shares come close to the weights, while a small number of calls can be uneven.

### Keep each contact on one version

With **Keep each contact on one version** on (`"sticky": true`), each contact gets one version at this node for its whole journey. Retries and later visits to the node use the same version, so every call a contact receives from the node sounds the same, and the comparison can credit the journey's outcome to that version.

With it off (`"sticky": false`), every visit to the node picks a version again by weight, so a contact retried three times can hear different versions. Use this when you are comparing single calls, such as how many complete or how long they last, rather than whole journeys. The comparison then shows call results only.

Each A/B-tested node picks on its own, and a contact who starts a new journey, for example in another campaign, can get a different version.

### Versions with different call settings

A call runs entirely on the version it was assigned. When the versions in a test differ in their telephony provider, **Total Call Timeout**, **Voicemail Detection** or [calling hours](https://www.bolna.ai/docs/outbound/calling-guardrails.md), each call uses the settings of the version that runs it.

***

## Prompt variables

All versions in the test share the node's **Variables**. Map every variable that any of the versions uses: validation warns `unmapped_agent_variable` for each one without a mapping and names the version that uses it, for example `agent prompt variable 'offer_code' used by version 'v2' has no mapping on this node`. In the editor, the node's **Variables** list every variable any version uses, and a variable that only some versions use is captioned with them, such as **Used by B only** or **Used by B and C only**.

When the other versions' variables can't be read yet, the list shows the live version's variables, with one line at the top saying why:

* **Showing the live version’s variables. Finish every A/B split in this workflow to see the rest.** This node's split, or another A/B split in the workflow, isn't complete yet.
* **Showing the live version’s variables while the other versions load.**
* **Showing the live version’s variables. Couldn’t load the other versions’ variables. It tries again on the next save.**

Mappings that no version in the test uses are listed under **No longer used by these versions**, where you can remove them. Until the versions have been checked they are listed under **Already mapped** and can't be removed yet, with one of these lines saying why:

* **Finish the split to check which versions use these.** The split still has an empty row, a weight outside 1 to 99, or a total other than 100.
* **Finish every A/B split in this workflow to check which versions use these.** Another node's A/B split isn't complete yet.
* **Checking which versions use these…** The check is running.
* **Couldn’t check which versions use these. It tries again on the next save.**

***

## Changing versions during a test

The test follows each version by its name, so ordinary work on the agent doesn't break it.

| You | What happens |
| - | - |
| Save edits into a version in the test | The next call that uses the version runs the edits, including in campaigns already running |
| Rename a version | The test keeps using it under the new name. In the comparison, calls before and after the rename appear as separate rows, and a case that compares `call.agent_version_name` with the old name stops matching |
| Make another version live | Nothing changes for the test. Each row keeps its version |
| Delete a version in the test | See [Deleting a version in a test](#deleting-a-version-in-a-test) |

### Deleting a version in a test

[Delete agent version](https://www.bolna.ai/docs/api-reference/agent/v2/versions/delete.md) returns `409` while a workflow call that has been accepted but hasn't connected yet is waiting on a save of that version, for example one held until calling hours, with the message `This version has workflow calls waiting to be placed. Take it out of the workflow and let those calls connect first.` It also returns `409` while an [A/B batch](https://www.bolna.ai/docs/agents/ab-testing.md#split-a-batch) that hasn't finished uses the version. Try again once those calls have connected.

After a version in a test is deleted:

* New calls on its arm run the live version instead. Workflows already published, and campaigns already running, keep going.
* The workflow can't be published until the node is fixed: validation reports `unknown_agent_version` at that arm's `config.version_split.arms[<i>].agent_version_id`, such as `A/B arm 2 (B): agent version '<id>' not found`. Choose another version for the arm, or turn the test off.
* Calls that ran on the live version in its place show in the comparison as their own row for that arm, under the live version's name, captioned for example **Ran live. B’s version was deleted.**

> [!NOTE]
>
> To take a version out of a test without deleting it, choose another version for its arm or turn the test off, and publish. A campaign keeps the split it was created with, so its calls on that arm keep using the version.

### Running campaigns keep their split

The split is part of the published workflow version. A campaign keeps the version it was created with, so changing the weights or the versions in the test and publishing again applies to new campaigns and new runs, not to campaigns already created. Edits saved into the versions themselves do reach running campaigns, as described above.

***

## Branch on the version

Every agent node, with or without an A/B test, writes two more outputs whenever it places a call:

| Output | Value |
| - | - |
| `call.agent_version_id` | The `id` of the saved version the call ran on |
| `call.agent_version_name` | That version's name, such as `v2-short-greeting` |

Branch on the name. Each save into a version gives it a new `id`, while its name stays the same until you rename it:

```json theme={"system"}
{ "when": { "cmp": "==", "left": { "var": "call.agent_version_name" }, "right": { "const": "v2-short-greeting" } },
  "then": { "to": "n_followup_short" } }
```

A later node reads the value as `node.<agent_node_id>.call.agent_version_name`, even after another agent node has run. See [Conditions and variables](https://www.bolna.ai/docs/workflows/conditions-and-variables.md#variable-namespaces).

***

## See which version a contact got

* **Journey timeline.** Open a contact's journey from the campaign page. A step of an A/B-tested node shows a chip with the arm's letter and the name of the version that ran, for example **B, v2-short-greeting**. The chip always names the version that ran: a call placed on the live version in place of a deleted one shows the live version's name, with no separate label. Steps of other agent nodes have no chip; their version is in the step's outputs.
* **Execution API.** The node's attempt in [Get workflow execution](https://www.bolna.ai/docs/api-reference/workflow-executions/get.md) carries the two `call.*` outputs.
* **Execution webhook.** The node's entry in the [webhook](https://www.bolna.ai/docs/workflows/execution-webhook) `trail` carries them too, and on an A/B-tested node `inputs` adds `version_split.agent_version_id`:

```json theme={"system"}
{
  "node_id": "n_call",
  "node_type": "agent",
  "attempt": 1,
  "status": "completed",
  "to_node_id": "n_extract",
  "matched_case_index": 0,
  "inputs": {
    "recipient": "+919999999999",
    "version_split.agent_version_id": "<id of v2-short-greeting>"
  },
  "outputs": {
    "call.status": "completed",
    "call.duration_s": 148,
    "call.agent_version_id": "<id of v2-short-greeting's latest save>",
    "call.agent_version_name": "v2-short-greeting"
  }
}
```

`inputs.version_split.agent_version_id` is the version the node picked, exactly as it appears in `arms`. The outputs are the version that ran: the picked version's latest save, so its `id` can differ from the one in `arms` after edits, or the live version when the picked one was deleted.

***

## Compare the versions

On the campaign page, each A/B-tested node has its own card, titled **A/B test:** and the node's name, such as **A/B test: Qualification call**. Open it to load the numbers, and click **Refresh** to update them; **Updated** shows how long ago they were loaded. **Set** shows the split you configured and, once calls have been placed, **Actual** shows the split calls really took, each with its shares such as 50 / 30 / 20. Until then the card reads **No calls yet. Results appear as calls finish.** The table has one row per version, and each column sorts:

| Column | What it shows |
| - | - |
| Version | The arm's letter and the version's name, with the live version marked **Live**. A row of calls that ran on the live version because the arm's version was deleted is captioned, for example **Ran live. B’s version was deleted.** An arm with no calls yet still has a row |
| Calls | Calls placed on the version |
| Completed | The share of those calls whose `call.status` is `completed` |
| Avg call | Total call time divided by completed calls |
| Settled | Journeys that have finished, credited to the version the contact got at this node |
| Success, Failure, Neutral | The share of settled journeys that ended at an end node with that `outcome` |

Hover a percentage to see the count behind it, such as 731 of 1,204 calls. A cell with nothing to divide by, such as **Avg call** for a version with no completed calls, shows **NA**.

Call columns update as calls finish. Journey columns count settled journeys only: a contact still partway through the workflow counts once its journey ends. Each contact counts toward the version of its first visit to the node, and only when that visit placed a call: a contact whose first visit placed none, for example because the dial was rejected, isn't counted in the journey columns, even if a later visit called. A journey that stopped without reaching an end node, such as one you canceled, is settled but has no outcome, so the three outcome columns can add up to less than all settled journeys. The card's footer reads **Calls update as they finish. Journey results count settled journeys.** With **Keep each contact on one version** off, the journey columns are left out and the footer reads **Calls update as they finish.** followed by **Journey results need ‘Keep each contact on one version’, because a journey can use several versions.**

Workflow calls are agent calls, so the other ways to compare versions work for them too: the **Version** filter and column in [call history](https://www.bolna.ai/docs/agent-setup/call-history), the `agent_version_id` filter on [Get agent executions](https://www.bolna.ai/docs/api-reference/executions/get_executions.md#filter-by-agent-version), and the `agent_version_id` column in **Download Records**. See [Compare the results](https://www.bolna.ai/docs/agents/ab-testing.md#compare-the-results).

### Comparison API

The same numbers come from [`GET /workflow-campaigns/{campaign_id}/ab-tests`](https://www.bolna.ai/docs/api-reference/workflow-campaigns/ab-tests), worked out each time you call it:

```bash request theme={"system"}
curl --location 'https://api.bolna.ai/workflow-campaigns/7c1d2e3f-4a5b-6c7d-8e9f-0a1b2c3d4e5f/ab-tests' \
--header 'Authorization: Bearer <api_key>'
```

```json response theme={"system"}
{
  "nodes": [
    {
      "node_id": "n_call",
      "node_name": "Qualification call",
      "agent_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "sticky": true,
      "arms": [
        { "agent_version_id": "0b6e2c1a-7f3d-4e8a-9c21-5d4f6a7b8c90", "weight": 50 },
        { "agent_version_id": "5e8f9a0b-1c2d-4e3f-8a4b-6c7d8e9f0a1b", "weight": 50 }
      ],
      "results": [
        {
          "agent_version_id": "0b6e2c1a-7f3d-4e8a-9c21-5d4f6a7b8c90",
          "agent_version_name": "v1",
          "calls": 1204,
          "completed": 731,
          "duration_s": 143280,
          "journeys": { "settled": 1010, "success": 286, "failure": 552, "neutral": 160 }
        },
        {
          "agent_version_id": "5e8f9a0b-1c2d-4e3f-8a4b-6c7d8e9f0a1b",
          "agent_version_name": "v2-short-greeting",
          "calls": 1188,
          "completed": 779,
          "duration_s": 131040,
          "journeys": { "settled": 996, "success": 331, "failure": 509, "neutral": 149 }
        }
      ]
    }
  ]
}
```

| Field | Meaning |
| - | - |
| `nodes` | Every A/B-tested agent node in the campaign's workflow version, in definition order. Empty when there are none |
| `nodes[].sticky` | The node's **Keep each contact on one version** setting |
| `nodes[].arms` | The configured split, in order: the first is A, the second B, and so on |
| `nodes[].results` | One row per version that ran for each arm. `agent_version_id` is the arm's id from `arms`, and `agent_version_name` the name of the version that ran. A deleted version's calls appear as a second row for its arm, under the live version's name |
| `results[].calls` | Calls placed |
| `results[].completed` | Calls whose `call.status` is `completed` |
| `results[].duration_s` | Total time of the completed calls, in seconds. The card's **Avg call** is `duration_s / completed` |
| `results[].journeys` | `settled` journeys, and how many of them ended with each `outcome`, counted by the version of each contact's first visit to the node. A contact whose first visit placed no call isn't counted. `null` when `sticky` is `false` |

All values are counts. Work out percentages and averages from them: the completion rate is `completed / calls`, and the average call is `duration_s / completed`.

***

## Next steps

> [card] [Agent versions](https://www.bolna.ai/docs/agent-setup/agent-versions.md)
>
> Save, edit and test versions of an agent

> [card] [Workflow nodes](https://www.bolna.ai/docs/workflows/nodes.md#agent)
>
> Every agent node setting and output

> [card] [Campaigns](https://www.bolna.ai/docs/workflows/campaigns.md)
>
> Run the workflow over your contacts

> [card] [Execution webhook](https://www.bolna.ai/docs/workflows/execution-webhook)
>
> Get each contact's journey, version included, when it finishes


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.