AssemblyAI real-time transcription for Bolna voice agents

AssemblyAI provides streaming speech-to-text through its Universal-3 model family. The current model, universal-3-6-pro, is trained on real voice-agent and phone audio, so it holds up on the turns that break voice agents: one-word answers, background voices, and callers who switch language mid-sentence. On Bolna it is also a strong choice when you need accurate proper nouns, because it accepts both a keyword list and a free-form description of the call.

Quick config

"transcriber": {
  "provider": "assembly",
  "model": "universal-3-6-pro",
  "language": "en",
  "endpointing": 150,
  "keywords": "Bolna, Plivo, Exotel",
  "context": "Inbound support line for a voice AI platform. Callers ask about telephony setup and billing.",
  "assemblyai_config": {
    "max_turn_silence": 2000
  }
}

Set this in your agent’s tools_config.transcriber block. See Create Agent API for the full schema.

Supported models

Model	Languages	Best for
`universal-3-6-pro`	32	Newest model; recommended default
`universal-3-5-pro`	19	Previous generation, still supported

Recommendation: use universal-3-6-pro. It is the most accurate of the family, it covers 13 languages that universal-3-5-pro does not, and AssemblyAI charges the same rate for both.

What `universal-3-6-pro` improves

AssemblyAI trained it on tens of thousands of hours of real voice-agent and phone conversations, so the gains land on conversational audio rather than clean dictation. Against universal-3-5-pro:

Area	Improvement
Short answers	45% fewer errors on replies like “yes” or “no thanks”, and 52% fewer in heavy background noise
Background voices	22% less background speech leaking into the transcript, 29% less in heavy noise
Language stability	74% fewer sessions transcribed in the wrong language
Turn handling	Same median latency, with more precise end-of-turn detection

Upgrading is a one-field change: set model to universal-3-6-pro. Nothing else in your transcriber block has to change, though check your language is still on the list below.

The older universal model is deprecated. It runs AssemblyAI’s legacy English-only streaming engine, which is not built for conversational speech, and it is no longer selectable on new agents. Existing agents on it keep running, but move them to universal-3-6-pro when you next edit them.

Supported languages

universal-3-6-pro code-switches across 32 languages, so a caller who mixes languages mid-sentence is handled without changing configuration.

Language	Code	Language	Code
Afrikaans	`af`	Korean	`ko`
Arabic	`ar`	Mandarin	`zh`
Cantonese	`yue`	Marathi	`mr`
Catalan	`ca`	Norwegian	`no`
Danish	`da`	Norwegian Nynorsk	`nn`
Dutch	`nl`	Persian	`fa`
English	`en`	Portuguese	`pt`
Estonian	`et`	Romanian	`ro`
Finnish	`fi`	Russian	`ru`
French	`fr`	Spanish	`es`
Galician	`gl`	Swedish	`sv`
German	`de`	Turkish	`tr`
Hebrew	`he`	Urdu	`ur`
Hindi	`hi`	Vietnamese	`vi`
Italian	`it`	Xhosa	`xh`
Japanese	`ja`	Zulu	`zu`

universal-3-5-pro covers 19 of these. The 13 it does not handle, and that universal-3-6-pro adds, are Afrikaans, Cantonese, Estonian, Galician, Korean, Marathi, Norwegian Nynorsk, Persian, Romanian, Russian, Urdu, Xhosa and Zulu. If your agent uses one of those, universal-3-6-pro is the only option. AssemblyAI keeps the canonical list at Multilingual transcription: supported languages.

Bolna’s language field takes the plain language code shown above, not a locale code: send "language": "en", not "en-IN". See Create Agent API for how model and language are validated as a pair.

Key settings

Setting	Type	Default	Description
`model`	string	`universal-3-6-pro`	Model variant
`language`	string	`en`	Language code, always set explicitly
`endpointing`	integer	`150`	ms the agent waits after a turn ends before replying
`keywords`	string	empty	Comma-separated terms to bias towards
`context`	string	empty	Plain description of the call
`assemblyai_config`	object	empty	AssemblyAI session settings, see Session settings

Keyword and context biasing

AssemblyAI is one of the few providers that accepts both fields, which is what makes it worth choosing when your calls are full of brand names or product codes.

"keywords": "Bolna, Plivo, Exotel, Twilio",
"context": "Inbound support line for a voice AI platform. Callers ask about telephony setup, call routing and billing."

keywords is sent as AssemblyAI’s keyterms prompt, up to 100 terms. Per-term weights are not supported, so Bolna strips any term:number suffix before sending.
context is sent as AssemblyAI’s prompt, up to 1,750 characters in any script, and is available on the Universal-3 models only.

See Keywords and context for how to write both well.

Session settings

assemblyai_config tunes the AssemblyAI session itself. Every field is optional: a field you leave out keeps AssemblyAI’s own default, so an agent without assemblyai_config behaves exactly as before.

"assemblyai_config": {
  "mode": "balanced",
  "max_turn_silence": 2000,
  "vad_threshold": 0.4,
  "voice_focus": "near-field"
}

Field	Values	Models	What it does
`mode`	`balanced`, `max_accuracy`, `min_latency`	Universal-3	Preset that sets the defaults of the turn settings below. `balanced` is the default and suits voice agents; `max_accuracy` transcribes more carefully but ends turns a few hundred milliseconds later
`max_turn_silence`	integer, ms	All	Silence after which the caller’s turn always ends. Raise it for callers who pause while reading out numbers or addresses
`min_turn_silence`	integer, ms	All	Silence before AssemblyAI checks whether the caller has finished. Raise it if brief pauses end turns too early
`interruption_delay`	`0` to `1000` ms	Universal-3	Wait before the first partial transcript. Lower lets callers interrupt sooner; higher makes that first partial more reliable
`vad_threshold`	`0` to `1`	All	How sure AssemblyAI must be that audio is speech. Raise it if line noise interrupts the agent; lower it if quiet callers are missed
`voice_focus`	`near-field`, `far-field`	Universal-3	Suppresses background voices and noise. `near-field` suits handsets and headsets, `far-field` speakerphones
`voice_focus_threshold`	`0` to `1`	Universal-3	How strongly voice focus suppresses audio that is not the main speaker. Requires `voice_focus`

When unset, AssemblyAI uses balanced, a VAD threshold of 0.2 on the Universal-3 models (0.4 on legacy models) and a longest pause of 1280 ms.
On the legacy universal model, mode, interruption_delay and the voice focus fields are not sent.
min_latency speeds up partial transcripts, not the end of the turn, so it does not make the agent reply sooner. Bolna replies once the turn ends; keep balanced for voice agents.
An invalid value, such as interruption_delay above 1000 or voice_focus_threshold without voice_focus, is rejected when you save the agent.

In the dashboard, Mode, VAD threshold and Longest pause in a turn sit under Advanced settings in the Transcription section of the Languages Tab. The other fields are available over the API.

Bolna Transcription settings with Provider set to assembly and Model set to universal-3-6-pro, Keywords and Context fields, and an expanded Advanced settings panel showing Mode set to Balanced, VAD threshold set to 0.2 and Longest pause in a turn set to 1280 — AssemblyAI transcription with Advanced settings expanded, showing Mode, VAD threshold and Longest pause in a turn at AssemblyAI's defaults

Agent context

On the Universal-3 models, Bolna gives AssemblyAI the agent’s last spoken reply as context for the caller’s next turn. When the agent asks “Can you spell your last name?” or “What are the last four digits of your card?”, AssemblyAI knows what kind of answer is coming, which helps on short answers, spelled-out names and numbers. AssemblyAI carries the caller’s earlier turns forward on its own. There is nothing to configure:

The context updates after every reply the agent speaks, including the welcome message.
If the caller interrupts, the context becomes the part of the reply they actually heard.
Long replies keep their last 1,750 characters, where the question usually is.
The legacy universal model does not support agent context, so nothing is sent there.

Turn handling

AssemblyAI’s streaming API is turn-based. It emits partial transcripts as the caller speaks and marks the end of a speaking turn explicitly, and Bolna treats only that end-of-turn signal as final, which keeps your agent from replying to half a sentence. Audio format and sample rate are set by Bolna from the channel the call arrives on, so there is nothing to configure there.

FAQ

Should I use universal-3-6-pro or universal-3-5-pro?

Use universal-3-6-pro unless you have measured universal-3-5-pro to be better on your own audio. universal-3-6-pro covers 32 languages against 19, is markedly better on short answers and noisy lines, and AssemblyAI charges the same rate for both, so there is rarely a reason to stay on the older model.

My agent is on the old universal model. Will calls break?

No. Existing agents keep running on the legacy engine. The model is hidden from new agent configuration, and saving an agent that still uses it prompts you to pick a replacement.

Why is my context being ignored?

Context is supported on the Universal-3 models only. On the legacy universal model the field is dropped. Check your agent is on universal-3-6-pro or universal-3-5-pro.

The agent replies before the caller has finished. What do I change?

AssemblyAI decides when the caller’s turn has ended, so tune that first. Raise max_turn_silence (Longest pause in a turn in the dashboard) for callers who pause while reading out numbers or addresses, and min_turn_silence if brief pauses end turns too early. See Session settings. endpointing is the extra pause the agent leaves after the turn has ended, before it speaks.

Line noise keeps interrupting the agent. What do I change?

Raise vad_threshold (VAD threshold in the dashboard) so AssemblyAI needs more confidence before it treats audio as speech, or set voice_focus to suppress background voices. If quiet callers start getting missed, lower vad_threshold again.

Can I turn off agent context?

No. It only adds context and is always on for the Universal-3 models. The legacy universal model does not support it.

Can I use an unlisted language?

No. A language outside the list above is not sent to AssemblyAI, and the model falls back to its own detection. Use Deepgram or Sarvam if you need a language AssemblyAI does not cover.

My language works on universal-3-6-pro but not on universal-3-5-pro. Why?

universal-3-5-pro only covers 19 of the 32 languages. Afrikaans, Cantonese, Estonian, Galician, Korean, Marathi, Norwegian Nynorsk, Persian, Romanian, Russian, Urdu, Xhosa and Zulu arrived with universal-3-6-pro. Move the agent to universal-3-6-pro.

Keywords and context: biasing transcription towards your vocabulary
Languages Tab: configure the transcriber in the dashboard
Deepgram transcriber: the default English option
Soniox transcriber: the other provider supporting both keywords and context

Telephony

AI models

No-code automation

Choose providers

AssemblyAI real-time transcription for Bolna voice agents

Quick config

Supported models

What `universal-3-6-pro` improves

Supported languages

Key settings

Keyword and context biasing

Session settings

Agent context

Turn handling

FAQ

​Quick config

​Supported models

​What universal-3-6-pro improves

​Supported languages

​Key settings

​Keyword and context biasing

​Session settings

​Agent context

​Turn handling

​FAQ

​Related

Quick config

Supported models

What `universal-3-6-pro` improves

Supported languages

Key settings

Keyword and context biasing

Session settings

Agent context

Turn handling

FAQ

Related