Last updated
What is Groq used for in a Bolna voice agent?
Groq is the LLM for a Bolna voice agent when reply speed matters most, because GroqCloud runs models on Groq's LPU hardware with very fast token output. A fast LLM shortens the gap between the caller finishing a sentence and the agent starting to speak, which is the gap callers notice most. Bolna connects to it as a custom LLM using the Groq preset.
How do I set up Groq with Bolna?
- Sign in to the GroqCloud console and create a key on the API Keys page.
- Check Settings → Limits in the console for your requests and tokens per minute and per day. Limits apply to the whole organisation.
- In Bolna, go to Settings → Integrations → LLM → Custom LLM → Add model, or choose + Add custom LLM in an agent's LLM Provider dropdown. Select Groq.
- Set Model name to a Groq model ID such as
llama-3.3-70b-versatile, Endpoint URL tohttps://api.groq.com/openai/v1, paste your key, and set Family tollamaorgpt-oss. - On the agent's Intelligence tab choose custom, pick the Groq model and save.
The Groq setup guide in the Bolna docs also covers rate limits and the OpenAI fields Groq handles differently.
Which Groq models should I use with Bolna?
| Model ID | Speed (tokens/s) | Good for |
|---|---|---|
llama-3.1-8b-instant | ~560 | Short, scripted flows where speed matters most |
llama-3.3-70b-versatile | ~280 | General-purpose agents that need better reasoning |
openai/gpt-oss-20b | ~1000 | Fast open-weight model with tool use |
openai/gpt-oss-120b | ~500 | Harder conversations and tool-heavy agents |
Speeds come from Groq's supported models page as quoted in the Bolna docs. Note the openai/ prefix on the GPT-OSS models. Groq rejects logprobs and top_logprobs with a 400, so leave them out of Extra request params.
What can you build with Groq and Bolna?
Fast scripted calls. Reminders and confirmations on llama-3.1-8b-instant, where a short pause before each reply keeps the call feeling natural.
Agents that call tools. Use openai/gpt-oss-20b or openai/gpt-oss-120b for agents that look up data or book slots mid-call. See the Cal.com integration for booking.
Batch campaigns with a shared prompt. Cached tokens don't count towards Groq's rate limits, so a long, stable system prompt shared by every call uses less of your quota. Pace large batches to stay under the requests-per-minute limit.
Speed comparisons. Register Groq and Cerebras side by side and compare first-reply time on web calls. The custom LLM page lists every provider preset.