State based vs Prompt Engineering
The Problem with One Big Prompt
A single prompt has to hold every rule the agent might ever need. "Greet the caller. Answer questions. If they want sales, collect their info. If they want support, route to the right person. If they ask about hours, look it up. If they ask to book, use Calendly. But only for existing customers. Except on weekends..." Every rule you add dilutes the earlier ones. At some point the model stops following them reliably.
In practice we see routing accuracy hover around 50% on prompt-only agents that have to branch to more than a handful of destinations. The agent gets it right sometimes and gets it wrong other times, and there is no clean lever to fix it.
What a Flow Changes
A flow splits the conversation into states. Each state is its own mini-prompt with its own set of tools. The agent only sees the instructions and tools relevant to where it is in the call.
- Shorter context means the agent pays attention to fewer rules at once.
- Fewer tools means fewer wrong choices. If the only tool in the "Collect Info" state is the info collector, the agent will not randomly decide to transfer.
- Explicit transitions replace hoping the prompt covers every branch. You write the trigger once, and the agent follows it.
Real-world result on the same routing problem: moving from prompt-only to state-based took accuracy from ~50% to near 100%.
When a Single Prompt Is Fine
Keep it simple when the agent's job is simple. A front-desk agent that only answers questions from company info and transfers to one receptionist does not need multiple states. Put everything in the initial Greeting / Assisting state and move on.
When to Reach for a Flow
- You have more than two or three places a caller might end up.
- You need to collect information before doing something (lead capture, booking, order intake).
- You need to verify the caller before a sensitive action (reschedule, cancel, account lookup).
- You have multiple integrations (calendar + POS + KB) and want the agent to use them at different points in the call.
- You are seeing the agent "freelance" — picking the wrong tool or skipping steps.
Rule of Thumb
If you are adding a sentence to the prompt that starts with "if the caller..." or "only do this when...", that is a candidate for a new state or transition. The prompt handles how the agent talks. The flow handles what the agent is allowed to do and when.
Related Articles
Call Routing Rules (Flow State)
A Call Routing state is a special node/state type that routes a call to a specific destination based on rules you define, instead of leaving the choice to the AI. It is the right tool whenever the destination must be exact — dialing the correct ...Transition Descriptions as Tools
Transitions are the arrows between states in your flow. To the AI Agent, each transition looks like a tool it can call. The description you write on a transition is what the agent reads to decide whether to fire it. A good description is the ...Knowledgebases
Your AI Agent's training data covers the basics — hours, address, services, a short FAQ. For anything deeper — product manuals, procedural answers, multi-step troubleshooting — use a Knowledge Base. Upload your documents once, and the agent can ...Building Conversation Flows
A flow is the conversation blueprint for your AI Agent. It defines the stages of a phone call, what the agent does at each stage, which tools it can use, and when the conversation moves forward. Think of it like a flowchart: each box is a stage the ...Multi-Language Support
Your AI Agent can handle calls in more than just English. Callers in Spanish, French, German, Italian, Dutch, or Japanese can be understood and answered in their own language, automatically and on the same call — no separate phone number required. ...