ServiceNow Virtual Agent Optimisation

Virtual agent optimization is the work of finding which part of the agent causes a failure and fixing that part. The common areas are conversation flow, intent recognition, knowledge sources and escalation rules. Each change should be tested so its effect is measured, not guessed.

Optimising a Virtual Agent means knowing when a tuning fix is the wrong fix, and sorting each failure by where the fix belongs: conversation flow, intent training, knowledge sources or escalation rules. This page covers the common optimisation areas, escalation tuning, a worked example, continuous improvement processes, metrics, testing a change without guessing its effect, prioritising fixes by impact against effort, an illustrative scenario about speeding up a status check, and keeping the agent honest about what it cannot do.

Common Optimization Areas

Conversation flow optimization improves how the agent asks clarifying questions. A poorly designed flow requires users to answer questions in a specific order even if some questions are irrelevant. An optimized flow asks only relevant questions and branches based on the user’s answers. If a user is asking about a password reset, the flow does not ask for network diagnostic details. This makes conversations faster and less frustrating.

Intent recognition tuning improves the agent’s ability to understand what users are asking. The agent is trained on many examples of how users phrase questions. If users say “I forgot my pin” or “password reset needed” or “can’t get in,” training examples teach the agent that all these phrases mean the same thing. Without training, the agent might think these are different requests.

Answer quality improves when knowledge sources are well-organized and current. If the agent consults outdated or incorrect knowledge articles, it gives wrong answers. Linking the agent to quality knowledge and removing outdated articles improves answer accuracy. Feedback loops where users mark answers as helpful or unhelpful teach the agent which answers work.

Escalation Optimization

Virtual agents should escalate to human agents only when necessary. An agent that escalates too quickly reduces efficiency and costs money. An agent that tries too hard to resolve issues beyond its capability frustrates users. Optimization finds the balance: the agent handles questions it can answer well and escalates only when human expertise is truly needed.

Escalation criteria should be clear. If a conversation has been going back and forth with no resolution after five exchanges, escalate. If the user explicitly asks to speak to a human, honor that request. If the agent detects strong emotion (frustration, anger), escalate because the user needs human empathy. These rules prevent the agent from becoming a blocker to human help.

A Worked Example

Your company deployed a chatbot for password resets. Initial performance: most users who start a conversation with the bot complete password reset without escalating, while some need to escalate to human agents. You analyze escalation conversations and find patterns: users whose user IDs differ from their email addresses get confused. Users who have multifactor authentication enabled do not receive reset codes at the email address the agent suggests.

You optimize the bot’s questions. It now asks “What is your user ID?” upfront and clarifies “Your email address is different; reset codes will be sent to…” before starting the process. Multifactor authentication is detected, and reset instructions mention the correct delivery channel. After optimization, most users complete password reset without escalation. Escalation rate drops significantly, saving significant call center cost.

You also notice that conversations about password resets for corporate systems ask for very different information than personal account resets. You configure separate conversation paths for each scenario. Users see a simple question upfront: “Is this a corporate system login or personal account?” This single routing question halves the questions needed because each path contains only relevant questions.

Continuous Improvement Processes

Virtual agent optimization is not a one-time project; it is an ongoing discipline. Every conversation provides data: which questions the agent handled well, which it got wrong, which led to escalation. Reviewing this data regularly (weekly or monthly) identifies improvement opportunities. You notice a pattern of questions the agent consistently misunderstands and add training examples for that pattern.

User feedback through ratings helps identify problems. If users consistently rate a particular answer as unhelpful, the knowledge content behind that answer may be wrong or out of date. Updating the knowledge or removing unhelpful answers improves the agent’s effectiveness immediately.

Metrics and Success Measures

Track conversation success rates: what percentage of conversations the agent handles end-to-end without escalation. Track resolution accuracy: of escalated conversations, what percentage did the human agent resolve as the chatbot suggested? Track user satisfaction: do users rate the chatbot experience positively? These metrics guide optimization. Declining satisfaction indicates the agent needs tuning.

Cost metrics matter: cost per conversation for chatbot versus cost per conversation for human agents. As escalation rates decrease and the agent handles more conversations, the cost per conversation decreases. Budget models should show the payoff from optimization effort.

Knowing when a tuning fix is the wrong fix

Optimizing a Virtual Agent is tempting because many of its failures can be fixed by adjusting wording or adding examples. Some failures cannot. If the business rule behind an answer is wrong, a better phrasing will only deliver the wrong rule more smoothly. If the process has changed and the agent describes the old one, no amount of intent tuning will help. Before changing the agent, decide which kind of problem you have, because the owner and the fix are different in each case.

This guidance is for the conversational designer, the knowledge owner and the service channel manager who are responsible for an agent that already answers questions but does not answer them well enough. It assumes the agent has its own conversation history to learn from.

Sorting failures by where the fix belongs

A quick triage separates the failures into groups that each have a single owner. The table is a working guide for that sorting.

Failure seen in conversations	Usually belongs to	Fix type
Agent gives a rule that no longer applies	Knowledge owner	Update the source content, then the topic that cites it
Agent misunderstands a common way of phrasing a request	Conversational designer	Add phrasings and check neighboring topics for overlap
Agent asks questions in an order that wastes the user’s time	Conversational designer	Reorder or branch the flow on the answers already given
Agent transfers too readily or never transfers	Service channel manager	Revisit the escalation rules agreed with the receiving team
Correct answer exists but the user cannot act on it	Service channel manager with the process owner	Fix the link, form or step the answer depends on

Many optimization programs start by fixing the first row and the last, because those failures damage trust the fastest, even though they are not the most frequent.

Testing a change without guessing its effect

Improving an agent by intuition tends to move the failures rather than remove them. A fix to one topic can make another topic misfire, and the team only notices when a user complains. The discipline is to hold back a set of real phrasings from the training effort, run them before and after each change, and compare the results. Keep the set separate from the examples used to tune, or the agent will look better than it is. Record which phrasings improved, which got worse, and why. A small held-out set run after every change is more reliable than a large one that nobody checks.

1. Collect a sample of real phrasings from failed conversations, grouped by topic.
2. Set aside a portion of them as a held-out set that is not used to tune the agent.
3. Record the agent’s current behavior on the held-out set before changing anything.
4. Make one change at a time, and run the held-out set again.
5. Keep the change only if the targeted failures improve and no other topic gets worse.

Prioritizing fixes by impact against effort

A long list of failures is not a plan. Rank each candidate fix by two things: how many users it affects, and how much effort it takes. A single phrasing fix that corrects a frequent misunderstanding can be worth more than a large rebuild of a rarely used flow. Put fixes that need the knowledge owner or a process change into a separate list, because they will run on the owner’s timetable and should not hold up the quick wins.

Illustrative scenario: a team that wants to speed up a status check

Consider a team whose agent answers travel expense status questions but asks for the claim number, the date and the cost center in a fixed order every time. Users who only remember the date are stuck, and they transfer to a person who then asks for all three anyway. The fix that would help is not more training examples. It is a change to the flow that accepts whichever detail the user has first and looks the claim up from that. Consider also whether the team needs the cost center at all for a status check, since asking for information the lookup does not use can be a source of friction.

Keeping the agent honest about what it cannot do

Optimization can drift towards making the agent answer everything. That often harms the users it is meant to help. A clear statement of what the agent does not handle, shown to users when a request falls outside its scope, sends people to the right route faster than a confident but wrong answer. Review the scope statement at each optimization cycle, and remove topics the agent has never handled well rather than leaving them to fail quietly.

Questions about tuning and its limits

How many failed conversations should be reviewed each cycle?

Enough to see the common patterns. A review that reads too few conversations overreacts to unusual ones, and one that reads too many stops being done. Pick a sample size the team can finish and keep it the same from cycle to cycle, so the trend is meaningful.

Can we use users’ thumbs-up or thumbs-down ratings as the only signal?

No. Ratings are useful for finding answers to examine, but they reflect mood as much as accuracy. Read the conversations behind low ratings before deciding that an answer is wrong.

Should every optimization change go to the content owner?

Changes to rules and to factual answers should. Changes to phrasing and flow usually do not need the content owner, provided the held-out set confirms they do no harm.

When should we stop optimizing a topic?

When the remaining failures need a process decision rather than a language fix. At that point the topic should be redesigned with the process owner, or handed to people, and more tuning will only consume effort.

Next step

The escalation handoff guide covers the transfer rules that often need revisiting. The Virtual Agent health check page describes how a wider review is scoped.

Related guides: ServiceNow Virtual Agent Topic Explained.

A first optimisation cycle, in the order that pays back fastest

Start with the conversations that ended without an answer, not the ones that scored badly. A thumbs-down tells you a user was unhappy. An abandoned or fallback conversation tells you where the agent had nothing to offer. Pull a sample of both from the same period and read them side by side before you touch a single topic.

Then sort the sample by what happened at the moment the conversation went wrong:

• The agent did not recognise the request at all. This points to missing phrasings, or to a topic that does not exist yet.
• The agent recognised the wrong request and opened a neighbouring topic. This usually means two topics overlap, and adding phrasings to one will make the other worse.
• The agent understood and answered, but the answer was stale or incomplete. The fix sits with the knowledge owner, not the designer.
• The agent understood and answered correctly, but the user wanted a person. That is an escalation-rule question, covered in the guide to when a Virtual Agent should hand off.

Change one group at a time. If you add phrasings, rewrite a flow and edit knowledge in the same release, next week’s numbers cannot tell you which change helped. One group per cycle keeps the cause visible, and the cycle can stay short because each group is a small edit.

Check overlap whenever a topic is added or retrained. A new topic with broad phrasings can quietly capture requests that a well-tuned neighbour used to handle. The explanation of how a Virtual Agent topic works helps when you are deciding whether two topics should be merged or kept apart.

Where the agent answers from articles, the quality of those articles caps the quality of the agent. If the sample keeps landing on weak content, the work belongs in knowledge and self-service optimisation before any more tuning.

Keep a short log of each cycle: the group you changed, the number of conversations it should affect, and what you expect to see. Review it against the next sample. A change that moved nothing is still a result, because it tells you the cause lay somewhere else.