ServiceNow AIOps Requirements Workshop Questions
Understanding AIOps Capability and Readiness
AIOps correlation and prediction depend on the quality of the monitoring events and incident records already in place, so a requirements workshop has to test that foundation before anyone commits to a design. The questions below cover monitoring and alert quality, ITSM process and data quality, team readiness, business goals and success measures, and integration and governance, followed by a two-session procedure, a decision table for reading the answers and the signs that the workshop has run its course.
Key questions address your data foundation. What monitoring systems do you have in place? How much historical data have you collected? Are your alerts tuned accurately or do you receive excessive false positives? How well do your incident records reflect what actually happened? Without good data, AIOps models will not work well.
Questions About Monitoring and Alert Quality
Does your organization use integrated monitoring or scattered point solutions? Integrated monitoring provides consistent data; scattered solutions create silos. How many monitoring systems feed data? How much of your infrastructure is covered by monitoring? Are there gaps where critical systems are not monitored?
How good is alert tuning? False positive alerts waste team time and make real issues harder to spot. Teams often disable poorly tuned alerts rather than fixing them. If your organization has high false positive rates, alert tuning should happen before AIOps implementation.
How are alerts currently routed to teams? Are routing rules clear and tested? Do teams know when they receive an alert whether it indicates a real problem requiring investigation? Alert routing logic directly affects whether AIOps alert correlation will work effectively.
Questions About ITSM Process and Data Quality
How consistently are incident records categorized? If incident classification is inconsistent, pattern recognition by AIOps models will fail. Do incident records capture enough information for analysis? Root cause, resolution time and impact level should be captured consistently. Do changes get linked to incidents when they fix problems? This linkage helps AIOps learn relationships between changes and incident reduction.
How quickly are incidents typically resolved? If incident resolution is slow, AIOps impact assessment may be delayed. How often do similar incidents occur? Recurring incidents indicate problems that AIOps should identify for proactive investigation.
Questions About Team Readiness and Skills
Does your organization have staff with machine learning and analytics skills? AIOps models require specialists to interpret results and tune models. Do you have staff who can understand AIOps recommendations and decide whether to act on them? AIOps provides recommendations; human judgment still decides whether to implement recommended changes.
Does your organization have automation capabilities? AIOps often recommends automating tasks; staff must have the skills and tools to implement automation. Do teams currently automate operational tasks or is everything manual? Starting from scratch with automation is harder than enhancing existing automation.
Questions About Business Goals and Success Metrics
What specific problems is the organization hoping AIOps will solve? Faster incident resolution, reduced alert fatigue, proactive problem detection? Clear goals help focus AIOps implementation on high-value areas. What metrics will demonstrate success? How will you measure whether AIOps improved operations?
What is your timeline for seeing results? Realistic expectations are important. Initial AIOps deployments take time before they deliver measurable results, and the length depends on scope and data quality. Setting overly aggressive timelines creates disappointment.
Questions About Integration and Governance
How will AIOps fit with existing ITSM processes? Will AIOps enhance incident management or replace parts of it? How will teams transition from current processes to AIOps-enhanced processes? How will governance and approval of AIOps recommendations work? Who decides whether to act on AIOps recommendations for automation or changes?
AIOps implementation is a journey rather than a one-time project requiring continued learning and adaptation as your organization develops machine learning capabilities.
Requirements workshops help prevent costly mistakes in AIOps implementation by identifying readiness gaps early when they can still be addressed through preparatory work before committing to full implementation. Careful assessment of readiness through structured workshop questions prevents organizations from investing in AIOps before foundations are solid enough for success.
Running the workshop so the answers can be trusted
A requirements workshop for AIOps produces either a real decision or a list of pleasant opinions, and the difference is mostly in how it is run. The person facilitating needs to separate questions that can be answered from records from questions that can only be answered by opinion. Where a question can be answered from records, ask for the record to be brought to the room. Where it can only be answered by opinion, record the opinion, name the person who gave it, and mark it as unverified. Workshops that skip this step tend to produce a requirement document that everyone signed and nobody can defend when the pilot starts.
Invite the people who will be on the receiving end of the output: the on-call lead who will see correlated groups at three in the morning, the monitoring owner whose alerts will be grouped, and the person who maintains service relationships. A senior sponsor is useful for setting the goal, but a workshop attended only by the sponsor will describe the program the sponsor hopes for, not the estate that exists.
Ordered procedure for a two-session workshop
- Before the first session, ask each invitee to bring one recent incident they consider mis-handled by the current process, with its ticket reference, so that examples drive the discussion.
- In the first session, walk through those incidents and ask what the platform would have needed to know at the time. Write each need down as a plain statement, such as which service an event belongs to.
- Sort the needs into three piles: data that exists and is trusted, data that exists but is not trusted, and data that does not exist. The third pile is the most expensive and should be discussed first.
- Between sessions, check the first pile against an actual extract. Confirm that a sample of records really contains the fields described.
- In the second session, decide which needs the pilot must satisfy, which can wait, and which should be dropped. Each dropped need needs a reason written beside it.
- Send the decisions back to the attendees within the same week, with the open items and an owner for each.
Reading the answers: a decision table
Answers to workshop questions tell you about readiness, and some answer patterns are more informative than the answers themselves. The table below lists common patterns and what a facilitator should do with each.
| Pattern in the room | What it often means | Suggested response |
|---|---|---|
| Everyone describes the same incident the same way | Shared understanding of the failure, possibly thin evidence | Ask for two other examples before accepting the pattern |
| The monitoring owner and the incident manager disagree on what an alert means | Alert semantics are undefined | Treat it as a blocking item for correlation requirements |
| Nobody can say where service relationships are kept | Topology is missing or unowned | Defer topology-based correlation and plan the data work first |
| Answers come only from the most senior attendee | Requirements reflect aspiration rather than practice | Ask the on-call lead the same questions in a separate short session |
| Requests keep growing with each example | Scope is becoming a catalog of wishes | Force a ranking and keep only the top few for the pilot |
Questions to ask about the people and their current workload
Many workshops ask about data and tools and skip the people who will absorb the change. Ask how many people currently take turns on the rota that would receive correlated groups, and how often they hand work over mid-incident. Ask which responders already keep private lists or spreadsheets to compensate for the current tooling, because those lists describe the real requirements better than any formal document. Ask who has the authority to tell a responder to ignore a suggested group. If that question has no clear answer, the requirement for governance is larger than the requirement for technology.
Signs the workshop has run its course
A workshop is finished when the open questions are all owned by someone outside the room, and when the attendees can repeat the agreed scope back without referring to their notes. Be wary of a workshop that ends with unanimous enthusiasm and no open items, because that often means the difficult questions were not asked. Equally, a workshop that ends with a long list of blockers is not a failure. It is an accurate picture of readiness, and the right next step is preparatory work before any pilot is committed.
Questions organizers ask after the first workshop
How long should the gap be between the two sessions?
Long enough for the records checks to happen, and short enough that the group still remembers the examples. The gap is often set by how quickly the data owners can produce extracts, so ask them for a date before the sessions are scheduled.
Should the workshop include vendors or implementation partners?
They can attend the second session to answer technical questions, but they should not set the requirements. A partner who writes the needs list is describing their preferred solution, which is useful information about the partner and not about your estate.
What do we do with requirements that depend on data we do not hold?
Record them as dependencies with an owner and a condition for reopening them. Do not drop them silently, because the same gap will reappear when the pilot is reviewed and nobody will remember why it was left out.
Is one workshop enough for a multi-team estate?
Rarely. Run the core workshop with the platform and incident leads, then hold shorter sessions with each team whose services will be correlated. Teams that were not in the room often reject groupings that the core group approved, and a short session earlier is cheaper than a dispute later.
The AIOps roles and RACI guide is the natural next document once the requirements are settled, because the same people will need named decisions. Teams that want a readiness review before the workshop can look at the assessments and readiness service as a starting point.
Frequently asked questions
Can we start AIOps without fixing alert tuning first?
AIOps works better with well-tuned alerts but does not require perfect tuning to start. Many organizations improve alert quality as a side effect of implementing AIOps. However, extremely high false positive rates may overwhelm AIOps analysis. Starting with basic tuning improvements helps.
How much historical data does AIOps need to learn from?
AIOps implementations benefit from a useful depth of historical monitoring and incident data, and the depth needed depends on how varied your incidents are. Models built on limited data are less accurate. If your history is thin, collect more before expecting AIOps to deliver value.
Do we need specialized skills to implement AIOps?
AIOps vendors provide tools that limit how much machine learning expertise is required. However, interpreting results, tuning models and implementing recommendations still benefits from analytics skills. Many organizations work with vendors or consultants during initial implementation.
Related guides
Related Oblytech services
Where this topic turns into delivery work, these are the Oblytech services that cover it.
Planning, running or fixing a ServiceNow programme? Tell us where you are and we will suggest a sensible next step.
Contact us