Common ServiceNow Major Incident Management Mistakes and How to Avoid Them
Mistake 1: No Clear Definition of What Qualifies as Major
Symptom: every incident is escalated as major. Your major incident process is triggered dozens of times a day. No one takes major incidents seriously anymore.
Root cause: your major incident criteria are undefined or so loose that anything important seems to qualify. Without clear criteria, people escalate based on gut feeling or customer pressure.
Fix: define major incident criteria in writing. For example, “an incident affecting a business-critical system and multiple users for an extended period” is major. “An isolated system affecting five users” is not. Apply the criteria consistently. If criteria change, update them and communicate the change.
Prevention: review major incident classification monthly. If you are classifying many incidents as major each month, your criteria are too loose. Tighten them or accept that your organisation has many critical systems.
Mistake 2: Unclear Escalation Path and Decision Rights
Symptom: when a major incident occurs, no one knows who is in charge. Multiple teams are waiting for direction. The incident drags on because decisions take hours to make.
Root cause: role definitions are vague. Who is the incident commander? Who decides whether to take a risky remediation action? Who communicates to the customer? Without clear answers, people guess, and the incident descends into chaos.
Fix: document your major incident response structure. Incident commander makes tactical decisions. Comms lead talks to the customer. Ops lead mobilises resources. Define these roles clearly and assign people to them before incidents occur.
Prevention: run a tabletop drill quarterly. Walk through a simulated major incident. See who acts and who waits. Identify confusion points and clarify roles.
Mistake 3: Poor Initial Communication and Unclear Timelines
Symptom: hours pass without any communication to affected customers. When you finally send an update, it contradicts what you said two hours earlier. Customer frustration spikes.
Root cause: communication is not prioritised during incident response. People focus on fixing the system and forget to update stakeholders. Or different teams make contradictory statements.
Fix: assign a comms lead immediately when an incident is escalated to major. This person’s only job is to send updates regularly (at intervals suited to the situation). Updates do not need to be detailed—just “we are investigating,” “we have a hypothesis,” “we are implementing a fix”—but they must be regular and consistent.
Prevention: have a comms template ready. Use the same structure for every update so people know what to expect. Include: what we know, what we are doing, ETA on resolution or next update.
Mistake 4: No Postmortem or Learning from the Incident
Symptom: the same major incident happens three times in six months. The response was the same each time; nothing changed. Teams feel like they are fighting the same fire repeatedly.
Root cause: no postmortem process or the postmortem identifies symptoms but not root causes. Without root cause analysis, you fix the immediate problem but not the underlying issue.
Fix: schedule a postmortem soon after resolution, while details are fresh. Invite the incident commander, the team members who responded, and a neutral facilitator. Focus on facts, not blame. Identify root cause (not just “the database was slow” but “we did not index the new query column”) and action items to prevent recurrence.
Prevention: track major incident postmortems in ServiceNow. Link action items to work requests. Follow up periodically to confirm they are completed. Measure the effectiveness: has the problem recurred?
Mistake 5: Not Having a Documented Runbook Before Things Break
Symptom: when the database goes down, the response team spends a long time figuring out how to restart it, though the fix itself is quick.
Root cause: critical runbooks are in people’s heads or scattered across wikis. When you need them under pressure, they are hard to find and remember.
Fix: document major incident response procedures for your critical systems. Include: how to detect the problem, diagnostic steps, remediation options, rollback procedure, and who to notify. Keep runbooks in your knowledge base and link them to your major incident management workflow.
Prevention: when a major incident occurs, document the resolution steps in the incident record. After postmortem, turn those steps into a runbook and publish it. Next time that problem occurs, you are faster.
Mistake 6: Escalating Without Context
Symptom: an incident is escalated to the infrastructure team without information about what was tried already. The infrastructure team starts from scratch, wasting time.
Root cause: escalation procedure does not require documentation. Teams pass incidents up the chain without clear context.
Fix: require a handoff note in every escalation. What symptoms was seen? What was already tried? What is the current hypothesis? Who is available on the escalated team? This note is included in the escalation ticket, not emailed separately.
Prevention: include a template in your major incident workflow. Force the escalating team to fill it in before they can escalate. This discipline saves time on the back end.
Declaring recovery before the service has really recovered
The all-clear can be the most dangerous message in an incident, because everyone stops watching when it arrives. A common pattern is that the immediate symptom disappears, a responder marks the service as restored, and the team stands down. An hour later a queue that was backing up during the outage begins to fail again, or a dependent service that was never checked reports errors. The fix for this is not a longer wait before every all-clear. It is a short, written check that names what was measured, by whom, and against which baseline. If the check cannot be written down, the service has not yet been shown to be restored.
The pressure to declare recovery early often comes from the people affected, who want the disruption to stop being discussed. Consider a team where the login page has returned after an outage and the operations channel has gone quiet, while the password reset flow behind it still fails for some users. If the record is closed on the strength of the login page alone, the reset failures will arrive as a new incident the next morning. A recovery checklist that names the dependent journeys can catch it before anyone stands down.
Parallel fixes with no single record of what is being tried
Large incidents attract several engineers who each start a fix on their own. Without a shared record, two people can roll back the same change, one can undo the other’s workaround, and neither can explain afterwards which action actually restored the service. The problem is not the number of people. It is the absence of a single place where each attempted action is written down with its owner, its time and its result. Make that place the incident record, and make the commander responsible for keeping it current rather than relying on a chat thread that will be hard to reconstruct.
| Mistake | Early warning sign | Check to run |
|---|---|---|
| Declaring recovery on a single signal | Only one symptom has been rechecked | Confirm each dependent journey named in the recovery checklist |
| Parallel fixes with no shared record | Two responders describe different current states | Reconcile the record against what each responder is doing before the next action |
| Closing the incident before the problem record exists | No problem reference on a major incident closed as resolved | Link or create the problem record before closure |
| Stakeholder list that is out of date | Updates go to people who have moved teams or left | Check the distribution list against current roles at the start of each event |
| Automation failure that goes unnoticed | Notifications stop but no one has been told | Confirm that automation runs have a visible success or failure status |
Stakeholder lists and channels that quietly go stale
Distribution lists are rarely reviewed until they are needed in an emergency, which is the worst time to discover that they are wrong. People change roles, business units reorganize, and a shared mailbox loses its owner. The effect during an incident is that updates reach the wrong people or nobody senior at all. Review the stakeholder list on a fixed schedule and at each change in the operating model, and make the list the source for notifications rather than a separate document. Where a channel is used for updates, check that it is the one the stakeholders actually read.
Pre-incident checks that prevent the common failures
Many of the mistakes above are visible before an incident starts. A short routine run each quarter, and before any planned high-risk change, will catch many of them.
- Confirm that the escalation criteria and the named commander role are current, and that the people named are still employed in those roles.
- Check the stakeholder distribution list against the current organization and remove anyone who no longer needs major incident updates.
- Test one notification path end to end and confirm that a failure would be visible to someone who can act on it.
- List the recovery checks for the three most important services, including the dependent journeys that customers use.
- Confirm that the problem management route for major incidents is working, so that a closed incident can be linked to a problem record without delay.
When a mistake is a training problem and when it is a design problem
A repeated mistake is a design problem when the process makes the wrong action easier than the right one, for example when closure is one click and the problem link is optional. It is a training problem when the process is sound but responders do not know what it asks of them. Telling the two apart matters, because training will not fix a form that allows the wrong closure, and a redesigned form will not fix a commander who has never practiced the role. The review after each incident should record which kind of problem occurred, so that the fix goes to the right owner.
Is a single missed step enough to call a process broken?
Often not. Look for whether the same step is missed in more than one event, or whether the miss had a material effect on customers. One-off misses are often handled through feedback to the responder, while repeated ones point to design.
Should responders be allowed to reopen a closed major incident?
Yes, if the service is found to be degraded again. Reopening is better than opening a new record, because the timeline and the earlier actions stay together and the review can see the whole picture.
How do we avoid alert fatigue among stakeholders during a long event?
Agree in advance which updates are needed and which are noise. Send scheduled updates with a clear next time, and reserve unscheduled messages for material changes. The major incident communication note covers update timing and message discipline.
What evidence should a review keep about the recovery decision?
The checks that were run, the people who ran them, the time of each check and the result. That evidence is what allows a later review to say whether the all-clear was justified, and the note on change validation evidence sets out the same kind of record for planned work.
Next step
The incident swarming coordination guide covers bringing specialists together with a shared lead and context, and the roles and RACI guide names who holds each decision.
Frequently asked questions
How long should a major incident postmortem take?
A brief session works best. You are not writing a doctoral thesis. Focus on: what happened, why it happened, and what we will do differently. Capture action items and assign them to owners. Done is better than perfect.
Who should attend the major incident response team?
Incident commander (who runs it), comms lead (who updates stakeholders), ops lead (who mobilises teams), and subject matter experts for the affected system. Keep it small enough to make decisions quickly. If you have 20 people in the incident room, you will spend an hour debating every decision.
What if we disagree about whether an incident is major?
Go with the conservative approach: treat it as major. You can always downgrade it if new information proves otherwise. Downgrading is cheaper than escalating a major incident after the fact and losing customer trust.
Related guides
Related Oblytech services
Where this topic turns into delivery work, these are the Oblytech services that cover it.
Planning, running or fixing a ServiceNow programme? Tell us where you are and we will suggest a sensible next step.
Contact us