OBLYTECH Logo
OBLYTECH Logo
Troubleshooting ServiceNow Major Incident Management: A Practical Guide
Home / Guides, Services and Solutions / IT Service Management: Guides and Services / Major Incident Management: Guides and Services / Troubleshooting ServiceNow Major Incident Management: A Practical Guide

Troubleshooting ServiceNow Major Incident Management: A Practical Guide

This guide helps service teams find out why a major incident response stalled or never started, and which fix belongs to each stage. It sets out common symptoms, the root causes behind them, a diagnosis path and a fix for each stage, from the first alert to the post-incident review.

Incident Not Being Created or Marked as Major

Symptom: An outage occurs, but no Major Incident record is created in ServiceNow. Monitoring systems may be firing alerts, but incident creation automation is not triggering. Root causes: workflow automation is disabled, event-to-incident rules are not matching, or incident routing rules are sending incidents to the wrong queue.

Diagnosis: Check the Event Management dashboard for raw monitoring events coming into ServiceNow. If events are arriving, check event-to-incident correlation rules. In ServiceNow, navigate to Monitoring > Event Rules and review the rule definitions. Is there an active rule matching the events coming from your monitoring tool? Has the rule been edited recently and disabled by accident? Test the rule by manually triggering a test event and verifying incident creation.

If events are not arriving, the issue is upstream: monitoring is not sending events to ServiceNow, or the integration connection is broken. Check ServiceNow integration logs (System Logs > Integration Hub logs) for connection errors. Verify that the monitoring tool has ServiceNow API credentials and that the target endpoint URL is correct and reachable.

Fix: Re-enable the event-to-incident rule or repair the integration connection. If the rule was disabled, check for error messages explaining why it was disabled (e.g., API rate limit exceeded, authentication failed). Address the root cause before re-enabling the rule.

Escalation Chain Not Triggering

Symptom: A major incident occurs, but executive notification or escalation does not occur. Senior IT leadership is surprised to learn about the outage from customer calls instead of incident notifications. Root causes: escalation workflow is disabled, notification recipients are incorrect, or escalation criteria are not matching the incident.

Diagnosis: Open the Major Incident workflow definition (Service Management > Major Incident Management > Workflow > Escalation). Is the workflow active? Is there an escalation task with notification rules? Check the recipient definition: Are email addresses current? Are distribution groups defined correctly?

Test the escalation manually: create a test major incident with high severity, and verify that notifications are sent. Check email logs (System Logs > Email Log) to confirm that emails were generated and sent. If emails were generated but not received, the issue is email delivery (SMTP configuration, firewall, spam filtering).

Fix: Verify notification recipients and re-enable the escalation workflow if it was disabled. Test end-to-end notification delivery with a non-critical test incident. If email delivery is slow or unreliable, consider alternative notification methods (SMS, Slack integration, mobile app push notifications).

Communication Updates Not Reaching Customers

Symptom: Incident managers post status updates in ServiceNow, but customers do not receive them. The Service Portal may show outdated information, or customer communication templates are not being sent. Root causes: Service Portal refresh is not configured, customer notification automation is disabled, or communication templates are missing or incorrectly configured.

Diagnosis: Verify that Major Incident updates are configured to trigger customer notifications. Check the Major Incident communication automation rules. Test by manually posting a status update and checking whether a notification is generated. If notifications are not generating, verify the rule conditions: Does the incident meet the criteria for customer notification? Is the notification workflow active?

Check the incident record’s customer contact information. Are customer email addresses and contact preferences recorded? If the incident has no customer contacts defined, notifications cannot be sent. Verify that the customer portal is configured to display incident updates in real-time or near-real-time.

Fix: Enable customer notification automation, verify customer contact information on the incident, and test notification delivery. Consider implementing customer status page integration (connecting to StatusPage.io or similar) so that customers see status updates even without email delivery.

CMDB Accuracy Affecting Impact Assessment

Symptom: An incident is raised as high-severity based on affected business services, but the CMDB does not reflect the actual service dependencies. For example, an outage affecting database servers may not be correctly linked to the customer-facing applications dependent on that database. Root causes: CMDB is out of date, service relationships are not defined, or incident impact calculation rules are using outdated data.

Diagnosis: Check the Configuration Management Database. Are business services defined? Are configuration items related to business services? Navigate to the affected application or service and verify the “depends on” relationships. Open the Major Incident record and check the “Affected Services” field. Are the correct services listed?

If CMDB relationships are missing or incorrect, impact assessment is unreliable. Run a CMDB audit: verify that high-criticality applications have documented dependencies, that infrastructure items are linked to applications, and that service relationships reflect the current architecture. Assign CMDB maintenance ownership to a team with access to architecture documentation.

Fix: Update CMDB relationships to reflect current architecture. Use ServiceNow Discovery or manual import to refresh CMDB data. Implement a CMDB review cycle (quarterly) to catch outdated relationships. Consider automation: if a new server is provisioned, ensure related configuration items are created and linked automatically.

Post-Incident Review Not Completed

Symptom: Major incidents close without a PIR, or PIRs are incomplete (missing root causes, action items left blank). Root causes: PIR template is not mandatory, workflow does not block closure until PIR is complete, or PIRs are created but abandoned before completion.

Diagnosis: Check the Major Incident workflow. Is there a workflow task for PIR creation that blocks incident closure? Open a closed incident and verify whether a PIR record exists. If PIRs exist, check whether they are complete: Do they have root cause analysis? Are action items defined? Are action items assigned to owners with target dates?

Review incident closure permissions. Who can close a major incident? If incident managers can close without PIR approval, the workflow is not enforced. Check role-based access control: only authorized personnel (management review) should be able to close without PIR.

Fix: Add a workflow task that requires PIR completion before incident closure. Make PIR template fields mandatory (root cause, impact assessment, action items). Assign PIR responsibility to an incident commander or quality team. Set a deadline for PIR completion (e.g., 5 business days) and escalate if overdue.

Integration Failures with Third-Party Tools

Symptom: Major Incident Management should integrate with monitoring tools, ITSM connectors, or incident communication platforms, but the integration is failing silently. Incidents are not synced, or updates are not propagated. Root causes: integration authentication has expired, API endpoints have changed, or integration workflow is disabled.

Diagnosis: Check IntegrationHub logs for the integration. Look for authentication errors, API timeouts, or malformed requests. Test the integration manually: manually trigger an integration action and check for errors. Verify that API credentials (OAuth tokens, API keys) are current and have required permissions.

If the integration was working previously and has stopped, check for recent changes: Did ServiceNow release a security update? Did the third-party API deprecate endpoints? Has the integration been disabled by a recent configuration change?

Fix: Re-authenticate the integration with current credentials. Update API endpoints if third-party services have been upgraded. Re-enable the integration if it was disabled. Test end-to-end data flow with a non-critical test incident.

Finding which stage of the response has stalled

Many failed major incident responses are not one broken setting. The response moves through a chain of hand-offs, and the failure sits at the first hand-off that nobody owns. Before you open any rule or workflow, decide which stage stopped. A declaration that never happened, a commander who was named but never engaged, and a bridge that opened without the right owners all look like “the process did not work”, yet each needs a different fix. The table below gives the first question to ask at each stage.

Stage that stalled First question to answer What to check before changing configuration Who should own the answer
Declaration Did anyone decide this was major? The incident history: was the record ever flagged, or did it stay at routine priority while the impact grew? Service desk lead
Commander assignment Who is in charge right now? Whether the assignment field is blank, and whether the named person’s shift and on-call status cover the hours of the outage Problem and major incident process owner
Stakeholder notice Did the people who must be told get told in time? The notification audience for the record, not the mailbox settings Service owners for each affected business service
Coordination Are the right technical groups working the problem together? Whether tasks were created for each group, or whether everyone is working from one chat thread Incident commander
Customer updates Is anyone outside IT hearing a current answer? Whether update intervals were agreed in advance, and whether the person posting them has access Communications lead
Closure and review Was the record closed on the basis of a finished review? Whether the review task was created at all, and who has the right to close Process owner with management sign-off

A fault-finding sequence to run after a stalled response

Work through the sequence in order. Skipping ahead to the workflow editor is a common reason a fix does not hold, because the real gap was a missing role or an unclear hand-off that no rule can supply.

  1. Rebuild the timeline from the record itself. Note the time the impact began, the time the record was first flagged, the time a commander was named and the time the first update was posted. Gaps in that timeline are your starting list.
  2. Ask the people who were on the bridge what they believed was happening at each gap. Their memory of who they expected to act is often more useful than any log.
  3. Match each gap to a stage in the table above, and write down the single owner for that stage.
  4. Check whether the gap came from automation that did not fire, or from a person who did not act. Automation problems leave traces in the integration logs. Human gaps often leave none and need a role or a checklist fix.
  5. Make one change at a time, then run a short rehearsal with a test record so the change is proven before the next one is made.
  6. Record the change and the reason for it in the process owner’s change log, so the next reviewer can see why the configuration looks the way it does.

Reading a failed rehearsal without blaming the tool

A rehearsal that stalls tells you where the process is weak, not necessarily that the platform is broken. Treat the first rehearsal after any change as a measurement. If the notice goes out but nobody on the bridge reacts for a long time, the notification is working and the on-call roster is the gap. If the notice never leaves the record, look at the trigger and the audience before anything else.

Signs the response is working, and what to do when they are not

  • The commander is named inside the response window the service owner has set, and the record shows that name before the first technical update.
  • Each affected technical group has a task on the record, and the tasks show who is working them.
  • Stakeholder notices are sent from the record rather than from personal email threads, so the audit trail is complete.
  • The review task exists before the record can be closed, and the review names owners for each action.

When these signs are missing, do not add more notification rules. Add ownership first. A notice to a larger audience does not help a response that has no one deciding what to do next.

When to leave the configuration alone

Some stalled responses are not configuration problems at all. If the same commander is the only person who ever runs the response, the process depends on one person’s availability, and no workflow edit will change that. Likewise, if the service owners disagree about which services are critical, the impact logic will keep producing different answers, and the right step is a decision meeting rather than a rule change. Recognizing this early saves a long cycle of tuning that never resolves the real question.

How often should a major incident rehearsal run?

Run a rehearsal after every material change to the roster, the notification audience or the workflow, and at a regular cadence your service owner can sustain. A rehearsal that is planned but skipped tells you the process is not yet a habit, which is itself a finding worth recording.

Should the commander be able to close the record alone?

That depends on how the review is governed. Many teams let the commander stop the active response but require a named reviewer to close the record, so that the commander can focus on recovery while the closure decision stays with someone who was not under pressure during the outage.

What if the monitoring team says the alert was sent?

Ask for the exact time and destination of the alert, then compare it with the record. An alert that reached a queue nobody watches is a routing problem, and it looks like a failed escalation from the business side.

When is it better to bring in outside help?

When the rehearsal keeps failing for reasons that sit across several teams, an independent reviewer can separate process gaps from tooling gaps faster than the internal teams that own both.

Next step

If a major incident response stalled and the cause is still unclear, get in touch to talk through the timeline. Bring the record, the rehearsal results and the list of owners, and the conversation can start from evidence rather than from a new set of settings. Related guidance on the major incident management hub and on problem management covers the work that follows a successful response.

Frequently asked questions

How do we quickly test the entire Major Incident Management workflow without triggering a false alarm?

Create a test major incident in ServiceNow with staging data (test customer, test services). Run through the entire workflow: escalation, communication, PIR creation, and closure. Verify that emails are sent, that Service Portal shows updates, and that PIR is created. Use a labeling system to mark test incidents so they are excluded from real metrics. After testing, delete test incidents or archive them separately.

What should we do if the Major Incident Management workflow is frequently timing out or slow?

Investigate workflow complexity. Is the workflow waiting for external system responses (API calls to monitoring tools, CMDB lookups)? Long timeouts can cause escalation delays. Implement asynchronous workflows: instead of waiting for an API response in the main workflow, spawn a parallel process that updates the incident when the response arrives. Monitor workflow execution times and optimize slow steps.

How do we escalate major incidents across time zones to ensure 24/7 coverage?

Configure escalation rules based on time of day and on-call schedules. ServiceNow integrates with on-call scheduling tools; escalation rules can route to the on-call engineer regardless of their location. Define escalation by geography: if an incident occurs in the Asia-Pacific region, escalate to the APAC on-call team. Implement handoff procedures so that the night-shift team can hand off to day-shift in their region.

Talk to Oblytech

Planning, running or fixing a ServiceNow programme? Tell us where you are and we will suggest a sensible next step.

Contact us