Troubleshooting ServiceNow Upgrades and Release Readiness: A Practical Guide
Pre-upgrade assessment and planning
ServiceNow releases new versions periodically. Upgrades bring new features, security patches, and sometimes breaking changes. CIO office analysts planning upgrades must assess readiness before committing to a date. ServiceNow provides an Upgrade Health Dashboard that identifies known compatibility issues with your current configuration. Review the dashboard report for your instance. Issues marked as high severity should be resolved before upgrade. Issues marked as low severity can often be addressed after upgrade if necessary. Also inventory custom code and integrations: third-party applications, REST API consumers, and custom flows depend on ServiceNow APIs. Check whether those applications are compatible with the target release. Vendor release notes often clarify compatibility.
Backup and recovery procedures
Before any upgrade, ensure a backup of the current instance exists and can be restored. ServiceNow automatically backs up instances before scheduled upgrades, but verify the backup is available in your backup retention system. Develop and test a rollback procedure: if the upgrade fails catastrophically, how will you restore to the previous version? Practice the rollback in a non-production environment first. Know how long restoration takes so you can plan maintenance windows. For critical systems, having a warm standby environment ready to assume traffic can minimize downtime if the primary upgrade fails.
Staging environment testing
A staging environment running the same configuration as production should be upgraded first to test all functionality. After the staging upgrade, run a comprehensive test plan covering critical workflows: incident creation and closure, change request approval, service catalog requests, user provisioning. Test integrations with external systems to ensure they still work. Test custom reports and dashboards to confirm they display correctly. Test authentication to ensure SSO or custom authentication still works. If any test fails, investigate before upgrading production. Failed tests in staging are likely to fail in production as well.
Custom code and plugin compatibility
Custom JavaScript, workflows, and plugins may break after upgrade if they depend on internal ServiceNow APIs that changed. ServiceNow maintains backward compatibility for public APIs but sometimes deprecates or changes internal features. Before upgrade, audit custom code for dependencies on potentially changing features. Check release notes for deprecation notices. If custom code depends on deprecated features, plan to refactor it. Some custom code may work as-is, but runs more slowly after upgrade due to internal optimizations changing. Performance regression testing is important: compare transaction timing before and after upgrade for critical workflows.
Instance configuration validation
After upgrade, the instance configuration may need adjustment. Security policies may have changed. New security features may be enabled by default, breaking existing workflows. Email server configurations might need reconfiguration. Check release notes for configuration changes. Re-test access controls to ensure users still have appropriate permissions. Re-test notification settings to ensure emails still go to the right recipients. If you run multiple instances (development, staging, production), ensure all are upgraded consistently or have a documented reason for version differences.
Cutover and monitoring
Schedule the production upgrade during a maintenance window when users cannot access the system. Communicate the maintenance window clearly so users plan accordingly. Monitor the upgrade process closely. Most upgrades proceed automatically, but unusual configurations sometimes cause issues. Have a ServiceNow support contact available during the upgrade in case problems arise. After upgrade completes, run a smoke test on critical functions before allowing users back on the system. A smoke test is a quick check that key workflows work: can users log in, can they create incidents, can they view dashboards? After users return to the system, monitor for issues. Have a clear escalation path so users can quickly report problems.
Post-upgrade issues and patches
Upgraded instances sometimes experience issues that only appear under real-world load. A workflow that works fine in testing may fail when thousands of users access it. If issues appear after upgrade, gather information: which workflow or feature is failing, what is the error, which users are affected? Compare to the Upgrade Health Dashboard and release notes to see if this is a known issue. ServiceNow releases patches to fix discovered issues, so stay current with patches. Also use release notes to understand any behavioral changes that affect users. A new default setting may change how users interact with forms or dashboards, requiring user communication and possibly training.
Start from the symptom, then choose which layer to inspect
Upgrade problems are easier to diagnose when they are sorted by what the team observes rather than by where they appear in the plan. The table groups the common symptoms by layer, so that the first place to look is set by what is failing rather than by the last test that ran.
| What is observed | Layer to inspect first | Evidence to collect before changing anything |
|---|---|---|
| The upgrade process halts or reports an incomplete step | The upgrade job itself and its step log | The step name, the time it stopped and the last successful step |
| An error appears only for certain users or roles | Access control rules and role inheritance | The affected role list and one failing record’s access trail |
| A script runs but produces a different result than before | Custom code that depends on a platform behavior that changed | The script’s input, its output on both versions and the release note that describes the change |
| Everything works but is noticeably slower | Query patterns, indexes and background loops | Timed transactions for the same action, before and after the window |
| An external system stops receiving or sending data | The integration’s credentials, endpoint and message definition | The last successful message and the response it received |
When an upgrade stalls partway through
A stalled step is tempting to re-run, and re-running is often the wrong first move. A second attempt against a half-changed instance can make the problem harder to read. The sequence below protects the evidence while the call is being made.
- Freeze the state. Stop any further changes on the instance, including fixes that someone is tempted to make in the middle of the window.
- Record the step name, the time it stopped and the last step that completed. Write these down before anyone speaks to the vendor, because the vendor will ask for them.
- Open a support case with the time-stamped evidence, and ask the support engineer to confirm whether the stalled step can be resumed or must be restored.
- Agree in advance how long the team will wait before it decides between resuming and restoring, and write that limit into the runbook so the call is not left to whoever is loudest.
- Communicate the status to the business owner at fixed intervals, including when there is nothing new to report.
When staging passes and production fails
A failure that appears only in production usually has one of four causes. Data volume differs, so a query that was instant on a small copy takes long on the full table. Concurrency differs, so a race condition that never occurred with one tester appears under real load. An environment property differs, such as a mail server name, a certificate or an endpoint address that was never copied across. Or a scheduled job runs at a time that the staging refresh did not reproduce. Start by checking which of the four applies, because the fix for each is different and the wrong one wastes the window.
When the problem is slowness rather than an error
Slowness is harder because nothing reports as failed. Compare timed transactions for the same action before and after the window, using the same user role and a similar record. If one form or list is slow in isolation, look at the queries it issues rather than the form itself. If the whole instance is slow, check background activity first, since a loop that was harmless on the old release may now consume more capacity. The platform performance guidance sets out the usual sources of this kind of load.
When to stop troubleshooting and call the rollback decision
Troubleshooting has a natural end point, and it should be named before the window opens. Call the rollback decision when a critical process cannot be made to work within the agreed time, when data integrity is in question, or when the cause is a platform defect with no workaround and no vendor date for a fix. Continuing to troubleshoot past those points tends to produce a longer outage and a rollback that is harder to perform, because the instance has accumulated changes that the restore will not reverse cleanly.
The testing and acceptance checklist for upgrades lists the processes that count as critical, and the roles view shows who makes the call. When the failure involves a workflow that moves work between teams, the guidance on coordinating a response across teams covers how to keep the response from fragmenting.
Frequently asked questions about upgrade troubleshooting
Should a restart be done after a failed step?
Not without the platform support team’s agreement. A restart can clear a transient fault, but it can also interrupt a step that was about to complete and leave the instance in a state that is harder to read. Ask before restarting, and record the answer.
How do we know a fix is complete?
Re-run the test that found the fault, then run the neighboring tests that depend on the same component. A fix that passes its own test but breaks a neighbor is a common outcome when the fault is traced to a shared setting.
Who should be on the bridge during a failed window?
The platform owner, the release lead, one developer for each failed component and a representative of the business process at risk. Keep the group small enough to make decisions quickly, and have someone whose only job is to keep the evidence log current.
Troubleshooting ServiceNow Discovery false failures after an upgrade
Discovery failures right after a ServiceNow upgrade are usually not real device problems. They typically trace to a MID Server that has not finished upgrading, a credential now tried in a different order, a customized pattern or probe that did not carry over, or a timeout the run now exceeds. Check them in that order.
Start with the MID Server, because every other check depends on it. After an upgrade it normally upgrades itself to match the instance, and a run started during that window can fail healthy devices. Confirm each MID Server is up, validated and on the instance’s version before reading a failure.
Then pick one device that failed but was discovered cleanly before the window, run Discovery against it alone and read its log. The error text sorts the problem.
| What the log shows | Likely cause | First check |
|---|---|---|
| Authentication failed on a device that worked before | Credential order changed, or a credential was altered | Which credential the log shows being tried |
| Device found but classified as a generic type | A customized pattern or probe was not applied | The skipped changes recorded during the upgrade |
| Timed out or no response | The network path or run length changed | The MID Server’s connection to that subnet and the time limit |
| Duplicate or missing configuration items | An identification or reconciliation rule changed | The rule set, tested on one record |
Widen the test only after that device passes. Re-run one subnet and compare the devices found with the last clean run. A gap of a few devices points to individual cases; a gap across a subnet points to the MID Server or the credential.
The Discovery guide explains credential and schedule settings, the MID Server guide covers its health and version, and Discovery and Service Mapping delivery shows how these runs are validated. If failures spread beyond Discovery, see platform performance and upgrade readiness.
Can a MID Server cause Discovery failures right after an upgrade?
Yes. A MID Server that is still upgrading, has lost its validation or runs a different version from the instance can fail healthy devices, so confirm its status and version first.
How do I tell a false failure from a real one?
Test the same device by a method that does not use Discovery, such as a direct sign-in with the same account from the MID Server’s network. If that works and Discovery fails, the cause sits in the Discovery configuration. If both fail, the device or the network has changed.
Should I re-run the full schedule to see whether the failures clear?
Not first. A full run on a half-understood problem adds noise and can change the records you want to compare. Fix the single device, confirm one subnet, then run the full schedule.
Frequently asked questions
What should we do if the Upgrade Health Dashboard shows high-severity issues?
Resolve high-severity issues before upgrade. They often indicate custom code or configurations incompatible with the new release. Work with ServiceNow support to understand the issue and fix it in staging before upgrading production.
How do we rollback if the upgrade fails?
ServiceNow provides backup snapshots taken before upgrade. If the upgrade fails, restore from the backup. Test this procedure in a non-production environment first to know how long restoration takes.
Why might custom workflows break after upgrade?
Custom workflows may depend on internal ServiceNow APIs that changed. Check release notes for deprecation notices. Audit custom code before upgrade to identify potential issues. Refactor code that depends on deprecated features.
Related guides
Related Oblytech services
Where this topic turns into delivery work, these are the Oblytech services that cover it.
Planning, running or fixing a ServiceNow programme? Tell us where you are and we will suggest a sensible next step.
Contact us