Troubleshooting ServiceNow Predictive Intelligence: A Practical Guide

Troubleshooting Predictive Intelligence starts from the symptom: no predictions at all, predictions that are consistently wrong, predictions for some incidents but not others, staff ignoring them, or a model that never retrains. Each section below pairs a symptom with the records to inspect (system properties, model status, training data), followed by an ordered isolation procedure, escalation guidance and how to stop a bad output while the fault is found.

Predictions Not Appearing for Any Incidents

If Predictive Intelligence is installed but no predictions appear in incidents, first verify that the module is enabled. Check System Properties for Predictive Intelligence settings; the module may be configured but not activated. If the module is active, check whether a model exists. Predictive Intelligence requires at least one trained model to make predictions. If no model exists, the module will not show predictions.

Check the model status. Open the Predictive Intelligence model configuration and verify the model’s status is ‘Active’ and ‘Trained’. A model in ‘Draft’ or ‘Failed’ status will not make predictions.

Predictions Consistently Wrong

If Predictive Intelligence makes predictions but they are often incorrect (category is wrong, time estimate is way off, assigned team cannot help), the model is likely trained on bad data or is out of date. Check the model’s training data: does it include only recent incidents, or does it include very old incidents from before the business process changed? If the training data is too old, the model is learning outdated patterns.

Retrain the model using only recent, high-quality incidents (incidents that were categorised correctly and were assigned to the right team). This may improve accuracy significantly.

Predictions Appearing for Some Incidents but Not Others

If Predictive Intelligence makes predictions for simple, common incidents but not for complex incidents, this is usually expected behaviour. Models are typically more accurate on high-volume incident types (password reset, simple requests) and less accurate on rare or complex incidents. This is not a failure; it is expected performance.

However, if predictions are missing for incidents where you would expect them (P1 incidents, for example), the issue may be a model configuration that restricts predictions to certain incident types. Check the model’s ‘Applicable To’ configuration to see which incidents the model covers.

Support Staff Ignoring Predictions

If support staff are not using Predictive Intelligence suggestions even though they appear in incidents, the issue is likely low adoption or low confidence in the predictions. Check prediction accuracy first: if accuracy is low, staff will naturally ignore suggestions. If accuracy is acceptable but adoption is still low, the issue is awareness or trust. Conduct training on how to use recommendations and build confidence through small wins (early successes with high-accuracy predictions).

Model Not Updating or Retraining

If a model is old and accuracy has degraded, it should be retrained on recent data. Check the model’s configuration to see if automatic retraining is enabled. If it is not, manually trigger a retrain. Go to the model record, click ‘Retrain’, and select recent incidents as the training data source.

Be careful when retraining: if recent incidents include incorrectly categorised or incorrectly assigned incidents, the model will learn from those mistakes. Clean the training data (verify that recent incidents were correctly categorised and assigned) before retraining.

Lookups and Inspection Points

• Predictive Intelligence Module: Check whether it is active in System Properties; if it is installed but not active, no predictions will appear
• Model Configuration: Status should be ‘Active’ and ‘Trained’; ‘Draft’ or ‘Failed’ models will not make predictions
• Model Applicable To: Check which incident types the model applies to; if a specific incident type is excluded, predictions will not appear for it
• Training Data Quality: If model accuracy is poor, investigate the training incidents; check whether they were correctly categorised and assigned
• Model Age: Check when the model was last trained; if training is old and incident patterns have changed, accuracy will degrade

When to Escalate

If a model is trained and active but no predictions appear, there may be a system issue with Predictive Intelligence. If predictions are being made but are consistently wrong and retraining does not help, the model may not be suitable for your incident patterns. Escalate to the platform team with model configuration and sample predictions showing the inaccuracy.

Preventive Measures

• Set up a retraining schedule: retrain models quarterly or when accuracy falls below target
• Monitor prediction accuracy weekly; catch degradation early
• Clean training data before retraining; remove miscategorised or misassigned incidents
• Track adoption by team; if one team ignores predictions, investigate why and provide targeted training

Testing Predictions in Staging Before Production

Before activating predictions in production incidents, run a pilot in a staging environment. Submit test incidents and verify that predictions appear and are reasonably accurate. This allows you to catch configuration errors or data quality issues before real support staff depend on the predictions.

Silent failures are harder than missing predictions

A prediction that never appears is annoying but obvious. The harder faults are the ones where output keeps appearing but has quietly stopped meaning what it used to. A scoring job can fall behind so that predictions reflect the state of a record from hours earlier. A field mapping can be altered during an unrelated form change, so the score is written to a field that users read as something else. A retrain can improve one category while degrading another, and the average looks fine. Troubleshooting should start by asking which of these faults could be present, because each one needs a different kind of evidence.

Keep a short fault log with the record reference, the time the prediction was shown, the value shown and the value the record held at that moment. Without that trio it is almost impossible to tell whether a complaint is about the model, the data or the display.

An ordered isolation procedure

Work from the outside in. Each step either confirms the layer is healthy or points to the layer that needs attention, and it is faster than reading configuration screens at random.

1. Reproduce the complaint on a single record, and capture the prediction value and the time it was displayed.
2. Check that the scoring step ran for that record after its last relevant change, using the job history rather than the output field alone.
3. Confirm that the output field the user sees is the field the model writes to, and that no later form or UI change has redirected it.
4. Compare the input values the model received with the current values on the record, to find stale or missing inputs.
5. Only then examine the model itself: its training window, its last approved version and its performance on the affected category.

Stopping at step two is a common mistake. Teams often rebuild a model to fix a problem that was really a job that never ran.

Symptoms that point to different causes

The table pairs common symptoms with the layer they usually point to. Treat it as a first guess to test, not as a diagnosis, because several causes often overlap.

When the fault is in the process, not the platform

Some complaints are real but the platform is working as designed. A prediction that is accurate about the historical pattern can still look wrong to an experienced agent who knows a new dependency has appeared. A user who overrides output without a reason makes the next training set harder to interpret. In these cases the fix is in the process: a reason field that is required on override, a short briefing when the underlying service changes, or a rule that tells reviewers when to ignore output during a known disruption.

Logging what an escalation needs

When the problem has to go to a platform or vendor team, a bare complaint wastes everyone’s time. Attach the fault log entries, the job history for the affected period, the model version identifier, a sample of five to ten records with their inputs and outputs, and a plain statement of what the user expected. An escalation with that evidence can be triaged in a single review. One without it tends to be sent back. Where the fault sits in an integration, the integration monitoring ownership notes cover who owns failures in an integration.

Stopping a bad output while the fault is found

Set a clear rule for pausing output during investigation. The pause should be a configuration change that the queue lead is authorized to make, it should be recorded in the fault log, and it should have a named person who reviews it the same day. Pausing output is safer than leaving a known-wrong prediction visible, but an unrecorded pause creates a second problem, because nobody knows later that the predictions were off for a period.

Questions that come up during an investigation

How do we tell a model problem from a data problem?

Look at whether the affected records had complete inputs at scoring time. If they did and the output is still poor, the model is the likelier cause. If inputs were missing, stale or recently changed, fix the data path first and then reassess.

Can we retrain as the first response to any complaint?

It is rarely the right first move. A retrain on the same data with the same fault will reproduce the problem and may hide the evidence. Confirm the cause, then decide whether a retrain is needed.

How long should an investigation stay open?

Set a review point after which the fault is either resolved, moved to a named owner, or accepted as a limitation with the output suppressed. An investigation with no review point tends to become the permanent state.

What should be told to frontline staff while the fault is open?

Tell them what is wrong, which categories are affected, whether output is paused, and what they should do instead. Silence can make staff distrust output even after a fix. The message pattern in major incident communication can help with this.

A focused review of that evidence is often the quickest way back to a working queue.

Related guides: ServiceNow Predictive Intelligence Testing and Acceptance Checklist.