ServiceNow Performance Analytics for ITSM Testing and Acceptance Checklist

Test Layers and Coverage

Testing Performance Analytics involves multiple layers. Unit testing validates that individual calculations are correct: does average resolution time calculation give the right result for sample data? Data validation testing checks that source data is complete, accurate, and in expected format. Integration testing verifies that Performance Analytics correctly reads data from incident and problem tables and applies business rules. Performance testing ensures analytics queries complete within acceptable time even with large datasets. User acceptance testing validates that reports and dashboards meet business requirements and are easy to use.

Entry and Exit Criteria

Entry criteria for analytics testing: reports are configured, sample data exists, test environment reflects production incident patterns. Exit criteria: calculations are validated as correct, data sources are verified, performance meets requirements, users have approved reports and dashboards. Do not move to production until exit criteria are clearly met.

Comprehensive Testing Checklist

Verify calculations: run tests with known data to confirm calculations are correct. Check data completeness: ensure all incidents and problems are included in analytics, no records are missed or duplicated. Validate business rules: confirm filtering, categorisation, and calculations match business requirements. Check performance: run analytics queries against production-sized datasets and confirm they complete within acceptable time. Test edge cases: incidents with no resolution time, incidents spanning multiple months, unusual data patterns. Verify dashboard refresh: confirm analytics refresh on schedule and data is current.

Evidence and Documentation

Capture test results: for each test, document what was tested, the expected result, the actual result, and whether it passed or failed. Keep test data: maintain sample data used for testing so tests can be repeated. Document calculations: for each metric, document how it is calculated and provide an example. This documentation becomes reference material for incident managers and IT staff using the analytics.

Regression Traps and Common Issues

A common trap is timezone handling: incidents created in one timezone may be reported in another. Verify that timezone conversions are correct. Another trap is daylight savings transitions: calculations may be off on the days when time changes. Test across timezone boundaries. A third trap is incomplete incident data: if many incidents lack resolution time or assignment data, metrics may be inaccurate. Validate data quality before relying on analytics.

Acceptance Sign-Off

Before moving to production, obtain formal sign-off from incident managers and business stakeholders confirming that analytics meet requirements. Create a sign-off document listing requirements, confirming each is met, and capturing approval. This protects both IT and business stakeholders by establishing clear expectations.

Post-Launch Monitoring and Refinement

After launch, monitor analytics for anomalies: do metrics make sense, do trends seem reasonable? Collect user feedback: are reports easy to use, do they answer business questions? Refine analytics based on feedback: add new metrics, adjust filters, improve visualisations. Many organisations find that the first version of analytics is a starting point; refinement after launch significantly improves value.

Establishing Performance Baselines

Before launching analytics, establish baselines: what is current average resolution time, what is current escalation rate, what is current first-contact resolution rate? These baselines allow you to measure improvement. After six months with analytics, compare current metrics to baseline to quantify impact. Positive improvements demonstrate value and justify continued investment in analytics.

Balancing Accuracy and Simplicity

Complex calculations produce precise results but are hard for users to understand and verify. Simple calculations are easier to understand but may be less precise. During testing, strike a balance: calculations should be accurate enough for decision-making and simple enough that incident managers can follow the logic. If a calculation is so complex that only a specialist can explain it, consider simplifying it.

Continuous Improvement and Iteration

Analytics is not a one-time implementation. After launch, commit to continuous improvement. Quarterly reviews assess whether analytics are still meeting needs. Stakeholder feedback is collected and prioritised. High-impact improvements are implemented incrementally. This iterative approach ensures analytics remain relevant and valuable as your incident handling processes evolve and business priorities change.

Proving a figure is right with a seeded dataset

Testing analytics is different from testing a transactional form. A form either saves the record or it does not. An analytics figure can look reasonable and still be wrong, and the only way to show otherwise is to know the answer in advance. The most reliable method is to seed a dataset whose correct results are known before the test starts, run the views against it, and compare each figure with the answer worked out by hand. The seeded records do not need to be realistic in volume. They need to be realistic in shape, with every state, group, category and boundary case the views will encounter.

Keep the seed dataset as a controlled artifact. A future change that alters a figure can then be tested against the same records, and any difference is either a defect or a deliberate change that should be recorded.

A test case table for the common measures

Each row in the table is a yes or no question for the tester, which keeps the test from becoming a subjective review of whether a chart looks sensible.

Running the reconciliation in order

1. Seed the dataset and record its identifiers, so the same records can be found again.
2. Work out the expected figure for every measure by hand, using the written definition and no reference to the views.
3. Run the views against the seed and export the results without editing them.
4. Compare each figure, and log every difference with the record identifiers responsible.
5. Classify each difference as a defect in the view, an error in the hand calculation, or an ambiguity in the definition, and fix it in that place only.
6. Repeat the full run after any fix, because a correction to one figure can change another.

Step two is often the step teams skip when they are short of time, and it is the one that makes the rest meaningful. A test compares a figure against something. If that something was produced by the view itself, the test proves only that the view agrees with itself.

Acceptance evidence that a reviewer can follow

Acceptance is a decision by the business, supported by evidence. The evidence should let a reviewer who was not in the test room reach the same conclusion. Include the seed description, the hand calculations, the platform results, the list of differences and how each was resolved, and the definition as it stood when the test ran. A reviewer who can follow that chain does not need to take the team’s word for anything.

Be explicit about what acceptance does not cover. A view accepted on a seeded dataset has not been shown correct against live data volumes or against live access patterns. Those are separate checks, and the acceptance note should say whether they were performed.

Retesting after a platform upgrade or configuration change

Upgrades and configuration changes can alter figures without touching the views. A change to a state model, a new pause condition or a modified category list can shift a measure while the dashboard looks unchanged. Rerun the seeded test after any such change, using the same records, and compare with the previous run rather than with the hand calculation alone. A difference that appears only after the change is a strong lead.

Keep the rerun proportionate. A cosmetic change to a chart does not need the full set. A change to a field, a state or a schedule does.

Signs the test set is no longer doing its job

• The seed dataset has been edited to make a failing figure pass.
• Test results are recorded as pass or fail without the expected figure written beside them.
• The same few scenarios are tested every cycle and new scenarios are never added.
• Acceptance was given on a test that did not include a reopened or moved record.
• Nobody can say which edition of the definition a test run was measured against.

How many seeded records do we need?

Enough to hit every scenario in the table at least once and to make each expected figure checkable by hand. Volume is less important than coverage, and a small dataset that is fully worked out is more useful than a large one that is not.

Who should do the hand calculations?

Someone other than the person who built the view. Independence is the point of the exercise, and a builder checking their own calculation is likely to reproduce the same assumption.

Should a test failure block go-live?

A defect in a measure that will drive decisions should block it. A cosmetic defect, or an ambiguity that the owner has accepted in writing, can be recorded and deferred.

Do we need the same tests for every view?

No. Views that share a measure can share the scenarios for that measure. Build the tests around the measures, not the screens, so that a new view on an existing measure needs only a short check. Access checks sit in their own set, described in the security and access design guidance, and should be run from separate test accounts.

Designing the acceptance test with a specialist

If the seed data has not been designed yet, that design work is the best use of a specialist’s time before sign-off. Get in touch before the sign-off meeting. The guidance on release acceptance criteria is worth reading for the evidence that acceptance criteria should describe.