ServiceNow and Datadog Integration Guide

A Datadog and ServiceNow integration sends monitoring alerts into ServiceNow as incidents and lets approved changes suppress alerting during maintenance. The most common pattern is inbound: a Datadog alert calls a webhook, which creates an incident with the alert context. Datadog remains the source of truth for monitoring, and ServiceNow holds process and change control.

Why Organisations Integrate ServiceNow with Datadog

Datadog is a monitoring and observability platform that tracks application performance, infrastructure metrics and logs. ServiceNow manages IT processes: incidents, changes and asset management. Integration between the two allows Datadog alerts to auto-create incidents in ServiceNow, and ServiceNow maintenance windows to suppress Datadog alerts during scheduled changes. Teams can also enrich Datadog alerts with ServiceNow context—such as the asset owner or service criticality—to ensure alerts reach the right people and are properly prioritised.

Common Integration Patterns

The most common pattern is inbound: Datadog monitors an application and detects elevated error rates, latency, or resource exhaustion. A Datadog alert is triggered and sends a webhook to ServiceNow, auto-creating an incident with the alert context (metric name, threshold, current value). ServiceNow ticket numbers are sent back to Datadog so alert timelines show the incident record link.

Outbound patterns allow ServiceNow changes to notify Datadog of maintenance windows. Before a deployment, ServiceNow’s change approval workflow triggers a call to Datadog to set a downtime window. Datadog suppresses alerting during the specified time, preventing false alarms during the maintenance. When the change window ends, ServiceNow tells Datadog to resume normal monitoring.

Bidirectional scenarios also sync asset and service metadata. ServiceNow’s CMDB can push tags to Datadog monitors, labelling them by business unit or criticality. Datadog can send recurring health metrics back to ServiceNow for dashboard and capacity planning purposes.

Data Ownership and Governance

Datadog is the source of truth for real-time monitoring and alerting; ServiceNow is the source of truth for process and change control. Establish upfront which Datadog monitors create incidents and which are handled as notifications only. Too many auto-created incidents create noise and increase ticket volume; too few miss opportunities to track real problems formally. Document which escalation rules apply to each alert type.

Authentication and API Credentials

Datadog provides API keys and application keys for authentication. Create a dedicated Datadog application key for the ServiceNow integration, with scopes limited to the downtime and monitor actions it performs, since Datadog API keys carry no scopes. In ServiceNow, store this credential in a secure vault, not as plain text in configuration. Use the same principle for ServiceNow credentials in Datadog: restrict permissions to create/read incidents and to query change data only.

Monitoring Sync Health

Track whether alerts create incidents successfully, whether change windows are set in Datadog, and whether tickets are resolved without incidents left open in ServiceNow. If Datadog alerts stop reaching ServiceNow, incidents will be missed. If change windows fail to suppress Datadog alerts, false alarms will distract the team during maintenance. Both failures should trigger notifications to the integration team.

Monitor API latency. A delay of a few seconds between a Datadog alert and a ServiceNow incident is normal; delays above 30 seconds suggest connectivity problems or quota limits.

Failure Handling and Runbooks

Datadog should continue alerting even if ServiceNow is unreachable. If the integration fails, configure Datadog to escalate to a backup channel (email, Slack) so the team responds even without a ServiceNow ticket. Document the fallback process for teams so they know to manually create a ServiceNow ticket if the integration breaks.

Design Checklist

• Which Datadog monitors should auto-create ServiceNow incidents? Which should only notify?
• How is incident priority determined from the Datadog alert severity?
• Who should be assigned to auto-created incidents, and how does that vary by monitor?
• When a ServiceNow change is scheduled, how is that communicated to Datadog, and how long does suppression last?
• How do you prevent duplicate incidents if the same alert fires multiple times?
• What is the SLA for integration response, and how is it monitored?

Managing Alert Suppression During Maintenance

When maintenance is scheduled in ServiceNow, Datadog should suppress alerts during that window. This requires ServiceNow to communicate maintenance windows to Datadog so the monitoring tool knows not to alert. Configure this integration carefully: if suppression window is too long, real problems during maintenance will be missed. If too short, legitimate maintenance alerts will still fire. Test the window by running a small maintenance task and verifying that expected alerts are suppressed.

Datadog and ServiceNow integration is most effective when both systems are kept current and accurate.

Sort alerts into three outcomes before any webhook is wired

An on-call lead who asks for every Datadog alert to become a ServiceNow incident will get an incident queue that nobody reads within a month. The design question is what each alert is for. Some alerts describe a condition a person must act on, and those should create a tracked incident. Some describe a condition that is worth recording but not worth waking anyone, and those belong in an event stream or a notification. A few describe expected states, such as a batch job running long during a known window, and those should be suppressed for that window rather than escalated. Assign every monitor to one of the three outcomes before the integration sends anything.

An ordered setup for the first monitor group

1. Choose one service with a clear owner group and a small set of monitors that the team already trusts.
2. Assign each monitor to an outcome class and record the owner who will answer for its threshold.
3. Map the Datadog service or host tags to the configuration item so that every incident opens against a known service record, not a free-text name.
4. Trigger a test alert and confirm that it creates or groups an incident with the right priority, assignment and a link back to the Datadog event.
5. Confirm that the return link is written to the Datadog timeline, so responders can move between the two without searching.
6. Add the next monitor group only after one on-call rotation has passed without manual cleanup of incidents.

How priority should be derived

Mapping Datadog severity directly to incident priority is a common shortcut, and it tends to produce incidents that are either all urgent or all routine. Priority should combine the alert severity with the service’s business criticality from the configuration record and whether the affected service is customer-facing. A medium-severity alert on a service that customers use directly can deserve a higher priority than a critical alert on an internal reporting tool. Write the combination as a small table that the service owner has approved, and review it when services are added or retired.

Duplicates, flapping and the storm that follows a shared dependency

A failing database or a network path can make dozens of monitors fire in the same minute. Without grouping, each one becomes an incident, and the responders spend their first ten minutes merging tickets. Group alerts by the shared dependency where the monitor tags allow it, and attach follow-on alerts to the open parent incident. Flapping monitors, which alternate between alert and recovery, need a stabilization rule so that each flip does not open and close an incident. The stabilization period should be chosen by the owner of the monitor, because a short period hides real intermittent faults and a long one delays genuine ones.

Maintenance suppression that does not hide real failures

Suppression is a feature that can easily cause a missed outage. A suppression window set too broadly, such as one covering a whole environment rather than the service being changed, can silence a real failure elsewhere. Scope each suppression to the tags of the configuration items named on the change, and set it to end automatically when the change closes, not at a fixed time that may be wrong. Test the scope before a large maintenance by running a small change and confirming that only the expected monitors were silenced.

Failure patterns that need an owner

• Tag drift, where a service is renamed in the platform but its monitors still carry the old tag, so new alerts open against a retired record.
• Alerts that create incidents but are never closed when the monitor recovers, which leaves the queue full of conditions that no longer exist.
• A fallback channel that was configured during setup but never tested, so the escalation fails at the moment it is needed.
• Priority changes made in the platform that are overwritten by the next alert, so the team cannot trust the ticket’s priority.

Tag drift is one of the more frequent of these. Assign a named owner to the tagging convention and check for retired tags after every service change.

Measures for the on-call side

Track the number of incidents created per monitor per week, the share of incidents closed without action, the time from alert to first acknowledgement and the number of suppressions that expired without a matching change. Low action rates point to monitors that should move to the event class. A high count of expired suppressions points to changes that were not closed in the platform. For grouping and deduplication, see the event management approach, and for how normal behavior is baselined per service, the operational baselines guide helps.

Should the on-call engineer be paged from the platform or from Datadog?

Decide per monitor class. Paging from the monitoring tool is usually faster for the most urgent alerts, while the platform is the better route for escalation and record keeping. Running both for the same alert doubles the noise, so choose one primary channel and use the other only as a recorded fallback.

What should a responder see first when an incident opens?

The service name, the monitor’s purpose, the link to the live graph and the link to the runbook for that monitor. A bare threshold message tells the responder that something crossed a line but not what to do about it.

Can an alert be promoted from event to incident automatically?

It can, when a repeat count or a duration rule is met. The rule should be stated in the monitor’s record and reviewed by the owner, because a promotion rule that nobody remembers writing will eventually surprise someone.

Next step for on-call alert design

Start with a month of monitor history and the escalation chain as the on-call team actually uses it. For the paging side, see the PagerDuty integration guide, with the IT operations management guide for context.