ServiceNow and Kubernetes Integration Guide

Why Organizations Integrate ServiceNow with Kubernetes

Organizations running containerized applications on Kubernetes need ServiceNow to provide operational visibility, incident correlation, and automated remediation. Common integration points include synchronizing Kubernetes cluster and pod inventory into the ServiceNow CMDB, ingesting Kubernetes monitoring data (container restarts, resource exhaustion, pod failures) as incidents, and triggering remediation flows (auto-scale clusters, restart pods, trigger deployment updates). The integration translates low-level Kubernetes events into service-level incidents that operational teams can act on.

This is essential for organizations running production Kubernetes clusters with multiple teams, where incident response coordination, compliance auditing, and root-cause analysis require a unified view of container infrastructure and the services it supports.

Common Integration Patterns

Inbound: Kubernetes monitoring systems (Prometheus, Datadog, New Relic) push alerts to ServiceNow, creating incidents linked to Kubernetes resources. Pod crash loops, node failures, and resource quota exhaustion trigger incidents. Outbound: ServiceNow triggers remediation in Kubernetes (scale deployments, update configurations, drain nodes) through Kubernetes API calls. Bidirectional: ServiceNow maintains a current inventory of clusters, nodes, deployments, and services from Kubernetes API; Kubernetes state changes trigger ServiceNow updates to keep CMDB and incident context current.

Data Ownership and Authentication

Kubernetes is the source of truth for container state and resource status. ServiceNow maintains a reference copy in the CMDB, synced via Kubernetes API queries or event listeners. Authentication typically uses Kubernetes service accounts with appropriate RBAC roles (read-only for inventory sync, controller-level for remediation). Store Kubernetes API credentials in ServiceNow Credential Store; never embed them in flows or scripts.

For large clusters, use Kubernetes namespaces to scope what ServiceNow ingests and manages. Isolate production-cluster integrations from dev/test to avoid accidental remediation in critical environments.

Monitoring and Failure Handling

Common failure points include Kubernetes API rate limiting (especially in large clusters), network latency or firewall rules blocking ServiceNow to API server connectivity, and RBAC permissions being too restrictive. Implement exponential backoff in flows making Kubernetes API calls. Monitor ServiceNow integration logs for Kubernetes API errors and set alerts if sync falls behind.

For remediation flows (auto-scaling, pod deletion), design them to be carefully gated: require manual approval for destructive actions like node drain or pod termination. Async flows are essential for long-running Kubernetes operations; synchronous flows will timeout waiting for deployment rollouts.

Design Checklist for Kubernetes Integration

• Define which Kubernetes resources (clusters, nodes, deployments, services, pods) will be synced into the CMDB and the sync frequency.
• Identify which Kubernetes alerts or events should create incidents (pod crashes, node-not-ready, quota exhaustion) and define escalation rules.
• Design remediation workflows: which actions are auto-remediated (restart pod) vs. manually approved (scale cluster down, drain node).
• Secure Kubernetes credentials: use Kubernetes RBAC service accounts, store credentials in Credential Store, rotate regularly, audit access.
• Plan for Kubernetes API rate limits: use batch queries, implement caching where possible, avoid redundant syncs.
• Design observability: log all Kubernetes API calls, sync events, and remediation actions; set up dashboards showing cluster health and incident correlation.
• Plan for multi-cluster scenarios: if you run multiple Kubernetes clusters, ensure ServiceNow CMDB scoping and incident routing handles cluster context correctly.

Testing Kubernetes integration is critical before enabling in production. Test discovery against a non-production cluster first to understand the data volume, verify asset filtering is configured correctly, and ensure no unexpected resources are being synced into your CMDB. Test provisioning or scaling workflows thoroughly in non-production environments before enabling in production systems. Test failover scenarios: what happens if the Kubernetes API server is temporarily unavailable? Ensure ServiceNow retries appropriately with exponential backoff, implements idempotency checks, and doesn’t accidentally provision duplicate resources during recovery. Test with representative data volumes to understand performance characteristics.

Common Pitfalls

Not separating production and non-production cluster credentials can lead to incidents being created for test environments or accidental remediation in production. Not implementing rate-limit handling causes sync to fall behind or API calls to fail. Not logging Kubernetes API activity makes troubleshooting integration failures difficult. Not filtering the initial CMDB sync creates thousands of pod records that clutter the CMDB; sync clusters and deployments, not individual pods.

Kubernetes integration helps teams understand their container infrastructure and its impact on services. When a Kubernetes pod fails, a ServiceNow incident is created with context about which deployment was affected and which services depend on it. This speeds diagnosis and response: teams know immediately what is affected and can prioritize remediation based on business impact.

Scoping which clusters and namespaces ServiceNow is allowed to see

The first Kubernetes decision is what ServiceNow is permitted to know about. A cluster can hold many short-lived objects, and most of them belong to the platform rather than to any service a business would recognize. Pulling everything in makes the CMDB unreadable and puts pressure on the API server. The sensible starting point is the namespace: each namespace should either map to a business service or be excluded explicitly. Agree that map with the platform team before any discovery runs, because everything after it depends on that agreement.

Namespaces are not always the right unit. Some organizations run one namespace per team with several services inside it, and others share a namespace across unrelated applications. In the first case, the service mapping has to descend into deployments. In the second, the mapping has to use labels that the application teams already set. Either way, the output should be a table of namespace or label to business service, and it should be reviewed when teams reorganise.

What to bring in, by scope

An order of work that keeps production clusters safe

1. Write the namespace-to-service map and get the platform and application leads to sign it.
2. Create a read-only service account in a non-production cluster, with RBAC rights limited to listing the objects in the table above.
3. Run discovery against that cluster and count the records it creates, comparing the figure with the expected inventory.
4. Repeat for production with the same read-only account, and keep remediation credentials out of the discovery configuration.
5. Connect alert ingestion from the monitoring tool, scoped to the same namespaces, so an incident is never created for a namespace nobody owns.
6. Add any remediation action last, starting with actions that are safe to repeat, such as restarting a single pod, and holding destructive actions behind an approval.

The failure that looks like a healthy sync

A sync can report success while the picture underneath is wrong. Labels change when a team renames a service, and the mapping then places a live workload under a retired service record. The discovery run still succeeds, but incidents from that workload are sent to a group that has stopped owning them. Watch for records with no owner, and for a rising count of workloads that match no service. Both are early signs that the map has drifted from the cluster. The thinking in governing relationships between configuration items applies directly, because a workload is only useful in the CMDB when its relationship to a service is correct.

Remediation needs its own gate

Scaling a deployment down or draining a node is an operational decision, even when a workflow makes it quickly. Separate remediation into three tiers. Safe and repeatable actions, such as restarting a pod that has crashed in a loop, can run automatically in non-production and with approval in production. Actions that change capacity need an operator to confirm the reason. Actions that remove workloads should always need a named approver and a rollback plan. Record which tier each action belongs to in the runbook, so that a new workflow cannot quietly start at the wrong tier. The automation exception handling guide is the companion read when those tiers are set.

Signs the Kubernetes link is trustworthy

• Every in-scope namespace has an owner and a service, and the list of namespaces without either is short and reviewed.
• No pod-level records exist in the CMDB.
• An alert from a namespace routes to the service owner recorded in the map, and a test alert proves it.
• Remediation actions show the tier, the approver and the outcome in the record.

Should the ServiceNow integration account have cluster-admin rights?

No. Discovery needs read access to the objects it records, and remediation needs rights limited to the actions in its tier. Cluster-admin rights can turn a compromised integration account into a compromise of every workload, and they are rarely justified by this work.

How do we keep production and non-production credentials apart?

Use separate service accounts and separate credential records for each environment, with different names, so that a flow configured for a test cluster cannot be pointed at production by a copied setting.

What happens when a cluster is rebuilt?

The cluster record should persist with a new version or a clear replacement reference, while its namespaces and services are re-mapped. A rebuild that creates a second cluster record with no link to the first can leave incidents orphaned.

Should auto-scaling events create change records?

Normal autoscaling should not. Changes to the scaling policy itself are configuration changes and belong in the normal change path. Mixing routine capacity movement into the change calendar can make the calendar less useful for the decisions it exists to support.

Next step

If the cluster estate is large and the namespace ownership is unclear, start with a scoping session before running discovery in production.