ServiceNow Import Set Explained

Definition and Purpose

An Import Set is a temporary staging table in ServiceNow that holds raw data from an external system before transformation and insertion into permanent tables. Instead of directly importing data into your incident table, change table, or user table, you import into a staging table first, validate and transform the data, and then push it into the real tables.

This two-stage approach prevents corrupted or malformed data from polluting your production tables. If a transformation script discovers an error, you can fix the source data and re-import without manually cleaning up partial records in your real tables.

Creating and Loading Import Sets

Import sets are created from a template table (`sys_import_set_row`). When you create an import set, you specify a target business process (such as users, incidents, or assets). ServiceNow creates a table with the prefix `imp_` (such as `imp_incident` or `imp_user`).

You load data into an import set via REST API, a scheduled integration, or manual CSV upload. The data lands in the staging table as-is, without transformation. Common errors during load: missing required fields, data type mismatches, encoding issues. The import set tracks errors and success count so you know how many rows loaded without issues.

Transformation Maps and Data Validation

A Transformation Map links an import set to a target table and defines how to transform each field. For example, a map from `imp_user` to the user table might specify: external field “emp_id” maps to the user table’s “employee_number” field, and a validation routine ensures employee_number is numeric.

You can include complex transformation logic: concatenating fields, looking up related records, applying regular expressions, or invoking scripts. If a transformation fails (for example, employee_number is non-numeric), the row is marked as error and does not insert into the target table.

Coalescing and Update vs Insert

When you run a transformation, you specify coalescing rules: which fields in the source data identify an existing record in the target table so you can update it instead of inserting a duplicate. For example, if the import set has employee_id, and you want to update existing users, you coalesce on employee_id. If an import row has an employee_id that already exists in the user table, the map updates that user; if not, it creates a new user.

Without correct coalescing, repeated imports create duplicates: you import 500 users on day one, import the same 500 (with some changes) on day two, and end up with 1000 users instead of 500 updated records.

Handling Reconciliation and Deletion

Import sets are useful for additions and updates but require careful handling for deletions. If you receive a daily user list from HR, but a user appears in Monday’s list but not Tuesday’s, does that mean the user was deleted? Import sets do not automatically delete records that disappear from source data. You must write logic to identify “deleted” records: query the target table for records older than the import date with no recent updates, then delete them.

This reconciliation logic is often the trickiest part of a data import: understanding whether absence from the source means the record was deleted, archived, or simply filtered out by an access rule.

Common Import Set Problems

Loading millions of rows into a single import set can overwhelm the instance. Break large imports into batches. Forgetting to clean up old import sets wastes database space. Configuring transformation maps incorrectly leads to data corruption: always test a small batch first. Failing to handle timezones or date formats often causes silent truncation (dates that looked correct in Excel show as NULL in ServiceNow).

Import Sets in Practice

Most organizations use import sets for recurring syncs: importing users daily from active directory, importing assets weekly from a CMDB tool, importing incidents from a legacy ticketing system during migration. Each import run should start fresh with a new import set table. Reusing the same import set across runs accumulates orphaned rows from failed previous attempts. Use an import set naming convention that includes the run date so import sets are self-documenting.

Monitoring and Logging

Import sets create audit trails: which rows succeeded, which failed, and why. Enable detailed logging on transformation maps so you can debug failures. Monitor import run times: if an import that usually takes ten minutes now takes forty, something is wrong (additional data, transformation inefficiency, or instance performance). Set up alerts for import failures so you know immediately when a daily sync does not complete. Document your coalescing logic and any custom transformation scripts so future team members can maintain the integration.

Designing the staging step before the first load

An import set is useful only if the staging step is designed on purpose. The staging table holds raw rows exactly as the source sent them, and that raw layer is what lets you explain a bad result later. Before loading anything, decide what a single row represents, how the source identifies it, and whether a later file will re-send the same rows. Those three answers determine the transform logic, the coalescing keys and whether the job can safely run again.

Load pattern	What the staging data looks like	Transform mode to plan for
One-off migration from a legacy extract	Full population, no repeats	Insert, with a post-load reconciliation count
Recurring full snapshot	Entire population every run	Update or insert on a stable key, plus a retirement pass
Recurring delta feed	Changed rows only	Update on the key, never a full overwrite
Corrections from a spreadsheet	Small, hand-edited batch	Update only, with a reviewed row count before the run

Running the first load as a controlled experiment

The first load should be treated as a test, even when it is the real migration. Load a slice of the source, run the transform, and inspect the result against the source row by row for a sample. Only then load the full population. The aim is to find mapping mistakes while they are cheap to fix, not after thousands of records are in the target.

1. Load a slice that includes the awkward cases: blank key fields, unusual characters, and rows that the source says are deleted.
2. Run the transform map on that slice only, and read the transform history for each row’s outcome, not just the summary count.
3. Compare a sample of target records to their source rows, checking reference fields as well as plain text.
4. Fix the mapping or the coalescing rule, then clear the slice from the target before rerunning it.
5. Repeat the full load only after two consecutive slices come through with no unexplained rows.

What goes wrong with references and lookups

Reference fields are where many imports quietly go wrong. A row can succeed with an empty reference because the lookup found no match, and the summary will still show success. Build the transform so that a failed lookup is written to an exception list, or sent to a default that a person has approved, rather than being left blank. For user and group references, check the match rule against the data you actually have. A source that holds email addresses in one case and the directory in another will match some people and silently miss the rest.

Keeping the load repeatable

A recurring import must be safe to run twice. Confirm this by running the same file a second time in a test copy and checking that the row counts in the target do not change. If they change, the coalescing key is wrong or the transform is inserting where it should update. Store the file name or batch identifier on each target record where your reporting needs it, so that a bad batch can be traced and reverted rather than hunted.

When a direct integration is the better choice

An import set suits batch data with a clear key and a tolerance for a delay between the source change and the platform update. It is the wrong choice for data that must be current within seconds, such as a status that drives an active incident. For that, a REST integration writing to the target is simpler and avoids a second copy of the data to keep in step. Choosing the batch route for real-time needs is a common source of complaints that the platform is out of date.

Checklist for a recurring load

• A documented key that the source guarantees to be unique and stable.
• A written rule for rows the source no longer sends, and an owner who approves retirement.
• Exception handling for failed lookups that gives someone a queue to work.
• A row count reconciliation between the source extract and the target after every run.
• A rollback path, such as a batch identifier that lets the last run be removed.

Frequently asked questions about staged loads

Can we skip the staging table and load straight into the target?

You can, but you lose the raw record of what arrived and the ability to rerun a transform without asking the source for the file again. For anything beyond a trivial load, the staging layer is worth keeping.

Why do some rows show as ignored rather than failed?

Often the transform found an existing record that matched the key and the mapping decided no change was needed, or a condition on the map excluded the row. Look at the transform history for that row to see which rule applied.

How should we handle a source that sends a field we did not expect?

Map it to a holding field in staging and decide on its handling before the next load. Ignoring unexpected columns silently is how a later report turns out to be missing data nobody remembered to ask for.

Do we need to keep import set rows after a successful load?

Keep them for as long as your audit or rerun requirements say. Purging staging rows removes the evidence needed to explain a discrepancy months later.

Next step

The guide to data migration validation and cutover covers how these loads are checked at the end of a migration, and the platform library shows where transform maps sit within the wider build.