Import Service Tutorial
The Import Service imports external master data into Emporix. A configuration groups one or more streams, where each stream extracts from a source connection, maps fields to an Emporix target type, and upserts idempotently. Imports run asynchronously and stream per-stream progress over Server-Sent Events (SSE).
This functionality is in preview mode - some of the features may not be fully operational yet.
This tutorial covers triggering, scheduling, monitoring, cancelling, and inspecting import runs. An import configuration must already exist for your tenant.
Creating and changing configurations, connections, streams, and mappings requires the importtool.import_manage scope.
The following diagram shows how the main Import Service resources relate to each other:
A typical operational workflow follows these steps:
List configurations and choose the
configIdyou want to run.Inspect streams to confirm source entities and target types.
Trigger a
DELTArun (or adryRunfirst if you want to validate without writing).Monitor progress by polling the run endpoint or subscribing to SSE events.
If the run ends as
PARTIALorFAILED, retrieve errors and fix the source data or mappings.Search imported records to confirm the expected outcomes.
The sections below walk through each step in detail. All operations use the importtool.import_trigger scope documented in the API Reference.
Prerequisites
Make sure you have the following:
A service OAuth2 token with the
importtool.import_triggerscope. For more information, see Authentication and Authorization.An enabled import configuration with active streams already exists for your tenant.
How to inspect import configurations
Before triggering a run, verify that the configuration you want to use exists and is enabled.
To list all import configurations for your tenant, send a request to the Retrieving all import configurations endpoint.
To retrieve a single configuration by its identifier, send a request to the Retrieving an import configuration endpoint.
The response includes fields such as name, enabled, deltaEnabled, and sourceConnId. Note the configuration id — you need it for scheduling and triggering runs.
To test the endpoint, open the API reference or check the example of a curl request.
How to inspect configuration streams
Each configuration contains one or more streams. A stream defines which source entity is extracted, how fields are mapped, and which Emporix target type receives the upserted records.
To list the streams for a configuration, ordered by sequence, send a request to the Retrieving all streams of a configuration endpoint.
To retrieve a single stream, including its resolved target types, send a request to the Retrieving a stream endpoint.
Each stream response includes sourceEntity, targetWriter, targetType, and mode. The mode field indicates how the stream contributes to the target:
STANDALONE— the stream upserts records independently.COMPOSITE_CHILD— the stream contributes child records to a composite parent.COMPOSITE_MERGE— the stream merges data into a composite parent.
To test the endpoint, open the API reference or check the example of a curl request.
How to schedule recurring imports
You can run a configuration automatically on a cron schedule.
To check whether a schedule already exists, send a request to the Retrieving a schedule endpoint. If no schedule is configured, the endpoint returns 204 No Content.
To create or update a schedule, send a request to the Scheduling an import run endpoint with a Spring cron expression (six fields), time zone, and enabled flag.
In this example, the configuration runs every day at 02:00 in the Europe/Berlin time zone. The saved schedule response includes nextFireAt when a next run time can be calculated.
To test the endpoint, open the API reference or check the example of a curl request.
How to trigger an import run
To start an import manually, send a request to the Triggering an import run endpoint. The endpoint returns immediately with the run in a RUNNING state; progress is available through polling or SSE.
Request body fields:
mode
No
FULL or DELTA. Defaults to DELTA.
dryRun
No
When true, maps and validates records without performing remote writes.
To validate a configuration without writing data, trigger a dry run:
If a run is already active for the configuration, the endpoint returns 409 Conflict with the message An import run is already active for this configuration.
On success, the response includes the run id, status, mode, and counters such as recordsRead, created, updated, skipped, failed, and deleted. Save the run id for monitoring and troubleshooting.
To list previous runs for a configuration, send a request to the Retrieving run history endpoint.
To test the endpoint, open the API reference or check the example of a curl request.
How to monitor an import run
You can monitor a run by polling its status or by subscribing to the SSE progress stream.
Poll run status
To retrieve a run together with per-stream progress, send a request to the Retrieving a run endpoint.
The response contains a run object and a streams array. Each stream entry includes its own status and counters. Terminal run statuses are SUCCEEDED, PARTIAL, FAILED, and CANCELLED.
Stream run progress over SSE
To receive live progress updates, open the Streaming run progress endpoint. The stream sends:
An initial
snapshotevent with the run and stream state.A
streamevent for each processed batch.A final
runevent when the run finishes.
To test the endpoint, open the API reference or check the example of a curl request.
How to cancel an import run
To request cancellation of an active run, send a request to the Cancelling a run endpoint.
By default, cancellation is cooperative — the run stops after the current batch completes. To stop the run immediately and unblock the configuration, pass force=true:
On success, the endpoint returns 202 Accepted with accepted: true. If the run is already finished or unknown, accepted is false.
To test the endpoint, open the API reference or check the example of a curl request.
How to inspect imported data
After a successful or partial run, verify what was imported.
List imported data types
To see which target types currently hold imported records, send a request to the Retrieving imported data types endpoint.
Search imported records by type
To search records of a given target type, send a request to the Searching imported records endpoint. The type query parameter is required. The optional search parameter filters by natural key.
The search parameter performs a case-insensitive substring match on the record's natural key only. It is not the Emporix query language, so field selectors and operators (for example field:value, comparisons, or boolean logic) are not supported.
Each record includes naturalKey, targetType, fields, outcome (for example CREATED or UPDATED), and importedAt.
Search records for a specific stream
To search only the records produced by one stream, send a request to the Searching a stream's imported records endpoint. The optional search parameter uses the same natural-key substring filter as the type-based search endpoint.
To test the endpoint, open the API reference or check the example of a curl request.
Troubleshooting
When a run finishes with status PARTIAL or FAILED, inspect the recorded errors.
To retrieve paginated errors for a run, send a request to the Retrieving run errors endpoint.
Each error record includes:
naturalKey— the source natural key of the offending record.streamId— the stream where the error occurred.stage— the pipeline stage, for example extraction or mapping.errorCode— a machine-readable error code.message— a human-readable description.
Use these fields to identify which source records failed and at which stage of the import pipeline.
To test the endpoint, open the API reference or check the example of a curl request.
Last updated
Was this helpful?