Configuring Streams

Configure stream source, target, key resolution, composite behavior, delete detection, routing, and health.

After you create a stream, expand its row to access the configuration tabs. Save each tab after you change it.

Source

On the Source tab, set:

• Connection – the connection that points to the source system
• Delta field format – the format used by the source field, for example Unix epoch (seconds). A Unix timestamp represents a point in time as the number of seconds elapsed since the Unix epoch: 1 January 1970 at 00:00 UTC.
• Delta field – a modified or created timestamp used to detect changes during delta runs
• Timestamp counts as a change – include a record when only its timestamp changes
• Max records – an optional limit on how many records to import
• Record filter (PQL) – optional filter for Celonis sources. PQL is Process Query Language, the Celonis query language for process data. Enter a condition here when you want to import only rows that match a rule, for example "o_Celonis_Vendor"."Country" = 'DE'. The Import Tool evaluates this filter in Celonis before records are extracted, so fewer rows are read from the source.

  Use PQL together with the run mode and limits on this tab. On a delta run, import only records that changed since the last watermark (Delta field). On a full run, import all eligible records, respecting Max records when you set a limit. When you add PQL, a row is imported only if it matches your condition and still qualifies under that run mode and those limits. For example, on a delta run you might import changed vendors, and PQL "Country" = 'DE' imports only the changed rows where the country is Germany.

For a PQL condition, use Insert column to add a qualified column reference ("table"."Column"), then choose Test to validate the condition before saving.

Source settings for a Celonis stream

Target

On the Target tab, set:

• Write strategy – choose Patch to update only mapped fields and keep the rest, or Replace PUT to overwrite the whole instance
• Stop a run if the target schema has changed – prevent standard runs when the mapping was created for an earlier target schema version; forced runs ignore this protection
• Depends on – select another stream that must run first, or None (full snapshot)

Use Patch when another process also manages the entity. Use Replace PUT when this import owns the complete instance.

Target settings for writing entities and controlling stream dependencies

HTTP endpoint

When the stream uses an HTTP / REST API connection, configure the request on the HTTP endpoint tab.

Endpoint

On the Endpoint sub-tab, set:

• Method – the HTTP method, for example GET
• Path – the path appended to the connection base URL
• Response format – the format returned by the source, for example JSON
• Records path – the path to the records array, or (response is the array) when the response body is the array

Path is appended to the connection Base URL. If the Base URL is only the host, enter the list path here, for example /countries/v5. Records path is a dotted JSON path to the array, for example data.objects.

Choose Save.

Endpoint settings for an HTTP source

Pagination

On the Pagination sub-tab, choose the pagination method used by the source API. For Cursor / token in response, configure:

• Cursor query param – the query parameter that sends the previous page's cursor in the next request
• Has-more path (body) – the dotted path to the response field that indicates whether more pages are available. Leave it empty to stop pagination when the API returns an empty page.
• Cursor path (body, optional) – the dotted path to the next cursor in the response body
• Cursor field (record) – the record field used as the next cursor when Cursor path is empty; the key field is the default

Review Request preview to verify how the Import Tool builds subsequent requests.

Cursor-based pagination with configurable query parameter and response paths

Parameters & headers

On the Parameters & headers sub-tab, configure values sent with every request:

• Request parameters – add query parameter names and values. Use {deltaSince} in a value to pass the last delta watermark during a delta run; the parameter is omitted during a full run.
• Headers – add request header names and values, for example an authorization or vendor-specific header.

Choose Save.

Query parameters and headers sent with each HTTP request

GraphQL query

When the stream uses a GraphQL connection, define which data to read on the GraphQL query tab. Build the operation against the connection endpoint and credentials, then run it to confirm the response before you save the stream.

• Query – write the GraphQL operation that returns the records to import. Use the schema list on the right to start from an existing query, or type field names in the editor. Press Alt+Space or Ctrl+Space to open field suggestions.
• Variables (JSON) – optional JSON variables for the query when the operation declares variables (for example filters or limits).
• Schema explorer – search and choose a root query to insert a starting operation, then adjust fields to match what you need in Emporix.
• Operation – enter the operation name only when the query text defines more than one named operation.
• Run query – execute the query against the live endpoint and check that the returned records look correct.

For a composite child stream, you can parameterize the query with placeholders:

• {parentKeys} – expands to every parent key as a list so the child can fetch related records in one request (for example where: { id: { _in: $ids } }).
• {parent.attribute} – expands to one parent’s value and runs the query once per parent.

Choose Save when the query and variables are ready.

GraphQL query editor with schema explorer and variables

Key resolution

On the Key resolution tab, select the Key field used as the primary key for matching and upserting records.

Optionally add Candidate key fields and rank them. The record is matched by the first non-empty candidate (COALESCE). For example, prefer an Emporix ID and fall back to an SAP number. The key field is the final fallback when candidate keys are empty.

Key resolution with a primary key and optional ranked candidate keys

Composite

On the Composite tab, choose how the stream is imported:

• Standalone (own records) – import the records as their own Emporix documents. This is the default.
• Composite child (embedded in a parent) – embed records as an array inside a parent stream. Typical for line items.
• Composite merge (enrich a base record) – merge mapped scalar fields onto a matching base stream instance instead of creating separate records.

Composite tab with Import as options: Standalone, Composite child, and Composite merge

Use composites when the source system has separate records, but you want to aggregate them into one Emporix document. For example, invoices and invoice lines can be separate source records, while in Emporix you want one invoice document with the lines embedded in it.

Composite child

Use Composite child (embedded in a parent) for array data such as line items. Child records are embedded as an array inside the parent and are not written as separate instances.

Configure:

• Parent stream – the stream that owns the parent document. For example, an invoice-line stream uses the invoice stream as its parent.
• Linking field (this record → parent) – the field on the child record that points to the parent. In database terms, this is the foreign key on the line, for example a payable item ID that references the invoice.
• Parent key field – the key field on the parent record that the linking field matches. This is often the same as the parent stream's key field, but you can set a different field when needed.
• Embed as attribute – the array attribute on the parent target type that receives the embedded children. If the parent type has more than one array, select the correct one, for example lineItems.
• Child strategy – how child data is applied during import. For example, Embed (parent writes the array) fetches the child records, stores them temporarily, and attaches them while the parent record is imported.

Composite child configuration with the parent stream, linking field, parent key, embed attribute, and child strategy

Check the relationship in Run order or on the Dependency graph tab. The parent stream must run before the composite child stream.

Composite merge

Use Composite merge (enrich a base record) when separate source records enrich one Emporix object with additional scalar fields. Merged records do not get their own instances; their mapped fields are patched onto the matching base stream instance.

Configure:

• Base stream – the stream whose instances are enriched
• Shared-key field (this record → base) – the field used to match this record to the base stream instance

Composite merge configuration with base stream and shared-key field for enriching a base record

Deletes

On the Deletes tab, use Delete detection to choose how deleted source records are handled:

• None (no delete tracking) – do not delete Emporix records based on source deletes
• Status field (marker) – delete when a source status field marks the record for deletion
• Tombstone record (delete table) – delete based on tombstone or delete-table records

Deletes tab with delete detection options

Type discriminator

On the Type discriminator tab, optionally route each source row to a specific target type based on a field value. For example, use an invoice type column to send PO and non-PO invoices to different target types. Values that are not listed use the stream target type.

Type discriminator for routing source records to different target types

Stream health

On the Health tab, set thresholds that determine this stream's health on the Dashboard. Leave a field blank to use the job default, which is the import-level setting shown in the UI.

Health tab with per-stream overrides for failure rate, freshness, and failed last run

You can override:

• Amber above failure rate
• Red above failure rate
• Amber after no fresh data for
• Red after no fresh data for
• A failed last run is red – turn off when occasional whole-run failures are expected and only the rates above decide health

After you configure the tabs, continue with Field Mapping, then run the import.