topic Incremental Load query in Data & Schema Discussions

Incremental Load query

mohamed_kamal — Sun, 14 Sep 2025 14:22:09 GMT

Hello Community,

We have a schema that is currently scheduled with a full load, and I would like to switch to incremental load for specific tables.

These tables have a timestamp column (for example: creation_date) that should be used for incremental load.
The source database is PostgreSQL.
I need guidance or documentation on:

How to correctly edit the Query and Update Query for incremental load.

How to choose the right parameters such as Incremental Type and Incremental Extract Using.

What I tried:

Following the Incorta course I have taken, I added a where condition in the query like:
WHERE "CREATION_DATE" > ?

But when I run discovery, I get this error: Failed discovery. Cannot retrieve sample data from [schema_name] due to [No value specified for parameter 1.]

so I tried:
WHERE "CREATION_DATE" >= :last_incremental_value
which also returned an error: Failed discovery. Cannot retrieve sample data from [schema_name] due to [ERROR: syntax at or near ":"]

Could you please share the correct approach, or point me to guides/docs

Thanks in advance for your help!

Re: Incremental Load query

enmaiya — Mon, 24 Nov 2025 00:42:58 GMT

I'm seeing a similar issue for the NetSuite. when we enable incrimental load for a table we should get the following options

For some reason, I did not have this option for NetSuite, and when I use '?' in the where condition, it errors out.

Re: Incremental Load query

flawrence — Wed, 26 Nov 2025 18:30:01 GMT

You can try the following:
FROM "transactionline" where lineLastModifiedDate > TO_TIMESTAMP(?, 'YYYY-MM-DD HH24:MI:SS.FF')

Another example to pull data that changed last 12 months for testing:
FROM "transactionline" where lineLastModifiedDate > ADD_MONTHS(TO_TIMESTAMP(?, 'YYYY-MM-DD HH24:MI:SS.FF'),-12)