Incorta Table load fails for a large Oracle table with 'Snapshot too old' error

[amit_kothari]

While trying to load an Incorta table with nearly 1 billion rows from Oracle the load fails with 'ORA-01555 ERROR MESSAGE “SNAPSHOT TOO OLD”' error.

[amit_kothari · Accepted solution]

While querying large tables from Oracle sometimes we see this load error from Oracle database. One possible solution is to enable chunking while extracting data from these huge tables .

Incorta chunking is a table data source configuration that allows for parallel data extraction. The parallel execution significantly helps extract rows from very large tables.  This is handled at a table level where we can specify the chunk size for a single table extract.  We support two modes in chunking -

1) Size of Chunk: Please refer to the screenshot below for an example of OE_ORDER_LINES_ALL table in Oracle, we specified LINE_ID which is the key column of this table for chunking. Please refer to https://docs.incorta.com/ for more information.

2) Timestamp field: Please refer to the screenshot below for an example of a table in Oracle where we specified CREATION_DATE as a chunking column for chunk period as Annually.