Incorta SQL , SQLi, Decimal data types, and UNIONing data

RADSr — Thu, 08 Jun 2023 23:48:32 GMT

I've got this table - it's huge, not by data volume standards ( ~ 45 million records and growing w/ projected exponential growth ), but by load-time standards ( ~ 25 hours to do a full load ).

I need a way to speed this up. ( I'm already doing incremental loads but situations arise where a full load is needed )

*** I have no influence over the source DB whatsoever, so "tune the view," while good advice, is not something I can do -- non starter **

Anyway - the ~ 25 hour load time is when I select * from view.

I did a test breaking that into 12 tables and adding a where clause to each to bring in a year's worth of data in ~ 2 hours. Huzzah!

So I can create a derived table to UNION those, right? <BZZZZZTTTTT> Incorta SQL doesn't like the DECIMAL data type.

----------

Decimal type is expected for select item: Txxxxxxxx

Currently decimal type is not supported!
Not supported query

-----------

OK - so the very quick engine isn't available, but surely a SQLi table will work ...

----------

<TABLE>

- INC_03010100: Unknown Error: Extraction failed [com.incorta.connector.errors.DiscoveryException: Error of type PSQLException while discovering columns from datasource [SQLInterface] due to: ERROR: Decimal type is expected for select item:

------------------------------------

So I'm learning that combining two Incorta tables, created by Incorta, with the Incorta assigned data types isn't as straightforward as one might expect.

I'm out for the evening, but if anyone has any experience w/ this type of exercise and can point me in the right direction I'm happy to learn!

topic Incorta SQL , SQLi, Decimal data types, and UNIONing data in Administrative Discussions

Incorta SQL , SQLi, Decimal data types, and UNIONing data