06-08-2023 04:48 PM
I've got this table - it's huge, not by data volume standards ( ~ 45 million records and growing w/ projected exponential growth ), but by load-time standards ( ~ 25 hours to do a full load ).
I need a way to speed this up. ( I'm already doing incremental loads but situations arise where a full load is needed )
*** I have no influence over the source DB whatsoever, so "tune the view," while good advice, is not something I can do -- non starter **
Anyway - the ~ 25 hour load time is when I select * from view.
I did a test breaking that into 12 tables and adding a where clause to each to bring in a year's worth of data in ~ 2 hours. Huzzah!
So I can create a derived table to UNION those, right? <BZZZZZTTTTT> Incorta SQL doesn't like the DECIMAL data type.
----------
Decimal type is expected for select item: Txxxxxxxx Currently decimal type is not supported! Not supported query
-----------
OK - so the very quick engine isn't available, but surely a SQLi table will work ...
----------
<TABLE>