Update ( add columns to ) a snapshot table

[RADSr]

So you've built yourself a snapshot table and things are going along swimmingly...   and then BAM!  you want to add another column to it.    

You can't do a full reload because all that sweet history would just disappear and never by properly appreciated.    Unfortunately altering the Incorta table structure *requires* a full load.

What I want though is the DB equivalent of altering the table and putting default values in for NULLs  ( although really what I'll be doing is manufacturing approximate data from context specific history to populate those fields ).

I'm sure I can do this in a series of tables using the original snapshot as a source, but that means every time I make a change I have to create another, new, snapshot table.   

I turn to you fellow Community members... how would you approach such a conundrum?    

[mhelmy]

Hello RADSr,

We are nearing the completion of a new feature designed to support schema evolution, allowing for the seamless addition or removal of columns. This enhancement directly addresses your request to eliminate the need for a full load after making changes to the schema.

With this new functionality, actions such as adding or deleting columns will no longer necessitate a full load or rewriting of the parquet files.

[RADSr]

Thank you @mhelmy  - is this a "take it easy for the holidays and expect this as a New Year's present" or a "find your workaround now and it'll be easier later" thing?   😉     

[mhelmy]

Expected in Q1 2024.