There are situations where columns are dynamically generated in PySpark MV, each time a column changes, Incorta Schema generates error and gives this recommendation: 'Please rerun table discovery'
Basically, you will have to manually validate the MV few times for Incorta to 'recognize' the updated column(s). It will be great, if there is a rediscovery button (that allows a one time fix) or better still for the MV to be able to accommodate and automatically accept changes in columns that are designed to be dynamic and save them.
Currently running into this problem with big datasets, take a long time to resolve, through manual re-validation.
Regards
David
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.