topic Re: Save only last X Period History in Incremental Tables in Data & Schema Discussions

Save only last X Period History in Incremental Tables

nikhil_cr — Mon, 13 Jan 2025 13:20:57 GMT

Hi,

I have an incremental table where I load data once a week. It’s been 6 months, and the data has already exceeded 30 GB (~500 million rows). Now, if I want to restrict the history to only the last 3 months, is there a way to do this? I don’t want to take a backup, remove the unnecessary history, and append it back. I’m looking for an automated solution. I see the load filter option, but is there any other way to achieve this?

Thanks,

Nikhil

Re: Save only last X Period History in Incremental Tables

JoeM — Mon, 13 Jan 2025 13:42:57 GMT

Hi @nikhil_cr, starting in 2024.7, we introduced better table management options. Below, you will see the option that can retain data based on a relative period:

Re: Save only last X Period History in Incremental Tables

nikhil_cr — Mon, 13 Jan 2025 14:12:20 GMT

@JoeM Thank you 🙂 Much appreciated your help.