I have a table w/ incremental loading enabled using " <timestampcolumn> > ? "
Troublingly, I have some duplicate rows with the same <timestampcolumn> values ( and all other values ). I'm not sure why, but the load time was the same time ( w/ in 4 seconds ) of the <timestampcolumn>
I see in the documentation there is a lookback setting for some sources - although I don't see that configuration in my SQL Database source.
If that's configurable, where? And/or what is the precision of "?" for the last load time.
Solved! Go to Solution.
1) Kudos to support who spent three hours on the phone w/ me trying to track down my issue!
2) As @amit_kothari mentions above, the "?" is stored in epoch time
3) It's precise to the millisecond
4) My specific mystery gets more and more interesting. The source table timestamp showed up in a file produced before the timestamp indicates the record was produced.
5) According to the source system support folks that timestamp is based on the time a record is uploaded.
6) The obvious next check would be time zone offsets, but there aren't enough records duplicated to make that likely. Nevertheless, it is on my list!
So I've found the precision is to the millisecond.