Data Lineage Latency

[mrossPM2]

I'm in the process of re-pointing dashboards to stop sourcing from a particular business schema. The data lineage functionality has been helpful in identifying which dashboards are using a particular business schema. I have noticed that sometimes after removing the references to the business schema, the dashboard still appears when I open the data lineage. I have confirmed that all references to the business schema have been removed from the dashboard by exporting the dashboard and searching the xml.

Is there a refresh interval for the data lineage? I'm on 2023.7.2. I can log a ticket if this seems like a bug.

[ayakandil]

Hello @mrossPM2 ,

Can you please answer few questions to understand a bit more about the issue you are facing?

1. In the times where you find the dashboard references still appear in the schema lineage, Do they keep appearing consistently in the lineage or they disappear after sometime? If they disappear from the lineage after sometime, have you noticed by any chance how long would you have to wait for until the lineage gets updated correctly?

2. Can you tell us some information about the times when the issue happened? What was the type of the view inside the schema and the column type as well?

3. Does the schema lineage stays incorrect after restarting the services?

We have faced an issue similar to the one you are facing here and we fixed it and should be delivered with the upcoming release but we need to make sure that both are the same.

[mrossPM2]

I should be able to give you more exact scenarios later this week.

Sometimes after removing the old business schema from the dashboard, the dashboard would immediately drop off the lineage report. But sometimes it wouldn't. I have not tried restarting services to see if that has an impact.

Good to hear that it might be fixed in the next release!

[mrossPM2]

I went through another round of re-pointing and deleting content today, but the lineage in the business schema always updated properly as I made the deletes. I wasn't able to replicate my issue from the other day, so I guess it was just a random occurrence.