topic Re: Prioritize a dashboard ( using cache )? in Administrative Discussions

Prioritize a dashboard ( using cache )?

RADSr — Mon, 24 Jul 2023 16:48:57 GMT

We have a dashboard ( might turn out to be a few ) which we need to be as near instantly rendered as possible when a consumer opens it.

I recall at some point there being a "warm up" setting but not that it was dashboard specific.

So - is there a way we can prioritize that dashboard ( or underlying tables? Queries? ) to be 1) read into memory in advance of a user opening the dashboard and 2) have it stay in memory so the dashboard always gets the highest priority performance?

Corollary - after reading @Tristan's excellent article ( https://community.incorta.com/t5/dashboards-analytics/dashboard-design-for-performance/ta-p/219 ) I understand each security filter version is cached individually. The article is from last year - is that still the case?

Given that we will have row-level security is there anything we can do to prioritize the single or group of dashboards?

Re: Prioritize a dashboard ( using cache )?

Prince — Tue, 25 Jul 2023 14:48:37 GMT

Hi @RADSr ,

We do have 2 scopes of caching that helps faster rendering of dashboards:

Column caching, this is security filter independent
Insight queries' result sets, this is per user session in case this data has a security filter, otherwise, it is a single cache for all users

For the column caching, columns are loaded to memory first time it is used and cached until evicted, either due to underlaying schema refresh or tight memory.

We do have column warmup on the start up level (from the CMC). This only helps with restarts of analytics service
There's also a force reload , but this works by reloading all evicted columns after each load. Not dashboard specific
We are also introducing soon this year some post-load actions to help in scheduling the warmup & caching post data refresh.

For the insight's query result sets caching, they are calculated first time, then cached until either the underlaying columns/schemas refresh or no enough caching memory. To rerun and cache those result sets, you can do the following:

Schedule a dummy dashboard send/download to run the dashboard after a scheduled underlaying schema refresh. This will warmup the columns and cache the result sets.
If you have security filter and you are looking for caching specific user's result set, then you can schedule a burst report to that specific user, thus generate a cache version for that that specific user.

Re: Prioritize a dashboard ( using cache )?

RADSr — Wed, 06 Sep 2023 21:52:33 GMT

@Prince I'm just starting down this road, so near-completely uninformed about how this might be possible in Incorta, but if I were to create a new analytics node w/in my cluster can I create routing rules per dashboard to make sure I have 1) reserved memory for those insights and 2) a much lower turnover of cached query results? I'm really curious about creating routing rules for both this use case and a case where we would have an outside API query against our cluster.

Even a pointer as to where to start learning about this will be helpful.

Thx