Incorta Copilot index_chroma

[mrossPM2]

I realize I'm probably going to need to log a ticket for this, but wanted a little bit more information around this command as well.

I was reading this post (https://community.incorta.com/t5/dashboards-analytics/copilot-in-interactive-notebooks/ta-p/5494) with different ways to use Copilot in the notebooks. It mentions running the index_chroma command. I've been getting some odd responses to my requests using Copilot, so I figured it wouldn't hurt to run that command.

It's been three hours, and it still says its at zero percent. This is in a cluster that has less than 500 megabytes of data in it. Something obviously is wrong (hence why I'll likely need to log a ticket).

Are there any benchmarks that can be provided as to how long this process would usually take relative to how much data is in the cluster?

Does it take a long time to run the first time, and then subsequent runs are faster (i.e. it is only looking for changes on subsequent runs)? Or does it take roughly the same amount of time each time you run the command?

Is this something we should run after any changes to the schema?

Does this process read anything done in the business schema? For example, if we added a new calculation to a business schema view based on existing schema columns, would we need to run this process again?

Any insight that could be offered up around what this command actually does would be appreciated.

[JoeM]

@mrossPM2 - In early iterations of the copilot in the notebook, indexing them is a manual step. This is only true of the copilot in the notebook, while the dashboard copilot updates as changes are made to business schemas. Automated indexing is on the short list for added features here.

Running @index_chroma will run against data not previously indexed, or add new schemas, business schemas, and tables. You can run @index_chroma_from_scratch to overwrite existing indexing and start over. Do note that it does index on business schemas - so it would be good to re-index after changes and formulas are added. 

In my small environments, index builds take roughly 10 minutes from scratch. I've heard rumblings of an issue where indexes are not accurately reading out progress. Since things are running slow, or not showing progress correctly, I would connect with support to investigate.

@oabdelkarim for visiblility.

[mrossPM2]

Thanks Joe for the commentary. So far, the index_chroma_from_scratch has returned the same 0% progress issue. I've logged a support ticket.