Incorta is a full stack data analytics platform that greatly accelerates the traditional BI development cycle; however, we still recommend following traditional software development life cycle (SDLC) best practices to ensure end users are leveraging accurate data and tested functionality. With a SDLC there are typically 5-8 steps but in Incorta best practices we usually focus on requirements, design, development, testing, and deployment. This article will focus on development, testing, and deployment.
We recommend that you be familiar with these Incorta concepts before exploring this topic further.
This article applies to any version of Incorta and includes both on-premises and cloud deployments.
In order to follow SDLC best practices application, development should be performed in a separate environment from the production application. Incorta strongly recommends that customers develop their data models, views, and dashboards in a dedicated Development environment and release changes to a separate Production environment. This would be considered the minimum infrastructure for following development best practices. Many customers choose more formal infrastructure topology such as a Dev -> Test -> Production, Dev -> Test -> User Acceptance Test (UAT) -> Production, Dev -> Test -> UAT -> Production -> Disaster Recovery (DR), etc. Regardless of the degree to which SDLC is followed, the crucial details are:
Some customers make a distinction between IT-managed content development and self-service content management. We will first focus on IT-managed content.
The Development environment for IT-managed content should be used and managed similarly to other in-house custom applications: gather requirements, connect data source(s), build out logical views, and then build the dashboards on the logical views.
Once there are existing use cases in higher level environments, Development will be used to work on changes and fixes for existing content as well as to build out new use cases.
It is very advantageous to have a dedicated environment for testing and user acceptance testing. This allows developers to continue working and not risk interruption to the testing. Once the development work has been validated, it is time to promote to the end user environment(s).
For IT-managed content, the production environment should be considered locked down with edit access limited to the administration team responsible for managing the environment. Content should not be edited directly in production; rather, it should be promoted via a regimented SDLC process.
Incorta makes it easy to allow business users the ability to create their own content including dashboards, business schemas, and physical schemas. The most common scenario for self-service is allowing business users to create dashboards based on IT-managed business schemas and physical schemas. This pattern allows data discovery on shared assets but protects the base objects from accidental errors. More advanced use cases have business users building business schemas with available physical schemas and even uploading their own data and managing data models.
User-owned content in downstream environments should be kept separate from IT-owned content. This is pretty easy for dashboards by leveraging folders and Group security and similarly for business and physical schemas using Group security. Ideally, any environment with user-generated content should be backed up at least daily to account for accidents (deletes, bad edits, etc). Backups can be achieved using the CMC tenant backup scheduler which can be configured to the customer’s preferences. With version 5, Incorta has introduced an Incorta Labs feature that automatically stores backups of dashboards and physical schemas allowing an administrator or developer to quickly revert to previous versions.
Generally, allowing business users the ability to build dashboards upon IT-managed business schemas in Production is low risk. For more advanced use cases involving business users adding high volume data sources, physical schemas, and data science on large data sets it is important to monitor the Production environment closely. If self-service work starts to disrupt other users’ dashboard performance, one option is to simply add another analytics node to the Incorta cluster and, using DNS routing, direct self-service users to their own dedicated query node. Should self-service work impact data-loading performance of the IT-managed objects, it is possible to add another loader node and dedicate it to self-service scenarios: with this arrangement, the IT-managed schemas will load on one dedicated Loader node and self-service schemas load on a separate dedicated Loader node.