Introduction
Incorta is a full stack data analytics platform that greatly accelerates the traditional BI development cycle; however, we still recommend following traditional software development life cycle (SDLC) best practices to ensure end users are leveraging accurate data and tested functionality. With a SDLC there are typically 5-8 steps but in Incorta best practices we usually focus on requirements, design, development, testing, and deployment. This article will focus on development, testing, and deployment.
What you should know before reading this article
We recommend that you be familiar with these Incorta concepts before exploring this topic further.
Applies to
This article applies to any version of Incorta and includes both on-premises and cloud deployments.
Let’s Go
Environments
In order to follow SDLC best practices application, development should be performed in a separate environment from the production application. Incorta strongly recommends that customers develop their data models, views, and dashboards in a dedicated Development environment and release changes to a separate Production environment. This would be considered the minimum infrastructure for following development best practices. Many customers choose more formal infrastructure topology such as a Dev -> Test -> Production, Dev -> Test -> User Acceptance Test (UAT) -> Production, Dev -> Test -> UAT -> Production -> Disaster Recovery (DR), etc. Regardless of the degree to which SDLC is followed, the crucial details are:
- Develop assets outside of production
- Develop/test with production data volumes
- Promote changes/new content/fixes from the lowest environment to the highest
Some customers make a distinction between IT-managed content development and self-service content management. We will first focus on IT-managed content.
Development
The Development environment for IT-managed content should be used and managed similarly to other in-house custom applications: gather requirements, connect data source(s), build out logical views, and then build the dashboards on the logical views.
Once there are existing use cases in higher level environments, Development will be used to work on changes and fixes for existing content as well as to build out new use cases.
Test/UAT
It is very advantageous to have a dedicated environment for testing and user acceptance testing. This allows developers to continue working and not risk interruption to the testing. Once the development work has been validated, it is time to promote to the end user environment(s).
Production
For IT-managed content, the production environment should be considered locked down with edit access limited to the administration team responsible for managing the environment. Content should not be edited directly in production; rather, it should be promoted via a regimented SDLC process.
Self-Service Considerations
Incorta makes it easy to allow business users the ability to create their own content including dashboards, business schemas, and physical schemas. The most common scenario for self-service is allowing business users to create dashboards based on IT-managed business schemas and physical schemas. This pattern allows data discovery on shared assets but protects the base objects from accidental errors. More advanced use cases have business users building business schemas with available physical schemas and even uploading their own data and managing data models.
Managing Content
User-owned content in downstream environments should be kept separate from IT-owned content. This is pretty easy for dashboards by leveraging folders and Group security and similarly for business and physical schemas using Group security. Ideally, any environment with user-generated content should be backed up at least daily to account for accidents (deletes, bad edits, etc). Backups can be achieved using the CMC tenant backup scheduler which can be configured to the customer’s preferences. With version 5, Incorta has introduced an Incorta Labs feature that automatically stores backups of dashboards and physical schemas allowing an administrator or developer to quickly revert to previous versions.
Performance Considerations
Generally, allowing business users the ability to build dashboards upon IT-managed business schemas in Production is low risk. For more advanced use cases involving business users adding high volume data sources, physical schemas, and data science on large data sets it is important to monitor the Production environment closely. If self-service work starts to disrupt other users’ dashboard performance, one option is to simply add another analytics node to the Incorta cluster and, using DNS routing, direct self-service users to their own dedicated query node. Should self-service work impact data-loading performance of the IT-managed objects, it is possible to add another loader node and dedicate it to self-service scenarios: with this arrangement, the IT-managed schemas will load on one dedicated Loader node and self-service schemas load on a separate dedicated Loader node.
Supporting Materials
Software development lifecycle
.png "Employee"){kind=icon}
-modified.png?version=preview "Employee Alumni"){kind=icon}
