Using The Inspector Tool

[ccastleman]

Introduction

For a given tenant, the Inspector Tool checks the lineage references of Incorta metadata objects including tables, schemas, business schemas, business schema views, dashboards, and session variables. It also checks for inconsistencies and validation errors in joins, tables, views, formulas, and dashboards.

 What you should know before reading this article

We recommend that you be familiar with these concepts.

• Enabling the Inspector Job
• Loading and Reviewing Inspector data
• Using Inspector Dashboards

Applies To

Applies to all Incorta versions version 4.8 and above.

Let's Go

Because the tool uses Incorta itself for reporting issues, the Inspector Tool consists of several components:

• A Cluster Management Console (CMC) scheduled job that generates .csv files in a given tenant directory as a local data files (one-time task to enable this for a given schema)
• The InspectorMetadata physical schema
• The incortaInspector business schema
• The InspectorTool dashboard folder and related dashboards

Once the Inspector job is enabled, data can be loaded on a periodic or scheduled basis to consume the data files generated by the Inspector CMC job.  

The InspectorMedata schema contains the following tables:

• BUSINESS_SCHEMA_VIEWS
• JOINS_DETAILS
• LINEAGE_REPORT
• MV_REFERENCED_TABLES
• SCHEMA_TABLES
• VALIDATION

The incortaInspector business schema contains the following folder and views:

• TenantHierarchy (folder)
  • DashboardLineage
  • Schemas
  • Joins
  • BusinessSchemas
  • MVs
• validation

In the InspectorTool folder, there are several Inspector Tool Dashboards:

0-Run Status

This dashboard captures runtimes of the Inspector Tool. In doing so, the dashboard captures basic counts for the various objects within a tenant for each runtime. The insights are:

• Tenant Validation
• Tenant_Dashboards
• Tenant_Joins
• Tenant_Schemas
• Tenant_BusinessSchemas
• Tenant_MVs

1 - Validation Use Cases 

The Validation UseCases dashboard identifies errors in the tenant such as broken references, duplicates, and invalid formulas. Employ the dashboard filters to quickly identify high severity issues. Focus on fixing all high severity (1 & 2) issues. The dashboard insights are:

• Summary
• Errors & Warnings
• Error & Warning Codes
• Errors & Warnings Per Schema
• Errors Details
• Warnings Details

2 - Unused Entities

This dashboard identifies columns and business schema columns that are not used in dashboards. You may consider removing unused columns from business schema or schemas. Removing unused columns can help improve schema load times, reduce storage needs, and free memory. The dashboard insights are:

• Unused Columns
• Unused business schema columns

3 - Schemas Details

The Schemas Details dashboard provides a detailed view of all the schemas in a tenant, including the tables, joins, and columns. The dashboard insights are:

• Schemas Summary
• Tables
• Joins
• Columns

4 - Dashboards Lineage Summary

Use the dashboard to get a quick view of data elements or calculations that could possibly be moved closer to the source. Employ the dashboard filter to help identify the lineage relationships of all the tenant’s dashboards. The details include the insights type as well as the source columns. The dashboard insights are:

• Summary
• Dashboards

5 - Tables Used in Business Views

Use this dashboard to identify all the business schemas in a tenant, the view names, and the columns. he dashboard insights are:

• Summary
• Business Schemas

6 - Tables Used in Materialized Views

Use this dashboard to identify all the materialized views in a tenant, including any referenced tables within the code or script of the materialized view.

Each of these dashboards provides insights into the status of the data modeling configurations in that Tenant and will often highlight issues which can be addressed through further configurations, mapping and so forth.  The Inspector Dashboards can also be used with the Metadata Analytics Dashboards to Identify systemic issues as well.

Best Practice: Using the Incorta Inspector prior to system upgrade

Prior to completing an upgrade of any Incorta implementation, we recommend using the Incorta Inspector (and especially the 1- Validation UseCases dashboard) to identify issues that will need to be corrected prior to beginning the upgrade process.  Re-running this dashboard until there are no issues and validating the changes made is strongly encouraged.  At a minimum, all Severity 1 issues need to be resolved to prevent issues with the upgrade process itself.  An example might be an invalid join or a column referenced in a formula that no longer exists in the system.