Data & Schemas Knowledgebase

Using Databricks External Locations to Access Incorta Schemas and Tables

This guide explains how to create External Locations in Databricks to read schemas and tables directly from your Incorta cluster. Overview Databricks' external locations allow you to access your Incorta data stored in Azure Data Lake Storage Gen2 wi...

MV Failing to Load Due to Executor Lost Errors

Introduction A Materialized View (MV) fails during the transformation stage in Incorta. The error shows executors repeatedly being lost during Spark job execution. Example Error Logs ERROR: [00:01:45] [default | 81d7 | ETL | EXECUTION | RUNNING | Sa...

Using Microsoft Fabric Shortcuts to Access Incorta Schemas and Tables

This guide explains how to create shortcuts in Microsoft Fabric to read schemas and tables directly from your Incorta cluster, whether stored in Azure Data Lake Storage Gen2 or Google Cloud Storage. Overview Microsoft Fabric shortcuts allow you to a...

Securing Data in Google BigQuery: Implementing ORG_ID-Based Security for Oracle EBS Data

When working with enterprise systems like Oracle E-Business Suite (EBS), ensuring that users only see the data relevant to their operating unit (ORG_ID) is a critical security and compliance requirement. In this post, we’ll walk through how you can ...

Discover and Extract Data From Microsoft Fabric

Overview Fabric provides two main data experiences: Warehouses: Traditional SQL-based data warehouses optimized for structured analytics and business intelligence workloadsLakehouses: Open data architecture that combines the flexibility of data lake...

MLflow for MLOps within Incorta

This article discusses how you can use Incorta's built-in integration with MLflow to manage your machine learning lifecycle.

Efficient Incremental Data Ingestion from AWS S3 to Incorta

Use boto3 to help you load AWS S3 files incrementally.

Decoupling Advanced SQL Interface from Null Handling (2024.7.3 Update)

Learn about the implications of de-activating NULL handling for your analytics.

Save Your Work: Essential Tips for Incorta MV Notebook

Never lose your Incorta Notebook work.

Screen Shot 2024-04-30 at 5.00.34 PM.png

Screen Shot 2024-04-30 at 5.06.44 PM.png

Load Plans and Sequential Groups

Learn tips and tricks for getting the most out of Load Plans!

Incorta Notebook Best Practices

We bet there is one thing in here you didn't know about working with notebooks in Incorta.

Use hash keys to compare two records in PySpark

Use hash keys to simplify duplicate record identification.

Query physical schema data from PySpark using the incorta_sql() function

Introduction The incorta_sql() function lets you directly query physical schema data in a PySpark MV without going through the old and cumbersome way. What you need to know before reading this article Previously, you could use spark.sql() to query ...

Incorta MV execution succeeds in the Incorta Notebook but fails to save

Your MV ran successfully in Incorta Notebook, but when you tried to validate and save the MV, it failed.

Data Harmonization for Multi-Source

Introduction When creating visualizations based on multiple source systems, tables must be aligned to achieve a wholistic view of a business. For example, organizations that migrate from one ERP to another align invoice tables from each ERP to see al...