Real-time Data Ingestion from Kinesis to Incorta with PySpark
Learn how to ingest real time Kinesis data with PySpark.
Knowledge Base Articles
How to Ingest Data from AWS S3 to Incorta using Incorta PySpark MV
Learn how to and why you would choose to ingest data from S3 with an MV
Enhancing Incremental Data Loads with Log-Based Approach
Introduction Managing incremental data loads efficiently is crucial for keeping your data warehouse up-to-date without putting unnecessary strain on your data sources. Incorta has traditionally relied on a query-based approach using either the LAST_...
Managing Data Growth with Incorta's New Data Retention Feature
Introduction As data volumes grow, especially with the proliferation of sensors and IoT devices or long-term usage of Incorta, it's essential to manage the amount of data stored on disk effectively. Increasing data size can lead to storage issues an...
Using Data Purge Feature For Handling Source Deletes
Incremental data loads in Incorta insert new records and update existing ones, but they don't automatically delete records when they're removed from the source system (like Oracle Cloud ERP). Since source systems often allow data deletion, this can ...
Parse a JSON string from column to flattened structure in Incorta
Question How do I parse a JSON string from column to flattened structure in Incorta? Answer Assuming a Incorta table has a column with JSON string then here is a sample pyspark code which you can use to create a Materialized view in Incorta - from...
Access AWS services from an Incorta MV
Learn how to access AWS services from an Incorta Materialized View.
Incorta MetaData is failing with Table 'incortametadata.CATALOG_LINK' doesn't exist
Symptoms After an upgrade, there is a possibility that incortaMetaData schema would fail with the following error: com.incorta.api.exceptions.DataException: INC_03070000: Failed retrieving sample data from [_IncortaMetadata] with properties [[increm...
Saving a Pandas DataFrame as a Materialized View (Convert to Spark)
Quickly save Pandas DataFrames as materialized views without manually defining each column's data type and function. These simple functions allow the switch between Spark and Pandas more efficiently.
Delta Sharing
Delta Sharing is a simple REST protocol that securely shares access to part of a cloud dataset. Learn more here!
Move Data from Dev/Test to Prod
Introduction It is a best practice to test configuration in the development/test environment(s) before moving it to production. In some cases, production data is loaded into the Dev/Test environments for validation purposes as well. This article di...
Create an ODBC Connection to Incorta
Applies to: Any Incorta version On-Prem/Cloud Question: How to configure an ODBC connection?Unable to connect to Incorta using ODBC Overview: Any program that uses Open Database Connectivity (ODBC) to connect to a remote database needs a way to iden...
