Real-time Data Ingestion from Kinesis to Incorta with PySpark
Learn how to ingest real time Kinesis data with PySpark.
Knowledge Base Articles
How to Ingest Data from AWS S3 to Incorta using Incorta PySpark MV
Learn how to and why you would choose to ingest data from S3 with an MV
Enhancing Incremental Data Loads with Log-Based Approach
Introduction Managing incremental data loads efficiently is crucial for keeping your data warehouse up-to-date without putting unnecessary strain on your data sources. Incorta has traditionally relied on a query-based approach using either the LAST_...
Managing Data Growth with Incorta's New Data Retention Feature
Introduction As data volumes grow, especially with the proliferation of sensors and IoT devices or long-term usage of Incorta, it's essential to manage the amount of data stored on disk effectively. Increasing data size can lead to storage issues an...
Using Data Purge Feature For Handling Source Deletes
Incremental data loads in Incorta insert new records and update existing ones, but they don't automatically delete records when they're removed from the source system (like Oracle Cloud ERP). Since source systems often allow data deletion, this can ...
Loading JSON data to Incorta using Materialized View - Part 1
SymptomsDiagnosisSolution1. Read Data into Spark2. Create JSON Schema3. Parse JSON and Create a Temporary Column4. Explode Nested JSON Array5. Save the Final DataFrame Download Symptoms JSON data are provided by the source system for sharing with ...
Decoupling Advanced SQL Interface from Null Handling (2024.7.3 Update)
Learn about the implications of de-activating NULL handling for your analytics.
How to convert complex Oracle sqls in the Incorta way
This article elaborates the steps to convert complex Oracle SQLS or views to Incorta.
OTBI Lineage
Use the Incorta OTBI Lineage dashboard to determine the mapping between Oracle tables and VOs.
Parse a JSON string from column to flattened structure in Incorta
Question How do I parse a JSON string from column to flattened structure in Incorta? Answer Assuming a Incorta table has a column with JSON string then here is a sample pyspark code which you can use to create a Materialized view in Incorta - from...
Access AWS services from an Incorta MV
Learn how to access AWS services from an Incorta Materialized View.
Issue with converting a Pandas DataFrame to a Spark DataFrame
What to do if you receive the 'iteritems' error when trying to convert a Pandas DataFrame to Spark DataFrame.
