To ensure that the Incorta cluster is properly sized for the data being ingested from Oracle Cloud ERP (Fusion) we need to ascertain the source data size. There is no direct way to run ad hoc SQL queries on the Oracle Cloud ERP database, unlike tradi...
Learn how to ingest real time Kinesis data with PySpark.
Learn how to and why you would choose to ingest data from S3 with an MV
Introduction Managing incremental data loads efficiently is crucial for keeping your data warehouse up-to-date without putting unnecessary strain on your data sources. Incorta has traditionally relied on a query-based approach using either the LAST_...
Introduction As data volumes grow, especially with the proliferation of sensors and IoT devices or long-term usage of Incorta, it's essential to manage the amount of data stored on disk effectively. Increasing data size can lead to storage issues an...
Incremental data loads in Incorta insert new records and update existing ones, but they don't automatically delete records when they're removed from the source system (like Oracle Cloud ERP). Since source systems often allow data deletion, this can ...
Question How do I parse a JSON string from column to flattened structure in Incorta? Answer Assuming a Incorta table has a column with JSON string then here is a sample pyspark code which you can use to create a Materialized view in Incorta - from...
Learn how to access AWS services from an Incorta Materialized View.
Symptoms After an upgrade, there is a possibility that incortaMetaData schema would fail with the following error: com.incorta.api.exceptions.DataException: INC_03070000: Failed retrieving sample data from [_IncortaMetadata] with properties [[increm...
Quickly save Pandas DataFrames as materialized views without manually defining each column's data type and function. These simple functions allow the switch between Spark and Pandas more efficiently.
Delta Sharing is a simple REST protocol that securely shares access to part of a cloud dataset. Learn more here!
Introduction It is a best practice to test configuration in the development/test environment(s) before moving it to production. In some cases, production data is loaded into the Dev/Test environments for validation purposes as well. This article di...
