This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Problem
You need to ingest data from S3 to Incorta. The data is in CSV format and requires transformations. While Incorta's Data Lake connector can read data from S3, it loads the data in raw format before transformations can be applied. Transformin...
SymptomsDiagnosisSolution1. Read Data into Spark2. Create JSON Schema3. Parse JSON and Create a Temporary Column4. Explode Nested JSON Array5. Save the Final DataFrame Download
Symptoms
JSON data are provided by the source system for sharing with ...
OverviewWhat is Chidori?Incorta and Sparkflows IntegrationConfigure the ConnectionsCreate Project and workflow in SparkflowsML Model RegistryCreate an Incorta MV using the PySpark generated by Sparkflows
Overview
Sparkflows provides a no code or low...
Introduction
Incorta MV Notebooks offer an alternative way to edit Spark code used to generate materialized views (MVs). This article provides a couple of tips that allows you to save your work while doing development.
What you need to know before r...
Scenario
You need to access AWS services, such as S3, Kafka, or Kinesis, from an Incorta Materialized View (MV) for extracting data.
Issues
You prefer not to list the credentials in the MV script.
Solution
You can set up AWS to use temporary secur...
You can consider using LEFT OUTER JOIN if the table may not have a matching record. If there are multiple such SELECT clause lookup table, you can use CTE.
WITH conversion_rate_LKP AS ( SELECT conversion_rate, ROW_NUMBER() OVER (PARTITION...
Manually editing the schema XML files may result in a corrupted Incorta metadata and can cause the entire system not workable. Even a small typo with the schema name, which can be sensitive, caused us a lot of time to debug as the symptom is on the ...
We support the migration using
CLI - You can access the command line interface to export and import objectsUI - You can manually export and import objects from the UI
.png "Employee"){kind=icon}
