This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
IntroductionSymptomsDiagnosisSolutionRead the rule tableDynamic SQL generation
Introduction
In data processing, you may encounter situations where you need to implement user-defined business rules. PySpark is a powerful tool for generating SQL dyn...
Problem
You need to ingest data from S3 to Incorta. The data is in CSV format and requires transformations. While Incorta's Data Lake connector can read data from S3, it loads the data in raw format before transformations can be applied. Transformin...
SymptomsDiagnosisSolution1. Read Data into Spark2. Create JSON Schema3. Parse JSON and Create a Temporary Column4. Explode Nested JSON Array5. Save the Final DataFrame Download
Symptoms
JSON data are provided by the source system for sharing with ...
OverviewWhat is Chidori?Incorta and Sparkflows IntegrationConfigure the ConnectionsCreate Project and workflow in SparkflowsML Model RegistryCreate an Incorta MV using the PySpark generated by Sparkflows
Overview
Sparkflows provides a no code or low...
Introduction
Incorta MV Notebooks offer an alternative way to edit Spark code used to generate materialized views (MVs). This article provides a couple of tips that allows you to save your work while doing development.
What you need to know before r...
We designed this purposely to avoid information leakage and compliance. An admin who can impersonate any one in the enterprise without notifications and can enable/disable this feature is not acceptable by security compliance.
Yes we have customers who have the same requirement and we use LLM capability to parse the PDF.
We created a custom recipe in the DataStudio for loading the pdf files from cloud storage like S3 or Azure. LLMs, like Gemini, have a good capability of ...
Your requirement should be addressed by building insights over a result set.
Building a result set is like creating an aggregate table insight, and the data will be available for building the insight.
The result set in your case will have the AVERAGE...
Incorta on-prem installation packaged Zookeeper, Spark, and Hadoop. The installation package and the installer will install these components
MySQL, Java, Python, and font families are not shipped with the software and you need to prepare your env be...
The reason why show the last schema refresh start time is to let the user know how fresh the data they are looking at.
What is the purpose of viewing loading completion time?
The JOB history table from the _IncortaMetadata schema has the information....
.png "Employee"){kind=icon}
