An Incorta job runs for more than 40 hours and none of us noticed until business users contacted us and asked why they do not see the updated data from the dashboard. Incorta provided us the Long Spinning Job Alert before. Why does it not work?
Have you ever needed to calculate a Z-score?
This article provides more information about the Avro Extractor tool that generates an Avro file from a provided JSON file.
Apache Kafka is a distributed publish-subscribe messaging system that can handle a high volume of data and enables you to pass messages from one end-point to another.
Use Incorta to prepare data and create an ML model in DataRobot. You can then send the data as part of an Incorta data pipeline to DataRobot for making predictions. The DataRobot Batch Prediction API can be called from an Incorta MV to complete the prediction job.
We would like to get the current hour, how can we get it in Spark SQL? We also need the time as of one hour ago, how do we do that?
Use select(), filter(), arrange(), and group_by() functions to handle Spark DataFrame on R.
A take-home guide for an introduction to Machine Learning in the Incorta platform.
IntroductionIncorta Materialized Views (MV) provide a way to run PySpark, Scala, and Spark R and can be used for building machine learning (ML) models. We will discuss PySpark, Scala, and Spark R separately. Here are the best practices of using Incor...
IntroductionWhat you should know before reading this articleApplied toLet's GoMulti-class classification data Introduction The train-test split procedure is used to evaluate the performance of machine learning (ML) algorithms. As part of data pre...
As a part of data cleaning in Machine Learning, you may need to convert the data from one data type to the other data type. In this article, you will learn how to use Incorta Notebook to covert the data. After you read data in Incorta Notebook, you c...
Introduction Incorta supports the Machine Learning(ML) model creation process by using Incorta Materialized Views (MV). While you can put the logic of applying the ML model testing and actually use of the ML model for inference in the same MV, the ...
Preview Data in Incorta Notebook In this article, you will learn how to preview data in Incorta Notebook. Incorta notebook provide methods to let you preview data. using show(), head(), printSchema(), and describe(). incorta.show(df) incorta.hea...
Find and Fill Missing Data in Incorta NotebookIn this article, you will learn how to find and fill missing values in Incorta Notebook. Copy below find null value reusable codes. from pyspark.sql.functions import isnan, when, count, col, lit def colu...
