Data Profiling

Can you please describe what are you looking for in data profiling?

Hi, IncortaVi

You can do data profiling using Spark Python in Incorta. 

Method 1:

Using df.describe() 

This function can provide min, max, count, mean, stddev. But only for data types of string and number. 

Method 2:

Calculate each metric ourselves. 

For more detail please find blow link: 

https://suziepyspark.blogspot.com/2021/02/how-to-do-data-profiling-in-incorta.html 

Hope it can help you.