Data & Schema Discussions

Load failures are sometimes not graceful

Hi,Some of our schema loads will occasionally fail due to memory errors. On this I would like to pass on some feedback as well as seek some advice.The feedback is that depending on how it falls over it might end up breaking all practical data access ...

Resolved! Trying To Fit A Linear Regression with pyspark.ml

I am trying to fit a simple linear regression with pyspark.ml in a materialized viewThe packages I am using are as follows: from pyspark.ml.feature import VectorAssembler from pyspark.ml.regression import LinearRegression my code looks like this: #...

Resolved! Java version for data agent?

Cloud documentation for data agent says to use OpenJDK11 ( snip below ) but the linked page doesn't have an OpenJDK11. Numerically it looks like JDK19 is the most recent GA release ( there's no "Open" on the link ) -- is that good for the data ag...

Spark Sql Execution Plan

Is there any way to get the Spark execution plan in an MV when selecting Spark SQL? I know I can take the below approach, but I prefer to do this in Spark SQL and not have to rebuild the Spark SQL queries. %pysparkdf = read("EBS_PARTY_COMMON.HZ_PARTI...

Data source in Incorta

HII need assistance in setting up a data source in Incorta. I have a colleague who updates an Excel file monthly on his device. I would like to know the best way to use this file as a data source in Incorta, allowing my colleague to continue updating...