This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
I am developing a pipeline for some regression modeling I am experimenting with and I've got a working script and output that I am reasonably happy with. However I am unable to write new scripts using the ml library. I'm not even able to copy and pas...
I am trying to refactor some tables that were previously using pyspark to use scala. However whenever I try to read in my data I am met with an error that states "table or view not found"My code looks something like this: import org.apache.spark.sql....
I am trying to fit a simple linear regression with pyspark.ml in a materialized viewThe packages I am using are as follows: from pyspark.ml.feature import VectorAssembler
from pyspark.ml.regression import LinearRegression my code looks like this: #...
Hi,
I am trying to get regex to work in a materialized view but keep running into "undefined function" errors
Is there something wrong with my format?
SELECT Name,
UniversalId,
Item_Description,
REG...
I am trying to create a session variable that returns a specific column value (say column X) from the row with the most recent date value. However the filter expression in my query does not work at all and I am returning all X column values.Why is my...
Solved.Was an issue with my order of operations between my vector assembly and some additional filtering I was doing before I fit my model (not pictured in my code snippets).Filter first, then create your vector.Thanks,Matt
Thanks for the quick response!I've confirmed that I am using SparkSQL, although I am still running into the following error message: org.apache.spark.sql.AnalysisException: Undefined function: 'regexp'. This function is neither a registered temporary...
