cancel
Showing results for 
Search instead for 
Did you mean: 

Spark Sql Execution Plan

Is there any way to get the Spark execution plan in an MV when selecting Spark SQL? I know I can take the below approach, but I prefer to do this in Spark SQL and not have to rebuild the Spark SQL queries.

 

%pyspark

df = read("EBS_PARTY_COMMON.HZ_PARTIES")
df.registerTempTable("TBL")
spark.sql("""
SELECT count(*)
FROM TBL
""").explain(mode="formatted")

1 REPLY 1

RADSr
Partner
Partner

<IncortaCluster>/applications will get you to the SPARK screen which is full of drillable links and contains all kinds of information including ( I think ) the explain plan if you drill far enough.