Spark Sql Execution Plan
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-07-2023 08:05 AM
Is there any way to get the Spark execution plan in an MV when selecting Spark SQL? I know I can take the below approach, but I prefer to do this in Spark SQL and not have to rebuild the Spark SQL queries.
%pyspark
df = read("EBS_PARTY_COMMON.HZ_PARTIES")
df.registerTempTable("TBL")
spark.sql("""
SELECT count(*)
FROM TBL
""").explain(mode="formatted")
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-13-2023 01:10 PM
<IncortaCluster>/applications will get you to the SPARK screen which is full of drillable links and contains all kinds of information including ( I think ) the explain plan if you drill far enough.
-- IncortaOne@PMsquare.com --
