cancel
Showing results for 
Search instead for 
Did you mean: 

Guidelines for SPARK property settings?

RADSr
Partner
Partner

I've got an MV using spark sql running for ~ 1.5 hours ( and counting )  - the same query ran in under 4 minutes using an Incorta SQL derived table.

For prior issues support has recommended adding some properties to the datasource screen.

Is there a guide as to when to use which?   It would be absolutely swell to be able to translate this ( screencap from drilling down in Spark /applications page ) into an informed action.

RADSr_0-1696960023640.png

* Incorta UI still shows this as "enriching."    Candidly I don't know whether all those "FAILED" messages are telling me to wait it out while it retries, or to give it up because something essential has already gone wrong...    

 

3 REPLIES 3

RADSr
Partner
Partner

I killed that load after about 2 hours, applied some settings from an earlier ( different MV ) support ticket and it went down to 1 minute 24 seconds.

I'm thinking I should just default to those settings for everything...    

The default values can be configured from the CMC.

Screen Shot 2023-10-24 at 6.00.12 AM.png

dylanwan
Employee
Employee

Incorta MV assistant feature was created for this purpose.

Those failed messages mean that the Spark cluster keeps trying to rerun a task but failed may be due to resource issue.