Save and Reuse Sklearn ML Models


I try to save and reuse ML model in Incorta. But this models builded with Sklearn. There is some ways to save Spark and Incorta ML models but I cnt find any about others libs like Sklearn. 

I tried to some things like below to save models. I don't get any error when saving but when try to load model I get not found error message.

save model:


pickle.dump(best_model, open(model_name, 'wb'))

load model:

loaded_model = pickle.load(open("time_series_model_2021_09_16_08_15_21.sav", 'rb'))

Fail to execute line 17: loaded_model = pickle.load(open("time_series_model_2021_09_16_08_15_21.sav", 'rb')) Traceback (most recent call last): File "/tmp/zeppelin_pyspark-7449725614426020440.py", line 383, in <module> exec(code, _zcUserQueryNameSpace) File "<stdin>", line 17, in <module> IOError: [Errno 2] No such file or directory: 'time_series_model_2021_09_16_08_15_21.sav'

Below link is work only for Spark ML and Incorta ML. And "model/xxx" is not work also.  



3replies Oldest first
  • Oldest first
  • Newest first
  • Active threads
  • Popular
  • Can you try to give the file a full path when you save and load it?

    Here is an example -https://suziepyspark.blogspot.com/2021/03/using-pickle-to-save-model-in-incorta.html

    Like 1
  • Yes I did same things on-prem Incorta and no problem with saving and loading models. But the case is about cloud Incorta.

    Below script handle saving of model with full path and other attributes. Incorta doing this things with itself for Incorta ML models and spark models. But for others it is not work. 



    • Omer AKGUL  I have a open bug logged for this

Like Follow
  • 2 mths agoLast active
  • 3Replies
  • 49Views
  • 3 Following

Product Announcement

A new community experience is coming! If you would like to have beta access to provide feedback, please contact us at community@incorta.com.