0

Save and Reuse Sklearn ML Models

Hi,

I try to save and reuse ML model in Incorta. But this models builded with Sklearn. There is some ways to save Spark and Incorta ML models but I cnt find any about others libs like Sklearn. 

I tried to some things like below to save models. I don't get any error when saving but when try to load model I get not found error message.

save model:

model_name="time_series_model_2021_09_16_08_15_21.sav"

pickle.dump(best_model, open(model_name, 'wb'))

load model:

loaded_model = pickle.load(open("time_series_model_2021_09_16_08_15_21.sav", 'rb'))
print(result)

Fail to execute line 17: loaded_model = pickle.load(open("time_series_model_2021_09_16_08_15_21.sav", 'rb')) Traceback (most recent call last): File "/tmp/zeppelin_pyspark-7449725614426020440.py", line 383, in <module> exec(code, _zcUserQueryNameSpace) File "<stdin>", line 17, in <module> IOError: [Errno 2] No such file or directory: 'time_series_model_2021_09_16_08_15_21.sav'

Below link is work only for Spark ML and Incorta ML. And "model/xxx" is not work also.  

https://community.incorta.com/t/m1h6wcq/save-and-reuse-ml-models

Thanks

2replies Oldest first
  • Oldest first
  • Newest first
  • Active threads
  • Popular
  • Can you try to give the file a full path when you save and load it?

    Here is an example -https://suziepyspark.blogspot.com/2021/03/using-pickle-to-save-model-in-incorta.html

    Like 1
  • Yes I did same things on-prem Incorta and no problem with saving and loading models. But the case is about cloud Incorta.

    Below script handle saving of model with full path and other attributes. Incorta doing this things with itself for Incorta ML models and spark models. But for others it is not work. 

    model.write().overwrite().save("models/modelname")

     

    Like
Like Follow
  • 1 mth agoLast active
  • 2Replies
  • 31Views
  • 2 Following

Product Announcement

Incorta 5 is now Generally Available