topic Data APIs for External Notebooks - save parquet in Data & Schema Discussions

Data APIs for External Notebooks - save parquet

marcioribeiro — Tue, 21 Jun 2022 17:36:22 GMT

Hi All,

I´m using external Jupyter notebook with python to capture data from Oracle database. Jupyter Notebook and Incorta server doesn´t share any storage location.

I´ve downloaded the sample-notebook.ipynb notebook and I´m using it as example to connect into Incorta tenant.

When I used the function incorta.save_parquet(df, 'test.parquet'), the parquet file is saved locally in Jupyter notebook server instead of saving in Incorta remote server.

Is this an expected behaviour or the parquet file should be saved in Incorta server?

If this is expected, what is the meaning of connecting to Incorta Tenant and use incorta package to generate a parquet file, if parquet file is stored locally, same as using python only script ?

Thanks,

Re: Data APIs for External Notebooks - save parquet

dylanwan — Tue, 21 Jun 2022 18:20:46 GMT

Thanks for trying to use the External Notebook to connect to Incorta. The Incorta data API for External Notebook will save to the tenant folder based on the tenant you specified in the property file. We made the assumption that the Incorta Data API is deployed to a server that can access the shared drive with the same path. If you don't have the shared storage, it will save to the same path but not as local files as you described.

Re: Data APIs for External Notebooks - save parquet

KailaT — Thu, 18 Aug 2022 12:53:27 GMT

Hi @marcioribeiro was Dylan's response able to answer your question?