3

Trigger a Incorta Load through a script or similar

We have an external scheduling tool from which can run shell/cmd scripts on a machine..

we want to trigger an incorta job load using this tool (Appworx,Autosys etx)

Can you please let me know how can we trigger the same.

15replies Oldest first
  • Oldest first
  • Newest first
  • Active threads
  • Popular
  • Incorta provides a command line interface for submitting an incorta job load.

    You can find those shell scripts under <incorta home>/IncortaNode/bin where it contains a sample: load_stagging_sample.sh

    #$incorta_cmd <load type> $session <Schema Name>

    The valid <load type> key words are
    load_schema
    load_schema_incremental

    Like
  • Thanks Dylan Wan

    can we also add parameter to load a specfic table in the schema ? 

    Like
    • Gnaneshwar Muthaluri  It supports load individual tables when you are using load_schema as the load type .

      #$incorta_cmd load_schema $session <Schema Name> <table name>

      Like
  • Thanks Dylan Wan

    I have tried it and it worked fine.

    but when we submit the command it does not wait until the load is completed and does not share the status of the load.

    Can you please guide if there is way to get the status of the load or any other statistics regarding the load.

    Like
  • We are having a similar issue. The script does not wait for the load to complete. How do we get the status of the load ?  In case of errors we prefer to send a notification email to our support team.

    Like
  • Ok, we have modified the load cli command to add few options like 

    1. Wait for the load to complete in synchronous mode before control returns

    2. Display some stats about the tables how many rows loaded etc, success, failure.

    3. Option to redirect output to a file instead of displaying on screen so that scheduling systems can read that file

    We are testing this script and will share with you shortly.

    Like 1
    • Mateen Mohammed Please see below.  Can you post the script here?  Mark Otto from Now Foods would like to put this new script with the "Wait for load to complete" feature to use. 

      Like
    • Dan Brock Mateen Mohammed here is a PDF of the Wiki, and also the shell script and python file.  Note the Filename for the .sh file had to be renamed in order to be uploaded here.

      Jeff

      Like
  • Great.. Thank you for you prompt response... will wait to hear from you.

    Like
  • Any update on this? We have been using the python script with cron and it has been working much better than the built-in Scheduler. Leveraging the script, we have more granularity/flexibility with the schedules. It would be great to have the ability to wait for one load to finish before the next load is kicked off.

    Like 1
    • Mark Otto we would also be very interested in this. 

      Another option would be in the UI to create something like "Load Groups". The idea is that you could create groups of schemas that would be controlled by a single start time. We have about 5 schemas that load every morning and it would sure help to speed up the load if we just said that they start at 5:00am and launch the first and proceed through the list. As it is right now, we probably take more than an hour of extra buffer time that is not usually needed. 

      I wonder if this would require a ticket/enhancement request to get it done.

      Like
    • Martin Carnegie  - Thanks for raising this request. We fully agree on the need and the value of this feature. We have plans to add this feature to the product in an upcoming release later this fall. 

      Like
      • Mark Otto
      • NOW Health Group, Inc.
      • Mark_Otto
      • 4 mths ago
      • Reported - view

      Ashwin Warrier It's great that the scheduler will get the enhancement mentioned by Martin , but can you provide an update on my post regarding the script that Mateen Mohammed mentioned in his post ~10 months ago? The current python script has been working much better than the current scheduler UI, but with the scripting we need the ability that Gnaneshwar Muthaluri asked about ~11 months ago.

      Like
Like3 Follow
  • Status Answered
  • 3 Likes
  • 3 mths agoLast active
  • 15Replies
  • 207Views
  • 15 Following

Product Announcement

Incorta 4.9 is now Generally Available (GA)!!!