0

Can I create a function ( UDF ) in Incorta? ( use case: csv files show percents, I need to convert to decimal - a *lot* ;-) )

The title sums it up - I have a need to take a text input ( e.g. 16.7% ) and conditionally format the output ( e.g. > 15% green, > 10 yellow, etc. )     Since the comparison operator wants a number I need to convert the text to a decimal.   Doing it isn't difficult, but I'd like to make it a standard and easily repeatable operation since we will have a lot of columns coming in this way.

5replies Oldest first
  • Oldest first
  • Newest first
  • Active threads
  • Popular
  • Bumping - I logged a support ticket asking this information as well ( Request #12541 ) but no answer yet.

    Like
  • Hello Dawson Sr,

                        I have come across the same scenario and below are the points which you can consider in your case.


    1.   if you have write access privileges , you can create a new aggregate column (e.g. > 15% green, > 10 yellow, etc. )   by using trim/substring Mysql functions. 

    2. if your table is already in incorta environment, you can make a MV on top of this table  ( C.T.E is best way) and extract the column by using Incorta supported functions 

    below point worked for me - 

    3.  Make a program/script which should run daily/hourly as per your requirement and place raw files in google drive so your program read and do modification and write back in google drive folder and that folder you can use as data source in incorta . 

     

    if you get inbuilt incorta functionality that's great  as I'm not aware of it.  Incorta support team would let you know about it.

    let me know if you have any question.

    Like
    • Arvind kumar   Thanks Arvind, creating the column isn't an issue, creating 50+ using the same formula is just painful  ;-)     

       

      I do like the idea of having another product to do some wrangling pre-Incorta into Google Drive but ideally this would be an Incorta feature.   Support didn't have an immediate answer, but offered to put in an enhancement request if we can't get there currently.

       

      I'll update here when I'm updated.

      Like
  • R. A. Dawson Sr - you could do this within an MV using Python.  Would that work?

     

    https://sparkbyexamples.com/pyspark/pyspark-udf-user-defined-function/

    Like
  • Thanks Dan - that would work.

     

    I did get a response from support and they've also logged an enhancement request:  INC-33138

    Like
Like Follow
  • Status Answered
  • 4 wk agoLast active
  • 5Replies
  • 47Views
  • 3 Following

Product Announcement

Incorta 5 is now Generally Available