0

Extracting ~28 million rows, loading zero, schema shows 28M rows in table, analyzer zero

I don't have direct access to the CMC, so I'm a bit limited in my troubleshooting, but what should I be looking at if a table isn't loading.  As in the title, it extracts ~28 million rows ( takes ~ 40 minutes ), loads zero, and analyzer returns zero rows.   The schema page shows it has 28M at 4.33 GB.    It's a straight select w/ a where clause, no formulae, no joins, no runtime filters, no security filters.

 

I replicated the schema and table and it runs and loads successfully which isn't a very good long term plan  ;-)   

 

I can get CMC access if needed, the environment is Incorta cloud.

 

Ideas welcome and thank you!

4replies Oldest first
  • Oldest first
  • Newest first
  • Active threads
  • Popular
  • Have you checked if this table is performance optimized?
    Can you also attach a screenshot of the schema table load details (attached an example)

    Like
  • It is not ( was not ) optimized by design since I won't be using this to analyze - just as a source for a MV.

    I did switch it to Optimized and am testing it now. 

    I understand that "optimized" just means read into memory at the conclusion of processing though, so I'm not sure that should impact the load process.  ( also understand that sometimes "should" doesn't indicate "does" !! )

    Like
    • R. A. Dawson Sr 
      In the Load Job Viewer, Load is for the Analytics Services. So by design, nothing will be available in the Analytics Service.

      You can try a MV with Spark SQL and with a SELECT * FROM TABLE LIMIT 1 statement.

      See if this returns 1 row from your non-optimized physical schema table. It should.

      If your MV is performance optimized, you will be able to use the Analyzer to explore data or create an insight. 

      Like 2
  • Thanks Seth -- I was under the ( apparently incorrect ) assumption that non-optimized just meant that the table wasn't in RAM, not that it couldn't be seen by analyzer.    

    This makes sense now and yep - the MV is optimized while the source tables are not.

    Like
Like Follow
  • 1 mth agoLast active
  • 4Replies
  • 33Views
  • 3 Following

Product Announcement

Incorta 5 is now Generally Available