Garbage Collection

[Incortaconsulta]

I Have 500 Gbbox , where I have assigned 365 Gb off-heap and 125 Gb on-heap memory, but while loading a table I m getting error message with the following.

- INC_03010102: Not enough memory while extracting table GC overhead limit exceeded. Please contact administrator to increase ON heap space.

Kindly suggest me suitable config for the above memory.

[JeffW]

Ram, a little more information please? 365+125 = 490 on a box having 500 limit.  So, please tell me a little more?  Are you Cloud or On-Prem?  If On-Prem, how many servers do you run?  Analytics? Loader? Spark? CMC? MySQL? Zookeeper?  On which servers do all of these critical components of Incorta run?  Understanding your installed physical architecture will assist in answering your question.

Jeff

[Incortaconsulta]

Hi Jeff - It's on-prem.

analytics = 500Gb( Zookeeper, spark,postgres, CMC)

loader = 500 Gb (Spark process I can see).

[JeffW]

How long has the On / Off Heap been in this config sizing = 490GB out of 500?  This is too high for these boxes and does not seem to be leaving much room for OS and other processes to run.  And if you are relying on Spark to load some of your data, I see no room allowed for Spark itself to run if it is shared on Loader box, or is it running on Analytics?  Have you opened a ticket for this, because it seems that you generally have too much memory allocated to the services and are not leaving enough room for other processes.

[ahmedezzeldeen]

Can you open a ticket for this. 125 GB seems to be enough for OnHeap memory. It strange with that onheap memory to have GC overhead error. The error generally means that OnHeap needs to be increased or decrease the concurrency of schema loading.