0

Onheap vs offheap memory

Team,

I am new to Incorta. Can anyone help me understand how the onheap vs offheap memory works. Any references for documentation with examples is much appreciated.

Also when the loader service is restarted, How do we ensure the loader service is available for the users to start using schema's.

What are the parameters we need to check to see the loader is service is available. Currently I see only a message saying loader service is not available when we try accessing the schema. The only place I see now is check the memory details. But certain times even the memory is completely loaded, the message appears as "loader is unavailable to access schema"

 

Thanks,

Srinivas

9replies Oldest first
  • Oldest first
  • Newest first
  • Active threads
  • Popular
  • Hello Srinivas, 

     

    For the On/off Heap point:

    • on-heap refers to having the data parts stored in the JVM memory heap 
    • off- heap refers to having all  the data parts are stored  outside the JVM memory heap

    For monitoring the Loader Service startup:

    In the CMC, the Loader Service status "started" reflects the Tomcat status and the tenant might not yet be available.

    Whether you are starting the Loader Service from the command line or from the CMC, in order to know that the loader service is completely started and the tenant is available, you must monitor the Incorta tenant log from the loader service backend:

      <Installation_Path>/IncortaNode/services/loaderServiceindex/logs/incorta/<tenant_name>/incorta.current_timestamp.log

    From this log (where "bi" is the tenant name), you must see a message similar to this to know that the Loader Service is completely started up.

      INFO: [12:07:28] [bi | System | Startup] Tenant: bi started in: 0.11 Mins [com.incorta.server.Server.startupTenant

    Once this is seen in the log, you can start the Analytic Service.

    This is simplified in the upcoming 4.4 release.

    Starting from 4.4 we've enhanced the CMC service status by introducing a new status called "Starting Tenants" which means that Tomcat is started but tenants are still being loaded and "started" status means that service is started and all tenants have been loaded and ready.

    Incorta 4.4, the Loader Service status are: 

    • At Startup: Processing -> Starting -> StartingTenants -> Started
    • At Shutdown: Stopping -> StoppingTenants -> processing -> Stopped
    Reply Like 1
  • Thank You !

    Reply Like
  • Srinivas  You are welcome.

    Reply Like
  • Hi Ayat,

    Do we have any best practices in configuring the On Heap and Off Heap memory ? Any references is much appreciated !

    Reply Like
    • Srinivas should be 25% for on heap and 75% for off heap for each service .  Let us take a single server example - if your data size in memory is say 100 GB then we should configure 2O0 GB memory for loader and 100 GB for analytic service . Now apply the above percentages for on heap and off heap  for each .

      Reply Like
    • Amit Kothari Thank You Amit, Trying to understand a little in detail. So when you are saying to apply the percentages for onheap and offheap. Are you saying out of 200 GB do we need to allocate 50 GB to onheap and 150 GB to offheap to loader service and 25 GB to onheap and 75 GB to offheap ?  I am looking for the best practice documents or a concrete different scenario examples to understand the configuration better.

      Reply Like
    • Srinivas you have the right idea of Amit's recommendation. Most customers are sizing their memory allocation by these guidelines and settling in that general area after testing with their own hardware, data, and use cases. For some more detail about the guidelines, please see this article: https://community.incorta.com/t/y70zja

      Reply Like
    • Dustin Basil Thank You Dustin. Here is a scenario we have. Please find attached snapshot where we allocated memory for the loader service and Analytics service in 4.3.3. What exactly drives the onheap memory vs offheap memory allocations here. We see the frequent change in the onheap vs offheap memory which is reaching to the maximum allocated. Is there any memory algorithm that drives this frequent change. Also we do not see the total memory size as shown in your link. 

      Reply Like
    • Srinivas Please review the information I sent you and ping me back if you want to set up a call.

      Reply Like
Like Follow
  • 1 mth agoLast active
  • 9Replies
  • 83Views
  • 4 Following

Welcome!

Welcome to Incorta's user community! Whether you are already an Incorta customer, or interested in becoming one, this is your place to come together and discuss the software, register for webinars, learn about events, learn about new product releases and get support from the community.