0

Memory Allocations for Incorta Instances

I'm working with an Incorta instance that has a total of 384GB of RAM allocated to the main Incorta service. This instance is running two tenants, and looking at the /incorta/memory.jsp page, there's a total of 253 GB allocated for all tenants (85GB Dictionary, 16GB Hash Table, 101 GB Java Heap, 151GB Est Off Heap).

However, I've started getting some error messages on incremental schema loads saying "Not enough memory to load schema ..." (INC_090001003). 

Are there any established guidelines out there for, given a specific data/schema size (i.e. 253GB in my case) how much TOTAL RAM should be allocated to the instance? If I have 250GB of RAM, do I need 300GB, 400GB, 500GB, or more of total RAM in order for the Incorta engine to work effectively? 

I need a way to monitor current memory usage for data, and know whether or not I'm reaching a limit based on the total memory allocated to the instance.

12replies Oldest first
  • Oldest first
  • Newest first
  • Active threads
  • Popular
  • To get an overview of your memory usage, you may use the following:

    1. To get information about system memory usage:
      free -gh

    2. To get a list of all processes owned by incorta user with their status and resource usage:
      ps ux --sort -rss

    3. To get the percentage of the system used memory:
      free | awk 'FNR == 3 {print $3/($3+$4)*100}'| cut -d '.' -f 1

    4. To get a report about tenant memory usage per tenant objects, you may browse the following report:
      http://<incorta_host>:<port>/incorta/memory.jsp

    5. To get a detailed report about the memory usage, you may browse the following report:
      http://<incorta_host>:<port>/incorta/test.jsp
      Scroll to the bottom of the page, enter 4 for the "Diagnostics" level and click on "Log"

    Reply Like 1
    • Ahmed Esmat  These are very handy. I knew about a few of them, but not all of them. Thanks!

      Reply Like 1
  • Dan,

    We are working on a new version which will allow users to load more data in Incorta than the amount of memory available.  We will keep you updated about the progress in this area.

    Currently, you may need to increase the memory by adding another 100GB for the current environment to function comfortably.  You can also update a parameter called Server Memory Percentage under the Incorta admin screen : http://<url>:port/incorta/admin then go to Genera/ServerConfigs page.  The default value is 75%, but you can increase it a little bit to fit more data (say 85%).  The performance may be affected a little bit though, so you need to test your dashboards after making this change.

    Reply Like
    • Hichem Sellami If I understand correctly, then this "Server Memory Percentage" setting is what controls the ratio of data to engine reserved memory, correct? So if this is set to the default of 75% and I have a total memory allocation of 384GB for the instance, then in theory I should be able to use up to 288GB (75% of 384GB) for data? 

      Do schema loads require additional memory outside of the allocated data memory to succeed? In other words, when I do a schema load (full or incremental) is there some amount of the "data" memory that is allocated for the schema load?

      Reply Like
    • Dan Nielsen That is correct.  Incorta does create some additional objects (in memory) based on the data loaded from source.

      Reply Like
  • We have also developed a script which will give you the list of fields not used in any dashboard.  This can be used to clean up some of the data currently loaded to memory but not used at all.  This script is in testing phase right now.  I hope we can share it with customers within a few weeks at most.

    Reply Like
    • Hichem Sellami This sounds great, I'd love to try it out when you have it ready!

      Reply Like
    • Hichem Sellami is this script available yet?

      Reply Like
    • Tom Fogarty Hichem Sellami is out of the office.  I will see if someone else is aware of the script.  Thanks

      Reply Like
  • Tom Fogarty  The tool is ready, I will send you the details over an email.

    Reply Like 1
    • Tom Fogarty Did the script work for you ?

      Reply Like
  • Matthew Halliday mostly–we did have some trouble with the newer build vs. the older build but I am working on it with Moneim. Thanks for the support!

    Reply Like
Like Follow
  • 1 yr agoLast active
  • 12Replies
  • 654Views
  • 9 Following