Memory consumption is higher after update
- Daria Samkova
Question
Ever since we updated the HiveMQ broker, we have observed high heap memory usage across cluster nodes. Shall we be concerned?
Answer
The observed behavior is normal and expected if there is no negative impact on your system except for the increased heap observation.
A Java application will use allocated heap as it is available and the Java Garbage Collector will only remove allocation as needed.
Here are many resources explaining the same:
If you migrate from the ParallelOldGC or CMS collector to G1, you will likely see a larger JVM process size. This is largely related to "accounting" data structures such as Remembered Sets and Collection Sets.
The first focus of G1 is to provide a solution for users running applications that require large heaps with limited GC latency. This means heap sizes of around 6GB or larger, and stable and predictable pause time below 0.5 seconds.
More explanation about the sizing of the heap
G1 respects standard rules when resizing the Java heap, using -XX:InitialHeapSize as the minimum Java heap size, -XX:MaxHeapSize as the maximum Java heap size, -XX:MinHeapFreeRatio for the minimum percentage of free memory, -XX:MaxHeapFreeRatio for determining the maximum percentage of free memory after resizing. The G1 collector considers to resize the Java heap during a the Remark and the Full GC pauses only. This process may release memory to or allocate memory from the operating system.
Important metrics to observe are:
com.hivemq.jvm.garbage-collector.G1-Old-Generation.count
and
com.hivemq.jvm.garbage-collector.G1-Old-Generation.time
as long as those are low there is no memory pressure on the system.