- Metaspace was monitored in test environments - there is steady increase in used memory when generating many reports and the memory is not freed what probably resulted in crash. Could you give any suggestion how the problem shuold be resolved? By now I tried to change language from 'groovy' to 'java' but it did not help.
- First of all, part of it can definitely be a malicious native code leaking memory. But in 99% of the cases it is completely normal behaviour by the JVM. What you have specified via the -Xmx switches is limiting the memory consumed by your application heap.
Perhaps this article should be updated with all latest information.
Also, there were obviously no classloader leaks (MetaSpace size was constant at around 130M). ByteBuffers looked good. We didn't use off-heap memory. Maybe garbage collection goes wild? First try: Native Memory Tracking Out of the box, Java offers Native Memory Tracking (NMT). Simply add at startup of your java application. The first step in detecting memory leaks is to monitor the live-set of the application. The live-set is the amount of Java heap being used after a full GC. If the live-set is increasing over time.
Java Metaspace Memory Leaks
It seems that all jar files on all classloaders are also part of RSS memory on Linux.
That adds some MBs to the memory consumption of the Java process too when running an application that has a lot of jar dependencies.
jar files are mmapped and the memory can be shared with other processes, but it still adds up to the RSS memory of the Java process.
I’m making this conclusion based on some pmap outputs from some java processes.
Here are examples of other parameters on JDK8 to control out-of-heap memory limits:
-XX:InitialCodeCacheSize=64M -XX:CodeCacheExpansionSize=1M -XX:CodeCacheMinimumFreeSpace=1M -XX:ReservedCodeCacheSize=200M
-XX:MinMetaspaceExpansion=1M -XX:MaxMetaspaceExpansion=8M -XX:MaxMetaspaceSize=200M
-XX:MaxDirectMemorySize=96M
-XX:CompressedClassSpaceSize=256M
-Xss1024K
This is an example from an app with -Xmx1445M -Xms1445M . The total consumption goes to about 2048M with these values in one case.
Java Metaspace Memory Leak Foam
It should be noted that Metaspace allocation in Java 8 doesn’t work like it did in previous JVMs.
I believe that you shouldn’t set the initial value for Metaspace since the JVM is not doing any preallocation based on that value (http://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/considerations.html#sthref66).
I haven’t come up with a proper equation to explain all of the RSS memory consuption of a Java process.
It would be nice if you could do more research on that subject.
The reason why I’ve had to dig in this is that on CloudFoundry there is an OOM killer (because it’s running the process in a Linux cgroups container which limits memory with OOM killer) when you step over the bounds.
Therefore it would be nice to be able to properly calculate and limit the memory consumption of a Java process without leaving too much safe boundaries (since that amount of memory would be “wasted” and you are paying for it).