Will OpenJDK JVM ever give heap memory back to Linux?

We have a long-lived server process, that infrequently needs a lot of RAM for a short while. We see that once the JVM has gotten the memory from the OS, it never returns it back to the OS. How do we ask the JVM to return heap memory back to the OS?

Typically, the accepted answer to such questions is to use -XX:MaxHeapFreeRatio and -XX:MinHeapFreeRatio. (See e.g. 1,2,3,4). But we're running java like this:

java -Xmx4G -XX:MaxHeapFreeRatio=50 -XX:MinHeapFreeRatio=30 MemoryUsage

and still see this in VisualVM:

Clearly, the JVM is not honoring -XX:MaxHeapFreeRatio=50 as the heapFreeRatio is very close to 100% and nowhere near 50%. No amount of clicking on "Perform GC" returns memory to the OS.

MemoryUsage.java:

import java.util.ArrayList;
import java.util.List;

public class MemoryUsage {

    public static void main(String[] args) throws InterruptedException {
        System.out.println("Sleeping before allocating memory");
        Thread.sleep(10*1000);

        System.out.println("Allocating/growing memory");
        List<Long> list = new ArrayList<>();
        // Experimentally determined factor. This gives approximately 1750 MB
        // memory in our installation.
        long realGrowN = 166608000; //
        for (int i = 0 ; i < realGrowN ; i++) {
            list.add(23L);
        }

        System.out.println("Memory allocated/grown - sleeping before Garbage collecting");
        Thread.sleep(10*1000);

        list = null;
        System.gc();

        System.out.println("Garbage collected - sleeping forever");
        while (true) {
            Thread.sleep(1*1000);
        }
    }
}

Versions:

> java -version
openjdk version "1.8.0_66-internal"
OpenJDK Runtime Environment (build 1.8.0_66-internal-b01)
OpenJDK 64-Bit Server VM (build 25.66-b01, mixed mode)

> uname -a
Linux londo 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u5 (2015-10-09) x86_64 GNU/Linux

> lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 8.2 (jessie)
Release:    8.2
Codename:   jessie

I've also tried OpenJDK 1.7 and Sun Java's 1.8. All behave similarly and none give memory back to the OS.

I do think I need this and that swap and paging won't "solve" this, because spending disk IO on paging close to 2GB garbage in and out is just a waste of resources. If you disagree, please enlighten me.

I've also written a little memoryUsage.c with malloc()/free(), and it does return memory to the OS. So it is possible in C. Perhaps not with Java?

Edit: Augusto pointed out that searching would've led me to -XX:MaxHeapFreeRatio and -XX:MinHeapFreeRatio only worked with -XX:+UseSerialGC. I was ecstatic and tried it, puzzled that I hadn't found this myself. Yes, it did work with my MemoryUsage.java:

However, when I tried -XX:+UseSerialGC with our real app, not so much:

I discovered that gc() after a while did help, so I made a thread that did more or less:

while (idle() && memoryTooLarge() && ! tooManyAttemptsYet()) {
    Thread.sleep(10*1000);
    System.gc();
}

and that did the trick:

I had actually previously seen the behavior with -XX:+UseSerialGC and multiple System.gc() calls in some of my many experiments but didn't like the need for a GC thread. And who knows if that'll continue to work as both our app and java evolves. There must be a better way.

What is the logic that forces me to call System.gc() four times (but not immediately), and where is this stuff documented?

In search of documentation for -XX:MaxHeapFreeRatio and -XX:MinHeapFreeRatio only working with -XX:+UseSerialGC, I read the documentation for the java tool/executable and it isn't mentioned anywhere that -XX:MaxHeapFreeRatio and -XX:MinHeapFreeRatio only works with -XX:+UseSerialGC. In fact, the fixed issue [JDK-8028391] Make the Min/MaxHeapFreeRatio flags manageable says:

To enable applications to control how and when to allow for more or less GC, the flags -XX:MinHeapFreeRatio and -XX:MaxHeapFreeRatio should be made manageable. Support for these flags should also be implemented in the default parallel collector.

A comment for the fixed issue says:

Support for these flags have also been added to the ParallelGC as part of the adaptive size policy.

I've checked, and the patch referenced in the fixed issue backported to openjdk-8 is indeed contained in the source package tarball for the openjdk-8 version I'm using. So it should apparently work in "the default parallel collector", but doesn't as I've demonstrated in this post. I haven't yet found any documentation that says it should only work with -XX:+UseSerialGC. And as I've documented here, even this is unreliable/dicey.

Can't I just get -XX:MaxHeapFreeRatio and -XX:MinHeapFreeRatio to do what they promise without having to go through all these hoops?

Solution 1:

G1 (-XX:+UseG1GC), Parallel scavenge (-XX:+UseParallelGC) and ParallelOld (-XX:+UseParallelOldGC) do return memory when the heap shrinks. I'm not so sure about Serial and CMS, they didn't shrink their heap in my experiments.

Both parallel collectors do require a number of GCs before shrinking the heap down to an "acceptable" size. This is per design. They are deliberately holding on to the heap assuming that it will be needed in the future. Setting the flag -XX:GCTimeRatio=1 will improve the situation somewhat but it will still take several GCs to shrink a lot.

G1 is remarkably good at shrinking the heap fast, so for the usecase described above I would say it's solvable by using G1 and running System.gc() after having released all caches and class loaders etc.

http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6498735