Java memory optimization in Docker containers

Nowadays it is very common to find applications of all kinds deployed with Docker, a technology that offers many advantages, such as a great flexibility to obtain reduced consumption of resources.

A well-sized Docker container allows the application hosted in it to be able to provide optimal service while not wasting resources due to unnecessary oversizing.

A while ago we were optimizing the memory consumption of a Java/Spring Boot application in a Docker container. If you want to know how we did it, keep reading!

Initial Docker Container

We can deploy containers running Java applications starting from an image that contains an implementation of the Java Platform. We chose an image of OpenJDK, an open-source implementation of the Java Platform (Standard Edition). Specifically, we chose 8-jdk-alpine, since it has a very small size because it is based on Alpine Linux.

Then, we created a Dockerfile that allowed us to build the final image with our Spring Boot application included.

FROM openjdk:8-jdk-alpine

RUN addgroup -S spring && adduser -S spring -G spring
USER spring:spring

ARG JAR_FILE=/build/*.jar
ADD --chown=spring:spring ${JAR_FILE} app.jar

ENV JAVA_OPTS=''

ENTRYPOINT java $JAVA_OPTS -jar /app.jar

As we can see, the Dockerfile is quite simple. We applied a few changes to the base image in order to include the .jar file of our Spring Boot application and set its execution as an entrypoint when running a container.

Furthermore, by setting the JAVA_OPTS environment variable we allow that, when creating a container from the image, it is possible to optionally set different flags on the Java Virtual Machine (JVM) to limit the resources consumption or enable certain functions.

From the previous image, a container can be deployed in different ways. For this example we define our container creation through Docker Compose:

version: '2.2'

services:

  springboot_app:
    container_name: 'springboot_app'
    image: 'springboot_app:1.0'
    ports:
      - '8080:8080'
    restart: always

By default, and by not setting any restrictions on it, Docker will deploy the container allowing it to consume as much memory as the host’s Kernel scheduler allows. On Linux systems, if the Kernel detects insufficient memory to carry out important tasks, an OOME (Out Of Memory Exception) will be thrown and it will start killing processes to free up space.

Setting a limit on the containers’ memory consumption will help us to have more control over memory and, at the same time, avoid infrastructure cost overruns.

We can check the memory limit assigned to the running containers by using docker stats command:

$ docker stats --format "table {{.Name}}\t{{.MemUsage}}"
NAME                 MEM USAGE / LIMIT
springboot_app       246.1MiB / 4.833GiB

In order to run Docker on my MacBook Pro I use Docker Desktop. Since we have not established any restrictions, the memory limit coincides with the maximum memory available for Docker Desktop, which in my case is 5Gb.

Now, how do we know how much memory the container really needs?

Our container runs a Java process, so a first approach is to understand how memory consumption is managed in this kind of processes.

Understanding the memory of a Java process

The total memory that a Java process takes can be divided into two main blocks: Heap Memory and Non-Heap Memory.

Heap Memory

Heap Memory is intended for Java objects. This memory is initialized at JVM startup and its size may vary during application execution.

When this memory becomes full, the Garbage Collector (GC) cleans up objects that are not used anymore, freeing space for new ones.

If we do not set a limit for Heap Memory, it will be set automatically. This automatic memory allocation is carried out taking into account the available physical memory of the system, among other configuration aspects, in addition to the Java Platform specific version. This limit can be set with the -Xmx flag.

We can check which is the limit of Heap Memory established in our container. The first thing to do is to open a shell session inside the container:

docker exec -it springboot_app /bin/sh

Then we execute the following command, which returns the total bytes corresponding to the memory limit allocated for Heap Memory in the container:

$ java -XX:+PrintFlagsFinal -version | grep -iE 'MaxHeapSize'
uintx MaxHeapSize := 1298137088 {product}

The java -XX:+PrintFlagsFinal -version command allows us to obtain the value of the different JVM flags. In our case, we use grep to filter the specific flag we are looking for.

The Heap Memory limit set in our container is about 1.3GB.

Non-Heap Memory

The memory consumed by a Java process is not only associated with Heap Memory. The JVM includes some necessary components that imply an increased memory consumption. The memory allocated for these components is independent of Heap Memory and is known as Non-Heap Memory.

Inside Non-Heap Memory we find: Garbage Collector, Class Metaspace, JIT compiler, Code Cache, threads, etc. There are flags to limit the memory consumption of some of these components, but not all of them have this option.

Because of the many factors to consider and some beyond our reach, there is no direct way to calculate the total size Non-Heap Memory occupies. Anyway, there are tools that can help us to monitor this memory block.

Monitoring our non-optimized Java process

In order to optimize the memory consumption of our Java process, firstly we monitored the process under “normal circumstances”. That is, without applying any memory limitation on it. In this way we were able to see how much the total Heap Memory consumed at runtime amounted to, as well as the consumption of other Non-Heap Memory components.

Our goal was that, based on this information, we could size the memory allocated to the process according to real needs.

There are different ways to monitor a Java process. From command line utilities like jcmd to visual tools like JConsole. In this example we will use JConsole to be able to show graphics.

In order to use JConsole we first have to activate Java Management Extensions (JMX). For this, it is necessary to set certain options in JAVA_OPTS within the definition of our container:

version: '2.2'

services:

  springboot_app:
    container_name: 'springboot_app'
    image: 'springboot_app:1.0'
    environment:
      JAVA_OPTS: '-Dcom.sun.management.jmxremote
                  -Dcom.sun.management.jmxremote.local.only=false
                  -Dcom.sun.management.jmxremote.authenticate=false
                  -Dcom.sun.management.jmxremote.port=9010
                  -Dcom.sun.management.jmxremote.rmi.port=9010
                  -Djava.rmi.server.hostname=0.0.0.0
                  -Dcom.sun.management.jmxremote.ssl=false'
    ports:
      - '8080:8080'
      - '9010:9010'
    restart: always

From this configuration, we can create a container which is accessible from JConsole through 9010 port.

To access JConsole, just execute the following command:

$ jconsole

Next, a window like the following one should appear:

To establish a connection with our container we must select Remote Access option and type the host and assigned port for monitoring.

For this example we have disabled authentication, so we can leave the user and password fields blank. We have also disabled SSL access, so a pop-up window may appear to confirm the connection.

Once connected, we will access Memory section:

In this section we can visualize graphs related to Heap Memory and Non-Heap Memory consumption. We can also access specific graphics of certain components (Code Cache, Metaspace, etc).

If we take a look at the Heap Memory graph, we can see that it remains at around 295Mb. This is due to the fact that the initial Heap Memory of the process is set by default to 25% of the maximum available (1.2Gb).

To visualize the Non-Heap Memory consumption, we select the corresponding graph:

As we can see, the use of Non-Heap Memory rises until it stabilizes around 92Mb.

Being able to get a graph which shows the Non-Heap Memory consumption is very useful since, as I mentioned before, calculating the total consumption of this memory block is not straightforward.

Obtaining the maximum values of the two graphs and summing them would not be a correct approach to determine the total memory needed for the process, since these graphs correspond to the memory consumption when our Spring Boot application is mainly idle.

In order to replicate a real-use scenario on our application, we are going to define a load test.

It is important to perform the load test based on a correct forecast of the traffic that we expect our application will have in production. In this way we can monitor the memory consumption of the Java process in a realistic scenario. For this example we are going to assume that 100 users making requests recurrently to our application at the same time replicate the expected traffic in production.

We use JMeter to perform this kind of tests. The following screenshot is of the JMeter project for the example’s Spring Boot application:

On the left side we can see the different functional tests on our Spring Boot application. Each test executes an operation and expects it to be successful.

On the right side we find a section that allows us to configure a load test, where among the available options we can set a number of threads (users) and a loop count, which corresponds to the number of times that each user, individually, requests each operation.

Based on the previous forecast, we set 100 users and in order to replicate the continuous usage we set a loop count of 20.

Once we run the test we can see how the graphs in JConsole start to change:

Heap Memory consumption reaches 550Mb during the test run and rises and falls suddenly in short periods of time, due to the action of the Garbage Collector. Once the test ends, we can see how the memory consumption starts to stabilize again.

Non-Heap Memory rises and stabilizes at around 158Mb, remaining stable in this range.

Finally we can see the JMeter report with the load test results, where we can see the throughput of our operations and the error rate, among other available data.

As we can see, the error rate is 0.00% on all operations and the throughputs sum is 124.1 operations/second.

It is important to take these values into account in order to understand the performance of our starting point and compare it with the performance after optimizations.

Optimizing the memory consumption of our application

Heap Memory consumption is directly related to the maximum available (1.2Gb), since the Garbage Collector is triggered taking this maximum into account. The higher the maximum available for Heap, the longer the Garbage Collector cycles will be, making cleanings less common and therefore increasing the consumption of Heap Memory. This can negatively impact our application’s performance.

In our example, having a maximum of 1.2Gb of Heap Memory and getting a peak of 550Mb during the load test, we can conclude that the Heap Memory is oversized, which implies longer Garbage Collector cycles. Therefore, the fact that the consumption of Heap Memory has risen up to 550Mb does not mean that our process requires as much memory.

On the other hand, taking a look at the Heap Memory graph above, we can see how the Garbage Collector acts on several occasions, lowering the memory consumption to approximately 100Mb, a value which is far below the initial Heap Memory established by default. This is an indicator that our process does not need that much initial Heap Memory.

Our approach was to gradually reduce the maximum size of Heap Memory, looking for memory consumption values closer to the limit during the load test and without penalizing performance and the success of the tests.

It is very important not to adjust too much the maximum Heap Memory. A good practice is to leave some room for extreme load spikes that may occur. A conservative approach, given the results obtained in this example, would be to repeat the load test for a maximum Heap Memory of 512Mb and analyze again the results.

With this new maximum of 512Mb, the initial Heap Memory, as it is set by default at 25% of the maximum, would be 128Mb. In the previous Heap Memory graph we have seen how on some occasions the Garbage Collector drops the memory consumption to around 100Mb. This means that our application does not need more than 100Mb to keep running when there is no load, so 128Mb for the initial Heap Memory is more than enough for a first optimization.

We can also set the initial Heap Memory through the -Xms flag, although we have preferred to avoid this practice, since it is recommended to be at 25-30% of the maximum Heap Memory.

Taking into account the maximum Heap Memory we establish and the Non-Heap Memory, which in our example reaches about 158Mb, we can set the memory limit of the Docker container. It is important not to limit the container memory only to the necessary by the Java process, since we must leave some space for other resources that are necessary for the container.

After some iterations on the example case, reducing the maximum of Heap Memory and analyzing results, we ended up with a value of 256Mb. Taking into account the 158Mb that Non-Heap Memory takes and keeping a small margin of 98Mb, we ended up setting up a limit of 512Mb for the container.

version: '2.2'

services:

  springboot_app:
    container_name: 'springboot_app'
    image: 'springboot_app:1.0'
    mem_limit: 512m
    environment:
      JAVA_OPTS: '-Dcom.sun.management.jmxremote
                  -Dcom.sun.management.jmxremote.local.only=false
                  -Dcom.sun.management.jmxremote.authenticate=false
                  -Dcom.sun.management.jmxremote.port=9010
                  -Dcom.sun.management.jmxremote.rmi.port=9010
                  -Djava.rmi.server.hostname=0.0.0.0
                  -Dcom.sun.management.jmxremote.ssl=false
                  -Xmx256m'
    ports:
      - '8080:8080'
      - '9010:9010'
    restart: always

By executing the load test on the new container, we obtain the following results:

Heap Memory consumption reaches spikes close to the maximum established, although a prudential margin is maintained for unforeseen events. We can see how the Garbage Collector acts more regularly to keep memory consumption below the new maximum.

Non-Heap Memory consumption stays exactly the same. This makes sense since we have not performed any optimization on this part of the memory.

The load test results are very positive, as the application continues to respond with an error rate of 0.00% while keeping the same throughput.

If we need to reduce the size of our container even more, we can try to further limit the size of Heap Memory. In this example we have not reduced the 256Mb limit, since in that case we find errors during the load test due to lack of memory.

A next step may be to consider the option of reducing the memory consumption of Non-Heap Memory components. I recommend you take a look at this video, where the Non-Heap Memory components and the available flags to limit their memory consumption are explained very well.

Conclusion

In this post I have shared with you a possible approach to optimize the memory consumption of a Docker container that runs a Java process.

The first step has been to understand the memory consumption of a Java process. Then we have monitored our process to analyze its associated memory consumption. Finally, based on our analysis, we have established memory limits for both the process and the Docker container that contains it.

By following these steps we can obtain a significant resources saving, directly affecting the size of the necessary infrastructure to hold our containers and thus reducing its cost.

I am convinced that there are different ways to approach this issue, so I encourage you to share your experiences and opinions in the comments section.

Also, I will be happy to answer any question you may have!