JProfiler Get rid of your performance problems and memory leaks! Modern Garbage Collection Tuning Java Performance Training Courses COURSES AVAILABLE NOW. We can provide training courses to handle all your Java performance needs Java Performance Tuning, 2nd ed The classic and most comprehensive book on tuning Java Java Performance Tuning Newsletter Your source of Java performance news. Subscribe now! Enter email: Training online Threading Essentials course JProfiler Get rid of your performance problems and memory leaks!

Published May 2003

TABLE OF CONTENTS

1. What is HPjtune?
2. How To Use HPjtune
3. Understanding Garbage Collection in HotSpot VM
4. Cumulative Object Retention (also known as Java Memory Leaks)
5. Young Generation Heap Sizing
6. Explicit GC Considered Harmful
7. Conclusion

What is HPjtune?

HPjtune helps you understand the behavior of the Java Heap and Garbage Collector in the Virtual Machine (VM) by presenting collected metrics graphically. The metrics presented in HPjtune's graphs make the patterns of good and bad behavior explicit, easily identifiable and easily corrected. HPjtune is free, 100% pure Java and displays metrics that include:

• Heap Size after each Garbage Collection (GC)
• Duration of each GC
• Object Creation Rate
• Number of Bytes Freed by each GC
• Type of GC
• and many others

How To Use HPjtune

HPjtune reads the GC metrics in the log files produced by the Java VM. Therefore, using HPjtune always requires two steps:

1. Run your application with a garbage collection logging option enabled (for example, add -Xloggc:filename or, when running on HP-UX, -Xverbosegc:file=filename to the command line)
2. Run HPjtune to analyze the file produced during the application execution

Download HPjtune (no cost) from the Hewlett-Packard Java website. Just unzip the downloaded bundle to get the jar file, and invoke it with:

   java -jar HPjtune.jar

Understanding Garbage Collection in HotSpot VM

As all Java developers know, Java Virtual Machines (JVMs) use automatic memory management, commonly known as Garbage Collection. Garbage Collection removes unreachable objects from the heap during the application execution. An object is unreachable if, in the future, the application can no longer use that object during execution, regardless of the path that the execution may take.

Most garbage collectors, including those used by the HotSpot VM, use object liveness to determine which objects are unreachable. An object is defined as live if there exists a path of references starting from one of the application variables (known as the "roots" of the application) that includes the given object. The unreachable objects are assumed to be those that are not live.

The design of the HotSpot garbage collector is based on the observation made during the analysis of Java applications, that most of the created objects are short-lived. This has led to splitting the heap into two generations:

• the young generation containing the recently created objects
• the old generation containing long-lived objects

The garbage collector strives to keep the newly created objects with others of the young generation in the New Space. Tenuring the long-lived objects moves them into the Old Space. We can see the logical structure of the HotSpot generational heap in the following diagram:

Objects are initially allocated into the Eden (or Nursery) area. When the Eden area is full, the garbage collector must find the live objects and move them to either the Survivor "To" area or the Old Space. The Garbage Collector will strive to keep the newly created objects in the New Space by copying the live objects between the From and To areas for a certain number of garbage collections. An object ages each time it survives a garbage collection event. It becomes designated as old after surviving a certain number of garbage collection events, at which point it is moved to the old area of the heap.

By separating the long-lived objects from the newly created objects, the garbage collector can, in most cases, avoid examining most long-lived objects during each garbage collection. A scavenge garbage collection event is one during which only short-lived unused objects are collected in the young generation. In contrast, a full garbage collection involves collection of objects in both the young and old generations.

A typical server application creates many short-lived objects, which are needed for transaction processing. Once the transaction is complete, these objects can be reclaimed from the heap. Some large objects will be allocated directly into the old generation (because they are too big to fit into the young generation), and some longer living objects will move to the old generation. Ideally, the GC log should contain many scavenge GCs and few full GCs.

Proper sizing of the old and the young heap areas is essential for the good GC performance. In the next sections we will show how HPjtune detects incorrect heap sizing and identifies other issues that can adversely affect your Java application performance.

Cumulative Object Retention (also known as Java Memory Leaks)

A frequent problem in Java applications is that some objects, because of either design or coding errors, remain live for very long periods of time, contrary to the programmer's intentions. Such objects have been called lingering objects. Lingering objects tend to accumulate over time, clogging the heap and causing multiple performance problems, eventually leading to an application "Out of Memory" problem.

In our first example we will demonstrate how HPjtune can be used to detect object retention. The application has been run with the additional option -Xloggc. After reading the created data file, and selecting the Heap usage panel, we see a picture typical for an application retaining objects.

The various types of garbage collection are shown using different colors. The X-axis shows the time, while the Y-axis shows the heap size after garbage collection. Even though some garbage collections are able to reclaim memory, the overall heap usage continues increasing. After some time, scavenges become impossible (the old generation becomes too full), and finally OutOfMemoryErrors will be thrown. Even if the application tries to handle them gracefully, it is paralyzed by the clogged heap.

HPjtune can also show the duration of each GC. After selecting the Duration panel, we see the following picture:

Here we see that the final garbage collections took much more time than the previous ones. The color coding also highlights the differences in time required for each of the types of garbage collection and is a simple graphical display of the reason why you want to have the shorter scavenge garbage collections predominate - they take less then one-tenth the amount of time!

We can also look at the number of bytes reclaimed by each garbage collection. Click on User defined panel. Then select Clock Time to be displayed along the X-axis, and Reclaimed Bytes to be displayed on the Y-axis.

We see that the effectiveness of the final full GCs in obtaining free space for object allocation declines until no space is reclaimed. At this time, the OutOfMemoryError was probably thrown.

If you see a similar pattern when displaying the GC data from your application, and the pattern repeats itself even after increasing the heap size, you will need to find and fix the object retention problem in the application. There are a number of commercial tools that support object retention/memory leak detection. There's also a free solution: use the standard -Xrunhprof option and HPjmeter (see the HPjmeter tool report).

Young Generation Heap Sizing

When tuning a commercial application which spent over 10% of its time in garbage collection, we see the following picture:

The garbage collections are frequent, so the individual points overlap. If you have difficulties with recognizing a particular pattern formed by the data points, you can zoom-in by dragging the mouse horizontally across the graph, with the left button depressed. This may result in a picture like this:

Overall, we see a pattern for the "Heap Usage After GC" that shows that with every scavenge GC, the Heap Usage increases. The heap is eventually sufficiently filled to cause an "Old too full" garbage collection - a Full GC. As you can see, after a Full GC, the heap size drops to a much lower level. The pattern of increasing heap size with each scavenge GC and then almost complete reclamation after a Full GC is caused by overflow of the young objects prematurely into the old space. Once identified and understood, the pattern is easy to recognize.

To verify our hypothesis, we can look at the Promoted Bytes, that is, the bytes moved from the young generation to the old generation with each garbage collection (this feature is unavailable for the GC logs created with the -Xloggc option).

We can assume that most of the short-lived objects that are promoted prematurely to the old generation would be garbage collected in the young generation if the young generation were large enough to accommodate all of the short-lived objects. To improve the performance of the garbage collector, we can increase the size of the young generation. This will make the scavenge GCs less frequent, and increase the probability that the short-lived objects become unreachable while still in the young generation. Therefore, the short-lived objects are more likely to be reclaimed before they are copied over to the old generation. Another approach is to increase the space reserved within the young generation for the surviving objects. We do this by decreasing the SurvivorRatio parameter (the default is 8). For example:

     -XX:NewSize=800m -XX:MaxNewSize=800m -XX:SurvivorRatio=4

     -Xmn800m -XX:SurvivorRatio=4

Explicit GC Considered Harmful

Some Java developers think that they need to help the JVM's garbage collector by calling system.gc(). But does it really help? HPjtune can help to answer this question. Let's analyze a GC log from a Java application that was using explicit garbage collection invocations. We examined the heap usage first, and noticed that there's no significant difference between the heap usage after a scavenge and after the forced full garbage collection.

When we look at the GC times, we notice again that the scavenges are much faster than the full GCs.

This is of course to be expected, scavenges are designed to be much faster because they are only working with a subset of the total allocated objects. So perhaps the scavenges reclaim a smaller amount of space? We look at the "Reclaimed Bytes" chart to figure this out.

Here we see that most of the explicit garbage collections actually reclaim less space than the scavenges. But some of the developers are still unconvinced. They claim that they invoke the garbage collector only when they know that their application is idle, so no harm is done. But is this really true? One of the side effects of the full GC is that all survivors from the young generation are moved to the old generation. We look at yet another graph, showing the survivors remaining in the young generation after each GC (this feature is available only with the -Xverbosegc option).

Here we see that after each scavenge about 1 MB of live objects were remaining in the young generation, while each System.gc() left the young generation empty. This means that some, potentially short-lived objects were moved to the old generation, and were made inaccessible for the scavenges. The old space would eventually overfill, making a full garbage collection necessary. What is particularly insidious is that the developers cannot control or predict the point of time it will happen. Therefore calling System.gc, even when the application is idle, can make HotSpot VM run additional full garbage collections at some unknown point in the future.

You can make the JVM ignore the explicit GC requests from the application by specifying the following option:

   -XX:+DisableExplicitGC

Conclusion

Heap tuning is an easy way to get better performance out of a Java application, without changing even a single line of code. The creation of GC logs introduces very little overhead, so the data can be collected in your production environment over long periods of time. At any point while the application is running, the logs can be analyzed with HPjtune to determine whether the Java heap is tuned properly.