Full-featured, built-in profilers like JConsole and VisualVM sometimes cost more than they're worth in performance overhead — particularly in systems running on production hardware. So, in this second article focusing on Java performance monitoring, I'll introduce five command-line profiling tools that enable developers to focus on just one aspect of a running Java process.
The JDK includes many command-line utilities that can be used to monitor and manage Java application performance. Although most of them are labeled "experimental" and therefore technically unsupported, they're still useful. Some might even be seed material for special-purpose tools that could be built using JVMTI or JDI (see Resources).
1. jps (sun.tools.jps)
Many command-line tools require that you identify the Java process that you want to monitor. This isn't so different from similar tools that monitor native-operating-system processes and also require a process identifier to work.
The "VMID" identifier is not always the same as the native operating system process identifier ("pid"), which is why we need the JDK's jps utility.
jps— whose name reflects the ps utility found on most UNIX systems — tells us the JVMID of a running Java application. As its name implies, jps returns the VMIDs for all discoverable Java processes running on a given machine. If jps doesn't discover a process, it doesn't mean that the Java process can't be attached or spelunked. It just means that it isn't advertising itself as available.
If the Java process can be found, jps will list the command-line used to launch it. This way of differentiating between Java processes is important because, as far as the operating system is concerned, all Java programs are "java." For most purposes, the VMID is the important number to note.
Getting started with profilers
The easiest way to get started with profiling utilities is to use a demo program like the SwingSet2 demo found at demo/jfc/SwingSet2. Doing this avoids potential hangups with processes running as background/daemon processes. Once you're familiar with the tool and its overhead, you can try it on your actual programs.
After you load up your demo app, run jps and note the returned vmid. For best effect, launch the Java program with the -Dcom.sun.management.jmxremote property set. If you don't use this setting, some data gathered by some of the tools below will be unavailable.
2. jstat (sun.tools.jstat)
The jstat utility can be used to gather a variety of different statistics. jstat statistics are sorted into "options" that are specified at the command-line as the first parameter. As of JDK 1.6, you can view the list of options available by running jstat with the command -options. Some options are shown in Listing 1:
Listing 1. jstat options
-class
-compiler
-gc
-gccapacity
-gccause
-gcnew
-gcnewcapacity
-gcold
-gcoldcapacity
-gcpermcapacity
-gcutil
-printcompilation
The utility's JDK documentation (see Resources) will tell you what each of the options in Listing 1 returns, but the majority of them are used to gather performance information about the garbage collector, or just parts of it. The -class option reveals loaded and unloaded classes (making it a great utility for detecting ClassLoader leaks within the app server or your code), and both -compiler and -printcompilation display information about the Hotspot JIT compiler.
By default, jstat will display the information at the moment you check it. If you want snapshots taken at regular intervals, specify the intervals in milliseconds following the -options command. jstat will continuously display snapshots of the monitored process's information. If you wantjstat to take a specific number of snapshots before terminating, specify that number after the interval/period value.
If 5756 were the VMID for a running SwingSet2 process started a few minutes ago, then the following command would tell jstat to produce a gc snapshot dump every 250 milliseconds for 10 iterations, then quit:
jstat -gc 5756 250 10
Note that Sun (now Oracle) reserved the right to change the output of the various options, or even the options themselves, without warning. That's the downside of using unsupported utilities. See the Javadocs for complete details about each of the columns in the jstat output.
3. jstack (sun.tools.jstack)
Knowing what's happening inside of a Java process vis-a-vis the executing threads is a common diagnostic challenge. For example, when an application has suddenly stopped processing, it's obvious that some kind of starvation has been reached, but it isn't obvious just by looking at the code where the starvation has occurred, or why.
jstack is a utility that returns a complete dump of the various threads running in an app, which you can then use to pinpoint the problem.
Running jstack with the VMID of the desired process will generate a stack dump. In this regard, jstack works the same as pressing Ctrl-Break within the console window in which a Java program is running, or calling Thread.getAllStackTraces() or Thread.dumpStack() on each of theThread objects in the VM. A jstack call also dumps information about non-Java threads running within the VM, which aren't always available asThread objects.
jstack's -l parameter offers up a slightly longer dump that includes more detailed information about the locks held by each of the Java threads, and can thus be invaluable in finding (and squashing!) deadlock or scalability bugs.
4. jmap (sun.tools.jmap)
Sometimes, the problem you're dealing with is an object leak, such as an ArrayList (which could hold thousands of objects) that just isn't getting released when it should. Another problem that is more general would be an expanding heap that never seems to shrink, despite active garbage collection.
When you're trying to locate an object leak, it's incredibly helpful to take a picture of the heap at a given moment in time, and then look through what's there. jmap provides the first part of that functionality by taking a snapshot picture of the heap. You can then analyze the heap data using the jhat utility described in the next section.
Using jmap is straightforward, like all the other utilities described here. Simply point jmap at the VMID of the Java process you want to snapshot, and give it some parameters to describe the resulting file produced. The options you'll pass to jmap consist of the name of the file to dump to and whether to use a text file or binary file. A binary file is the most useful option, but only when combined with some kind of indexing tool — manually slogging through several megabytes of text filled with hexadecimal values is not the best way to spend your day.
For a more casual glance at the Java heap, jmap also supports a -histo option. -histo produces a text histogram of the objects currently strongly referenced in the heap, sorted by the total number of bytes consumed by that particular type. It also gives the total number of instances of that particular type, which allows for some primitive calculations and guesses about relative cost per instance.
Unfortunately, jmap doesn't have a period-and-max-count option like jstat does, but it's relatively easy to put calls to jmap (or to jmap.main()) into a loop in either a shell script or other class to take snapshots periodically. (In fact, this would be a good extension to add to jmap, either as a source patch to the OpenJDK itself or as an extension to another utility.)
5. jhat (com.sun.tools.hat.Main)
Once you've dumped the heap into a binary file, you can use jhat to analyze the binary heap dump file. jhat creates an HTTP/HTML server that can be surfed in a browser, giving an object-by-object view of the heap, frozen in time. While it could be amusing to walk through the heap, object reference by object reference, you're probably better served by doing some kind of automated analysis of the whole mess. Fortunately, jhatsupports an OQL syntax for doing that analysis.
For example, running an OQL query for all Strings with more than 100 characters would look like this:
select s from java.lang.String s where s.count >= 100
The results are displayed as links to the objects, which then display the complete contents of that object, the field references to other objects as additional links that can be dereferenced. OQL queries can also invoke methods on the objects themselves, use regular expressions as part of the query, and use built-in query tools. One query tool, the referrers() function, displays all the referrers that refer to an object of a given type. Here's the query to find all the referrers to File objects:
select referrers(f) from java.io.File f
You can find the full syntax of OQL and its features under the "OQL Help" page inside the jhat browser environment. Combining jhat with OQL is a powerful way to do a targeted investigation of a misbehaving heap.
In conclusion
The JDK's profiling extensions can be very useful when you need to get a closer look at what's going on inside a Java process. All of the tools introduced in this article can be used by themselves from the command line. They also can be powerfully combined with JConsole or VisualVM. Whereas JConsole and VisualVM provide an overarching view of the Java virtual machine, specifically focused tools like jstat and jmap let you fine-tune your investigation.