Monitoring your Java Processes with jcmd tool

In this tutorial we will learn more of the jcmd diagnostic command utility which is available since Java 8. We will see how we can gather some useful diagnostic information of a running instance of WildFly.

Currently we have already several diagnostic tools for the JVM like jmap, jstack, jstat. As you will see in this tutorial it’s enough to learn just one: jcmd which allows to perform most diagnostic commands just using a single tool. Please note that jcmd must be however used on the same machine where the JVM is running.

The shell jcmd is available into the $JAVA_HOME/bin folder. If you execute it without any parameter it will dump the list of Java Processes which are running:

$ jcmd
6328 sun.tools.jcmd.JCmd
28430 /home/jboss/wildfly-11.0.0.Final/jboss-modules.jar -mp /home/jboss/wildfly-11.0.0.Final/modules org.jboss.as.standalone -Djboss.home.dir=/home/jboss/wildfly-11.0.0.Final -Djboss.server.base.dir=/home/jboss/wildfly-11.0.0.Final/standalone

In order to use jcmd you need to execute some commands against the Java Process (you can reference it either by PID or by its name). The list of available commands depend on the actual JDK you are using. The “help” command however will display which commands are built-in:

$ jcmd 28430 help
28430:
The following commands are available:
JFR.stop
JFR.start
JFR.dump
JFR.check
VM.native_memory
VM.check_commercial_features
VM.unlock_commercial_features
ManagementAgent.stop
ManagementAgent.start_local
ManagementAgent.start
GC.rotate_log
Thread.print
GC.class_stats
GC.class_histogram
GC.heap_dump
GC.run_finalization
GC.run
VM.uptime
VM.flags
VM.system_properties
VM.command_line
VM.version
help

As you can see, there’s plenty of useful commands available in the current JDK I’m using (OpenJDK 9). Let’s see for example how to gather a Thread dump of a Java process:

$ jcmd 28430 Thread.print
28430:
2018-01-22 09:25:00
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode):

"Attach Listener" #178 daemon prio=9 os_prio=0 tid=0x0000000003755000 nid=0x1538 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
. . . . .

How to print all the system properties set for a VM

$ jcmd 28430 VM.system_properties

jboss.qualified.host.name=localhost.localdomain
jboss.host.name=localhost
javax.xml.transform.TransformerFactory=__redirected.__TransformerFactory
jboss.server.name=localhost
. . . . .

Print all the flags used for a VM (This is quite useful to gather quickly the initial and maximum heap size)

$ jcmd 28430 VM.flags
28430:
-XX:CICompilerCount=3 -XX:InitialHeapSize=67108864 -XX:MaxHeapSize=536870912 -XX:MaxMetaspaceSize=268435456 -XX:MaxNewSize=178782208 -XX:MetaspaceSize=100663296 -XX:MinHeapDeltaBytes=524288 -XX:NewSize=22020096 -XX:OldSize=45088768 -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseFastUnorderedTimeStamps -XX:+UseParallelGC 

You can check what is the Java process uptime in seconds

$ jcmd 28430 VM.uptime
28430:
28416.487 s

Another useful feature is creating a Class histogram, which provides at your finger tips how many instances of a Class is around. This command can be a bit verbose so you can redirect it to a file or use a grep to filter just on some classes:

$ jcmd 28430 GC.class_histogram | grep "java.lang.Thread"
  59:            75          28200  java.lang.Thread
  89:           553          17696  java.lang.ThreadLocal$ThreadLocalMap$Entry
 103:           190          15712  [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;
 215:           190           4560  java.lang.ThreadLocal$ThreadLocalMap
 386:             5           1152  [Ljava.lang.Thread;
 454:            17            816  java.lang.ThreadGroup
 455:            51            816  java.lang.ThreadLocal
 695:            14            336  java.lang.ThreadLocal$SuppliedThreadLocal
1074:             3            160  [Ljava.lang.ThreadGroup;
1136:             6            144  java.lang.Thread$State
1592:             2             80  [Ljava.lang.Thread$State;

You can create as well an heap dump (hprof dump) with jcmd, by providing the filename:

$ jcmd 28430 GC.heap_dump filename=Myheapdump
28430:
Heap dump file created

You can run a full Garbage Collector cycle as well:

$ jcmd 28430 GC.run
28430:
Command executed successfully

Additionally, jcmd can be used to produce a Java Flight Recorder diagnostic output. The Java Flight Recorder is a great tool to investigate performance issues, however it is a commercial tool from Oracle JDK, therefore check its license agreements: https://docs.oracle.com/javase/9/troubleshoot/troubleshoot-performance-issues-using-jfr.htm

Please note that these commands are available only if the Java application was started with the Java Flight Recorder enabled, that is, using the following options: -XX:+UnlockCommercialFeatures -XX:+FlightRecorder

Let’s see how we can start a recording of a JFR session:

Here is how to start a 10-minute recording on the running Java process with the identifier 28430 and save it to session1.jfr in the current directory, use the following:

$ jcmd 28430 JFR.start name=Session1 duration=10m filename=session1.jfr

To check the status of all recordings running for the specified process, including the recording identification number, file name, duration use the JFR.check diagnostic command:

$ jcmd 28430 JFR.check

The JFR.stop diagnostic command stops a running recording and has the option to discard the recording data. For example:

$ jcmd 28430 JFR.stop

Finally, you can dump the data collected so far by the recording with a specific identification number

$ jcmd 28430 JFR.dump