How to Monitor Datamer via Java Management Extensions (JMX)

Goal

Ability to monitor Java applications such as Hadoop and Datameer with JMX and monitor the underlying OS with SNMP to watch the machines CPU activity levels, memory usage, network traffic levels, disk IO, and more. 

Learn

JMX is a Java technology that supplies tools for managing and monitoring applications, system objects, and devices. It pulls information from Hadoop and monitors Java applications.

[datameer@<host> current]$ bin/conductor.sh start --jmx
Java 7 found
DeployMode: live
DAS Options: -Dcom.sun.management.jmxremote.port=6789 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Ddb.name=dap
Java native libraries: /opt/datameer/current/lib/native
Note: The variable DAS_PORT overrides the port 8080 defined by jetty.port in start.ini
Starting jetty using port 8080
Starting Jetty: STARTED Jetty Do 18. Jun 06:22:01 EDT 2015 under PID 1383

You see that the service open an additional management port 6789 and will listen on it.

[datameer@<host> current]$ netstat -apn | grep 1383
...
tcp 0 0 :::8080 :::* LISTEN 1383/java
tcp 0 0 :::6789 :::* LISTEN 1383/java
tcp 0 0 ::ffff:<host>:8080 ::ffff:<ip>:59443 VERBUNDEN 1383/java
tcp 0 0 ::ffff:127.0.0.1:40010 ::ffff:127.0.0.1:3306 VERBUNDEN 1383/java
...

Now you can connect to it via jconsole.

[datameer@<host> current]$ jconsole localhost:6789

On the tab MBean you should see the Datameer RestController where you can access operations as described under REST API and JMX.

There is also command line (CLI) version cmjx available, which you can start after download via

[datameer@<host> current]$ java -cp $JAVA_HOME/lib/tools.jar:/<yourPath>/cjmx_2.10-2.3.0-app.jar cjmx.Main

To gather in example the open file descriptors you would use

[datameer@<host> current]$ java -cp $JAVA_HOME/lib/tools.jar:/<yourPath>/cjmx_2.10-2.3.0-app.jar cjmx.Main "<DatameerPID>" "mbeans 'java.lang:type=OperatingSystem' select OpenFileDescriptorCount"

java.lang:type=OperatingSystem
------------------------------
  OpenFileDescriptorCount: <#>