Monitoring JMX metrics with Metricbeat and Elastic Search
Some time ago we did a Kibana dashboard for JMX monitoring in Alfresco via Logstash. Today I’ll briefly let some notes for a similar configuration but with other tool of the Elastic Stack, Metricbeat, using a jolokia plugin for HTTP – JMX bridge. The main difference is that in the previous example, we had a logstash process for getting JMX information in a Tomcat node, and now we have a lighter agent, Metricbeat, which is ideally better suited for this purpose.
Besides the Metricbeat client, as con, we have to add / deploy jolokia.war under webapps directory in our Tomcat node, and to configure basic authentication and role for jolokia in the usual tomcat-users.xml file. We are using Elastic Search, MetricBeat and Logstash in version 7.2.0 and jolokia 1.6.2 (and Liferay 7.0 DXP deployed in Tomcat).
In tomcat-users.xml
<role rolename="jolokia"/> <user username="monitor" password="secret" roles="jolokia"/>
For Metricbeat configuration (jolokia.jmx), we need to set JMX endpoint and the corresponding JMX beans to monitor (once the module is enabled):
- module: jolokia metricsets: ["jmx"] period: 30s hosts: ["localhost:8080"] namespace: "metrics" path: "/jolokia/?ignoreErrors=true&canonicalNaming=false" username: "monitor" password: "secret" jmx.mappings: - mbean: 'java.lang:type=Memory' attributes: - attr: HeapMemoryUsage field: memory.heap_usage - mbean: 'java.lang:type=Threading' attributes: - attr: ThreadCount field: threading.thread_count - mbean: 'java.lang:type=OperatingSystem' attributes: - attr: SystemCpuLoad field: system.system_cpu_load - attr: SystemLoadAverage field: system.system_load_average - attr: OpenFileDescriptorCount field: system.opened_file_descriptors - mbean: 'Catalina:type=ThreadPool,name="ajp-nio-8009"' attributes: - attr: currentThreadCount field: threads.current_threads_count - attr: currentThreadsBusy field: threads.current_threads_busy - mbean: 'com.zaxxer.hikari:type=Pool (HikariPool-1)' attributes: - attr: TotalConnections field: pool.total_connections - attr: ActiveConnections field: pool.active_connections jmx.application: jmx.instance: fields: type: liferay-jmx fields_under_root: true
Then, we get JMX metrics via beats in logstash, having a custom index for JMX in Elastic Search:
input { beats { port => 5044 } } output { #stdout { codec => rubydebug } if [type] == "liferay-jmx" { elasticsearch { index => "metricbeat-jmx-%{+YYYY.MM.dd}" hosts => ["http://localhost:9200"] } } }
Finally, we can create a custom dashboard representation of the obtained JMX data, for example JVM Used Heap or Java Threads, as shown below:
External links: