Monitoring Java based Applications: Metrics+Graphite+Grafana

As developers when we write a piece of code we try our best to make sure we have carefully chosen the best algorithms, the data structures are carefully tuned to match the situation at hand and the piece of software we wrote just works out of the box. Unfortunately life is not ideal and there are some hiccups that are bound to hit us however smooth the ride be. Some nasty bugs will inevitably show up on the production server as the software is used for a length of time. The best one can do then is to monitor the software being used in production environment, anticipate the problems before it hits the user and hampers their experience.

This post will talk about such a stack for monitoring java based applications involving:

1. Dropwizard Metrics
2. Graphite and
3. Grafana

We will use dropwizard metrics to instrument our Java code and push the data to graphite. Graphite will store our time-series data meaning values such as latency, performance, throughput, counts etc corresponding to say our APIs. Though Graphite renders graphs built on top of the time-series data as and when required, Graphite on its own doesn't provide the best visual experience and dashboard properties. We will thence use Grafana to build and view our dashboards. So the pipeline on a high level will be this:

Here is the documentation of dropwizard metrics listing its various capabilities. Their documentation is not that great, in my opinion, but workable. The good news is there is also a spring library that provides annotations you can use directly to enable instrumentation. Just add an annotation on top of a method and thats it. Here is an example:

@Metered(absolute = true, name= "RuntimeError")

public ErrorMessage handleThrowable(HttpServletRequest request, Throwable t) {

            ………..

            }

As you see here, there is an annotation (@Metred) which is responsible for instrumenting the following method- handleThrowable(...). The metred annotation gives the timed average meaning number of errors in a unit of time. As simple as that! The code in bold is all you need to write to instrument the following method, for instance.

Next, we need to tell our metrics where Graphite resides so data can be sent to Graphite. Here is how you configure this. The prefixedWith() attribute defines how Graphite will show this. Here is a screenshot from my Graphite dashboard showing the tree-based structure that we just set.

On the right of this pane shown above you ca see the graph you select. The visual features in Graphite is rather limited so we will forward the Graphit input to Grafana. Here is a video introducing Grafana interface.e

Make your dashboards the way you wish to see your graphs and keep monitoring to stay ahead of the bugs!