Metrics

Introduction

You can think of metrics as a stream of tiny, time-stamped events, that flow into a special kind of database, organized to efficiently query such a stream, and aggregate the results in multiple useful ways. A full explanation of metrics is beyond the scope of this document, however.

Metrics are a key part of observability, the ability to see how your application, running as tens or hundreds (or thousands) of servers, is operating as a whole. Where logging can give you information about a specific request, and can somewhat grudgingly aggregate data together - metrics services such as Prometheus exist to aggregate vast amounts of data points together, and give you the ability to see what’s happening to your application now, or over time, broken up by any number of dimensions. Dimensions are used to select events by application-supplied values, that can include such things as the server’s hostname, a request’s URI, the environment (staging vs. production), or the AWS region.

At the core, your application will include extra code to emit metrics; depending on the underlying metrics service you are using, these are either stored in-memory until scraped by another collecting service, or pushed to the collecting service at intervals.

Pedestal’s default metrics library is DropWizard, and the default configuration stores the results internally in JMX.

API

The metrics functions are defined in the io.pedestal.log namespace, along with supporting protocols and utility functions.

Generally, there is no need to define a metric before using it; it will be registered or created, from its name, automatically when first used.

All of these metrics-producing functions have two forms: the canonical form accepts a recorder, the metric name, and additional arguments; the common form omits the recorder, and uses the default-recorder.

A metric name can be any object that can be converted to a string; usually a fully qualified symbol or keyword. For a keyword, the leading : stripped off.

`counter`

Counters are simply values that increase over time; the argument is a delta value, a long to increment the counter by.

A common counter metric might be the total number of requests processed.

`gauge`

Gauges are used to track a value that varies over time; a callback function is passed to the gauge function; this callback will be periodically invoked to return a value for the metric.

A common gauge metric might be the number of entries stored in a cache.

`histogram`

A histogram is somewhat like a gauge; it’s a value that varies over time. The difference is, you supply the values (there isn’t as callback) and the metrics that are produced are broken out in a number of ways: by min, max, mean, median, standard deviation, and by various percentiles.

A common histogram metric might be the latency of database reads or updates.

`meter`

Meters are used to track the rate at which operations occur.

meter is much like counter, it is invoked with a value indicating how many operations occurred (this value is typically 1). It produces moving averages for the one-, five-, and fifteen-minute rates based on sampling this value.

A common meter might be the number of total number of requests processed.

`default-recorder`

This is a var, not a function; it stores a recorder, an object that extends the MetricRecorder protocol; this is usually an instance of com.codahale.metrics.MetricRegistry.

By default, the registry is created via metric-registry, and uses a jmx-reporter to store metrics.

However, this can be overridden at application startup; the JVM system property io.pedestal.log.defaultMetricsRecorder or environment variable PEDESTAL_METRICS_RECORDER can specify the fully-qualified name of a function to invoke (with no arguments), that returns the default recorder.