Export HiveMQ Metrics from either Prometheus or a Grafana Datasource

It is recommended to install and monitor HiveMQ broker with Prometheus and Grafana. Metrics collected in this fashion can be batch exported with the HiveMQ grafana-metrics-exporter tool.

Instructions

Request from the HiveMQ Support Team to provide you with the HiveMQ grafana-metrics-exporter tool. The tool is provided as a JAR file named hivemq-metrics-exporter.jar
Run the exporter.
You can run the exporter with (Java 11+):
java -jar hivemq-metrics-exporter.jar <options>
You can run a quick test if the parameters are correct by adding the --test param, then only the first metric is exported. Debug output can be enabled with -v

Exporting from Grafana

Find the Grafana Datasource Id.
To export the metrics from a Grafana Datasource first find the Datasource Id. To find the ID for the desired Grafana Datasource, first, log in to Grafana and next go to Menu→Configuration→Data Sources* and click on the desired Datasource, then the ID is part of the URL
Example URL: https://my.grafana-url.com:port/datasources/edit/1/.
In the Example URL, the ID for the Datasource is 1.
1. Note: If you cannot reach the Menu→Configuration→Data Sources, it indicates that you do not have necessary access rights to the Grafana. In this case, you need to contact the admin of Grafana and get the necessary access rights.
2. In case the ID is alphanumeric(e.g. AB123BC), please find the numerical ID via https://hivemq.atlassian.net/wiki/spaces/KB/pages/2391441409
Run the exporter. A full command should look like the following
1. In case you are using localhost via port-forwarding
  1. please use --url "http://localhost:port" , where port is the target port of Grafana service
  2. username: admin, password: prom-operator
2. Time is in UTC and ends with letter Z

java -jar hivemq-metrics-exporter.jar \
--url "https://my.grafana-url.com:port/grafana" \
-u "myusername" \
-p "mypassword" \
-ds 1 \
-f "tag1=\"mytagvalue1\",tag2=\"mytagvalue2\"" \
--from "2021-08-01T10:00:00Z" \
--to "2021-08-01T19:00:00Z" \
--step 15s

The result is a ZIP file that contains a JSON for each HiveMQ metric.

Exporting from Prometheus

Sometimes, Prometheus is installed but Grafana is not.

To export directly from Prometheus use the option “--direct“.

A full command looks like this:

java -jar hivemq-metrics-exporter.jar \
--url "https://my.prometheus-url.com:port" \
-u "myusername" \
-p "mypassword" \
--direct \
-ds 1 \
-f "tag1=\"mytagvalue1\",tag2=\"mytagvalue2\"" \
--from "2021-08-01T10:00:00Z" \
--to "2021-08-01T19:00:00Z" \
--step 15s

The result is a ZIP file that contains a JSON for each HiveMQ metric.

Note, you can specify any numeric Datasource id with option --direct, as with direct Prometheus export the Datasource id is ignored.

Params

Name	Description	Required	Example

Name	Description	Required	Example
`--url`	Grafana URL (Without trailing slash)	x	`--url https://my.grafana.url:3000`
`-ds`, `--datasource`	Grafana Datasource ID	x	`-ds 1`
`--from`	Start timestamp in ISO Format. If it ends with `Z` at the end be aware that this means UTC and you might have to correct the timestamp manually to match the local time	x	`--from "2021-08-01T10:00:00Z"`
`--to`	End timestamp in ISO Format. If it ends with `Z` at the end be aware that this means UTC and you might have to correct the timestamp manually to match the local time	x	`--from "2021-08-01T10:00:00Z"`
`-u`, `--user`	Grafana Username		`-u "myusername"`
`-p`, `--password`	Grafana Password. Can be passed as param directly `-p "mypass"` or in interactive mode when not passing a param after `-p`		`-p "mypass"`
`-f`, `--filter`	Prometheus Filter to only pull HiveMQ metrics with specific tags. Example: If you would get the HiveMQ metrics with the query `com_hivemq_messages_retained_current{tagone="valueone", job=~"valuetwo"}` then the filter would be `-f "tagone=\"valueone\", job=~\"valuetwo\""`		`-f "tagone=\"valueone\", job=~\"valuetwo\""`
`--step`	Step size, the distance between measured points, should be greater or equal to the Prometheus scrape interval. Default: `15s`		`--step 1m`
`--test`	Test Mode stops after the first metric. Good for testing if the params are correct		`--test`
`-v`	Enable verbose mode, good for debugging. Very verbose!		`-v`

To get help on all available parameters of the command, run the command with the option --help:

java -jar hivemq-metrics-exporter.jar --help

This will output the full list of available options and values.

Troubleshooting

In case the Datasource Id is unknown, but the name of the Datasource is known, follow the steps of the following article to find the Datasource id by the Datasource name:

https://hivemq.atlassian.net/wiki/spaces/KB/pages/2391441409

What are ‘tag1’ and ‘tag2’ arguments?

In Prometheus, labels are key-value pairs that are associated with metrics. When a Prometheus server scrapes a target, it collects metric data from the target and attaches labels to the collected data to provide additional information about the metric.

The labels displayed in the Prometheus UI's Targets page are the labels that have been attached to the metrics collected from the target. These labels can provide information about the source of the metric data, such as the host, application, or process that generated the metric. Labels can also be used to organize and filter metrics, allowing users to more easily query and analyze the data.

Same labels and values can be used as a ‘tag1’ and ‘tag2’ arguments of the metrics-exporter tool to refine the metrics you want to export from the metrics you do no want to export.