Enable Prometheus and Grafana monitoring

Owned by Daria Samkova, created
Last updated: Nov 08, 2023
3 min read

Monitoring the HiveMQ broker is crucial in cluster environments.

  • HiveMQ Prometheus Monitoring Extension – exposes an endpoint for Prometheus scraping data from every HiveMQ pod.

  • Prometheus – is scraping the metrics data from the monitoring targets.

  • Grafana – visualises metrics from the Prometheus Datasource.

This article explains how to enable monitoring with HiveMQ Operator.

 Instructions

HiveMQ Kubernetes Operator supports flexible options for Prometheus.

Option 1: Enable HiveMQ to expose an endpoint for scraping metrics by Prometheus.

  1. Edit the values.yaml file of the HiveMQ operator and enable the deployment of the HiveMQ Prometheus Monitoring Extension along with the deployment of the HiveMQ operator.
    When the monitoring.enabled key in the values.yaml then each HiveMQ pod will have the HiveMQ Monitoring Extension installed and enabled. The operator will also create a hivemq-...-metrics service and expose its endpoint by default on port 9399. This makes HiveMQ broker metrics ready for scraping by the Prometheus application.

    monitoring: # Deploy ServiceMonitor for Prometheus Operator and enable Prometheus enabled: true # Deploy a dedicated instance of the prometheus operator, including grafana, as sub-chart dedicated: false

  2. Install HiveMQ with the release name "hivemq" from the Helm chart hivemq-operator to the namespace hivemq and use parameter values from the file values.yaml

    helm upgrade "hivemq" \ --install hivemq/hivemq-operator \ --namespace hivemq \ --values values.yaml

  3. Make sure that alongside the HiveMQ cluster, the Kubernetes Service hivemq-metrics was created:

    kubectl get services --namespace 'hivemq' | grep 'metrics'

  4. Install Prometheus on your own and configure it to scrape from the hivemq:9399/metrics.

  5. Install Grafana on your own and configure it to use the Prometheus Datasource and the HiveMQ Cluster Dashboard.

Option 2: Install Prometheus and Grafana alongside HiveMQ.

  1. Edit the values.yaml of the HiveMQ operator and set the parameter monitoring.dedicated to true. This will make the HiveMQ Operator install Prometheus and Grafana applications along with the installation of the HiveMQ. This will also add a HiveMQ Cluster dashboard to Grafana.
    Both Prometheus and Grafana are installed from the kube-prometheus-stack Helm chart.

  2. Install HiveMQ with the release name hivemq from the Helm chart hivemq-operator to the namespace hivemq and use parameter values from the file values.yaml

  3. Alternatively, you can override the installation parameter values directly in the command line of the helm upgrade command:

     

If an error arises while using the parameter value monitoring.dedicated=true, note that both Prometheus and Grafana are installed from the kube-prometheus-stack Helm chart.

The kube-prometheus-stack Helm chart version might be incompatible with Kubernetes 1.25

To be sure, where the installation failure originates from: hivemq-operator or kube-prometheus-stack, you need to think:
If there no error while installing HiveMQ Operator with monitoring.dedicated=false, but there is an error while installing HiveMQ Operator with monitoring.dedicated=true, this error originates from the kube-prometheus-stack Helm chart and not from the HiveMQ Operator Helm chart.
Such an error should be reported to the issues of the https://github.com/prometheus-community/helm-charts/issues rather than the HiveMQ.