No data in Grafana after installing HiveMQ Kubernetes Operator with monitoring
Fixed in the hivemq/hivemq-operator:4.7.10
.
Chart version
0.11.14
App version
4.13.0
App image
hivemq/hivemq4:k8s-4.13.0
Operator image
hivemq/hivemq-operator:4.7.10
Problem
No data is displayed in the Grafana “HiveMQ Cluster” Dashboard, after installing the hivemq-operator with monitoring.dedicated=true
:
helm upgrade ${hivemqRelease} --install 'hivemq/hivemq-operator' \
--namespace ${namespace} \
--set monitoring.enabled=true \
--set monitoring.dedicated=true
where:
${hivemqRelease}
is your release name for the hivemq deployment;${namespace}
is the namespace you want to install the hivemq-operator.
Solution
It might happen because Prometheus is missing Service Monitor – a custom resource defining from where to scrape the data.
Prepare a manifest definition for the Service Monitor,
monitorService.yaml
:apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: labels: hivemq-cluster: ${hivemqRelease} release: ${hivemqRelease} name: hivemq spec: endpoints: - port: metrics jobLabel: hivemq selector: matchLabels: hivemq-cluster: ${hivemqRelease} app: hivemq
Do substitute
${hivemqRelease}
with the actual name you gave to the hivemq-operator deployment.labels
here do command the Prometheus to take the ServiceMonitor into account.selector
here specifies which pods to scrape the data from.port
here specifies the name of the port to which the metrics are exposed.Apply the manifest
kubectl apply -f monitorService.yaml --namespace ${namespace}
When all hivemq-pods are running and all their containers are ready, go to the Grafana and check if the data is updating in the “HiveMQ Cluster” dashboard.
This is tested with Kubernetes API v1.23 and hivemq-operator versions up to Chart 0.11.11, App 4.11.0, hivemq/hivemq-operator:4.7.9