Data Intelligence Clustering v2 Upgrades and Running

Data Intelligence Clustering v2 Upgrades and Running

Affected Versions

4.53 [including] onwards.

Context

To start Data Intelligence in HiveMQ EE Broker, one of the nodes in the cluster needs to be designated as a bootstrap node. It must be done only once for the lifetime of the deployment, that is, the bootstrap designation of the node needs to be removed until next rolling restart or upgrade is performed.

image-20260702-154907.png
Lifetime of the cluster is split into a bootstrap phase and a normal operations phase.
Bootstrap node is only required during the bootstrap phase. It must be removed afterwards by removing the bootstrap configuration.

As of version 4.53, the bootstrap node is designated via environment variable HIVEMQ_CLUSTERING_BOOTSTRAP or, alternatively, via JVM system property hivemq.clustering.bootstrap. To mark the node as a bootstrap node, either of these has to be set to true. If the variable is not set or is set to a different value, then the node will be started in a non-bootstrap mode.

The legacy setting via configuration file is also supported but NOT RECOMMENDED for use.

Known Issues

After first deployment of Data Intelligence in HiveMQ EE Broker, the bootstrap option on the node MUST be removed. Attempting to start the node with this option the second time or to add another node to the cluster with this option set to true will fail the startup of the node. This may manifest in a stuck rolling upgrade/rolling restart.

Normal Path

  • If you run Kubernetes deployment using HiveMQ Platform Operator version 2.2.1 and above, you are safe from this issue.

  • If you run non-Kubernetes deployment, on the first deployment or upgrade to the Data Intelligence version, you need to set the environment variable HIVEMQ_CLUSTERING_BOOTSTRAP on one of the nodes to true. After successful start, you need to remove this environment variable.

Kubernetes deployments (StatefulSet) without using HiveMQ Platform Operator are currently not supported.

Failure Case

If the recommendations above are not followed, the rolling restart/rolling update may end up in a stuck state. To resolve this condition, the failed node must be restarted with the bootstrap option removed.