Challenge
The HiveMQ operator (since 0.9.0) as well as the HiveMQ Docker images (since 4.6.4) introduced some breaking changes. The. following manual shows how to execute a zero downtime rolling upgrade from older operator and HiveMQ versions.
Prerequisites:
k8s version 1.16+ is installed
Helm version 3 is installed
Hivemq k8s operator repo is already added
Expected Result:
Here we will show the necessary steps for a zero-downtime upgrade of both the HiveMQ k8s operator as well as HiveMQ itself.
Upgrade hivemq version without downtime. Hivemq version 4.6.4 should be running without error using k8s operator version 0.9.2
Steps to follow:
Hivemq cluster values file:
apiVersion: hivemq.com/v1 kind: HiveMQCluster metadata: name: hivemq-dev spec: env: - name: TEST_ENV value: 'set'
operator values file:
monitoring: enabled: true operator: admissionWebhooks: enabled: false failurePolicy: Fail deployCr: false logLevel: DEBUG
Deploy Hivemq operator with 0.8.7
helm upgrade --version 0.8.7 --install -f operator.yaml <release name> hivemq/hivemq-operator
Apply Hivemq cluster with hivemq 4.6.2
kubectl apply -f <hivemq cluster.yaml>
Simply apply the new CRD using the following command (considering k8s version is 1.16+)
Note: This step can be skipped if you are using BladeCenter.kubectl apply -f https://raw.githubusercontent.com/hivemq/helm-charts/prerelease-4.6.4/charts/hivemq-operator/crds/hivemq-cluster.yaml
Scale operator to 0 so it doesn’t immediately reconcile. This will only remove the operator pod, but leave the HiveMQ nodes as remaining
kubectl scale —replicas=0 deployment <operator name>
Edit hivemq cluster yaml to update the hivemq version as 4.6.4
Apply the changes
Kubectl apply -f <hivemq cluster.yaml>
check the pods status(no errors)
kubectl get pods
Update operator version i.e 0.9.2 in cluster(operator config File)
Run the following command for rolling upgrade
helm upgrade --version 0.9.2 --install -f operator.yaml <release name> hivemq/hivemq-operator