Challenge
The HiveMQ operator (since 0.9.0) as well as the HiveMQ Docker images (since 4.6.4) introduced some breaking changes. The. following manual shows how to execute a zero downtime rolling upgrade from older operator and HiveMQ versions.
Prerequisites:
k8s version 1.16+ is installed
Helm version 3 is installed
Hivemq k8s operator repo is already added
Expected Result:
Here we will show the necessary steps for a zero-downtime upgrade of both the HiveMQ k8s operator as well as HiveMQ itself.
Upgrade hivemq version without downtime. Hivemq version 4.6.4 should be running without error using k8s operator version 0.9.2
Steps to follow:
Example of hivemq-cluster.yaml file:
apiVersion: hivemq.com/v1
kind: HiveMQCluster
metadata:
name: hivemq-dev
spec:
env:
- name: TEST_ENV
value: 'set'
Example of HiveMQ operator values.yaml file:
monitoring:
enabled: true
operator:
admissionWebhooks:
enabled: false
failurePolicy: Fail
deployCr: false
logLevel: DEBUG
Deploy Hivemq operator with 0.8.7
helm upgrade --version 0.8.7 --install -f operator.yaml <release name> hivemq/hivemq-operator
Apply Hivemq cluster with hivemq 4.6.2
kubectl apply -f <hivemq-cluster.yaml>
Simply apply the new CRD using the following command (considering k8s version is 1.16+)
Note: This step can be skipped if you are using BladeCenter.kubectl apply -f https://raw.githubusercontent.com/hivemq/helm-charts/prerelease-4.6.4/charts/hivemq-operator/crds/hivemq-cluster.yaml
Scale operator to 0 so it doesn’t immediately reconcile. This will only remove the operator pod, but leave the HiveMQ nodes as remaining
kubectl scale —replicas=0 deployment <operator name>
Edit hivemq cluster yaml to update the hivemq version as 4.6.4
Apply the changes
Kubectl apply -f <hivemq cluster.yaml>
check the pods status(no errors)
kubectl get pods
Update operator version i.e 0.9.2 in the cluster(operator config File or command line)
Run the following command for a rolling upgrade
helm upgrade --version 0.9.2 --install -f operator.yaml <release name> hivemq/hivemq-operator