Consul-template sidecar | HiveMQ Platform Operator (new)

\uD83D\uDCD8 Instructions

These instructions assume that the secrets are put into the vault in the following way and it is enabled to read them via policy “hivemq”:

vault secrets enable -path=hivemq-poc1 kv-v2

cd /tmp

vault kv put hivemq-poc1/opt/hivemq/conf keystore_base64="$(base64 </tmp/broker-keystore.jks)" \
  keystore.password=changeme key.passphrase=changeme
vault kv put hivemq-poc1/opt/hivemq/conf truststore_base64="$(base64 </tmp/broker-truststore.jks)"
vault kv put hivemq-poc1/opt/hivemq/license license_base64="$(base64 </tmp/hivemq4.lic)"

vault policy write hivemq - <<EOF
path "hivemq-poc1/data/opt/hivemq/license" {
   capabilities = ["read"]
}
path "hivemq-poc1/data/opt/hivemq/conf" {
   capabilities = ["read"]
}
EOF

This means that the secrets in the vault have the following structure:

hivemq-poc1
└── opt
    └── hivemq
        ├── conf
        │   ├── key.passphrase
        │   ├── keystore.password
        │   └── keystore_base64
        └── license
            └── license_base64

In this example, the path in the vault is hivemq-poc1/data/opt/hivemq/license and it contains keys keystore_base64, keystore.password, and key.passphrase, and path hivemq-poc1/data/opt/hivemq/license contains key license_base64.
If your secrets in the vault are stored differently, you must adopt the consul-template.hcl accordingly.

Create the HCL template to fetch and decode the secrets:
consul-template.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: consul-template
  namespace: hivemq
data:
  consul-template.hcl: |
    template {
      contents = "{{ with secret (printf \"%s/opt/hivemq/license\" (mustEnv \"TARGET_ENV\")) }}{{ base64Decode .Data.data.license_base64 }}{{ end }}"
      destination = "/opt/hivemq/license/hivemq4.lic"
    }
    template {
      contents = "{{ with secret (printf \"%s/opt/hivemq/conf\" (mustEnv \"TARGET_ENV\")) }}{{ base64Decode .Data.data.keystore_base64 }}{{ end }}"
      destination = "/opt/hivemq/keystore/broker-keystore.jks"
    }
    template {
      contents = "{{ with secret (printf \"%s/opt/hivemq/conf\" (mustEnv \"TARGET_ENV\")) }}{{ base64Decode .Data.data.truststore_base64 }}{{ end }}"
      destination = "/opt/hivemq/keystore/broker-truststore.jks"
    }

kubectl apply -f consul-template.yaml --namespace hivemq

Add init container to the hivemq-platform values.yaml

additionalInitContainers:
  - name: init-consul-template
    image: hashicorp/consul-template:latest
    command: [ "consul-template", "-once", "-config", "/consul-template/consul-template.hcl", "-kill-signal", "SIGTERM", "-log-level", "debug" ]
    env:
      - name: VAULT_ADDR
        value: http://vault.vault.svc.cluster.local:8200
      - name: VAULT_TOKEN
        value: root
      - name: TARGET_ENV
        value: hivemq-poc1/data
    volumeMounts:
      - name: consul-template
        mountPath: /consul-template
      - name: hivemq-license
        mountPath: /opt/hivemq/license
      - name: hivemq-keystore
        mountPath: /opt/hivemq/keystore

Add the additional container to the hivemq-platform values.yaml:

additionalContainers:
  - name: sidecar-consul-template
    image: hashicorp/consul-template:latest
    command: [ '/bin/sh', '-c' ]
    args:
      - |
        trap 'echo "Terminating container"; exit 0' SIGTERM
        consul-template -config /consul-template/consul-template.hcl -kill-signal SIGTERM -log-level debug"  &
        while true; do sleep 1; done
    env:
      - name: VAULT_ADDR
        value: http://vault.vault.svc.cluster.local:8200
      - name: VAULT_TOKEN
        value: root
      - name: TARGET_ENV
        value: hivemq-poc1/data
    volumeMounts:
      # Volume consul-template to read the consul-template.hcl from
      - name: consul-template
        mountPath: /consul-template
      # Volume hivemq-license to store the decoded license
      - name: hivemq-license
        mountPath: /opt/hivemq/license
      # Volume hivemq-keystore to store the keystore and truststore
      - name: hivemq-keystore
        mountPath: /opt/hivemq/keystore

Add additional volumes to the hivemq-platform values.yaml:

additionalVolumes:
  # Volume hivemq-license to store the decoded license
  - name: hivemq-license
    path: /opt/hivemq/license
    type: emptyDir
    containerName: hivemq
  
  # Volume hivemq-keystore to store the keystore and truststore
  - type: emptyDir
    name: hivemq-keystore
    containerName: hivemq
    path: /opt/hivemq/keystore
    
  # Volume consul-template with consul-template.hcl file
  - type: configMap
    name: consul-template
    containerName: sidecar-consul-template
    path: /consul-template

Install HiveMQ Platform Operator

helm upgrade op --install hivemq/hivemq-platform-operator --set logLevel=DEBUG

Install HiveMQ Platform broker

helm upgrade broker --install hivemq/hivemq-platform --values values-hivemq-platform.yaml

If Helm upgrade broker fails, for example, because of a mistake in a test configuration, do not immediately force-delete broker pods. Instead, first uninstall the release:

helm uninstall broker

Only if the uninstallation cannot terminate broker pods, force delete them:

kubectl delete pod --selector hivemq-platform=broker --grace-period=0 --force

Tested with:

Helm v3.15.4
Kubernetes v1.29.7
Kubectl v1.29.2
Kubernetes provider: AKS 2xStandard_A8_v2 Kubernetes v1.29.7
HiveMQ 4.31.0 (Helm chart: hivemq/hivemq-platform-0.2.21)
HiveMQ Platform Operator 1.5.1 (Helm chart: hivemq-platform-operator-0.2.9)

Consul-template sidecar | HiveMQ Platform Operator (new)

\uD83D\uDCD8 Instructions

Tested with:

\uD83D\uDCCB Related articles