Consul-template sidecar | HiveMQ Platform Operator (new)

\uD83D\uDCD8 Instructions

These instructions assume that the secrets in the vault are in the following structure:

hivemq-poc1
└── opt
    └── hivemq
        ├── conf
        │   ├── key.passphrase
        │   ├── keystore.password
        │   ├── keystore_base64
        │   ├── truststore.password
        │   └── truststore_base64
        └── license
            └── license_base64

To achieve such a structure in the Hashicorp Vault at a kv-2 path, and enable hivemq to read from the path, the following commands in the Hashicorp Vault are used:

# Enable Vault to store key-value pairs at path hivemq-poc1
vault secrets enable -path=hivemq-poc1 kv-v2

# Create vault secrets from files

cd /tmp

vault kv put hivemq-poc1/opt/hivemq/conf keystore_base64="$(base64 </tmp/broker-keystore.jks)" \
  keystore.password=changeme key.passphrase=changeme
vault kv put hivemq-poc1/opt/hivemq/conf truststore_base64="$(base64 </tmp/broker-truststore.jks)"
vault kv put hivemq-poc1/opt/hivemq/license license_base64="$(base64 </tmp/hivemq4.lic)"

# Enable hivemq to read from paths
vault policy write hivemq - <<EOF
path "hivemq-poc1/data/opt/hivemq/license" {
   capabilities = ["read"]
}
path "hivemq-poc1/data/opt/hivemq/conf" {
   capabilities = ["read"]
}
EOFIf your secrets in the vault are stored differently, you must adopt the consul-template.hcl accordingly.

Create the HCL template to fetch and decode the secrets:
consul-template.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: consul-template
  namespace: hivemq
data:
  consul-template.hcl: |
    template {
      contents = "{{ with secret (printf \"%s/opt/hivemq/license\" (mustEnv \"TARGET_ENV\")) }}{{ base64Decode .Data.data.license_base64 }}{{ end }}"
      destination = "/opt/hivemq/license/hivemq4.lic"
    }
    template {
      contents = "{{ with secret (printf \"%s/opt/hivemq/conf\" (mustEnv \"TARGET_ENV\")) }}{{ base64Decode .Data.data.keystore_base64 }}{{ end }}"
      destination = "/opt/hivemq/keystore/broker-keystore.jks"
    }
    template {
      contents = "{{ with secret (printf \"%s/opt/hivemq/conf\" (mustEnv \"TARGET_ENV\")) }}{{ base64Decode .Data.data.truststore_base64 }}{{ end }}"
      destination = "/opt/hivemq/keystore/broker-truststore.jks"
    }

kubectl apply -f consul-template.yaml --namespace hivemq

Add init container to the hivemq-platform values.yaml

additionalInitContainers:
  - name: init-consul-template
    image: hashicorp/consul-template:latest
    command: [ "consul-template", "-once", "-config", "/consul-template/consul-template.hcl", "-kill-signal", "SIGTERM", "-log-level", "debug" ]
    env:
      - name: VAULT_ADDR
        value: http://vault.vault.svc.cluster.local:8200
      - name: VAULT_TOKEN
        value: root
      - name: TARGET_ENV
        value: hivemq-poc1/data
    volumeMounts:
      - name: consul-template
        mountPath: /consul-template
      - name: hivemq-license
        mountPath: /opt/hivemq/license
      - name: hivemq-keystore
        mountPath: /opt/hivemq/keystore

Add the additional container to the hivemq-platform values.yaml:

additionalContainers:
  - name: sidecar-consul-template
    image: hashicorp/consul-template:latest
    command: [ '/bin/sh', '-c' ]
    args:
      - |
        trap 'echo "Terminating container"; exit 0' SIGTERM
        consul-template -config /consul-template/consul-template.hcl -kill-signal SIGTERM -log-level debug"  &
        while true; do sleep 1; done
    env:
      - name: VAULT_ADDR
        value: http://vault.vault.svc.cluster.local:8200
      - name: VAULT_TOKEN
        value: root
      - name: TARGET_ENV
        value: hivemq-poc1/data
    volumeMounts:
      # Volume consul-template to read the consul-template.hcl from
      - name: consul-template
        mountPath: /consul-template
      # Volume hivemq-license to store the decoded license
      - name: hivemq-license
        mountPath: /opt/hivemq/license
      # Volume hivemq-keystore to store the keystore and truststore
      - name: hivemq-keystore
        mountPath: /opt/hivemq/keystore

Add additional volumes to the hivemq-platform values.yaml:

additionalVolumes:
  # Volume hivemq-license to store the decoded license
  - name: hivemq-license
    path: /opt/hivemq/license
    type: emptyDir
    containerName: hivemq
  
  # Volume hivemq-keystore to store the keystore and truststore
  - type: emptyDir
    name: hivemq-keystore
    containerName: hivemq
    path: /opt/hivemq/keystore
    
  # Volume consul-template with consul-template.hcl file
  - type: configMap
    name: consul-template
    containerName: sidecar-consul-template
    path: /consul-template

Install HiveMQ Platform Operator

helm upgrade op --install hivemq/hivemq-platform-operator --set logLevel=DEBUG

Install HiveMQ Platform broker

helm upgrade broker --install hivemq/hivemq-platform --values values-hivemq-platform.yaml

If Helm upgrade broker fails, for example, because of a mistake in a test configuration, do not immediately force-delete broker pods. Instead, first uninstall the release:

helm uninstall broker

Only if the uninstallation cannot terminate broker pods, force delete them:

kubectl delete pod --selector hivemq-platform=broker --grace-period=0 --force

Tested with:

Helm v3.15.4
Kubernetes v1.29.7
Kubectl v1.29.2
Kubernetes provider: AKS 2xStandard_A8_v2 Kubernetes v1.29.7
HiveMQ 4.31.0 (Helm chart: hivemq/hivemq-platform-0.2.21)
HiveMQ Platform Operator 1.5.1 (Helm chart: hivemq-platform-operator-0.2.9)

Consul-template sidecar | HiveMQ Platform Operator (new)

\uD83D\uDCD8 Instructions

Tested with:

\uD83D\uDCCB Related articles