Integrating HiveMQ Kafka Extension with Azure Key Vault for Environment Variables

Overview

This article provides guidance on integrating the HiveMQ Enterprise Extension for Kafka with Azure Key Vault when deploying on Azure Kubernetes Service (AKS). It covers the use of environment variables and secure secret management.

Issue Description

When deploying HiveMQ with the Kafka Extension on Azure Kubernetes Service, users may need to securely manage sensitive configuration data such as:

Kafka cluster credentials (username/password)
TLS certificates and keystores
Authentication tokens

Prerequisites

Azure CLI installed and configured
kubectl installed
Helm package manager installed
Running Azure Kubernetes Service cluster
HiveMQ Platform Operator installed
Valid HiveMQ Enterprise Extension for Kafka license

Solution Overview

1. Environment Variables in Kafka Extension Configuration

The HiveMQ Kafka Extension supports environment variables in its configuration file.

<?xml version="1.0" encoding="UTF-8" ?>
<kafka-configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:noNamespaceSchemaLocation="config.xsd">
    <kafka-clusters>
        <kafka-cluster>
            <id>cluster01</id>
            <bootstrap-servers>${ENV:KAFKA_BOOTSTRAP_SERVERS}</bootstrap-servers>
            <authentication>
                <plain>
                    <username>${ENV:KAFKA_USERNAME}</username>
                    <password>${ENV:KAFKA_PASSWORD}</password>
                </plain>
            </authentication>
        </kafka-cluster>
    </kafka-clusters>
</kafka-configuration>

Important: Environment variables cannot be hot reloaded and must be set before HiveMQ startup.

2. Azure Key Vault Integration Steps

Step 1: Create Resource Group and AKS Cluster

# Create resource group
az group create --name <resource-group-name> --location <location>

# Create AKS cluster with Key Vault integration
az aks create \
  --resource-group <resource-group-name> \
  --name <cluster-name> \
  --enable-managed-identity \
  --enable-addons azure-keyvault-secrets-provider

Step 2: Create and Configure Key Vault

az keyvault create \
  --name <keyvault-name> \
  --resource-group <resource-group-name> \
  --enable-rbac-authorization false

Step 3: Add Secrets to Key Vault

az keyvault secret set \
  --vault-name <keyvault-name> \
  --name kafka-username \
  --value <your-kafka-username>

az keyvault secret set \
  --vault-name <keyvault-name> \
  --name kafka-password \
  --value <your-kafka-password>

Step 4: Configure Access Permissions

kubectl get azureidentity -n kube-system

az keyvault set-policy \
  --name <keyvault-name> \
  --object-id <secrets-provider-object-id> \
  --secret-permissions get list

Step 5: Create SecretProviderClass

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: kafka-secrets
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true"
    userAssignedIdentityID: <client-id>
    keyvaultName: <keyvault-name>
    tenantId: <tenant-id>
    objects: |
      array:
        - |
          objectName: kafka-username
          objectType: secret
        - |
          objectName: kafka-password
          objectType: secret

Step 6: Deploy Kafka Extension Configuration

kubectl create configmap kafka-configuration \
  --from-file=config.xml=kafka-config.xml \
  -n <namespace>

Step 7: Configure HiveMQ Platform with Extension

hivemq:
  nodeCount: "1"
  extensions:
    - name: hivemq-kafka-extension
      extensionUri: preinstalled
      enabled: true
      supportsHotReload: true
      configMapName: "kafka-configuration"

3. TLS Configuration for Kafka

If connecting to Confluent Cloud or other TLS-enabled Kafka clusters:

<kafka-cluster>
    <id>cluster01</id>
    <bootstrap-servers>${ENV:KAFKA_BOOTSTRAP_SERVERS}</bootstrap-servers>
    <tls>
        <enabled>true</enabled>
        <hostname-verification>false</hostname-verification>
    </tls>
    <authentication>
        <plain>
            <username>${ENV:KAFKA_USERNAME}</username>
            <password>${ENV:KAFKA_PASSWORD}</password>
        </plain>
    </authentication>
</kafka-cluster>

Verification Steps

Verify the extension is loaded:

kubectl logs <hivemq-pod-name> -n <namespace> | grep kafka

Check for successful initialization:

INFO - Starting extension with id "hivemq-kafka-extension"

Verify Kafka connection in HiveMQ Control Center under the Kafka tab.

Common Issues and Troubleshooting

Authentication Errors

If you see errors like Connection to node terminated during authentication:

Verify TLS is enabled in configuration if required
Confirm API key/secret are correct
Ensure environment variables are set before HiveMQ startup

Extension Not Loading

Remove the DISABLED file from the extension directory
Verify the config.xml file exists at HIVEMQ_HOME/extensions/hivemq-kafka-extension/conf/config.xml
Check that a valid license file is present

Best Practices

Use Environment Variables: Store sensitive data like passwords and API keys as environment variables rather than hardcoding in configuration files
Enable TLS: Always use TLS encryption when connecting to production Kafka clusters
Hot Reload Support: The Kafka extension supports hot reload for configuration changes, but environment variables require a restart
Schema Validation: Enable schema validation for the config.xml to catch configuration errors early

Prevention and Recommendations

Store all sensitive credentials in Azure Key Vault
Use managed identities for authentication instead of service principals where possible
Implement proper RBAC policies on Key Vault access
Document environment variable requirements before deployment
Test configuration changes in non-production environments first
Monitor extension metrics in HiveMQ Control Center