Enabling HiveMQ Enterprise Extension for Kafka in HiveMQ Cloud Enterprise Clusters

Overview

This article addresses the process of enabling the HiveMQ Enterprise Extension for Kafka in HiveMQ Cloud Enterprise clusters. The HiveMQ Enterprise Extension for Kafka implements the native Kafka protocol inside your HiveMQ MQTT broker, enabling seamless integration of MQTT messages into Kafka messaging flows. (1)

Issue Description

Users with HiveMQ Cloud Enterprise clusters may need to enable the HiveMQ Enterprise Extension for Kafka for bi-directional MQTT data movement between IoT devices and Kafka clusters. (2) This requires verification of license coverage and deployment to the specific cluster.

Prerequisites

A valid license for the HiveMQ Enterprise Extension for Kafka (1)
HiveMQ Cloud Enterprise Plan cluster (4)
Running Kafka Cluster (versions 0.10.2 and higher)

Key Features

The HiveMQ Enterprise Extension for Kafka provides:

Forward MQTT messages from IoT devices connected to your HiveMQ MQTT broker to topics in one or more Kafka clusters
Poll information from a Kafka topic and publish this information as MQTT messages to one or more MQTT topics
Use placeholders to dynamically generate MQTT messages at runtime with values extracted from a Kafka record
Multiple MQTT topic filters with full support of wildcards to route MQTT messages to the desired Kafka topics

Resolution Steps

Step 1: Verify License Coverage

Contact HiveMQ Support to confirm that the HiveMQ Enterprise Extension for Kafka is included in your current license.

Step 2: Request Extension Deployment

Submit a support ticket to HiveMQ Support requesting deployment of the Kafka extension to your specific HiveMQ Cloud cluster (provide the cluster URL).

Step 3: Prepare Configuration

Before deployment, prepare your {{config.xml}} configuration file. (1) The configuration file must be located at {{HIVEMQ_HOME/extensions/hivemq-kafka-extension/conf/config.xml}}.

Example minimal configuration:

<?xml version="1.0" encoding="UTF-8" ?>
<kafka-configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                     xsi:noNamespaceSchemaLocation="config.xsd">
    <kafka-clusters>
        <kafka-cluster>
            <id>cluster01</id>
            <bootstrap-servers>127.0.0.1:9092</bootstrap-servers>
        </kafka-cluster>
    </kafka-clusters>
    <mqtt-to-kafka-mappings>
        <mqtt-to-kafka-mapping>
            <id>mapping01</id>
            <cluster-id>cluster01</cluster-id>
            <mqtt-topic-filters>
                <mqtt-topic-filter>mytopic/#</mqtt-topic-filter>
            </mqtt-topic-filters>
            <kafka-topic>my-kafka-topic</kafka-topic>
        </mqtt-to-kafka-mapping>
    </mqtt-to-kafka-mappings>
</kafka-configuration>

Step 4: Verify Deployment

After HiveMQ Support deploys the extension:

Check the HiveMQ Control Center for the newly-created Kafka tab
Review HiveMQ logs to verify proper installation and configuration
Verify the extension is listed in the Active License Features section of the Control Center (5)(https://docs.hivemq.com/hivemq/4.28/control-center/information.html)

Configuration Requirements

The HiveMQ Enterprise Extension for Kafka configuration is divided into four sections:

Kafka Clusters: General information and connection settings for one or more Kafka clusters
MQTT-to-Kafka Mappings: Routing information for one or more MQTT topic filters to a Kafka topic
Kafka-to-MQTT Mappings: Routing information for one or more records in a Kafka topic to one or more MQTT topics
Schema Registry: Schema used to deserialize values in the Kafka topic

Important Notes

HiveMQ Cloud is a fully-managed service, so extension deployment is handled by HiveMQ Support (4)(https://docs.hivemq.com/hivemq-cloud/index.html)
The extension supports hot reload of configuration changes while running (1)(https://docs.hivemq.com/hivemq-kafka-extension/4.28/index.html)(3)(https://docs.hivemq.com/hivemq-kafka-extension/latest/index.html)
The Kafka dashboard becomes visible in the HiveMQ Control Center after successful installation (1)(https://docs.hivemq.com/hivemq-kafka-extension/4.28/index.html)

Prevention and Best Practices

To streamline future extension enablement requests:

Verify license coverage for required extensions before deployment planning
Prepare configuration files in advance, following the official documentation
Test configurations in staging environments before production deployment
Document cluster-specific requirements (Kafka cluster details, topic mappings, authentication settings)
Review the extension documentation for version-specific features and requirements

Related Resources

Support Contact

For assistance with enabling extensions in HiveMQ Cloud Enterprise clusters, contact HiveMQ Support through your support portal.