File realm - ESE - legacy Kubernetes hivemq-operator

\uD83D\uDCD8 Prerequisites

1. You have a Kubernetes cluster with API version >= 1.25 Setting up AKS Cluster in Azure

2. You have a HiveMQ broker cluster with version >= 4.2x installed in the Kubernetes cluster Install HiveMQ on the AKS cluster

3. You have added a valid license to the HiveMQ broker Add a valid license to HiveMQ Cluster

4. You have added a load balancer and verified the connection Enable load balancer and test with MQTT CLI

 Instructions

1. Place your HiveMQ Enterprise Security Extension license file (.elic) in the license folder of your HiveMQ installation (skip this step if you are using a trial version of the extension).

   1. Get the values.yaml file from the Helm chart (skip this step if you are already in possession of the values.yaml file)

      helm show values hivemq/hivemq-operator

      This will output the values.yaml file to the stdout. You can save it to a file:

      helm show values hivemq/hivemq-operator > my-values.yaml

   2. HiveMQ Enterprise Security Extension requires a separate license file, e.g. ese-license.elic, in the $HIVEMQ_HOME/license directory. You can skip this step. If you skip this step, then the HiveMQ Enterprise Security Extension will start in trial mode, limited to 5h, and will be automatically disabled by the HiveMQ broker after 5h. To add the ese-license.elic along with the hivemq-license.lic, create a new configMap hivemq-license including all desired license files:

      kubectl create configmap hivemq-license --namespace=hivemq \
        --from-file hivemq-licesen.lic \
        --from-file kafka-license.elic

   3. Edit the values.yaml file of the hivemq-operator, section hivemq.configMaps. Update this:

        configMaps: []
        # ConfigMaps to mount to the HiveMQ pods. These can be mounted to existing directories without shadowing the folder contents as well.
        #- name: hivemq-license
        #  path: /opt/hivemq/license

      To this:

        configMaps: 
          - name: hivemq-license
            path: /opt/hivemq/license

      This will mount the content of the configMap hivemq-license to the directory /opt/hivemq/license of the hivemq-broker pods.

2. Prepare your HiveMQ Enterprise Security Extension configuration files.
   HiveMQ Enterprise Security Extension is preinstalled with HiveMQ so once you enable it, it will look for its configuration file. You must prepare this file before enabling the extension. If you skip this step, the extension will not find its configuration file and will not load any configuration.

   1. config.xml

      <?xml version="1.0" encoding="UTF-8" ?>
      <enterprise-security-extension
              xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xsi:noNamespaceSchemaLocation="config.xsd"
              version="1">
          <realms>
              <file-realm>
                  <name>file-realm</name>
                  <enabled>true</enabled>
                  <configuration>
                      <file-path>conf/ese-file-realm.xml</file-path>
                  </configuration>
              </file-realm>
          </realms>
          <pipelines>
              <!-- secure access to the mqtt broker -->
              <listener-pipeline listener="ALL">
                  <!-- authenticate over a file -->
                  <file-authentication-manager>
                      <realm>file-realm</realm>
                  </file-authentication-manager>
                  <!-- authorize over a file -->
                  <file-authorization-manager>
                      <realm>file-realm</realm>
                  </file-authorization-manager>
              </listener-pipeline>
              <!-- secure access to the control center -->
              <control-center-pipeline>
                  <!-- authenticate over a file -->
                  <file-authentication-manager>
                      <realm>file-realm</realm>
                  </file-authentication-manager>
                  <!-- authorize over a file -->
                  <file-authorization-manager>
                      <realm>file-realm</realm>
                  </file-authorization-manager>
              </control-center-pipeline>
              <!-- secure access to the REST API -->
              <rest-api-pipeline listener="ALL">
                  <authentication-preprocessors>
                      <http-headers-preprocessor>
                          <basic-auth-extraction/>
                      </http-headers-preprocessor>
                  </authentication-preprocessors>
                  <!-- authenticate over a file -->
                  <file-authentication-manager>
                      <realm>file-realm</realm>
                  </file-authentication-manager>
                  <!-- authorize over a file -->
                  <file-authorization-manager>
                      <realm>file-realm</realm>
                  </file-authorization-manager>
              </rest-api-pipeline>
          </pipelines>
      </enterprise-security-extension>
      

   2. file-realm.xml

      <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
      <ese-file-realm xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                      xsi:noNamespaceSchemaLocation="ese-file-realm.xsd">
          <mqtt>
              <!-- users are fetched via AUTHENTICATION_KEY-->
              <users>
                  <user>
                      <name>mqtt-user-1</name>
                      <password>mqtt-password-1</password>
                      <permissions>
                          <permission>
                              <topic>#</topic>
                              <qos>ALL</qos>
                              <activity>ALL</activity>
                              <retain>ALL</retain>
                              <shared-subscription>ALL</shared-subscription>
                              <shared-group>group-1</shared-group>
                          </permission>
                      </permissions>
                  </user>
      
                  <user>
                      <name>mqtt-user-2</name>
                      <!-- <password>mqtt-password-2</password> -->
                      <!-- https://docs.hivemq.com/ese/latest/enterprise-security-extension/ese-helper.html#hash -->
                      <password encoding="Base64">aDGGvi/kW+ba1ZlQxObGZT7kNK7Vg4qkFR7HvHapCGk=</password>
                      <iterations>10</iterations>
                      <salt>mqtt-password-2-salt</salt>
                      <algorithm>PKCS5S2</algorithm>
                      <roles>
                          <role>publisher</role>
                          <role>subscriber</role>
                      </roles>
                  </user>
              </users>
      
              <!-- roles are fetched via AUTHENTICATION_ROLE_KEY-->
              <roles>
                  <role>
                      <id>publisher</id>
                      <permissions>
                          <permission>
                              <topic>topic-1</topic>
                              <qos>ALL</qos>
                              <activity>PUBLISH</activity>
                          </permission>
                          <permission>
                              <topic>topic-2</topic>
                              <qos>ALL</qos>
                              <activity>PUBLISH</activity>
                          </permission>
                      </permissions>
                  </role>
      
                  <role>
                      <id>subscriber</id>
                      <permissions>
                          <permission>
                              <topic>topic-3</topic>
                              <qos>ALL</qos>
                              <activity>SUBSCRIBE</activity>
                          </permission>
                      </permissions>
                  </role>
              </roles>
          </mqtt>
      
          <control-center>
              <!-- list of available permissions:
                     https://www.hivemq.com/docs/ese/latest/enterprise-security-extension/ese.html#control-center-access-control-permissions -->
              <users>
                  <user>
                      <name>cc-user-1</name>
                      <password>cc-password-1</password>
                      <permissions>
                          <permission>HIVEMQ_SUPER_ADMIN</permission>
                      </permissions>
                  </user>
      
                  <user>
                      <name>cc-user-2</name>
                      <!-- <password>cc-password-2</password> -->
                      <!-- https://docs.hivemq.com/ese/latest/enterprise-security-extension/ese-helper.html#hash -->
                      <password encoding="Base64">XW5ESC/iKdtYWIDpcgeFOIXhjA9reoCBzNcFm/SQhWk=</password>
                      <iterations>10</iterations>
                      <salt>cc-password-2-salt</salt>
                      <algorithm>PKCS5S2</algorithm>
                      <roles>
                          <role>topics</role>
                          <role>shared-subscriptions</role>
                      </roles>
                  </user>
              </users>
      
              <roles>
                  <role>
                      <id>topics</id>
                      <permissions>
                          <permission>HIVEMQ_VIEW_DATA_TOPIC</permission>
                      </permissions>
                  </role>
                  <role>
                      <id>shared-subscriptions</id>
                      <permissions>
                          <permission>HIVEMQ_VIEW_DATA_TOPIC</permission>
                          <permission>HIVEMQ_VIEW_DATA_SUBSCRIPTION</permission>
                          <permission>HIVEMQ_VIEW_PAGE_SHARED_SUBSCRIPTION_DETAIL</permission>
                          <permission>HIVEMQ_VIEW_PAGE_SHARED_SUBSCRIPTION_LIST</permission>
                      </permissions>
                  </role>
              </roles>
          </control-center>
      
          <rest-api>
              <!-- list of available permissions:
                      https://docs.hivemq.com/ese/latest/enterprise-security-extension/ese.html#rest-api-access-permissions -->
              <users>
                  <user>
                      <name>rest-api-user-1</name>
                      <password>rest-api-password-1</password>
                      <permissions>
                          <permission>HIVEMQ_SUPER_ADMIN</permission>
                      </permissions>
                  </user>
      
                  <user>
                      <name>rest-api-user-2</name>
                      <!-- <password>rest-api-password-2</password> -->
                      <!-- https://docs.hivemq.com/ese/latest/enterprise-security-extension/ese-helper.html#hash -->
                      <password encoding="Base64">apukEACJ+UeyE+onxnseWRZi/QgWOeSTtFwxHJTswxc=</password>
                      <iterations>10</iterations>
                      <salt>rest-api-password-2-salt</salt>
                      <algorithm>PKCS5S2</algorithm>
                      <roles>
                          <role>backups</role>
                          <role>subscriptions</role>
                      </roles>
                  </user>
              </users>
      
              <roles>
                  <role>
                      <id>backups</id>
                      <permissions>
                          <permission>HIVEMQ_MANAGEMENT_BACKUPS_GET</permission>
                          <permission>HIVEMQ_MANAGEMENT_BACKUPS_POST</permission>
                          <permission>HIVEMQ_MANAGEMENT_BACKUPS_BACKUPID_GET</permission>
                          <permission>HIVEMQ_MANAGEMENT_BACKUPS_BACKUPID_POST</permission>
                          <permission>HIVEMQ_MANAGEMENT_FILES_BACKUPS_BACKUPID_GET</permission>
                      </permissions>
                  </role>
                  <role>
                      <id>subscriptions</id>
                      <permissions>
                          <permission>HIVEMQ_MQTT_CLIENTS_CLIENTID_SUBSCRIPTIONS_GET</permission>
                      </permissions>
                  </role>
              </roles>
          </rest-api>
      </ese-file-realm>

3. Place your HiveMQ Enterprise Security Extension configuration files in the conf folder of your HiveMQ Enterprise Security Extension.

   1. Create a new configMap ese-config including all desired config files:

      kubectl create configmap ese-config --namespace=hivemq \
        --from-file config.xml \
        --from-file ese-file-realm.xml

   2. Edit the values.yaml file of the hivemq-operator, section hivemq.extensions. Update this:

      hivemq:
        extensions:
        ...
        
          - name: hivemq-enterprise-security-extension
            extensionUri: preinstalled
            enabled: false
            # Note that this is just an example initialization routine. Make sure this points to the current JDBC version you require for your configuration.
            initialization: |
              # Download JDBC driver for PostgreSQL
              [[ ! -f drivers/postgres-jdbc.jar ]] &&
              curl -L https://jdbc.postgresql.org/download/postgresql-42.2.14.jar --output drivers/jdbc/postgres.jar

      To this:

      hivemq:
        extensions:
        ...
        
          - name: hivemq-enterprise-security-extension
            extensionUri: preinstalled
            enabled: true
            configMap: ese-config
            initialization: |
              [[ ! -f conf/config.xml ]] &&
              [[ -f /conf-override/extensions/hivemq-enterprise-security-extension/config.xml ]] &&
              ln -s /conf-override/extensions/hivemq-enterprise-security-extension/config.xml conf/config.xml &&
              [[ ! -f conf/ese-file-realm.xml ]] &&
              [[ -f /conf-override/extensions/hivemq-enterprise-security-extension/ese-file-realm.xml ]] &&
              ln -s /conf-override/extensions/hivemq-enterprise-security-extension/ese-file-realm.xml conf/ese-file-realm.xml 

4. Finally, disable the default security extension. By default, the HiveMQ distribution comes with the allow-all extension that permits all MQTT connections without requiring authentication. Before you use HiveMQ in production, add an appropriate security extension and remove the HiveMQ allow-all extension.
   To disable the extension, set the HIVEMQ_ALLOW_ALL_CLIENTS environment variable to false.
   Edit the values.yaml file of the hivemq-operator, section hivemq.env. Update this:

   hivemq:
   ...
   
     env: []
     ## Skip config validation
     #     - name: "HIVEMQ_SKIP_CONFIG_VALIDATION"
     #      value: "true"
     ## Add custom environment variables (e.g. for your extension) here.
     # - name: MY_CUSTOM_ENV
     #   value: some-value

   To this:

     env:
       - name: "HIVEMQ_ALLOW_ALL_CLIENTS"
         value: "false"

5. Install HiveMQ with updated configuration.

   helm upgrade hivemq --install hivemq/hivemq-operator --values my-hivemq-values.yaml --namespace hivemq

6. Verify logs
   If everything is correct, then

   1. The HiveMQ log contains info about using the correct license:

      kubectl logs deployment/hivemq | grep 'Using valid'

      INFO  - Using valid Enterprise Edition CPU license (hivemq-license.lic) issued to HiveMQ - Internal for max 9999 CPU cores, valid until 2024-03-31.
      INFO  - Using valid license (ese-license.elic) for enterprise extension with name "HiveMQ Enterprise Security Extension", valid until 2024-03-31.

   2. The HiveMQ log contains info about successful extension start:

      kubectl logs  deployments.apps/hivemq -f | grep -i 'Security'

      INFO  - Starting extension with id "hivemq-enterprise-security-extension" at /opt/hivemq/extensions/hivemq-enterprise-security-extension
      INFO  - HiveMQ Enterprise Security Extension: Successfully loaded configuration from '/opt/hivemq/extensions/hivemq-enterprise-security-extension/conf/config.xml'.
      INFO  - Starting HiveMQ Enterprise Security Extension.
      INFO  - Started HiveMQ Enterprise Security Extension successfully in 774ms.
      INFO  - Extension "HiveMQ Enterprise Security Extension" version 4.24.0 started successfully.

7. Get the external IP of the MQTT load balancer

   mqttHost=$(kubectl get svc hivemq-hivemq-mqtt -o jsonpath='{.status.loadBalancer.ingress[0].ip}{"\n"}')
   mqttPort=$(kubectl get svc hivemq-hivemq-mqtt -o jsonpath='{.spec.ports[0].port}{"\n"}')

8. End-to-end testing of mqtt clients

   1. Subscribe a mqtt client:

      mqtt subscribe -h $mqttHost -p $mqttPort -t '#' -q 1 -u mqtt-user-1 -pw mqtt-password-1

      Do not close this terminal session. This will allow you to see the results.

   2. From a different terminal session, publish a message to the topic “test”:

      mqtt publish -h $mqttHost -p $mqttPort -t topic-1 -m Hello -q 1 -u mqtt-user-2 -pw mqtt-password-2

9. If everything is correct, the subscriber will receive the message:

   mqtt subscribe -h $mqttHost -p $mqttPort -t '#' -q 1 -u mqtt-user-1 -pw mqtt-password-1

   Hello

\uD83D\uDCCB Related articles