Setup ESE with Postgres using k8s operator

Prerequisites:

Helm version v3+
Running Kubernetes cluster version 1.18.0 or higher
kubectl latest version

Instructions

Create a Namespace for the HiveMQ/Postgres deployment.
You can skip this step you want to run everything in “default” namespace.
Execute the following command to create a namespace:
kubectl create namespace <namespace name>
Switch to the newly created namespace:
kubectl config set-context --current --namespace=<namespace name>
Deploy Postgres
Add the Bitnami Helm repository:
helm repo add bitnami https://charts.bitnami.com/bitnami
Create a postgres_values.yaml file to configure Postgres deployment:
global: #storageClass: "rook-ceph-block" postgresql: auth: password: password postgresPassword: password username: admin primary: initdb: scriptsConfigMap: ese-db-init
Create a ConfigMap called ese-db-init containing the ese-db-init.sql script, which creates tables and inserts data for testing purposes:

kubectl create configmap ese-db-init --from-file 0_ese-db-init.sql --from-file 1_permissions.sql
Deploy Postgres using Helm:
helm upgrade postgres --install bitnami/postgresql --values postgres_values.yaml
Verify the status of the pod:
kubectl get pods
If an error occurs, check the pod logs:
kubectl logs <pod name>
Connect to the Postgres pod to verify the connection:
psql --host 127.0.0.1 -U postgres -d postgres -p 5432
Use the following commands in the Postgres shell to interact with the database:
- \l: List the databases.
- \c <db name>: Connect to a specific database.
- \dt: List the tables from the connected database.
- select * from users;
Deploy HiveMQ with Enterprise Security Extension (ESE)

Create a ConfigMap for the HiveMQ license (skip this step if you don't have a license yet):

kubectl create configmap hivemq-license --from-file=hivemq-ese-2021.lic

Create a config.xml configuration file for the Enterprise Security Extension: You can also find examples of this file in the extension folder under conf/examples

configure sql-realm.

db-name - you can find this in the ese-db-init.sql, default it is postgres

db-host - this should be your postgres service name, you can get this via kubetctl get svc command

db-username - This should be from postgres_values.yaml auth.username block

db-password - This should be from postgres_values.yaml auth.password block

Configure the listener-pipeline. Since here we are using role-based authorization we need to set <use-authorization-key> to false and <use-authorization-role-key> to true.

<?xml version="1.0" encoding="UTF-8" ?>
<enterprise-security-extension
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="config.xsd"
        version="1">
    <realms>
        <!-- a postgresql db-->
        <sql-realm>
            <name>postgres-backend</name>
            <enabled>true</enabled>
            <configuration>
                <db-type>POSTGRES</db-type>
                <db-name>hivemq</db-name>
                <db-host>postgres-servicename</db-host>
                <db-port>5432</db-port>
                <db-username>hivemq</db-username>
                <db-password>password</db-password>
            </configuration>
        </sql-realm>
    </realms>
    <pipelines>
        <!-- secure access to the mqtt broker -->
        <listener-pipeline listener="ALL">
            <!-- authenticate over a sql db -->
            <sql-authentication-manager>
                <realm>postgres-backend</realm>
            </sql-authentication-manager>
            <!-- authorize over a sql db -->
            <sql-authorization-manager>
                <realm>postgres-backend</realm>
                <use-authorization-key>false</use-authorization-key>
                <use-authorization-role-key>true</use-authorization-role-key>
            </sql-authorization-manager>
        </listener-pipeline>
    </pipelines>
</enterprise-security-extension>

Create a ConfigMap for the ESE configuration:

kubectl create configmap enterprise-security-extension-config --from-file config.xml

Create a hivemq_values.yaml file to deploy HiveMQ using the Kubernetes operator. Ensure that the ESE extension is preinstalled: Full values of the operator can be found here

(Note: CPU and Memory configs we have set for demo purposes. For production, we recommend qualifying our minimum hardware requirements for HiveMQ to run as expected. )

hivemq:
  cpu: 2
  memory: 2Gi
  nodeCount: "2"
  
  extensions:
  - enabled: true
    extensionUri: preinstalled
    initialization: |
      # A little hack because k8s configMaps can't handle sub-directories
      [[ -e /conf-override/extensions/hivemq-enterprise-security-extension/config.xml ]] &&
      rm -f $(pwd)/conf/config.xml &&
      cp -s /conf-override/extensions/hivemq-enterprise-security-extension/config.xml $(pwd)/conf/config.xml
      [[ ! -f drivers/postgres-jdbc.jar ]] &&
      curl -L https://jdbc.postgresql.org/download/postgresql-42.2.14.jar --output drivers/jdbc/postgres.jar
    name: hivemq-enterprise-security-extension
    configMap: enterprise-security-extension-config
  
  ports:
    - name: "mqtt"
      port: 1883
      expose: true
      patch:
        - '[{"op":"add","path":"/spec/selector/hivemq.com~1node-offline","value":"false"},{"op":"add","path":"/metadata/annotations","value":{"service.spec.externalTrafficPolicy":"Local"}}]'
        # If you want Kubernetes to expose the MQTT port to external traffic
        # - '[{"op":"add","path":"/spec/type","value":"LoadBalancer"}]'
    - name: "cc"
      port: 8080
      expose: true
      patch:
        - '[{"op":"add","path":"/spec/sessionAffinity","value":"ClientIP"}]'
        # If you want Kubernetes to expose the MQTT port to external traffic
        # - '[{"op":"add","path":"/spec/type","value":"LoadBalancer"}]'
  configMaps:
    - name: hivemq-license
      path: /opt/hivemq/license
operator:
  admissionWebhooks:
    enabled: false

Deploy the HiveMQ cluster using Helm:

helm upgrade --install -f hivemq_values.yaml <release name> hivemq/hivemq-operator

Check the status of the pods:

kubectl get pods

Verify the extension logs if it has started successfully:

kubectl logs <pod name>

Use the MQTT CLI to perform quick tests.