Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents

...

  1. Create a namespace for Kafka and switch the context to it:

    Code Block
    languagebash
    kubectl create namespace kafka;  
    Code Block
    kubectl config set-context --current --namespace=kafka
  2. Add the repository for the Kafka Helm chart to your package manager.

    Code Block
    languagebash
    helm repo add bitnami https://charts.bitnami.com/bitnami
    Code Block
    helm repo update
  3. Deploy the Kafka server using the Helm chart. The below command deploys Kafka with 2 brokers (replicas).

    Code Block
    languagebash
    helm upgrade --install kafka bitnami/kafka --namespace=kafka --set replicaCount=2 

    With EKS cluster:
    You will need to configure the Amazon EBS CSI driver. You can find detailed steps How to configure Amazon EBS CSI driver for working PersistentVolumes in EKS

  4. Please notice the output of the command above, it provides critical data that is used for the next steps

    1. Consumers can access Kafka via port 9092 on the following DNS name from within your cluster: kafka.kafka.svc.cluster.local

    2. The CLIENT listener for Kafka client connections from within your cluster has been configured with the following security settings: SASL authentication

    3. To connect a client to your Kafka:

      1. username="user1"

      2. To get the password execute the command below:(skip % at the end)

        Code Block
        languagebash
        kubectl get secret kafka-user-passwords --namespace kafka \
          -o jsonpath='{.data.client-passwords}' | base64 -d | cut -d , -f 1;

...

  1. Prepare a simple configuration file for kafka-extension as in the example below.

    • this example configuration will map all incoming MQTT publish packets to the topic “test” in Kafka; and will map the topic “test” in Kafka to the topic “test” in the HiveMQ broker

    • Use your password in <password>here_is_your_password</password>, that you successfully retrieved with this command a few steps ago:

      Code Block
      languagebash
      kubectl get secret kafka-user-passwords --namespace kafka \
        -o jsonpath='{.data.client-passwords}' | base64 -d | cut -d , -f 1;
    • Here is the file:

...