AWS NLB Configuration Recommendations

The following recommendations assume a deployment utilizing Amazon Elastic Kubernetes Service (Amazon EKS) on AWS (Amazon Web Services).

In following with HiveMQ best practices, we would first recommend using separate Load Balancer entities for MQTT and HTTP traffic. More specifically, on AWS, this entails a dedicated NLB for the MQTT services, and an ingress controller that manages the HTTP services that run on another Load Balancer. This configuration overcomes the problem of bad pod distribution and assumptions made on their placement within availability zones.

When configuring the MQTT-specific Load Balancer, we recommend introducing an additional internal proxy layer like HAProxy, Envoy, or Emissary. In addition to some other beneficial management features, this proxy layer provides a more stable layer with quicker health checks.

For load balancer setup, we recommend using the aws-loadbalancer-controller and annotating the Service of the internal Proxy tier:

Using clusterIP (within Kubernetes documentation here) works very well for us within our testing.

We also recommend incorporating cross-AZ traffic, which offers the benefit of allowing for highly resilient, long-lived flows without forcing connection restarts. Referring to AWS documentation, this will also not introduce significant latency, as “Roundtrip latency between two instances in the same AZ is generally sub-millisecond”.

It is worth mentioning that cross-AZ traffic can introduce a slightly higher cost, as every AZ ‘hop’ can be considered an additional network charge, depending on your deployment.