Why is my queue growing

Question

Why is my queue growing?

Answer

Queues may experience growth when clients are unable to consume messages at a rate equal to or greater than the messages arriving for their subscriptions. This commonly occurs due to too slow or offline clients (the offline clients with persistent sessions). It's important to note that queues can only serve as temporary buffers if the incoming message stream persists, as the broker operates within finite resources.

The configuration parameter max-queue-size plays a crucial role in managing individual client queue sizes for QoS 1 and QoS 2 messages. By default, it is set to 1000, limiting the size of queues. Once the queue for a specific client reaches this threshold, the broker will drop a message queued by that client. The determination of whether it drops the newest or oldest message depends on the configured strategy, elaborated in our detailed documentation.

It's essential to understand that merely increasing the queue size in the configuration won't prevent messages from being dropped indefinitely, as the disk may eventually become full.

When faced with a growing queue and dropped messages, take the following steps:

1. Identify Responsible Client Subscriptions:

   • Determine which client subscriptions are responsible for dropping messages.

2. Ensure Subscribed Clients Are Connected:

   • Confirm that the consuming clients are actively connected to the broker and are subscribed to their topics.

3. Verify Clients' Capabilities:

   • Ensure that the capabilities of the clients are sufficient to handle the inflow of messages.

4. Consider Load Distribution:

   • Alternatively, distribute the load on the client side by leveraging shared subscriptions.

By following these steps, you can effectively manage and address the reasons behind your growing queue, promoting a smoother and more efficient system operation. For further guidance, refer to our comprehensive documentation.