Using regular expressions in Trace Recordings

With the help of regular expressions, textual search patterns can be defined.

When used as filters for Trace Recordings the resulting output can be vastly more helpful.

Regular Expressions are a broad topic and cannot be covered in full here. This guide will demonstrate the use of HiveMQ’s pattern matching capabilities by creating a Trace Recording, matching all clients with a clientId beginning with publisher0, followed by a single character. Further the special case of creating a recording for topic tree patterns with use of the + character will be shown.

The key to utilising RegEx in Trace Recording filters is knowing that client identifiers and topics are treated as regular strings by the filters, with no MQTT specific treatment of tokens

Prerequisites

In order to verify, that our messages are being delivered as we expect we will subscribe to a test topic pattern.

In our example we will capture messages by clients including their identifiers in the topic structure. This is a common pattern in production environments.

HiveMQ’s MQTT CLI lets us subscribe to all of these this in one command by using the wildcard operator:

mqtt sub -t 'our/+/topic' -h localhost -i subscriber00

To have an easier time differentiating our clients, we gave this one the identifier subscriber00.

Let this session remain active throughout the test.

Configuring the Trace Recording

Initiating, managing and downloading Trace Recordings is done in the HiveMQ Control Center.

Here we must Add New Trace Recording in order to reach the configuration mask.

First, choose a new, unique name. This need not match a topic name and only serves to help identify the recording we are about to create.

Client Filter

Next, we have to define a Client Filter. Here we will make use of the wildcard character . that acts as a quantifier for “any single character”.

After clicking Add Client Filter it should become visible as an existing filter:

Filtering for topics using the wildcard operator

In order for our recording to match the subscription of subscriber00, which uses MQTT’s single level wildcard character, a Regular Expression will allow us to do so. Here foreslashes need to be escaped with \.

our\/.+\/topic

The + in this expression is not to be confused with MQTT’s wildcard operator. In this case it acts as a RegEx quantifier

Message Type

From here we can select the types of messages we are interested in. For this example, we are only interested in PUBLISH messages.

Finally, after making sure that Start Date is set to a past time and the selected End Date is far enough into the future to cover our efforts, we are ready to Start Trace Recording.

Don’t worry about timing it just right or otherwise having to wait for your recording to end. Trace Recordings can be stopped at any time

Upon success, your new recording will show up in the overview with the name you specified:Upon success, your new recording will show up in the overview with the name you specified:

Generate test data

In order to verify our recording works as expected, we will publish messages.

You can use the following loop to send messages using both clientIDs containing a single character after publisher0 as well ones with an additional second character.

for i in `seq 5 15`; do sleep 1; mqtt pub -t "our/publisher0$i/topic" -h 127.0.0.1 -m "message by publisher0$i" -i publisher0$i; done

Once $i reaches 10, the clientID will contain a character more than we specified in our filter. We expect to not see any messages from publisher010 to publisher015 in our recording.

Inspecting the Trace Recording

Once our publishing run is complete, we may

stop the Trace Recording by clicking stop
have HiveMQ prepare the resulting output by clicking download
and click save to receive the resulting compressed trace recording

After unpacking the downloaded file, the resulting .trace file can be inspected with any text editor.

You should see a Sent and corresponding Received for each message, similar to the following:

2020-04-09 15:52:43,526 - [publisher05] - Received PUBLISH message (topic: our/publisher05/topic, QoS: 0, retain: false, duplicate delivery: false)
2020-04-09 15:52:45,297 - [publisher06] - Received PUBLISH message (topic: our/publisher06/topic, QoS: 0, retain: false, duplicate delivery: false)
[...]
2020-04-09 15:53:16,852 - [publisher09] - Received PUBLISH message (topic: our/publisher09/topic, QoS: 0, retain: false, duplicate delivery: false)

Please refer to HiveMQ Docs for a list of available regular expression quantifiers.

Information on filtering subscriber groups can be found Create Trace Recordings for a shared subscription