Scrape prometheus metrics on EC2 instance using HiveMQ Prometheus Extension

Owned by Harsh Bansal
Last updated: Mar 14, 2024
4 min read

Prerequisites

  1. A running EC2 instance.

  2. A running HiveMQ setup.

  3. SSH and sudo access to the EC2 instance.

Prometheus Setup

Install Prometheus on EC2 instance

  1. SSH into your EC2 instance and switch to root user

  2. Download Prometheus (e.g. wget …) and untar the file (tar -xzfv …)

  3. Rename the unzipped folder. e.g.
    mv prometheus-2.50.1.linux-amd64 prometheus-files

  4. Create a Prometheus user, and required directories, and make Prometheus the user as the owner of those directories.

    sudo useradd --no-create-home --shell /bin/false prometheus sudo mkdir /etc/prometheus sudo mkdir /var/lib/prometheus sudo chown prometheus:prometheus /etc/prometheus sudo chown prometheus:prometheus /var/lib/prometheus

  5. Copy Prometheus and promtool binary from prometheus-files folder to /usr/local/bin and change the ownership to prometheus user.

sudo cp prometheus-files/prometheus /usr/local/bin/ sudo cp prometheus-files/promtool /usr/local/bin/ sudo chown prometheus:prometheus /usr/local/bin/prometheus sudo chown prometheus:prometheus /usr/local/bin/promtool

  1. Move the consoles and console_libraries directories from prometheus-files to /etc/prometheus folder and change the ownership to prometheus user.

sudo cp -r prometheus-files/consoles /etc/prometheus sudo cp -r prometheus-files/console_libraries /etc/prometheus sudo chown -R prometheus:prometheus /etc/prometheus/consoles sudo chown -R prometheus:prometheus /etc/prometheus/console_libraries

Setup Prometheus Configuration

All the Prometheus configurations should be present in /etc/prometheus/prometheus.yml file.

  1. Create the prometheus.yml file.

  1. Copy the following contents to the prometheus.yml file.

  1. Change the ownership of the file to prometheus user.

Setup Prometheus Service File

Step 1: Create a prometheus service file.

Step 2: Copy the following content to the file.

Step 3: Reload the systemd service to register the prometheus service and start the prometheus service.

Check the prometheus service status using the following command.

The status should show the active state.

Access Prometheus Web UI

Now you will be able to access the Prometheus UI on 9090 port of the Prometheus server.

In case of EC2, replace this IP with Auto-assigned IP address

Setup HiveMQ Prometheus Extension

  1. On your EC2 instance, go to your hivemq folder and cd into the extensions folder

  2. Download and unzip the latest HiveMQ Prometheus Extension. The default configuration is already set in prometheusConfiguration.properties. This usually does not need to be changed

  3. Start hivemq by executing run.sh under hivemq/bin folder(please make sure it is executable (chmod +x run.sh)

  4. You can confirm if the extension worked well by:

    1. Checking HiveMQ logs.

    2. Go to http://<ec2-auto-assigned-ip>/metrics, it should show scrapped metrics page.

Connecting Prometheus to scrape HiveMQ Metrics

  1. Access Your Prometheus Configuration File: Log into your EC2 instance where Prometheus is running. You'll typically find the prometheus.yml configuration file in the directory where you have Prometheus installed, or under /etc/prometheus/ if you used a package manager to install Prometheus.

  2. Edit the Configuration File: Open prometheus.yml in a text editor. You can use nano, vi, or any editor you're comfortable with. For example:

  3. Add a New Job to Scrape HiveMQ Metrics: In the scrape_configs section of the configuration file, add a new job to scrape metrics from the HiveMQ Prometheus operator. You'll specify the metrics path and the address of the server where HiveMQ is running. For example, we used EC2 IP and port 9399 for HiveMQ job. Note that the default binding IP address can be 0.0.0.0

  4. Save and Close the File: After adding the new job, save your changes and exit the text editor.

  5. Reload Prometheus Configuration: Prometheus needs to reload its configuration to start scraping metrics from the new endpoint. You can do this by sending a SIGHUP signal to the Prometheus process. If you're using systemd, you can reload the Prometheus service using:

    If you're not using systemd, find the Prometheus process ID and send a SIGHUP signal to it:

  6. Verify Configuration: After reloading the configuration, you can check that Prometheus is successfully scraping metrics from HiveMQ by accessing the Prometheus UI and navigating to the ‘Targets’ page.(http://<ec2-auto-assigned-ip>/targets) You should see your new job listed with a status indicating whether the scrape is successful.

  7. Going back to http://<ec2-auto-assigned-ip>/graph, you should be able to see HiveMQ metrics scrapped.

     

This concludes the tutorial. As the next steps, you can connect Prometheus to Grafana for better visualization and dashboards