Inject license via Hashicorp Vault Agent Init Container | HiveMQ Platform Operator (new)
Vault Agent provides a number of different helper features, specifically addressing the following challenges:
Automatic authentication
Secure delivery/storage of tokens
Lifecycle management of these tokens (renewal & re-authentication)
Vault Agent can help authenticate to Hashicorp Vault, retrieving a base64 encoded HiveMQ license, decoding the license, and saving it to a file in the HiveMQ container.
Prerequisites
These instructions require the following tools on the local machine:
Kubernetes command-line interface (CLI)
Helm CLI
The kubectl context should be set to the Kubernetes cluster where the HiveMQ broker will be installed.
Instructions
Install the Hashicorp Vault Helm Chart
Add the HashiCorp Helm repository.
helm repo add hashicorp https://helm.releases.hashicorp.com
Update all the repositories to ensure
helm
is aware of the latest versions.helm repo update hashicorp
Install the latest version of the Hashicorp Vault server running in development mode.
Development mode: Running a Hashicorp Vault server in development is automatically initialized and unsealed. This is ideal in a learning environment but NOT recommended for a production environment.kubectl create namespace vault kubectl config set-context --current --namespace=vault helm install vault hashicorp/vault --set "server.dev.enabled=true" --namespace vault
The vault pod and vault Agent Injector pod are deployed in the default namespace.
Display all the pods in the default namespace.
Successful output from the command resembles this example:
The
vault-0
pod runs a vault server in development mode. Thevault-agent-injector
pod performs the injection based on the annotations present or patched on a deployment.Wait until the
vault-0
pod andvault-agent-injector
pod are running and ready1/1
.
Set a secret in Hashicorp Vault
Prepare your HiveMQ license in the file named
hivemq.lic
.Run the command on the
vault-0
pod to enable kv-v2 secrets at the pathhivemq
. The command is executed on thevault-0
container.Successful output from the command resembles this example:
Create a new secret at path
hivemq/test/license
with ahivemq_license_b64
key and base64-encodedhivemq.lic
file.Successful output from the command resembles this example:
Verify that the secret is defined at the path
hivemq/test/license
.Successful output from the command resembles this example:
The secret is ready for the application.
Configure Kubernetes authentication
Hashicorp Vault provides a Kubernetes authentication method that enables clients to authenticate with a Kubernetes Service Account Token. This token is provided to each pod when it is created.
Enable the Kubernetes authentication method.
Successful output from the command resembles this example:
Hashicorp Vault accepts a service token from any client in the Kubernetes cluster. During authentication, Hashicorp Vault verifies that the service account token is valid by querying a token review Kubernetes endpoint.
Configure the Kubernetes authentication method to use the location of your HiveMQ Kubernetes cluster.
Note: Ensure you are using Hashicorp Vault v1.13.3 or greater for the best compatibility with recent Kubernetes versions.
Successful output from the command resembles this example:
The environment variable
KUBERNETES_PORT_443_TCP_ADDR
is defined and references the internal network address of the Kubernetes host.Verify the kubernetes_host of the created kubernetes authentication method.
For a client to read the secret data defined at
hivemq/test/license
, the read capability must be granted for the pathhivemq/data/test/license
. A policy defines a set of capabilities.Create a new policy named
hivemq
that grants read access to secrets at the pathhivemq/data/test/license
.Create a Kubernetes authentication role named
hivemq
.Successful output from the command resembles this example:
The role connects the Kubernetes service account, hivemq-platform-pod-broker, and namespace,
hivemq
, with the Hashicorp Vault policy,hivemq
. The tokens returned after authentication are valid for 24 hours.Lastly, exit the
vault-0
pod.
Inject secrets into the pods
Switch the kubectl context to the HiveMQ Kubernetes cluster.
Install hivemq
Check the pods in the hivemq namespace.
Wait until the deployed
broker-0
pod reports that it isRunning
and ready.Check service accounts. Copy the name of the service account of hivemq platform pods. In this example:
hivemq-platform-pod-broker
Create a new secret required for the Hashicorp Vault authentication. vault-auth-secret.yaml:
Create a config map definition file
configmap.yaml
with a Vault Agent configuration file,vault-agent-config.hcl
This creates a config map with a Vault Agent configuration file,
vault-agent-config.hcl
. Notice that the Vault Agent Auto-Auth (auto_auth
block) is configured to use thekubernetes
auth method enabled at theauth/kubernetes
path on the Vault Server. The Vault Agent will use thehivemq
role that you created in your Hashicorp Vault.The
sink
block specifies the location on the disk where to write tokens. In this example, thesink
is set to/home/vault/.vault-token
.The template block creates a file that retrieves the
hivemq_license_b64
secret at thehivemq/data/test/license
path, decodes it from base64, and writes the decoded text to the/etc/secrets/hivemq.lic
file.Create a ConfigMap containing a Vault Agent configuration.
Update HiveMQ Platform helm chart values by adding a new Init Container for the Vault Agent. You can get the original values by running the command
Add a new Init Container for the Vault Agent.
If your Hashicorp Vault is external to the HiveMQ Kubernetes cluster, replace the domain
vault.vault.svc.cluster.local
with the external IP of your Hashicorp Vault.Update HiveMQ Platform helm chart values by adding the volume required by the Vault Agent Init Container configuration file and the destination volume for the HiveMQ license secret.
Update the HiveMQ Platform
Test end-to-end
When the HiveMQ pod has started successfully the vault-agent init container will be run. The container shall authenticate to the Vault, fetch the secret, and decode the secret to the file hivemq.lic.
Check the log of the Vault Agent Init Container:
The following log illustrates a successful run of the Vault Agent Init Container.
Check the log of the HiveMQ container:
The following illustrates a successful use of the HiveMQ license.