In this knowledge base article, we set up Hashicorp Vault and injector service with the Hashicorp Vault Helm chart and store a HiveMQ License as a secret. Then we will deploy the HiveMQ broker cluster with the hivemq-operator Helm chart to demonstrate how the Hashicorp Vault injector service retrieves, decodes and writes the secret to /opt/hivemq/license/hivmq.lic file on the pod for the HiveMQ application to use.
Prerequisites
These instructions require the following tools on the local machine:
...
The kubectl context should be set to the Kubernetes cluster where the HiveMQ broker will be installed.
\uD83D\uDCD8 Instructions
Install the Hashicorp Vault Helm Chart
Add the HashiCorp Helm repository.
Code Block language bash helm repo add hashicorp https://helm.releases.hashicorp.com
Update all the repositories to ensure
helm
is aware of the latest versions.Code Block language bash helm repo update hashicorp
Install the latest version of the Hashicorp Vault server running in development mode.
Development mode: Running a Hashicorp Vault server in development is automatically initialized and unsealed. This is ideal in a learning environment but NOT recommended for a production environment.Code Block language bash helm install vault hashicorp/vault --set "server.dev.enabled=true"
The vault pod and vault Agent Injector pod are deployed in the default namespace.
Display all the pods in the default namespace.
Code Block language bash kubectl get pods
Code Block language text NAME READY STATUS RESTARTS AGE vault-0 1/1 Running 0 80s vault-agent-injector-5945fb98b5-tpglz 1/1 Running 0 80s
The
vault-0
pod runs a vault server in development mode. Thevault-agent-injector
pod performs the injection based on the annotations present or patched on a deployment.Wait until the
vault-0
pod andvault-agent-injector
pod are running and ready (1/1
).
Set a secret in Hashicorp Vault
Copy the hivemq-license file to the
vault-0
pod.Code Block language bash kubectl cp hivemq.lic vault-0:/tmp/
Verify that the file is copied.
Code Block language bash kubectl exec -it vault-0 -- ls /tmp
Code Block language text hivemq.lic
Start an interactive shell session on the
vault-0
pod.Code Block language bash kubectl exec -it vault-0 -- /bin/sh
Your system prompt is replaced with a new prompt
/ $
. Commands issued at this prompt are executed on thevault-0
container.Enable kv-v2 secrets at the path
hivemq
.Code Block language bash vault secrets enable -path=hivemq kv-v2
Code Block language text Success! Enabled the kv-v2 secrets engine at: hivemq/
Create a secret at path
hivemq/myenv/license
with ahivemq_license_b64
key and base64-encoded/tmp/hivemq.lic
file.Code Block language bash cd /tmp vault kv put hivemq/myenv/license hivemq_license_b64="$(base64 -w 0 hivemq.lic)"
Code Block language text ====== Secret Path ====== hivemq/data/myenv/license ======= Metadata ======= Key Value --- ----- created_time 2024-02-21T17:34:39.261249639Z custom_metadata <nil> deletion_time n/a destroyed false version 1
Verify that the secret is defined at the path
hivemq/myenv/license
.Code Block language bash vault kv get hivemq/myenv/license
Code Block language text ====== Secret Path ====== hivemq/data/myenv/license ======= Metadata ======= Key Value --- ----- created_time 2024-02-21T14:57:01.446984026Z custom_metadata <nil> deletion_time n/a destroyed false version 1 ========= Data ========= Key Value --- ----- hivemq_license_b64 SCFNUSRbM10.......
The secret is ready for the application.
Lastly, exit the
vault-0
pod.Code Block language bash exit
Configure Kubernetes authentication
Hashicorp Vault provides a Kubernetes authentication method that enables clients to authenticate with a Kubernetes Service Account Token. This token is provided to each pod when it is created.
Start an interactive shell session on the
vault-0
pod.Code Block language bash $ kubectl exec -it vault-0 -- /bin/sh / $
Your system prompt is replaced with a new prompt
/ $
. Commands issued at this prompt are executed on thevault-0
container.Enable the Kubernetes authentication method.
Code Block language bash $ vault auth enable kubernetes
Code Block language text Success! Enabled kubernetes auth method at: kubernetes/
Hashicorp Vault accepts a service token from any client in the Kubernetes cluster. During authentication, Hashicorp Vault verifies that the service account token is valid by querying a token review Kubernetes endpoint.
Configure the Kubernetes authentication method to use the location of the Kubernetes API.
Note: For the best compatibility with recent Kubernetes versions, ensure you are using Hashicorp Vault v1.13.3 or greater.
Code Block language bash $ vault write auth/kubernetes/config \ kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443"
Successful output from the command resembles this example:
Code Block language text Success! Data written to: auth/kubernetes/config
The environment variable
KUBERNETES_PORT_443_TCP_ADDR
is defined and references the internal network address of the Kubernetes host.For a client to read the secret data defined at
hivemq/myenv/license
, requires that the read capability be granted for the pathhivemq/data/myenv/license
. This is an example of a policy. A policy defines a set of capabilities.Write out the policy named
hivemq
that enables theread
capability for secrets at pathhivemq/data/myenv/license
.Code Block language bash $ vault policy write hivemq - <<EOF path "hivemq/data/myenv/license" { capabilities = ["read"] } EOF
Create a Kubernetes authentication role named
hivemq
.Code Block language bash $ vault write auth/kubernetes/role/hivemq \ bound_service_account_names=hivemq-hivemq-operator-hivemq \ bound_service_account_namespaces=hivemq \ policies=hivemq \ ttl=24h
Successful output from the command resembles this example:
Code Block language text Success! Data written to: auth/kubernetes/role/hivemq
The role connects the Kubernetes service account,
hivemq-hivemq-operator-hivemq
, and namespace,hivemq
, with the Hashicorp Vault policy,hivemq
. The tokens returned after authentication are valid for 24 hours.Lastly, exit the
vault-0
pod.Code Block language bash $ exit
Inject secrets into the pods
If you do not have values.yaml file yet, you can get the latest version from the Helm chart repository and store it as a file, for example,
values-hivemq.yaml
:Code Block language bash helm show values hivemq/hivemq-operator > values-hivemq.yaml
Edit the
values-hivemq.yaml
file. Add annotations to the hivemq pods.Code Block language yaml hivemq: # Annotations to add to the HiveMQ Pods podAnnotations: vault.hashicorp.com/agent-inject: "true" vault.hashicorp.com/role: "hivemq" vault.hashicorp.com/agent-inject-status: 'update' vault.hashicorp.com/agent-inject-secret-hivemq.lic: "hivemq/data/myenv/license" vault.hashicorp.com/secret-volume-path-hivemq.lic: "/opt/hivemq/license/" vault.hashicorp.com/agent-inject-template-hivemq.lic: | {{- with secret \"hivemq/data/myenv/license\" -}} {{- $hivemq_broker_license := base64Decode .Data.data.hivemq_license_b64 -}} {{- $hivemq_broker_license -}} {{- end -}}
(Re)install hivemq
Code Block language bash helm upgrade hivemq --install hivemq/hivemq-operator -n hivemq -f values-hivemq.yaml
Get all the pods in the hivemq namespace.
Code Block language bash $ kubectl get pods --nnamespace hivemq
Code Block language text NAME READY STATUS RESTARTS AGE hivemq-599cb74d9c-s8hhm 0/2 Init:0/1 0 23s hivemq-69697d9598-l878s 1/1 Running 0 20m vault-0 1/1 Running 0 78m vault-agent-injector-5945fb98b5-tpglz 1/1 Running 0 78m
Wait until the re-deployed
hivemq
pod reports that it isRunning
and ready (2/2
).This new pod now launches two containers. The application container, named
hivemq
, and the Hashicorp Vault Agent container, namedvault-agent
.Display the logs of the
vault-agent
container in the newhivemq
pod.Code Block language bash $ kubectl logs \ $(kubectl get pod -l app=hivemq -o jsonpath="{.items[0].metadata.name}") \ --container vault-agent
Hashicorp Vault Agent manages the token lifecycle and the secret retrieval. The secret is rendered in the
hivemq
container at the path/opt/hivemq/license/
.Display the secret written to the
hivemq
container.Code Block language bash $ kubectl exec \ $(kubectl get pod -l app=hivemq -o jsonpath="{.items[0].metadata.name}") \ --container hivemq -- cat /opt/hivemq/license/hivemq.lic
The base64-decoded secret data is present on the container
\uD83D\uDCCB Related articles
...