Managing secrets on Azure
It may be necessary to change or recreate secrets stored in AKS. Typical use cases are as follows:
-
A password change in PostgreSQL
-
Accidental removal of secret values
Warning
Changing or deleting the values of AKS secrets may break kdb Insights Enterprise on Azure. If you are not sure about the correct values and/or secrets to set, please reach out to KX Support team for assistance.
Pre-requisites
To change or modify secrets on AKS, a command line tool like kubectl must be configured to communicate with the cluster. Before you make the modifications you will need the following:
-
Access to the Azure Portal
-
Access to the Kubernetes Cluster Overview
-
Access to Cloud Shell or Azure CLI
-
Kubectl CLI
Saving secrets
Saving secrets
Depending on your company policy and needs, you may wish to save the different secret values to another secret store in case of corruption or accidental deletion.
Listing secrets
List all secrets across all namespaces:
kubectl get secrets --all-namespaces
List secrets for a particular namespace:
kubectl get secrets --namespace "insights"
Decoding a secret
Secrets need to be formatted (in this example with JSONPath) and decoded, as they are base64 encoded. The namespace where the current secret resides also needs to be defined, or we can iterate over all namespaces with -A (short for --all-namespaces).
kubectl get secrets/kxi-acr-pull-secret --namespace "insights" --output="jsonpath={.data.\.dockerconfigjson}" | base64 --decode
This will output the content of the secret in a decoded and readable format.
To save the secret in base64 encoded format, just omit the decode command after the pipe to have the output still encoded:
kubectl get secrets/kxi-acr-pull-secret --namespace "insights" --output="jsonpath={.data.\.dockerconfigjson}"
Editing secrets
Since the kxi-acr-pull-secret mentioned as an example earlier is a Dockercfg secret that is used to authenticate against a Docker registry, we need to tell kubectl it is of type "docker-registry".
Changing ACR URL
kxi-acr-pull-secret's --docker-server attribute will change which ACR is used for pulling images. Currently changing the ACR to any other than what was the original URL at the time of deployment is not supported. Especially rook-ceph is affected negatively by this change.
kubectl edit --namespace "insights" secret docker-registry kxi-acr-pull-secret \
--docker-username=<ACR_USERNAME> \
--docker-password=<ACR_PASSWORD> \
--docker-server=<ACR_ADDRESS> \
--dry-run=client -o yaml | kubectl apply -f -
If the secret needs to be changed across multiple namespaces, it is possible to iterate over the command, for example:
kubectl get secret -A | grep kxi-acr-pull-secret | sed 's/ .*//' | while read -r ns ; do
kubectl edit --namespace "${ns}" secret docker-registry kxi-acr-pull-secret \
--docker-username=<ACR_USERNAME> \
--docker-password=<ACR_PASSWORD> \
--docker-server=<ACR_ADDRESS> \
--dry-run=client -o yaml | kubectl apply -f -
done
Creating a secret
This section is applicable if the secret has been removed by accident.
To create a new secret, the namespace and type of secret needs to be defined.
kubectl create --namespace "insights" secret generic my-secret \
--from-literal=username=username \
--from-literal=password=topsecret
To recreate the above kxi-acr-pull-secret in the "insights" namespace:
kubectl create --namespace "insights" secret docker-registry kxi-acr-pull-secret \
--docker-username=<ACR_USERNAME> \
--docker-password=<ACR_PASSWORD> \
--docker-server=<ACR_ADDRESS> \
--dry-run=client -o yaml | kubectl apply -f -
Warning
As there usually isn't a reason to delete a secret on the kdb Insights Enterprise AKS, we have intentionally left the command out of the examples written here. KX advises using caution and recommends that you should be careful when deleting or modifying a secret.