Author : MD TAREQ HASSAN | Updated : 2021/05/20
Prerequisites
- Provisioning AKS Cluster
- Pod identity
- Azure KeyVault
- Test secret (name: ‘test secret’, value: ‘My test keyvault secret’)
KeyVault Access Permission
- User managed identity (i.e. MyManagedIdentity) will be used to create pod identity
- Give MyManagedIdentity access permission to KeyVault (read, list)
Give KeyVault access permission in Azure portal or using Azure CLI
$Env:ResourceGroup = "MyResourceGroup"
$Env:ManagedIdentityName = "MyManagedIdentity"
$Env:ManagedIdentityClientId = $(az identity show `
--resource-group $Env:ResourceGroup `
--name $Env:ManagedIdentityName `
--query clientId -o tsv)
#
# Assign keyvault access to MyManagedIdentity
#
az keyvault set-policy -n $Env:ResourceGroup --key-permissions get list --spn $Env:ManagedIdentityClientId
az keyvault set-policy -n $Env:ResourceGroup --secret-permissions get list --spn $Env:ManagedIdentityClientId
az keyvault set-policy -n $Env:ResourceGroup --certificate-permissions get list --spn $Env:ManagedIdentityClientId
Using Managed Pod Identity
Implement KeyVault access code in your app.
See: ASP .Net Core - Implementing Azure KeyVault Access
Get ‘selector’ (i.e. MyPodIdentity) of MyManagedIdentity-binding
kubectl describe "azureidentitybinding.aadpodidentity.k8s.io/MyPodIdentity-binding" -n test
Use that ‘selector’ (MyPodIdentity) in Pod deployment manifest: deployment-demo.yaml
apiVersion: v1
kind: Service
metadata:
name: demo-api
labels:
app: demo-api
spec:
selector:
app: demo-api
type: LoadBalancer
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo-api
spec:
replicas: 2
selector:
matchLabels:
app: demo-api
template:
metadata:
labels:
app: demo-api
aadpodidbinding: my-pod-identity # 'selector' of MyPodIdentity-binding
spec:
containers:
- name: demo-api
image: myacr.azurecr.io/demoapi:latest
ports:
- name: http
containerPort: 80
protocol: TCP
Deploy: kubectl apply -f deployment-demo.yaml -n test
Using AKS-AzureKeyvaultSecretsProvider Add-on
aks-preview
extension
az extension add --name aks-preview
az extension update --name aks-preview
Register features AzureKeyvaultSecretsProvider
az feature register --namespace "Microsoft.ContainerService" --name AKS-AzureKeyvaultSecretsProvider
#
# Check registered (may take time, wait and execute again)
#
az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/AKS-AzureKeyvaultSecretsProvider')].{Name:name,State:properties.state}"
Need to re-register the ContainerService provider to enable the new feature
az provider register --namespace Microsoft.ContainerService
Enable add-on
- Will install required things (i.e. driver, provider etc.)
- Will create ‘User Managed Identity’ in AKS managed resource group (AKS managed resource group ->
MC_x_y_z_...
)
az aks enable-addons --resource-group clusterresourcegroup --name clustername --addons azure-keyvault-secrets-provider
Give AKS Managed Identity Access To KeyVault
- When AKS cluster is provisioned, a managed identity is created
- Azure portal > AKS > Cluster configuration > AKS-managed Azure Active Directory: Enabled
- Normally called “[clustername]-agentpool”
- About “[clustername]-agentpool”
- It’s a managed identity (user managed identity)
- It is assigned to the AKS clsuter VMs
- Allow managed identity of AKS clsuter to access KeyVault
- Azure portal > Resource group ‘MC_resourcegroupname_clustername_region’ (this resource group was created automatically when AKS was provisioned)
- Find managed identity ‘clustername-agentpool’ (was cerated automatically when AKS was provisioned)
- Go to KeyVault (you resource group) > Settings: Access policies > Add access policy
- Select principle > Search ‘clustername-agentpool’
- Add > Save
SecretProviderClass
- SecretProviderClass object will live in the cluster
- This object will let the cluster know about the key vault and what identity to use to get the information from key vault
- SecretProviderClass will use managed identity (“[clustername]-agentpool”) of AKS clsuter VMs and therefore
useVMManagedIdentity: "true"
will be used
SecretProviderClass.yaml
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
name: demo-keyvault-user-msi
spec:
provider: azure
parameters:
usePodIdentity: "false"
useVMManagedIdentity: "true"
userAssignedIdentityID: 71f71709-a424-4de0-bf42-xxxxxxxxx
keyvaultName: "demo-keyvault"
cloudName: "" # [OPTIONAL for Azure] if not provided, azure environment will default to AzurePublicCloud
objects: |
array:
- |
objectName: test-secret
objectType: secret # object types: secret, key or cert
objectVersion: "" # [OPTIONAL] object versions, default to latest if empty
tenantId: da6d0388-8495-4722-b411-xxxxxxxxxx # the tenant ID (directory id) of the KeyVault
Deployment: demo-deploymnet.yaml
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
name: demo-keyvault-user-msi
spec:
provider: azure
parameters:
usePodIdentity: "false"
useVMManagedIdentity: "true"
userAssignedIdentityID: 71f71709-a424-4de0-bf42-xxxxxxxxx
keyvaultName: "demo-keyvault"
cloudName: "" # [OPTIONAL for Azure] if not provided, azure environment will default to AzurePublicCloud
objects: |
array:
- |
objectName: test-secret
objectType: secret # object types: secret, key or cert
objectVersion: "" # [OPTIONAL] object versions, default to latest if empty
tenantId: da6d0388-8495-4722-b411-xxxxxxxxxx # the tenant ID (directory id) of the KeyVault
---
kind: Pod
apiVersion: v1
metadata:
name: busybox-secrets-store-inline-user-msi
spec:
containers:
- name: busybox
image: k8s.gcr.io/e2e-test-images/busybox:1.29
command:
- "/bin/sleep"
- "10000"
volumeMounts:
- name: secrets-store01-inline
mountPath: "/mnt/secrets-store"
readOnly: true
volumes:
- name: secrets-store01-inline
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "demo-keyvault-user-msi"
Courtesies:
- https://pumpingco.de/blog/use-azure-keyvault-with-asp-net-core-running-in-an-aks-cluster-using-aad-pod-identity/
- https://pixelrobots.co.uk/2021/04/azure-key-vault-access-on-azure-kubernetes-service-using-the-new-aks-add-on/
- https://medium.com/dzerolabs/kubernetes-saved-today-f-cked-tomorrow-a-rant-azure-key-vault-secrets-%C3%A0-la-kubernetes-fc3be5e65d18
Links
- https://pixelrobots.co.uk/2021/04/azure-key-vault-access-on-azure-kubernetes-service-using-the-new-aks-add-on/
- https://blog.nillsf.com/index.php/2021/01/05/trying-out-the-preview-of-azure-active-directory-pod-managed-identities-in-azure-kubernetes-service/
- https://docs.microsoft.com/en-us/azure/aks/use-azure-ad-pod-identity