Fetching Certificates from Azure Key Vault and Creating Kubernetes Secrets

Prerequisites

Azure Subscription and Permissions
- You must have the necessary permissions to remove role assignments at the root scope or management group level. Typically, this requires a high-level role like Owner, Key Vault Administrator, or equivalent permissions.
Install Azure cli
- Run az login to authenticate with your Azure account.
Install OpenSSL for Linux or windows

Intro

I have a scenario where I want to enable HTTPS communication between API Management (APIM) and an AKS cluster. There are several ways to set this up, but I decided to use Istio (you could also achieve this with the NGINX add-on), as it is offered as an add-on for AKS. This setup requires a Kubernetes secret that contains the certificate and private key used for securing HTTPS communication.

Below, I have a Bash script that uses Azure CLI to fetch the certificate and private key from Azure Key Vault. The script will also encrypt the private key using OpenSSL, and based on the output, it will create Kubernetes secrets in your AKS cluster. If you want to automate this further, you can integrate the process into your Azure Pipeline, as I will show below.

Step 1: Run Locally

First, run the script below locally to ensure that it works as expected. Also, make sure that you have the correct permissions on Azure and AKS.

# Set the environment
ENV=$1

K8S_SECRET_NAME=${2:- "istio-credentials"}  # Use 'istio-credentials' as default if no argument is provided

# Define variables for Key Vault access and Kubernetes integration
CERT_NAME=""
KV_NAME=""
K8S_NAMESPACE="aks-istio-ingress"
SUBSCRIPTION_NAME=""

# Set the subscription
az account set --subscription $SUBSCRIPTION_NAME

echo "[*] Create a certificate temporary folder"
FOLDER_NAME="cert_temp"
mkdir $FOLDER_NAME

echo "[*] Download certificate from Key Vault"
az keyvault certificate download \
    --vault-name "$KV_NAME" \
    --name "$CERT_NAME" \
    --file $FOLDER_NAME/tls.crt

echo "[*] Download private key from Key Vault"
az keyvault secret download \
    --vault-name "$KV_NAME" \
    --name "$CERT_NAME" \
    --file $FOLDER_NAME/encrypted-tls.key \
    --encoding base64

echo "[*] Decrypting the private key (removing passphrase)"
openssl rsa -in $FOLDER_NAME/encrypted-tls.key -out $FOLDER_NAME/tls.key

# *** This code snippet will be commented because it will be handled by the Azure pipeline. ***
echo "[*] Create secret $K8S_SECRET_NAME on cluster"
kubectl create secret tls $K8S_SECRET_NAME \
   --cert=$FOLDER_NAME/tls.crt \
   --key=$FOLDER_NAME/tls.key \
   --namespace $K8S_NAMESPACE

echo "[*] Remove temporary folder"
rm -rf $FOLDER_NAME

# ****************************************************************

You can trigger it through your terminal with the command:

bash create-k8s-secrets-from-keyvault-local.sh <ENV> <K8S_SECRET_NAME>

Do you want the whole code? Click on the GitHub link below.

Step 2: Run it in an Azure pipeline


# Deploy to Azure Kubernetes Service
# Build and push image to Azure Container Registry; Deploy to Azure Kubernetes Service
# https://docs.microsoft.com/azure/devops/pipelines/languages/docker

parameters:
  - name: environment
    type: string
    values:
      - test
      - prod
    default: test
  - name: secretName
    type: string
    default: istio-credentials

trigger:
- none

variables:
  agent: <AGENT_POOL_NAME>
  armServiceConnection: <ARM_SERVICE_CONNECTION>
  buildName: update-k8s-secrets-$(Build.BuildId)
  folderName: cert_temp
  kubernetesServiceConnection: <KUBERNETES_SERVICE_CONNECTION>
  workingDirectoryForCertFolder: $(Build.SourcesDirectory)/aks-secret-automation

name: ${{ variables.buildName }}

pool: $(agent)

stages:
- stage: Deploy
  displayName: Deploy ${{ parameters.environment }}
  jobs:
  - deployment: Deploy
    displayName: Deploy ${{ parameters.environment }}
    environment: ${{ parameters.environment }}
    strategy:
      runOnce:
        deploy:
          steps:
          - checkout: self
          - task: AzureCLI@2
            inputs:
              azureSubscription: '${{ variables.armServiceConnection }}'
              scriptType: 'bash'
              scriptLocation: 'inlineScript'
              inlineScript: 'bash create-k8s-secrets-from-keyvault.sh  ${{ parameters.environment }} ${{ parameters.secretName }}'
              workingDirectory: '${{ variables.workingDirectoryForCertFolder }}'
            displayName: Fetch secret from Key vault
          - task: Kubernetes@1
            displayName: Create secrets
            inputs:
              connectionType: "Kubernetes Service Connection"
              kubernetesServiceEndpoint: ${{ variables.kubernetesServiceConnection }}
              namespace: aks-istio-ingress
              secretType: generic
              secretArguments: --from-file=tls.key=${{ variables.folderName }}/tls.key --from-file=tls.crt=${{ variables.folderName }}/tls.crt
              secretName: ${{ parameters.secretName }}
              workingDirectory: '${{ variables.workingDirectoryForCertFolder }}'
          - script: rm -rf ${{ variables.folderName }}
            displayName: Remove folder ${{ variables.folderName }}

💡 Extra Bonus Tip!

You could also skip the Kubernetes@1 task with the secretType: generic that creates the Kubernetes secret and just use the Bash script in your pipeline. However, then you will need to modify the script slightly because you cannot run kubectl create secret tls twice. If you try, you'll get an error message saying, "the secret is already created", so you may want to use kubectl apply instead.

If you manage to solve the issue, then you need to use the Kubernetes@1 task to log into the cluster with your Kubernetes service connection in your Azure DevOps project and simply run the Bash script.

- task: Kubernetes@1
  # This is needed to run kubectl command in script below.
  displayName: 'Kubernetes Login'
  inputs:
    connectionType: 'Kubernetes Service Connection'
    kubernetesServiceEndpoint: <KUBERNETES_SERVICE_CONNECTION>
    command: 'login'
- script: bash $(Pipeline.Workspace)/create-k8s-secrets-from-keyvault-local.sh
  continueOnError: false
  displayName: Run bash script

I hope this guide helps you out! If you have any questions, don’t hesitate to reach out.

Alexander Arana.E

References

GitHub Link

Kubernetes@1

kubectl create secret generic