Skip to content

Key Vault

aws sm

Azure Key vault

External Secrets Operator integrates with Azure Key vault for secrets, certificates and Keys management.

Authentication

We support Service Principals, Managed Identity and Workload Identity authentication.

To use Managed Identity authentication, you should use aad-pod-identity to assign the identity to external-secrets operator. To add the selector to external-secrets operator, use podLabels in your values.yaml in case of Helm installation of external-secrets.

We support connecting to different cloud flavours azure supports: PublicCloud, USGovernmentCloud, ChinaCloud and GermanCloud. You have to specify the environmentType and point to the correct cloud flavour. This defaults to PublicCloud.

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: azure-backend
spec:
  provider:
    azurekv:
      # PublicCloud, USGovernmentCloud, ChinaCloud, GermanCloud
      environmentType: PublicCloud # default

Minimum required permissions are Get over secret and certificate permissions. This can be done by adding a Key Vault access policy:

KUBELET_IDENTITY_OBJECT_ID=$(az aks show --resource-group <AKS_CLUSTER_RG_NAME> --name <AKS_CLUSTER_NAME> --query 'identityProfile.kubeletidentity.objectId' -o tsv)
az keyvault set-policy --name kv-name-with-certs --object-id "$KUBELET_IDENTITY_OBJECT_ID" --certificate-permissions get --secret-permissions get

Service Principal key authentication

A service Principal client and Secret is created and the JSON keyfile is stored in a Kind=Secret. The ClientID and ClientSecret should be configured for the secret. This service principal should have proper access rights to the keyvault to be managed by the operator

Managed Identity authentication

A Managed Identity should be created in Azure, and that Identity should have proper rights to the keyvault to be managed by the operator.

If there are multiple Managed Identitites for different keyvaults, the operator should have been assigned all identities via aad-pod-identity, then the SecretStore configuration should include the Id of the idenetity to be used via the identityId field.

apiVersion: v1
kind: Secret
metadata:
  name: azure-secret-sp
type: Opaque
data:
  ClientID: bXktc2VydmljZS1wcmluY2lwbGUtY2xpZW50LWlkCg==  #service-principal-ID
  ClientSecret: bXktc2VydmljZS1wcmluY2lwbGUtY2xpZW50LXNlY3JldAo= #service-principal-secret

Workload Identity

You can use Azure AD Workload Identity Federation to access Azure managed services like Key Vault without needing to manage secrets. You need to configure a trust relationship between your Kubernetes Cluster and Azure AD. This can be done in various ways, for instance using terraform, the Azure Portal or the az cli. We found the azwi cli very helpful. The Azure Workload Identity Quick Start Guide is also good place to get started.

This is basically a two step process:

  1. Create a Kubernetes Service Account (guide)

azwi serviceaccount create phase sa \
  --aad-application-name "${APPLICATION_NAME}" \
  --service-account-namespace "${SERVICE_ACCOUNT_NAMESPACE}" \
  --service-account-name "${SERVICE_ACCOUNT_NAME}"
2. Configure the trust relationship between Azure AD and Kubernetes (guide)

azwi serviceaccount create phase federated-identity \
  --aad-application-name "${APPLICATION_NAME}" \
  --service-account-namespace "${SERVICE_ACCOUNT_NAMESPACE}" \
  --service-account-name "${SERVICE_ACCOUNT_NAME}" \
  --service-account-issuer-url "${SERVICE_ACCOUNT_ISSUER}"

With these prerequisites met you can configure ESO to use that Service Account. You have two options:

Mounted Service Account

You run the controller and mount that particular service account into the pod. That grants everyone who is able to create a secret store or reference a correctly configured one the ability to read secrets. This approach is usually not recommended. But may make sense when you want to share an identity with multiple namespaces. Also see our Multi-Tenancy Guide for design considerations.

apiVersion: v1
kind: ServiceAccount
metadata:
  # this service account was created by azwi
  name: workload-identity-sa
  annotations:
    azure.workload.identity/client-id: 7d8cdf74-xxxx-xxxx-xxxx-274d963d358b
    azure.workload.identity/tenant-id: 5a02a20e-xxxx-xxxx-xxxx-0ad5b634c5d8
---
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: example-secret-store
spec:
  provider:
    azurekv:
      authType: WorkloadIdentity
      vaultUrl: "https://xx-xxxx-xx.vault.azure.net"
      # note: no serviceAccountRef was provided
Referenced Service Account

You run the controller without service account (effectively without azure permissions). Now you have to configure the SecretStore and set the serviceAccountRef and point to the service account you have just created. This is usually the recommended approach. It makes sense for everyone who wants to run the controller withour Azure permissions and delegate authentication via service accounts in particular namespaces. Also see our [Multi-Tenancy Guide] for design considerations.

apiVersion: v1
kind: ServiceAccount
metadata:
  # this service account was created by azwi
  name: workload-identity-sa
  annotations:
    azure.workload.identity/client-id: 7d8cdf74-xxxx-xxxx-xxxx-274d963d358b
    azure.workload.identity/tenant-id: 5a02a20e-xxxx-xxxx-xxxx-0ad5b634c5d8
---
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: example-secret-store
spec:
  provider:
    azurekv:
      authType: WorkloadIdentity
      vaultUrl: "https://xx-xxxx-xx.vault.azure.net"
      serviceAccountRef:
        name: workload-identity-sa

Update secret store

Be sure the azurekv provider is listed in the Kind=SecretStore

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: example-secret-store
spec:
  provider:
    # provider type: azure keyvault
    azurekv:
      # azure tenant ID, see: https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/active-directory-how-to-find-tenant
      tenantId: "d3bc2180-xxxx-xxxx-xxxx-154105743342"
      # URL of your vault instance, see: https://docs.microsoft.com/en-us/azure/key-vault/general/about-keys-secrets-certificates
      vaultUrl: "https://my-keyvault-name.vault.azure.net"
      authSecretRef:
        # points to the secret that contains
        # the azure service principal credentials
        clientId:
          name: azure-secret-sp
          key: ClientID
        clientSecret:
          name: azure-secret-sp
          key: ClientSecret
NOTE: In case of a ClusterSecretStore, Be sure to provide namespace in clientId and clientSecret with the namespaces where the secrets reside.

Or in case of Managed Idenetity authentication:

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: example-secret-store
spec:
  provider:
    # provider type: azure keyvault
    azurekv:
      authType: ManagedIdentity
      # Optionally set the Id of the Managed Identity, if multiple identities are assigned to external-secrets operator
      identityId: "<MI_clientId>"
      # URL of your vault instance, see: https://docs.microsoft.com/en-us/azure/key-vault/general/about-keys-secrets-certificates
      vaultUrl: "https://my-keyvault-name.vault.azure.net"

Object Types

Azure KeyVault manages different object types, we support keys, secrets and certificates. Simply prefix the key with key, secret or cert to retrieve the desired type (defaults to secret).

Object Type Return Value
secret the raw secret value.
key A JWK which contains the public key. Azure KeyVault does not export the private key. You may want to use template functions to transform this JWK into PEM encoded PKIX ASN.1 DER format.
certificate The raw CER contents of the x509 certificate. You may want to use template functions to transform this into your desired encoding

Creating external secret

To create a kubernetes secret from the Azure Key vault secret a Kind=ExternalSecret is needed.

You can manage keys/secrets/certificates saved inside the keyvault , by setting a "/" prefixed type in the secret name, the default type is a secret. Other supported values are cert and key.

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: example-external-secret
spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: SecretStore
    name: example-secret-store

  target:
    name: secret-to-be-created
    creationPolicy: Owner

  data:
  # name of the SECRET in the Azure KV (no prefix is by default a SECRET)
  - secretKey: dev-secret-test
    remoteRef:
      key: dev-secret-test

  # explicit type and name of secret in the Azure KV
  - secretKey: dev-another-secret-test
    remoteRef:
      key: secret/dev-secret-test

  # metadataPolicy to fetch all the tags in JSON format
  - secretKey: dev-secret-test
    remoteRef:
      key: dev-secret-test
      metadataPolicy: Fetch

  # metadataPolicy to fetch a specific tag which name must be in property
  - secretKey: dev-secret-test
    remoteRef:
      key: dev-secret-test
      metadataPolicy: Fetch
      property: tagname

  # type/name of certificate in the Azure KV
  # raw value will be returned, use templating features for data processing
  - secretKey: dev-cert-test
    remoteRef:
      key: cert/dev-cert-test

  # type/name of the public key in the Azure KV
  # the key is returned PEM encoded
  - secretKey: dev-key-test
    remoteRef:
      key: key/dev-key-test

The operator will fetch the Azure Key vault secret and inject it as a Kind=Secret. Then the Kubernetes secret can be fetched by issuing:

kubectl get secret secret-to-be-created -n <namespace> | -o jsonpath='{.data.dev-secret-test}' | base64 -d

To select all secrets inside the key vault or all tags inside a secret, you can use the dataFrom directive:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: example
spec:
  refreshInterval: 1h           # rate SecretManager pulls Azure Key Vault
  secretStoreRef:
    kind: SecretStore
    name: example               # name of the SecretStore (or kind specified)
  target:
    name: secret-to-be-created  # name of the k8s Secret to be created
    creationPolicy: Owner
  dataFrom:
  - find:
      name:
        regexp: "^example"
  - find:
      tags:
        author: seb
        environment: dev
  # secret value is in JSON format and we unmarshall it into multiple key/values in k8s secret
  - extract: 
      key: test
  # get all tags and the tags in JSON format will be unmarshall 
  - extract: 
      key: test
      metadataPolicy: Fetch

To get a PKCS#12 certificate from Azure Key Vault and inject it as a Kind=Secret of type kubernetes.io/tls:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: mycert
spec:
  refreshInterval: 24h
  secretStoreRef:
    kind: ClusterSecretStore
    name: kv-mycert
  target:
    template:
      type: kubernetes.io/tls
      engineVersion: v2
      data:
        tls.crt: "{{ .mycert | b64dec | pkcs12cert }}"
        tls.key: "{{ .mycert | b64dec | pkcs12key }}"
  data:
  - secretKey: mycert
    remoteRef:
      # Azure Key Vault certificates must be fetched as secret/cert-name
      key: secret/mycert