Advanced Templating v2

With External Secrets Operator you can transform the data from the external secret provider before it is stored as Kind=Secret. You can do this with the Spec.Target.Template. Each data value is interpreted as a golang template.

Note

Consider using camelcase when defining .'spec.data.secretkey', example: serviceAccountToken

If your secret keys contain - (dashes), you will need to reference them using index
Example: \{\{ index .data "service-account-token" \}\}

Helm

When installing ExternalSecrets via helm, the template must be escaped so that helm will not try to render it. The most straightforward way to accomplish this would be to use backticks (raw string constants):

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: template
spec:
  # ...
  target:
    template:
      engineVersion: v2
      data:
        name: admin
        # password: "{{ .mysecret }}"               # If you are using plain manifests or gitops tools
        password: "{{ `{{ .mysecret }}` }}"         # If you are using helm
  data:
  - secretKey: mysecret
    remoteRef:
      key: /credentials

Examples

You can use templates to inject your secrets into a configuration file that you mount into your pod:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: template
spec:
  # ...
  target:
    name: secret-to-be-created
    # this is how the Kind=Secret will look like
    template:
      engineVersion: v2
      data:
        # multiline string
        config: |
          datasources:
          - name: Graphite
            type: graphite
            access: proxy
            url: http://localhost:8080
            password: "{{ .password }}"
            user: "{{ .user }}"
        # using replace function to rewrite secret
        connection: '{{ .dburl | replace "postgres://" "postgresql://" }}'

  data:
  - secretKey: user
    remoteRef:
      key: /grafana/user
  - secretKey: password
    remoteRef:
      key: /grafana/password
  - secretKey: dburl
    remoteRef:
      key: /database/url

Another example with two keys in the same secret:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: template
spec:
  # ...
  target:
    template:
      engineVersion: v2
      data:
        name: admin
        password: "{{ .mysecret }}"                   # If you are using plain manifests or gitops tools
        # password: "{{ `{{ .mysecret }}` }}"         # If you are using templated tools like helm
  data:
  - secretKey: mysecret
    remoteRef:
      key: /credentials

MergePolicy

By default, the templating mechanism will not use any information available from the original data and dataFrom queries to the provider, and only keep the templated information. It is possible to change this behavior through the use of the mergePolicy field. mergePolicy currently accepts two values: Replace (the default) and Merge. When using Merge, data and dataFrom keys will also be embedded into the templated secret, having lower priority than the template outcome. See the example for more information:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: template
spec:
  # ...
  target:
    template:
      mergePolicy: Merge
      engineVersion: v2
      data:
        name: admin
        password: "{{ .password | b64dec }}" # Overwrites the password from the data call and use this output
  data:
  - secretKey: password
    remoteRef:
      key: /credentials/password
  - secretKey: username # Preserves the username in the templated Secret
    remoteRef:
      key: /credentials/username

TemplateFrom

You do not have to define your templates inline in an ExternalSecret but you can pull ConfigMaps or other Secrets that contain a template. Consider the following example:

# define your template in a config map
apiVersion: v1
kind: ConfigMap
metadata:
  name: grafana-config-tpl
data:
  config.yaml: |
    datasources:
      - name: Graphite
        type: graphite
        access: proxy
        url: "{{ .uri }}"
        password: "{{ .password }}"
        user: "{{ .user }}"
  templated: |
     # key and value templated
     my-application-{{ .user}}: {{ .password | b64enc }}
     # conditional keys
     {{- if hasPrefix "oci://" .uri }}
     enableOCI: true
     {{- else }}
     enableOCI: false
     {{- end }}
     # Fixed values
     application-type: grafana
  annotations: |
     #dynamic timestamp generation
     last-synced-for-user/{{ .user }}: {{ now }}
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: my-template-example
spec:
  # ...
  target:
    name: secret-to-be-created
    template:
      engineVersion: v2
      templateFrom:
      - target: Data
        configMap:
          # name of the configmap to pull in
          name: grafana-config-tpl
          # here you define the keys that should be used as template
          items:
          - key: config.yaml
            templateAs: Values
          - key: templated
            templateAs: KeysAndValues
      - target: Annotations
        configMap:
          # name of the configmap to pull in
          name: grafana-config-tpl
          # here you define the keys that should be used as template
          items:
          - key: annotations
            templateAs: KeysAndValues
  data:
  - secretKey: user
    remoteRef:
      key: /grafana/user
  - secretKey: password
    remoteRef:
      key: /grafana/password
  - secretKey: uri
    remoteRef:
      key: /grafana/uri

TemplateFrom also gives you the ability to Target your template to the Secret's Annotations, Labels or the Data block. It also allows you to render the templated information as Values or as KeysAndValues through the templateAs configuration:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: my-template-example
spec:
  # ...
  target:
    name: secret-to-be-created
    template:
      engineVersion: v2
      templateFrom:
      - target: Annotations
        literal: "last-sync-for-user/{{ .user }}: {{ .now }}"
  data:
  - secretKey: user
    remoteRef:
      key: /grafana/user
  - secretKey: password
    remoteRef:
      key: /grafana/password

Lastly, TemplateFrom also supports adding Literal blocks for quick templating. These Literal blocks differ from Template.Data as they are rendered as a a key:value pair (while the Template.Data, you can only template the value).

See an example, how to produce a htpasswd file that can be used by an ingress-controller (for example: https://kubernetes.github.io/ingress-nginx/examples/auth/basic/) where the contents of the htpasswd file needs to be presented via the auth key. We use the htpasswd function to create a bcrytped hash of the password.

Suppose you have multiple key-value pairs within your provider secret like

{
  "user1": "password1",
  "user2": "password2",
  ...
}

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: my-template-example
spec:
  # ...
  target:
    name: secret-to-be-created
    template:
      engineVersion: v2
      templateFrom:
      - target: Data
        literal: |-
          {{- $creds := list }}
          {{- range $user, $pw := . }}
            {{- $creds = append $creds (printf "%s" (htpasswd $user $pw)) }}
          {{- end }}
          auth: {{ $creds | join "\n" | quote }}
  dataFrom:
  - extract:
      key: /ingress-controller/valid-users

Extract Keys and Certificates from PKCS#12 Archive

You can use pre-defined functions to extract data from your secrets. Here: extract keys and certificates from a PKCS#12 archive and store it as PEM.

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: template
spec:
  # ...
  target:
    template:
      type: kubernetes.io/tls
      engineVersion: v2
      data:
        tls.crt: "{{ .mysecret | pkcs12cert }}"
        tls.key: "{{ .mysecret | pkcs12key }}"

        # if needed unlock the pkcs12 with the password
        tls.crt: "{{ .mysecret | pkcs12certPass "my-password" }}"

Extract from JWK

You can extract the public or private key parts of a JWK and use them as PKCS#8 private key or PEM-encoded PKIX public key.

A JWK looks similar to this:

{
  "kty": "RSA",
  "kid": "cc34c0a0-bd5a-4a3c-a50d-a2a7db7643df",
  "use": "sig",
  "n": "pjdss...",
  "e": "AQAB"
  // ...
}

And what you want may be a PEM-encoded public or private key portion of it. Take a look at this example on how to transform it into the desired format:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: template
spec:
  # ...
  target:
    template:
      engineVersion: v2
      data:
        # .myjwk is a json-encoded JWK string.
        #
        # this template will produce for jwk_pub a PEM encoded public key:
        # -----BEGIN PUBLIC KEY-----
        # MIIBI...
        # ...
        # ...AQAB
        # -----END PUBLIC KEY-----
        jwk_pub: "{{ .myjwk | jwkPublicKeyPem }}"
        # private key is a pem-encoded PKCS#8 private key
        jwk_priv: "{{ .myjwk | jwkPrivateKeyPem }}"

Filter PEM blocks

Consider you have a secret that contains both a certificate and a private key encoded in PEM format and it is your goal to use only the certificate from that secret.

-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCvxGZOW4IXvGlh
 . . .
m8JCpbJXDfSSVxKHgK1Siw4K6pnTsIA2e/Z+Ha2fvtocERjq7VQMAJFaIZSTKo9Q
JwwY+vj0yxWjyzHUzZB33tg=
-----END PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
MIIDMDCCAhigAwIBAgIQabPaXuZCQaCg+eQAVptGGDANBgkqhkiG9w0BAQsFADAV
 . . .
NtFUGA95RGN9s+pl6XY0YARPHf5O76ErC1OZtDTR5RdyQfcM+94gYZsexsXl0aQO
9YD3Wg==
-----END CERTIFICATE-----

You can achieve that by using the filterPEM function to extract a specific type of PEM block from that secret. If multiple blocks of that type (here: CERTIFICATE) exist, all of them are returned in the order specified. To extract a specific type of PEM block, pass the type as a string argument to the filterPEM function. Take a look at this example of how to transform a secret which contains a private key and a certificate into the desired format:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: template
spec:
  # ...
  target:
    template:
      type: kubernetes.io/tls
      engineVersion: v2
      data:
        tls.crt: "{{ .mysecret | filterPEM "CERTIFICATE" }}"
        tls.key: "{{ .mysecret | filterPEM "PRIVATE KEY" }}"

Templating with PushSecret

PushSecret templating is much like ExternalSecrets templating. In-fact under the hood, it's using the same data structure. Which means, anything described in the above should be possible with push secret as well resulting in a templated secret created at the provider.

apiVersion: external-secrets.io/v1beta1
kind: PushSecret
metadata:
  name: template
spec:
  # ...
  template:
    engineVersion: v2
    data:
      token: "{{ .token | toString | upper }} was templated"
  data:
    - match:
        secretKey: token
        remoteRef:
          remoteKey: create-secret-name
          property: token

Helper functions

Info

Note: we removed env and expandenv from sprig functions for security reasons.

We provide a couple of convenience functions that help you transform your secrets. This is useful when dealing with PKCS#12 archives or JSON Web Keys (JWK).

In addition to that you can use over 200+ sprig functions. If you feel a function is missing or might be valuable feel free to open an issue and submit a pull request.

Function	Description
pkcs12key	Extracts all private keys from a PKCS#12 archive and encodes them in PKCS#8 PEM format.
pkcs12keyPass	Same as pkcs12key. Uses the provided password to decrypt the PKCS#12 archive.
pkcs12cert	Extracts all certificates from a PKCS#12 archive and orders them if possible. If disjunct or multiple leaf certs are provided they are returned as-is. Sort order: leaf / intermediate(s) / root.
pkcs12certPass	Same as pkcs12cert. Uses the provided password to decrypt the PKCS#12 archive.
pemToPkcs12	Takes a PEM encoded certificate and key and creates a base64 enoded PKCS#12 archive.
pemToPkcs12Pass	Same as pemToPkcs12. Uses the provided password to encrypt the PKCS#12 archive.
filterPEM	Filters PEM blocks with a specific type from a list of PEM blocks.
jwkPublicKeyPem	Takes an json-serialized JWK and returns an PEM block of type PUBLIC KEY that contains the public key. See here for details.
jwkPrivateKeyPem	Takes an json-serialized JWK as string and returns an PEM block of type PRIVATE KEY that contains the private key in PKCS #8 format. See here for details.
toYaml	Takes an interface, marshals it to yaml. It returns a string, even on marshal error (empty string).
fromYaml	Function converts a YAML document into a map[string]any.

Migrating from v1

If you are still using v1alpha1, You have to opt-in to use the new engine version by specifying template.engineVersion=v2:

apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
  name: secret
spec:
  # ...
  target:
    template:
      engineVersion: v2
  # ...

The biggest change was that basically all function parameter types were changed from accepting/returning []byte to string. This is relevant for you because now you don't need to specify toString all the time at the end of a template pipeline.

apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
# ...
spec:
  target:
    template:
      engineVersion: v2
      data:
        # this used to be {{ .foobar | toString }}
        egg: "new: {{ .foobar }}"

Functions removed/replaced

• base64encode was renamed to b64enc.
• base64decode was renamed to b64dec. Any errors that occur during decoding are silenced.
• fromJSON was renamed to fromJson. Any errors that occur during unmarshalling are silenced.
• toJSON was renamed to toJson. Any errors that occur during marshalling are silenced.
• pkcs12key and pkcs12keyPass encode the PKCS#8 key directly into PEM format. There is no need to call pemPrivateKey anymore. Also, these functions do extract all private keys from the PKCS#12 archive not just the first one.
• pkcs12cert and pkcs12certPass encode the certs directly into PEM format. There is no need to call pemCertificate anymore. These functions now extract all certificates from the PKCS#12 archive not just the first one.
• toString implementation was replaced by the sprig implementation and should be api-compatible.
• toBytes was removed.
• pemPrivateKey was removed. It's now implemented within the pkcs12* functions.
• pemCertificate was removed. It's now implemented within the pkcs12* functions.