Advanced Templating v2
With External Secrets Operator you can transform the data from the external secret provider before it is stored as Kind=Secret
. You can do this with the Spec.Target.Template
. Each data value is interpreted as a golang template.
Examples
You can use templates to inject your secrets into a configuration file that you mount into your pod:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: template
spec:
# ...
target:
name: secret-to-be-created
# this is how the Kind=Secret will look like
template:
engineVersion: v2
data:
# multiline string
config: |
datasources:
- name: Graphite
type: graphite
access: proxy
url: http://localhost:8080
password: "{{ .password }}"
user: "{{ .user }}"
# using replace function to rewrite secret
connection: '{{ .dburl | replace "postgres://" "postgresql://" }}'
data:
- secretKey: user
remoteRef:
key: /grafana/user
- secretKey: password
remoteRef:
key: /grafana/password
- secretKey: dburl
remoteRef:
key: /database/url
Another example with two keys in the same secret:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: template
spec:
# ...
target:
template:
engineVersion: v2
data:
name: admin
password: "{{ .mysecret }}" # If you are using plain manifests or gitops tools
# password: '{{ printf "{{ .mysecret }}" }}' # If you are using templated tools like helm
data:
- secretKey: mysecret
remoteRef:
key: /credentials
TemplateFrom
You do not have to define your templates inline in an ExternalSecret but you can pull ConfigMaps
or other Secrets that contain a template. Consider the following example:
# define your template in a config map
apiVersion: v1
kind: ConfigMap
metadata:
name: grafana-config-tpl
data:
config.yaml: |
datasources:
- name: Graphite
type: graphite
access: proxy
url: "{{ .uri }}"
password: "{{ .password }}"
user: "{{ .user }}"
templated: |
# key and value templated
my-application-{{ .user}}: {{ .password | b64enc }}
# conditional keys
{{- if hasPrefix "oci://" .uri }}
enableOCI: true
{{- else }}
enableOCI: false
{{- end }}
# Fixed values
application-type: grafana
annotations: |
#dynamic timestamp generation
last-synced-for-user/{{ .user }}: {{ now }}
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: my-template-example
spec:
# ...
target:
name: secret-to-be-created
template:
engineVersion: v2
templateFrom:
- target: Data
configMap:
# name of the configmap to pull in
name: grafana-config-tpl
# here you define the keys that should be used as template
items:
- key: config.yaml
templateAs: Values
- key: generated
templateAs: KeysAndValues
- target: Annotations
configMap:
# name of the configmap to pull in
name: grafana-config-tpl
# here you define the keys that should be used as template
items:
- key: annotations
templateAs: KeysAndValues
data:
- secretKey: user
remoteRef:
key: /grafana/user
- secretKey: password
remoteRef:
key: /grafana/password
- secretKey: uri
remoteRef:
key: /grafana/uri
TemplateFrom
also gives you the ability to Target your template to the Secret's Annotations, Labels or the Data block. It also allows you to render the templated information as Values
or as KeysAndValues
through the templateAs
configuration:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: my-template-example
spec:
# ...
target:
name: secret-to-be-created
template:
engineVersion: v2
templateFrom:
- target: Annotations
literal: "last-sync-for-user/{{ .user }}: {{ .now }}"
data:
- secretKey: user
remoteRef:
key: /grafana/user
- secretKey: password
remoteRef:
key: /grafana/password
Lastly, TemplateFrom
also supports adding Literal
blocks for quick templating. These Literal
blocks differ from Template.Data
as they are rendered as a a key:value
pair (while the Template.Data
, you can only template the value).
Extract Keys and Certificates from PKCS#12 Archive
You can use pre-defined functions to extract data from your secrets. Here: extract keys and certificates from a PKCS#12 archive and store it as PEM.
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: template
spec:
# ...
target:
template:
type: kubernetes.io/tls
engineVersion: v2
data:
tls.crt: "{{ .mysecret | pkcs12cert }}"
tls.key: "{{ .mysecret | pkcs12key }}"
# if needed unlock the pkcs12 with the password
tls.crt: "{{ .mysecret | pkcs12certPass "my-password" }}"
Extract from JWK
You can extract the public or private key parts of a JWK and use them as PKCS#8 private key or PEM-encoded PKIX public key.
A JWK looks similar to this:
{
"kty": "RSA",
"kid": "cc34c0a0-bd5a-4a3c-a50d-a2a7db7643df",
"use": "sig",
"n": "pjdss...",
"e": "AQAB"
// ...
}
And what you want may be a PEM-encoded public or private key portion of it. Take a look at this example on how to transform it into the desired format:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: template
spec:
# ...
target:
template:
engineVersion: v2
data:
# .myjwk is a json-encoded JWK string.
#
# this template will produce for jwk_pub a PEM encoded public key:
# -----BEGIN PUBLIC KEY-----
# MIIBI...
# ...
# ...AQAB
# -----END PUBLIC KEY-----
jwk_pub: "{{ .myjwk | jwkPublicKeyPem }}"
# private key is a pem-encoded PKCS#8 private key
jwk_priv: "{{ .myjwk | jwkPrivateKeyPem }}"
Filter PEM blocks
Consider you have a secret that contains both a certificate and a private key encoded in PEM format and it is your goal to use only the certificate from that secret.
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCvxGZOW4IXvGlh
. . .
m8JCpbJXDfSSVxKHgK1Siw4K6pnTsIA2e/Z+Ha2fvtocERjq7VQMAJFaIZSTKo9Q
JwwY+vj0yxWjyzHUzZB33tg=
-----END PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
MIIDMDCCAhigAwIBAgIQabPaXuZCQaCg+eQAVptGGDANBgkqhkiG9w0BAQsFADAV
. . .
NtFUGA95RGN9s+pl6XY0YARPHf5O76ErC1OZtDTR5RdyQfcM+94gYZsexsXl0aQO
9YD3Wg==
-----END CERTIFICATE-----
You can achieve that by using the filterPEM
function to extract a specific type of PEM block from that secret. If multiple blocks of that type (here: CERTIFICATE
) exist then all of them are returned in the order they are specified.
Helper functions
Info
Note: we removed env
and expandenv
from sprig functions for security reasons.
We provide a couple of convenience functions that help you transform your secrets. This is useful when dealing with PKCS#12 archives or JSON Web Keys (JWK).
In addition to that you can use over 200+ sprig functions. If you feel a function is missing or might be valuable feel free to open an issue and submit a pull request.
Function | Description |
---|---|
pkcs12key | Extracts all private keys from a PKCS#12 archive and encodes them in PKCS#8 PEM format. |
pkcs12keyPass | Same as pkcs12key . Uses the provided password to decrypt the PKCS#12 archive. |
pkcs12cert | Extracts all certificates from a PKCS#12 archive and orders them if possible. If disjunct or multiple leaf certs are provided they are returned as-is. Sort order: leaf / intermediate(s) / root . |
pkcs12certPass | Same as pkcs12cert . Uses the provided password to decrypt the PKCS#12 archive. |
filterPEM | Filters PEM blocks with a specific type from a list of PEM blocks. |
jwkPublicKeyPem | Takes an json-serialized JWK and returns an PEM block of type PUBLIC KEY that contains the public key. See here for details. |
jwkPrivateKeyPem | Takes an json-serialized JWK as string and returns an PEM block of type PRIVATE KEY that contains the private key in PKCS #8 format. See here for details. |
toYaml | Takes an interface, marshals it to yaml. It returns a string, even on marshal error (empty string). |
fromYaml | Function converts a YAML document into a map[string]interface{}. |
Migrating from v1
If you are still using v1alpha1
, You have to opt-in to use the new engine version by specifying template.engineVersion=v2
:
apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
name: secret
spec:
# ...
target:
template:
engineVersion: v2
# ...
The biggest change was that basically all function parameter types were changed from accepting/returning []byte
to string
. This is relevant for you because now you don't need to specify toString
all the time at the end of a template pipeline.
apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
# ...
spec:
target:
template:
engineVersion: v2
data:
# this used to be {{ .foobar | toString }}
egg: "new: {{ .foobar }}"
Functions removed/replaced
base64encode
was renamed tob64enc
.base64decode
was renamed tob64dec
. Any errors that occurr during decoding are silenced.fromJSON
was renamed tofromJson
. Any errors that occurr during unmarshalling are silenced.toJSON
was renamed totoJson
. Any errors that occurr during marshalling are silenced.pkcs12key
andpkcs12keyPass
encode the PKCS#8 key directly into PEM format. There is no need to callpemPrivateKey
anymore. Also, these functions do extract all private keys from the PKCS#12 archive not just the first one.pkcs12cert
andpkcs12certPass
encode the certs directly into PEM format. There is no need to callpemCertificate
anymore. These functions now extract all certificates from the PKCS#12 archive not just the first one.toString
implementation was replaced by thesprig
implementation and should be api-compatible.toBytes
was removed.pemPrivateKey
was removed. It's now implemented within thepkcs12*
functions.pemCertificate
was removed. It's now implemented within thepkcs12*
functions.