Google Secret Manager
Google Cloud Secret Manager
External Secrets Operator integrates with GCP Secret Manager for secret management.
Authentication
Workload Identity
Your Google Kubernetes Engine (GKE) applications can consume GCP services like Secrets Manager without using static, long-lived authentication tokens. This is our recommended approach of handling credentials in GCP. ESO offers two options for integrating with GKE workload identity: pod-based workload identity and using service accounts directly. Before using either way you need to create a service account - this is covered below.
Creating Workload Identity Service Accounts
You can find the documentation for Workload Identity here. We will walk you through how to navigate it here.
Search the document for this editable values and change them to your values:
Note: If you have installed ESO, a serviceaccount has already been created. You can either patch the existing external-secrets
SA or create a new one that fits your needs.
CLUSTER_NAME
: The name of your clusterPROJECT_ID
: Your project ID (not your Project number nor your Project name)K8S_NAMESPACE
: For us following these steps here it will bees
, but this will be the namespace where you deployed the external-secrets operatorKSA_NAME
: external-secrets (if you are not creating a new one to attach to the deployment)GSA_NAME
: external-secrets for simplicity, or something else if you have to follow different naming convetions for cloud resourcesROLE_NAME
: should beroles/secretmanager.secretAccessor
- so you make the pod only be able to access secrets on Secret Manager
Using Service Accounts directly
Let's assume you have created a service account correctly and attached a appropriate workload identity. It should roughly look like this:
apiVersion: v1
kind: ServiceAccount
metadata:
name: external-secrets
namespace: es
annotations:
iam.gke.io/gcp-service-account: example-team-a@my-project.iam.gserviceaccount.com
You can reference this particular ServiceAccount in a SecretStore
or ClusterSecretStore
. It's important that you also set the projectID
, clusterLocation
and clusterName
. The Namespace on the serviceAccountRef
is ignored when using a SecretStore
resource. This is needed to isolate the namespaces properly.
apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
name: gcp-store
spec:
provider:
gcpsm:
projectID: alphabet-123
auth:
workloadIdentity:
# name of the cluster region
clusterLocation: europe-central2
# name of the GKE cluster
clusterName: alpha-cluster-42
# projectID of the cluster (if omitted defaults to spec.provider.gcpsm.projectID)
clusterProjectID: my-cluster-project
# reference the sa from above
serviceAccountRef:
name: team-a
namespace: team-a
You need to give the Google service account the roles/iam.serviceAccountTokenCreator
role so it can generate a service account token for you (not necessary in the Pod-based Workload Identity bellow)
Using Pod-based Workload Identity
You can attach a Workload Identity directly to the ESO pod. ESO then has access to all the APIs defined in the attached service account policy. You attach the workload identity by (1) creating a service account with a attached workload identity (described above) and (2) using this particular service account in the pod's serviceAccountName
field.
For this example we will assume that you installed ESO using helm and that you named the chart installation external-secrets
and the namespace where it lives es
like:
helm install external-secrets external-secrets/external-secrets --namespace es
Then most of the resources would have this name, the important one here being the k8s service account attached to the external-secrets operator deployment:
# ...
containers:
- image: ghcr.io/external-secrets/external-secrets:vVERSION
name: external-secrets
ports:
- containerPort: 8080
protocol: TCP
restartPolicy: Always
schedulerName: default-scheduler
serviceAccount: external-secrets
serviceAccountName: external-secrets # <--- here
The pod now has the identity. Now you need to configure the SecretStore
.
You just need to set the projectID
, all other fields can be omitted.
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: gcp-store
spec:
provider:
gcpsm:
projectID: alphabet-123
GCP Service Account authentication
You can use GCP Service Account to authenticate with GCP. These are static, long-lived credentials. A GCP Service Account is a JSON file that needs to be stored in a Kind=Secret
. ESO will use that Secret to authenticate with GCP. See here how you manage GCP Service Accounts.
After creating a GCP Service acount go to IAM & Admin
web UI, click ADD ANOTHER ROLE
button, add Secret Manager Secret Accessor
role to this service account.
The Secret Manager Secret Accessor
role is required to access secrets.
apiVersion: v1
kind: Secret
metadata:
name: gcpsm-secret
labels:
type: gcpsm
type: Opaque
stringData:
secret-access-credentials: |-
{
"type": "service_account",
"project_id": "external-secrets-operator",
"private_key_id": "",
"private_key": "-----BEGIN PRIVATE KEY-----\nA key\n-----END PRIVATE KEY-----\n",
"client_email": "test-service-account@external-secrets-operator.iam.gserviceaccount.com",
"client_id": "client ID",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/test-service-account%40external-secrets-operator.iam.gserviceaccount.com"
}
Update secret store
Be sure the gcpsm
provider is listed in the Kind=SecretStore
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: gcp-store
spec:
provider:
gcpsm: # gcpsm provider
auth:
secretRef:
secretAccessKeySecretRef:
name: gcpsm-secret # secret name containing SA key
key: secret-access-credentials # key name containing SA key
projectID: alphabet-123 # name of Google Cloud project
NOTE: In case of a ClusterSecretStore
, Be sure to provide namespace
for SecretAccessKeyRef
with the namespace of the secret that we just created.
Creating external secret
To create a kubernetes secret from the GCP Secret Manager secret a Kind=ExternalSecret
is needed.
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: database-credentials
spec:
refreshInterval: 1h # rate SecretManager pulls GCPSM
secretStoreRef:
kind: SecretStore
name: gcp-store # name of the SecretStore (or kind specified)
target:
name: database-credentials # name of the k8s Secret to be created
creationPolicy: Owner
data:
- secretKey: database_username
remoteRef:
key: database_username # name of the GCPSM secret key
- secretKey: database_password
remoteRef:
key: database_password # name of the GCPSM secret key
The operator will fetch the GCP Secret Manager secret and inject it as a Kind=Secret
kubectl get secret secret-to-be-created -n <namespace> | -o jsonpath='{.data.dev-secret-test}' | base64 -d