AWS Secrets Manager
Secrets Manager
A SecretStore points to AWS Secrets Manager in a certain account within a
defined region. You should define Roles that define fine-grained access to
individual secrets and pass them to ESO using spec.provider.aws.role. This
way users of the SecretStore can only access the secrets necessary.
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: aws-secretsmanager
spec:
provider:
aws:
service: SecretsManager
# define a specific role to limit access
# to certain secrets.
# role is a optional field that
# can be omitted for test purposes
role: arn:aws:iam::123456789012:role/external-secrets
region: eu-central-1
auth:
secretRef:
accessKeyIDSecretRef:
name: awssm-secret
key: access-key
secretAccessKeySecretRef:
name: awssm-secret
key: secret-access-key
ClusterSecretStore, Be sure to provide namespace in accessKeyIDSecretRef and secretAccessKeySecretRef with the namespaces where the secrets reside.
NOTE: When using dataFrom without a path defined, the provider will fall back to using ListSecrets. ListSecrets
then proceeds to fetch each individual secret in turn. To use BatchGetSecretValue and avoid excessive API calls define
a path prefix or use Tags filter.
IAM Policy
Create a IAM Policy to pin down access to secrets matching dev-*.
For Batch permissions read the following post https://aws.amazon.com/about-aws/whats-new/2023/11/aws-secrets-manager-batch-retrieval-secrets/.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:GetResourcePolicy",
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:ListSecretVersionIds"
"secretsmanager:BatchGetSecretValue"
],
"Resource": [
"arn:aws:secretsmanager:us-west-2:111122223333:secret:dev-*"
]
}
]
}
Permissions for PushSecret
If you're planning to use PushSecret, ensure you also have the following permissions in your IAM policy:
{
"Effect": "Allow",
"Action": [
"secretsmanager:CreateSecret",
"secretsmanager:PutSecretValue",
"secretsmanager:TagResource",
"secretsmanager:DeleteSecret"
],
"Resource": [
"arn:aws:secretsmanager:us-west-2:111122223333:secret:dev-*"
]
}
Here's a more restrictive version of the IAM policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:CreateSecret",
"secretsmanager:PutSecretValue",
"secretsmanager:TagResource"
],
"Resource": [
"arn:aws:secretsmanager:us-west-2:111122223333:secret:dev-*"
]
},
{
"Effect": "Allow",
"Action": [
"secretsmanager:DeleteSecret"
],
"Resource": [
"arn:aws:secretsmanager:us-west-2:111122223333:secret:dev-*"
],
"Condition": {
"StringEquals": {
"secretsmanager:ResourceTag/managed-by": "external-secrets"
}
}
}
]
}
In this policy, the DeleteSecret action is restricted to secrets that have the specified tag, ensuring that deletion operations are more controlled and in line with the intended management of the secrets.
Additional Settings for PushSecret
Additional settings can be set at the SecretStore level to control the behavior of PushSecret when interacting with AWS Secrets Manager.
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: aws-secretsmanager
spec:
provider:
aws:
service: SecretsManager
role: arn:aws:iam::123456789012:role/external-secrets
region: eu-central-1
secretsManager:
# Additional parameters can be added to the AWS Secrets Manager DeleteSecret API call.
# These parameters are only relevant when the deletionPolicy is set to Delete.
# See: https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_DeleteSecret.html#API_DeleteSecret_RequestSyntax
forceDeleteWithoutRecovery: true
# recoveryWindowInDays: 9 (conflicts with forceDeleteWithoutRecovery)
Additional Metadata for PushSecret
It's possible to configure AWS Secrets Manager to either push secrets in binary format or as plain string.
To control this behaviour set the following provider metadata:
apiVersion: external-secrets.io/v1alpha1
kind: PushSecret
metadata:
name: pushsecret-example # Customisable
namespace: teamb # Same of the SecretStores
spec:
deletionPolicy: Delete
refreshInterval: 1h # Refresh interval for which push secret will reconcile
secretStoreRefs: # A list of secret stores to push secrets to
- name: teamb-secret-store
kind: SecretStore
selector:
secret:
name: my-secret # Source Kubernetes secret to be pushed
data:
- match:
secretKey: key1 # Source Kubernetes secret key to be pushed
remoteRef:
remoteKey: teamb-my-first-parameter-3 # Remote reference (where the secret is going to be pushed)
metadata:
secretPushFormat: string
secretPushFormat takes two options. binary and string, where binary is the default.
JSON Secret Values
SecretsManager supports simple key/value pairs that are stored as json. If you use the API you can store more complex JSON objects. You can access nested values or arrays using gjson syntax:
Consider the following JSON object that is stored in the SecretsManager key friendslist:
{
"name": {"first": "Tom", "last": "Anderson"},
"friends": [
{"first": "Dale", "last": "Murphy"},
{"first": "Roger", "last": "Craig"},
{"first": "Jane", "last": "Murphy"}
]
}
This is an example on how you would look up nested keys in the above json object:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: example
spec:
refreshInterval: 1h
secretStoreRef:
name: aws-secretsmanager
kind: SecretStore
target:
name: friends
creationPolicy: Owner
data:
- secretKey: my_name
remoteRef:
key: friendslist
property: name.first # Tom
- secretKey: first_friend
remoteRef:
key: friendslist
property: friends.1.first # Roger
# metadataPolicy to fetch all the labels in JSON format
- secretKey: tags
remoteRef:
metadataPolicy: Fetch
key: database-credentials
# metadataPolicy to fetch a specific label (dev) from the source secret
- secretKey: developer
remoteRef:
metadataPolicy: Fetch
key: database-credentials
property: dev
Secret Versions
SecretsManager creates a new version of a secret every time it is updated. The secret version can be reference in two ways, the VersionStage and the VersionId. The VersionId is a unique uuid which is generated every time the secret changes. This id is immutable and will always refer to the same secret data. The VersionStage is an alias to a VersionId, and can refer to different secret data as the secret is updated. By default, SecretsManager will add the version stages AWSCURRENT and AWSPREVIOUS to every secret, but other stages can be created via the update-secret-version-stage api.
The version field on the remoteRef of the ExternalSecret will normally consider the version to be a VersionStage, but if the field is prefixed with uuid/, then the version will be considered a VersionId.
So in this example, the operator will request the same secret with different versions: AWSCURRENT and AWSPREVIOUS:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: versioned-api-key
spec:
refreshInterval: 1h
secretStoreRef:
name: aws-secretsmanager
kind: SecretStore
target:
name: versioned-api-key
creationPolicy: Owner
data:
- secretKey: previous-api-key
remoteRef:
key: "production/api-key"
version: "AWSPREVIOUS"
- secretKey: current-api-key
remoteRef:
key: "production/api-key"
version: "AWSCURRENT"
While in this example, the operator will request the secret with VersionId as abcd-1234
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: versioned-api-key
spec:
refreshInterval: 1h
secretStoreRef:
name: aws-secretsmanager
kind: SecretStore
target:
name: versioned-api-key
creationPolicy: Owner
data:
- secretKey: api-key
remoteRef:
key: "production/api-key"
version: "uuid/123e4567-e89b-12d3-a456-426614174000"
AWS Authentication
Controller's Pod Identity
Note: If you are using Parameter Store replace service: SecretsManager with service: ParameterStore in all examples below.
This is basicially a zero-configuration authentication method that inherits the credentials from the runtime environment using the aws sdk default credential chain.
You can attach a role to the pod using IRSA, kiam or kube2iam. When no other authentication method is configured in the Kind=Secretstore this role is used to make all API calls against AWS Secrets Manager or SSM Parameter Store.
Based on the Pod's identity you can do a sts:assumeRole before fetching the secrets to limit access to certain keys in your provider. This is optional.
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: team-b-store
spec:
provider:
aws:
service: SecretsManager
region: eu-central-1
# optional: do a sts:assumeRole before fetching secrets
role: team-b
Access Key ID & Secret Access Key
You can store Access Key ID & Secret Access Key in a Kind=Secret and reference it from a SecretStore.
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: team-b-store
spec:
provider:
aws:
service: SecretsManager
region: eu-central-1
# optional: assume role before fetching secrets
role: team-b
auth:
secretRef:
accessKeyIDSecretRef:
name: awssm-secret
key: access-key
secretAccessKeySecretRef:
name: awssm-secret
key: secret-access-key
NOTE: In case of a ClusterSecretStore, Be sure to provide namespace in accessKeyIDSecretRef, secretAccessKeySecretRef with the namespaces where the secrets reside.
EKS Service Account credentials
This feature lets you use short-lived service account tokens to authenticate with AWS. You must have Service Account Volume Projection enabled - it is by default on EKS. See EKS guide on how to set up IAM roles for service accounts.
The big advantage of this approach is that ESO runs without any credentials.
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/team-a
name: my-serviceaccount
namespace: default
Reference the service account from above in the Secret Store:
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: secretstore-sample
spec:
provider:
aws:
service: SecretsManager
region: eu-central-1
auth:
jwt:
serviceAccountRef:
name: my-serviceaccount
NOTE: In case of a ClusterSecretStore, Be sure to provide namespace for serviceAccountRef with the namespace where the service account resides.
Custom Endpoints
You can define custom AWS endpoints if you want to use regional, vpc or custom endpoints. See List of endpoints for Secrets Manager, Secure Systems Manager and Security Token Service.
Use the following environment variables to point the controller to your custom endpoints. Note: All resources managed by this controller are affected.
| ENV VAR | DESCRIPTION |
|---|---|
| AWS_SECRETSMANAGER_ENDPOINT | Endpoint for the Secrets Manager Service. The controller uses this endpoint to fetch secrets from AWS Secrets Manager. |
| AWS_SSM_ENDPOINT | Endpoint for the AWS Secure Systems Manager. The controller uses this endpoint to fetch secrets from SSM Parameter Store. |
| AWS_STS_ENDPOINT | Endpoint for the Security Token Service. The controller uses this endpoint when creating a session and when doing assumeRole or assumeRoleWithWebIdentity calls. |
