AWS Secrets Manager
Secrets Manager
A SecretStore
points to AWS Secrets Manager in a certain account within a
defined region. You should define Roles that define fine-grained access to
individual secrets and pass them to ESO using spec.provider.aws.role
. This
way users of the SecretStore
can only access the secrets necessary.
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: aws-secretsmanager
spec:
provider:
aws:
service: SecretsManager
# define a specific role to limit access
# to certain secrets.
# role is a optional field that
# can be omitted for test purposes
role: arn:aws:iam::123456789012:role/external-secrets
region: eu-central-1
auth:
secretRef:
accessKeyIDSecretRef:
name: awssm-secret
key: access-key
secretAccessKeySecretRef:
name: awssm-secret
key: secret-access-key
ClusterSecretStore
, Be sure to provide namespace
in accessKeyIDSecretRef
and secretAccessKeySecretRef
with the namespaces where the secrets reside.
IAM Policy
Create a IAM Policy to pin down access to secrets matching dev-*
.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:GetResourcePolicy",
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:ListSecretVersionIds"
],
"Resource": [
"arn:aws:secretsmanager:us-west-2:111122223333:secret:dev-*"
]
}
]
}
JSON Secret Values
SecretsManager supports simple key/value pairs that are stored as json. If you use the API you can store more complex JSON objects. You can access nested values or arrays using gjson syntax:
Consider the following JSON object that is stored in the SecretsManager key friendslist
:
{
"name": {"first": "Tom", "last": "Anderson"},
"friends": [
{"first": "Dale", "last": "Murphy"},
{"first": "Roger", "last": "Craig"},
{"first": "Jane", "last": "Murphy"}
]
}
This is an example on how you would look up nested keys in the above json object:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: example
spec:
refreshInterval: 1m
secretStoreRef:
name: aws-secretsmanager
kind: SecretStore
target:
name: friends
creationPolicy: Owner
data:
- secretKey: my_name
remoteRef:
key: friendslist
property: name.first # Tom
- secretKey: first_friend
remoteRef:
key: friendslist
property: friends.1.first # Roger
# metadataPolicy to fetch all the labels in JSON format
- secretKey: tags
remoteRef:
metadataPolicy: Fetch
key: database-credentials
# metadataPolicy to fetch a specific label (dev) from the source secret
- secretKey: developer
remoteRef:
metadataPolicy: Fetch
key: database-credentials
property: dev
Secret Versions
SecretsManager creates a new version of a secret every time it is updated. The secret version can be reference in two ways, the VersionStage
and the VersionId
. The VersionId
is a unique uuid which is generated every time the secret changes. This id is immutable and will always refer to the same secret data. The VersionStage
is an alias to a VersionId
, and can refer to different secret data as the secret is updated. By default, SecretsManager will add the version stages AWSCURRENT
and AWSPREVIOUS
to every secret, but other stages can be created via the update-secret-version-stage api.
The version
field on the remoteRef
of the ExternalSecret will normally consider the version to be a VersionStage
, but if the field is prefixed with uuid/
, then the version will be considered a VersionId
.
So in this example, the operator will request the same secret with different versions: AWSCURRENT
and AWSPREVIOUS
:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: versioned-api-key
spec:
refreshInterval: 1h
secretStoreRef:
name: aws-secretsmanager
kind: SecretStore
target:
name: versioned-api-key
creationPolicy: Owner
data:
- secretKey: previous-api-key
remoteRef:
key: "production/api-key"
version: "AWSPREVIOUS"
- secretKey: current-api-key
remoteRef:
key: "production/api-key"
version: "AWSCURRENT"
While in this example, the operator will request the secret with VersionId
as abcd-1234
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: versioned-api-key
spec:
refreshInterval: 1h
secretStoreRef:
name: aws-secretsmanager
kind: SecretStore
target:
name: versioned-api-key
creationPolicy: Owner
data:
- secretKey: api-key
remoteRef:
key: "production/api-key"
version: "uuid/123e4567-e89b-12d3-a456-426614174000"
AWS Authentication
Controller's Pod Identity
Note: If you are using Parameter Store replace service: SecretsManager
with service: ParameterStore
in all examples below.
This is basicially a zero-configuration authentication method that inherits the credentials from the runtime environment using the aws sdk default credential chain.
You can attach a role to the pod using IRSA, kiam or kube2iam. When no other authentication method is configured in the Kind=Secretstore
this role is used to make all API calls against AWS Secrets Manager or SSM Parameter Store.
Based on the Pod's identity you can do a sts:assumeRole
before fetching the secrets to limit access to certain keys in your provider. This is optional.
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: team-b-store
spec:
provider:
aws:
service: SecretsManager
region: eu-central-1
# optional: do a sts:assumeRole before fetching secrets
role: team-b
Access Key ID & Secret Access Key
You can store Access Key ID & Secret Access Key in a Kind=Secret
and reference it from a SecretStore.
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: team-b-store
spec:
provider:
aws:
service: SecretsManager
region: eu-central-1
# optional: assume role before fetching secrets
role: team-b
auth:
secretRef:
accessKeyIDSecretRef:
name: awssm-secret
key: access-key
secretAccessKeySecretRef:
name: awssm-secret
key: secret-access-key
NOTE: In case of a ClusterSecretStore
, Be sure to provide namespace
in accessKeyIDSecretRef
, secretAccessKeySecretRef
with the namespaces where the secrets reside.
EKS Service Account credentials
This feature lets you use short-lived service account tokens to authenticate with AWS. You must have Service Account Volume Projection enabled - it is by default on EKS. See EKS guide on how to set up IAM roles for service accounts.
The big advantage of this approach is that ESO runs without any credentials.
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/team-a
name: my-serviceaccount
namespace: default
Reference the service account from above in the Secret Store:
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: secretstore-sample
spec:
provider:
aws:
service: SecretsManager
region: eu-central-1
auth:
jwt:
serviceAccountRef:
name: my-serviceaccount
NOTE: In case of a ClusterSecretStore
, Be sure to provide namespace
for serviceAccountRef
with the namespace where the service account resides.
Custom Endpoints
You can define custom AWS endpoints if you want to use regional, vpc or custom endpoints. See List of endpoints for Secrets Manager, Secure Systems Manager and Security Token Service.
Use the following environment variables to point the controller to your custom endpoints. Note: All resources managed by this controller are affected.
ENV VAR | DESCRIPTION |
---|---|
AWS_SECRETSMANAGER_ENDPOINT | Endpoint for the Secrets Manager Service. The controller uses this endpoint to fetch secrets from AWS Secrets Manager. |
AWS_SSM_ENDPOINT | Endpoint for the AWS Secure Systems Manager. The controller uses this endpoint to fetch secrets from SSM Parameter Store. |
AWS_STS_ENDPOINT | Endpoint for the Security Token Service. The controller uses this endpoint when creating a session and when doing assumeRole or assumeRoleWithWebIdentity calls. |