Secrets Automation

1Password Secrets Automation

External Secrets Operator integrates with 1Password Secrets Automation for secret management.

Important note about this documentation

The 1Password API calls the entries in vaults 'Items'. These docs use the same term.

Behavior

How an Item is equated to an ExternalSecret:
- remoteRef.key is equated to an Item's Title
- remoteRef.property is equated to:
  - An Item's field's Label (Password type)
  - An Item's file's Name (Document type)
  - If empty, defaults to the first file name, or the field labeled password
- remoteRef.version is currently not supported.
- One Item in a vault can equate to one Kubernetes Secret to keep things easy to comprehend.
Support for 1Password secret types of Password and Document.
- The Password type can get data from multiple fields in the Item.
- The Document type can get data from files.
- See creating 1Password Items compatible with ExternalSecrets.
Ordered vaults
- Specify an ordered list of vaults in a SecretStore and the value will be sourced from the first vault with a matching Item.
- If no matching Item is found, an error is returned.
- This supports having a default or shared set of values that can also be overriden for specific environments.
dataFrom:
- find.path is equated to Item Title.
- find.name.regexp is equated to field Labels.
- find.tags are not supported at this time.

Prerequisites

1Password requires running a 1Password Connect Server to which the API requests will be made.
- External Secrets does not run this server. See Deploy a Connect Server.
- One Connect Server is needed per 1Password Automation Environment.
- Many Vaults can be added to an Automation Environment, and Tokens can be generated in that Environment with access to any set or subset of those Vaults.
1Password Connect Server version 1.5.6 or higher.

Setup Authentication

Authentication requires a 1password-credentials.json file provided to the Connect Server, and a related 'Access Token' for the client in this provider to authenticate to that Connect Server. Both of these are generated by 1Password.

Setup an Automation Environment at 1Password.com, or via the op CLI.
- Note: don't be confused by the op connect server create syntax. This will create an Automation Environment in 1Password, and corresponding credentials for a Connect Server, nothing more.
- This will result in a 1password-credentials.json file to provide to a Connect Server Deployment, and an Access Token to provide as a Secret referenced by a SecretStore or ClusterSecretStore.

Create a Kubernetes secret with the Access Token

---
apiVersion: v1
kind: Secret
metadata:
  name: onepassword-connect-token-staging
type: Opaque
stringData:
  token: my-token

Reference the secret in a SecretStore or ClusterSecretStore

---
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: staging
spec:
  provider:
    onepassword:
      connectHost: https://onepassword-connect-staging
      vaults:
        staging: 1  # look in this vault first
        shared: 2   # next look in here. error if not found
      auth:
        secretRef:
        connectTokenSecretRef:
          name: onepassword-connect-token-staging
          key: token

Create a Kubernetes secret with the Connect Server credentials

---
apiVersion: v1
kind: Secret
metadata:
  name: connect-server-credentials
type: Opaque
stringData:
  # NOTE: This secret value must be base64 encoded after it becomes the OP_SESSION env var in the Connect Server Deployment, that means double base64 encoded here. (Or single w/ stringData.)
  1password-credentials.json: |-
    eyJ2ZXJpZmllciI6eyJzYWx0IjoiZXhhbXBsZSIsImxvY2FsSGFzaCI6ImV4YW1wbGUifSwiZW5jQ3JlZGVudGlhbHMiOnsia2lkIjoiZXhhbXBsZSIsImVuYyI6ImV4YW1wbGUiLCJjdHkiOiJleGFtcGxlIiwiaXYiOiJleGFtcGxlIiwiZGF0YSI6ImV4YW1wbGUifSwidmVyc2lvbiI6IjIiLCJkZXZpY2VVdWlkIjoiZXhhbXBsZSIsInVuaXF1ZUtleSI6eyJhbGciOiJleGFtcGxlIiwiZXh0Ijp0cnVlLCJrIjoiZXhhbXBsZSIsImtleV9vcHMiOlsiZW5jcnlwdCIsImRlY3J5cHQiXSwia3R5Ijoib2N0Iiwia2lkIjoiZXhhbXBsZSJ9fQ==

Reference the secret in a Connect Server Deployment

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: onepassword-connect-staging
spec:
  template:
    spec:
      containers:
      - name: connect-api
        image: 1password/connect-api:1.5.0
        env:
        - name: OP_SESSION
          valueFrom:
            secretKeyRef:
              name: connect-server-credentials
              key: 1password-credentials.json
        ...
      - name: connect-sync
        image: 1password/connect-sync:1.5.0
        env:
        - name: OP_SESSION
          valueFrom:
            secretKeyRef:
              name: connect-server-credentials
              key: 1password-credentials.json
        ...
      ...

Deploy a Connect Server

Follow the remaining instructions in the Quick Start guide.
- Deploy at minimum a Deployment and Service for a Connect Server, to go along with the Secret for the Server created in the Setup Authentication section.
The Service's name will be referenced in SecretStores/ClusterSecretStores.
Keep in mind the likely need for additional Connect Servers for other Automation Environments when naming objects. For example dev, staging, prod, etc.
Unencrypted secret values are passed over the connection between the Operator and the Connect Server. Encrypting the connection is recommended.

Creating Compatible 1Password Items

Also see examples below for matching SecretStore and ExternalSecret specs.

Manually (Password type)

Click the plus button to create a new Password type Item.
Change the title to what you want remoteRef.key to be.
Set what you want remoteRef.property to be in the field sections where is says 'label', and values where it says 'new field'.
Click the 'Save' button.

create-password-screenshot

Manually (Document type)

Click the plus button to create a new Document type Item.
Choose the file to upload and upload it.
Change the title to match remoteRef.key
Click the 'Add New File' button to add more files.
Click the 'Save' button.

create-document-screenshot

Scripting (Password type with op CLI)

Create file.json with the following contents, swapping in your keys and values. Note: section.name's and section.title's values are ignored by the Operator, but cannot be empty for the op CLI

{
  "sections": [
    {
      "fields": [
        {
          "k": "concealed",
          "n": "MY_ENV_VAR1",
          "t": "MY_ENV_VAR1",
          "v": "value1"
        },
        {
          "k": "concealed",
          "n": "MY_ENV_VAR2",
          "t": "MY_ENV_VAR2",
          "v": "value2"
        }
      ],
      "name": "EXTERNAL-SECRETS",
      "title": "EXTERNAL-SECRETS"
    }
  ]
}

Run op create item password --template file.json --vault my-vault --title my-item

Scripting (Document type)

Unfortunately the op CLI doesn't seem to support uploading multiple files to the same Item, and the current Go lib has a bug. op can be used to create a Document type Item with one file in it, but for now it's necessary to add multiple files to the same Document via the GUI.

In-built field labeled `password` on Password type Items

TL;DR if you need a field labeled password, use the in-built one rather than the one in a fields Section.

password-field-example

1Password automatically adds a field labeled password on every Password type Item, whether it's created through a GUI or the API or op CLI.
There's no problem with using this field just like any other field, just make sure you don't end up with two fields with the same label. (For example, by automating the op CLI to create Items.)
The in-built password field is not otherwise special for the purposes of ExternalSecrets. It can be ignored when not in use.

Examples

Examples of using the my-env-config and my-cert Items seen above.

Note: with this configuration a 1Password Item titled my-env-config is correlated to a ExternalSecret named my-env-config that results in a Kubernetes secret named my-env-config, all with matching names for the key/value pairs. This is a way to increase comprehensibility.

---
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: staging
spec:
  provider:
    onepassword:
      connectHost: https://onepassword-connect-staging
      vaults:
        staging: 1  # look in this vault first
        shared: 2   # next look in here. error if not found
      auth:
        secretRef:
        connectTokenSecretRef:
          name: onepassword-connect-token-staging
          key: token

---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: my-env-config
spec:
  secretStoreRef:
    kind: SecretStore
    name: staging
  target:
    creationPolicy: Owner
  data:
  - secretKey: MY_ENV_VAR1
    remoteRef:
      key: my-env-config
      property: MY_ENV_VAR1
  - secretKey: MY_ENV_VAR2
    remoteRef:
      key: my-env-config
      property: MY_ENV_VAR2
  # OR
  dataFrom:
  - extract:
      key: my-env-config
      property: MY_ENV_VAR1  # optional field Label to match exactly
  # OR
  - find:
      path: my-env-config  # optional Item Title to match exactly
      name:
        regexp: "^MY_ENV_VAR.*"

---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: my-cert
spec:
  secretStoreRef:
    kind: SecretStore
    name: staging
  target:
    creationPolicy: Owner
  data:
  - secretKey: cert.crt
    remoteRef:
      key: my-cert
      property: cert.crt
  - secretKey: cert.key
    remoteRef:
      key: my-cert
      property: cert.key
  # OR
  dataFrom:
  - extract:
      key: my-cert
      property: cert.key  # optional field Label to match exactly
  # OR
  - find:
      path: my-cert  # optional Item Title to match exactly
      name:
        regexp: "^cert.*"

Additional Notes

General

It's intuative to use Document type Items for Kubernetes secrets mounted as files, and Password type Items for ones that will be mounted as environment variables, but either can be used for either. It comes down to what's more convenient.

Why no version history

1Password only supports version history on their in-built password field. Therefore, implementing version history in this provider would require one Item in 1Password per remoteRef in an ExternalSecret. Additionally remoteRef.property would be pointless/unusable.
For example, a Kubernetes secret with 15 keys (say, used in envFrom,) would require 15 Items in the 1Password vault, instead of 15 Fields in 1 Item. This would quickly get untenable for more than a few secrets, because:
- All Items would have to have unique names which means secretKey couldn't match the Item name the remoteRef is targeting.
- Maintenance, particularly clean up of no longer used secrets, would be significantly more work.
- A vault would often become a huge list of unorganized entries as opposed to a much smaller list organized by Kubernetes Secret.
To support new and old versions of a secret value at the same time, create a new Item in 1Password with the new value, and point some ExternalSecrets at a time to the new Item.

Keeping misconfiguration from working

One instance of the ExternalSecrets Operator can work with many Connect Server instances, but it may not be the best approach.
With one Operator instance per Connect Server instance, namespaces and RBAC can be used to improve security posture, and perhaps just as importantly, it's harder to misconfigure something and have it work (supply env A's secret values to env B for example.)
You can run as many 1Password Connect Servers as you need security boundaries to help protect against accidental misconfiguration.

Patching ExternalSecrets with Kustomize

An overlay can provide a SecretStore specific to that overlay, and then use JSON6902 to patch all the ExternalSecrets coming from base to point to that SecretStore. Here's an example overlays/staging/kustomization.yaml:

---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../../base/something-with-external-secrets
- secretStore.staging.yaml

patchesJson6902:
- target:
    kind: ExternalSecret
    name: ".*"
  patch: |-
    - op: replace
      path: /spec/secretStoreRef/name
      value: staging