How to use External Secrets Operator to sync secrets from AWS Secrets Manager

What if you use AWS Secrets Manager and would like to synchronize the secrets into OpenShift? This is exactly what we will look at today by using an External Secrets Operator (ESO). Here, we will be using the OpenShift manifest files that can be found in my GitHub repo at https://github.com/wengkee/external-secrets-operator-demo

AWS Secrets Manager

Next, you will need to create a programmatic access user in the AWS cloud service. Using this, generate the access key and its secret access key (yes… the naming is a mouthful). Moving on, we will need to create a custom policy (you may refer to the aws-policy.json file) and assign it to this user.

Almost there, now we just need to create a dummy secret in AWS Secrets Manager.

Cool, we have our first secret in the AWS Secrets Manager, let’s test it out in the AWS CLI. First, create an AWS profile

$ aws configure --profile sample-profile
 AWS Access Key ID [None]: <YOUR-ACCESS-KEY>
 AWS Secret Access Key [None]: <YOUR-SECRET-ACCESS-KEY>
 Default region name [None]: <YOUR-REGION>
 Default output format [None]: yaml

$ export AWS_PROFILE=sample-profile

Here, let’s try getting the secret that we created earlier.

$ aws secretsmanager get-secret-value --secret-id wk-first-aws-secret
 ARN: arn:aws:secretsmanager:<YOUR-ARN>
 CreatedDate: '2023-07-06T20:54:21.074000+08:00'
 Name: wk-first-aws-secret
 SecretString: '{"my-key":"my-value"}'
 VersionId: a2004df5-b147-476d-973c-6f512492bdf5
 VersionStages:
 - AWSCURRENT

Spin up the ESO in your OpenShift cluster

This is fairly simple, you can install it through the Operator Hub

or apply it by applying the subscription.yaml file.

 oc apply -f subscription.yaml

Next, we will also need to apply the OperatorConfig file.

oc apply -f operator-config.yaml

See the magic in OpenShift

Essentially, the ESO will create the OpenShift Secret based on ExternalSecret which will pull the actual secrets from the AWS Secrets Manager. And of course, ESO will need the credentials to pull from AWS, which is why we will also need SecretStore.

Now, let’s create a dummy project called external-secrets

oc new-project external-secrets

Here, we need to create a secret that holds the access key and secret access key of your programmatic user. Use the YAML and create it in OpenShift.

apiVersion: v1
kind: Secret
metadata:
  name: aws-secret
  namespace: external-secrets
stringData:
  access-key : YOUR-ACCESS-KEY
  secret-access-key: YOUR-SECRET-ACCESS-KEY
type: Opaque

Next, create the SecretStore that will hold the reference to the secret that we just created.

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: secret-store
  namespace: external-secrets
spec:
  provider:
    aws:
      auth:
        secretRef:
          accessKeyIDSecretRef:
            key: access-key
            name: aws-secret
          secretAccessKeySecretRef:
            key: secret-access-key
            name: aws-secret
      region: <YOUR-REGION>
      service: SecretsManager
  retrySettings:
    maxRetries: 3
    retryInterval: '10s'
  refreshInterval: 3600

Lastly, create ExternalSecret, almost there!

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: external-secret
  namespace: external-secrets
spec:
  refreshInterval: 1m
  secretStoreRef:
    name: secret-store-1
    kind: SecretStore
  target:
    name: wk-first-aws-secret-actual # DEST - name of the k8s secret that will be mapped to
    creationPolicy: Owner
  data:
  - secretKey: my-key # DEST - name of the key in the k8s secret that will be mapped to
    remoteRef:
      key: wk-first-aws-secret # SOURCE - name of the aws secret
      property: my-key # SOURCE - key of the key-value pair inside the aws secret

Voila, you should see thatwk-first-aws-secret-actual is now created in OpenShift and synchronized with the wk-first-aws-secret AWS Secrets Manager.

Thanks for reading, happy coding!

References:

https://github.com/wengkee/external-secrets-operator-demo