{"id":1009,"date":"2024-11-18T04:15:00","date_gmt":"2024-11-18T04:15:00","guid":{"rendered":"https:\/\/blog.ngocha.biz\/?p=1009"},"modified":"2024-11-18T04:15:00","modified_gmt":"2024-11-18T04:15:00","slug":"kubernetes-external-secrets-operator","status":"publish","type":"post","link":"https:\/\/blog.ngocha.biz\/?p=1009","title":{"rendered":"External Secrets Operator Setup for EKS using Secrets Manager"},"content":{"rendered":"

In this blog we will look at Kubernetes External secrets operator setup on AWS EKS<\/a> and integrate with AWS secrets manager for fetching secrets.<\/p>\n

External Secrets Operator is an operator for Kubernetes<\/a> that manages Kubernetes Secrets on external secrets managers like AWS Secrets Manager, Google Secrets Manager, Azure Key Vault, etc.<\/p>\n

External Secrets Operator Workflow<\/h2>\n

Given below is the diagrammatic workflow of the External Secrets Operator, which gets secrets from the AWS Secrets Manager and saves them as a secret in the EKS cluster.<\/p>\n

\"External<\/figure>\n

Here is how it works.<\/p>\n

    \n
  1. An External Secrets Object<\/strong> is created in a namespace you want to use the secret.<\/li>\n
  2. The External Secrets Object<\/strong> contains information such as the refresh interval, the secret store to refer to, the target where the secret needs to be stored, and the AWS Secrets Manager name and secret key.<\/li>\n
  3. The SecretStore<\/strong> contains the authentication details for the External Secrets Managers<\/strong>, which helps the External Secrets Operator<\/strong> to access the secret.<\/li>\n
  4. External Secret Operator<\/strong> fetches the secrets stored in AWS Secrets Manager<\/strong> and saves them as Kubernetes Secrets<\/strong> on the EKS cluster.<\/li>\n
  5. The Kubernetes Secrets<\/strong> created by the External Secrets Operator<\/strong> get the required secrets from the AWS Secrets Manager<\/strong> and refresh the Secrets<\/strong> with a time period to keep it up to date.<\/li>\n
  6. After the Kubernetes Secret<\/strong> is created you can use the secrets in the application by specifying the Secret name<\/strong> and Secret key<\/strong> in which the secret values are mapped.<\/li>\n<\/ol>\n

    Prerequisites<\/h2>\n

    The prerequisites are given below.<\/p>\n

      \n
    1. AWS CLI<\/a> with access to IAM and EKS<\/li>\n
    2. EKS cluster<\/a> with Amazon EKS Pod Identity Agent addon<\/li>\n
    3. Helm<\/a><\/li>\n
    4. kubectl<\/li>\n
    5. eksctl<\/li>\n<\/ol>\n

      Steps to Setup External Secrets Operator on EKS<\/h2>\n

      If you are ready with the prerequisites, follow the steps below to set up an External Secrets Operator on EKS.<\/p>\n

      In this setup, we will get the secrets on AWS Secrets Manager and save them as a Kubernetes secret using External Secrets Operator and keep them in sync.<\/p>\n

      Step 1: Create an IAM Policy for the External Secrets Operator<\/h3>\n

      Let’s start with creating a policy for assigning permission to the external secrets operator to get secrets from the AWS Secrets Manager.<\/p>\n

      The policy we are going to create only gives read permission of AWS Secrets Manager to the external secrets manager.<\/p>\n

      Use the policy command to create a JSON file with permission for the policy.<\/p>\n

      cat << EOF > policy.json\n{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Sid\": \"AllowAccessToSecretsManager\",\n            \"Effect\": \"Allow\",\n            \"Action\": [\n                \"secretsmanager:ListSecrets\",\n                \"secretsmanager:GetSecretValue\",\n                \"secretsmanager:DescribeSecret\",\n                \"secretsmanager:ListSecretVersionIds\"\n            ],\n            \"Resource\": \"arn:aws:secretsmanager:$Region:$Account-ID:secret:*\"\n        }\n    ]\n}\nEOF<\/code><\/pre>\n
      Update the region where you have the secret and your account ID in the above command.<\/p>\n
      Note:<\/strong> As a best practise, use the specific secret name instead of \u2018*\u2019, to avoid giving permission for all the secrets.<\/p>\n
      For example: “Resource”: “arn:aws:secretsmanager:$Region:$Account-ID:secret:testing-secrets-manager”<\/p><\/blockquote>\n
      Run the following command to create the policy with the policy.json<\/strong> file.<\/p>\n
      aws iam create-policy \\\n    --policy-name external-secrets-policy \\\n    --policy-document file:\/\/policy.json<\/code><\/pre>\n
      Once the policy is created, run the following command to get the ARN of the policy and save it as a variable. This will be useful when associating the policy to a role.<\/p>\n
      export POLICY_ARN=$(aws iam list-policies --query \"Policies[?PolicyName=='external-secrets-policy'].Arn\" --output text)<\/code><\/pre>\n
      Now, run the following command to check if the ARN is saved as a variable.<\/p>\n
      echo $POLICY_ARN<\/code><\/pre>\n
      This command will show the policy’s ARN.<\/p>\n
      Step 2: Create an IAM Role for the Service Account<\/h3>\n
      We will be using pod identity mapping to attach a role to a service account.<\/p>\n
      Using the following command, write a trust relationship for pod identity in a JSON file.<\/p>\n
      cat <<EOF > trust-policy.json\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Effect\": \"Allow\",\n      \"Principal\": {\n        \"Service\": \"pods.eks.amazonaws.com\"\n      },\n      \"Action\": [\n        \"sts:AssumeRole\",\n        \"sts:TagSession\"\n      ]\n    }\n  ]\n}\nEOF<\/code><\/pre>\n
      Now, run the following command to create a role with the trust relationship in the trust-policy.json<\/strong> file.<\/p>\n
      aws iam create-role \\\n    --role-name  external-secrets-role \\\n    --assume-role-policy-document file:\/\/trust-policy.json<\/code><\/pre>\n
      Once the role is created, run the following command to associate the policy with the role.<\/p>\n
      aws iam attach-role-policy \\\n    --role-name external-secrets-role \\\n    --policy-arn $POLICY_ARN<\/code><\/pre>\n
      Run the following command to get the role’s ARN and save it as a variable. This will be useful when assigning the role to the service account.<\/p>\n
      export ROLE_ARN=$(aws iam get-role --role-name external-secrets-role --query \"Role.Arn\" --output text)<\/code><\/pre>\n
      Now, run the following command to check if the ARN is saved as a variable.<\/p>\n
      echo $ROLE_ARN<\/code><\/pre>\n
      This command will show the role’s ARN.<\/p>\n
      Step 3: Create a Service Account and Assign the Role<\/h3>\n
      The next step is to create a service account and assign the role we created in the previous step using pod identity.<\/p>\n
      First, create a namespace, we will be creating every resource inside the external-secrets <\/strong>namespace.<\/p>\n
      kubectl create ns external-secrets<\/code><\/pre>\n
      Now, create a service account inside the namespace using the following command.<\/p>\n
      kubectl create sa external-secrets-sa -n external-secrets<\/code><\/pre>\n
      Once the service account is created, run the following command to assign the role to the service account using pod identity.<\/p>\n
      eksctl create podidentityassociation \\\n    --cluster $Cluster-Name \\\n    --namespace external-secrets \\\n    --service-account-name external-secrets-sa \\\n    --role-arn $ROLE_ARN<\/code><\/pre>\n
      In the above command, update the name of your cluster.<\/p>\n
      To check if the role has been attached to the service account, run the following command.<\/p>\n
      eksctl get podidentityassociation --cluster $Cluster-Name <\/code><\/pre>\n
      If the role is successfully attached to the service account, you will get the following output:<\/p>\n
      \"checking<\/figure>\n
      Step 4: Install External Secrets Operator<\/h3>\n
      Now, we are going to install External Secrets Operator using Helm.<\/p>\n
      Add the repo for External Secrets Operator on your system using the command given below.<\/p>\n
      helm repo add external-secrets https:\/\/charts.external-secrets.io<\/code><\/pre>\n
      After adding the repo, get the values.yaml<\/strong> file of the helm chart so that we can specify the service account we created in the <\/strong>previous step.<\/p>\n
      helm show values external-secrets\/external-secrets > values.yaml<\/code><\/pre>\n
      Open the values.yaml file set the create a service account to false because we have already created it and specify the name of the service account as shown below<\/p>\n
      \"updating<\/figure>\n
      This makes the service account we created earlier the default service account of the External Secrets pod.<\/p>\n
      Now run the following command to install External Secrets Operator with the values.yaml file<\/p>\n
      helm install external-secrets external-secrets\/external-secrets \\\n  --namespace external-secrets \\\n  --set installCRDs=true \\\n  -f values.yaml<\/code><\/pre>\n
      This command installs the External Secrets Operator on the external-secrets namespace.<\/p>\n
      The CRDs are set to true so that the Helm can install the required custom resources for the External Secret Operator on the cluster.<\/p>\n
      Use the following command to check if the pods of external secrets operator are up and running.<\/p>\n
      kubectl get po -n external-secrets<\/code><\/pre>\n
      You will get the following output.<\/p>\n
      \"to<\/figure>\n
      Step 5: Create a SecretStore<\/h3>\n
      To create a SecretStore, create a YML file secretstore.yml<\/strong> and copy the below content<\/p>\n
      apiVersion: external-secrets.io\/v1beta1\nkind: SecretStore\nmetadata:\n  name: aws-secret-store\n  namespace: external-secrets\nspec:\n  provider:\n    aws:\n      service: SecretsManager\n      region: $Region<\/code><\/pre>\n
      Make sure to update <\/strong>the region<\/strong> where you have the secret.<\/p>\n
      Run this YML file to create a SecretStore aws-secret-store<\/strong> on the namespace external-secrets.<\/p>\n
      The SecretStore is the resource that tells the External Secrets controller how to access external resources.<\/p>\n
      It uses the role we attached to the Service Account external-secrets-sa<\/strong> for authentication.<\/p>\n
      You may have doubts about how it knows the correct service account to use.<\/p>\n
      Since we are using pod identity, we cannot specify the service account with serviceAccountRef; it only works when using OIDC.<\/p>\n
      Without serviceAccountRef<\/strong>, it will make use of the default service account of External Secrets pod, which is external-secrets-sa<\/strong> that we created in Step 3<\/strong>.<\/p>\n
      Now, run the following command to create the secret store.<\/p>\n
      kubectl apply -f secretstore.yaml<\/code><\/pre>\n
      Run the following command to check if the secret store has been created successfully.<\/p>\n
      kubectl get secretstore -n external-secrets<\/code><\/pre>\n
      You will get the following output as shown below.<\/p>\n
      \"checking<\/figure>\n
      You can see in the above image, the status of the secret store is valid and it is ready to use.<\/p>\n
      Step 6: Create an External Secret<\/h3>\n
      To create a Secret, create a YML file externalsecret.yaml<\/strong> and copy the below content<\/p>\n
      apiVersion: external-secrets.io\/v1beta1\nkind: ExternalSecret\nmetadata:\n  name: external-secret\n  namespace: external-secrets\nspec:\n  refreshInterval: 1h\n  secretStoreRef:\n    name: aws-secret-store\n    kind: SecretStore\n  target:\n    name: aws-secret\n    creationPolicy: Owner\n  data:\n  - secretKey: aws-secrets-manager\n    remoteRef:\n      key: {secret-name}\n      property: {secret-key}<\/code><\/pre>\n
      Make sure to replace secret-name<\/strong> and secret-key<\/strong> with your Secrets Manager Secrets name and key of the value you want to fetch.<\/p>\n
      The above YML file stores the secret on the target aws-secret<\/strong> on the namespace external-secrets<\/strong>.<\/p>\n
      For every 1hr, it refreshes and updates the secrets from the SecretStore.<\/p>\n
      It fetches the specific secret from AWS Secrets Manager<\/strong> and stores it in a Kubernetes secret as aws-secret<\/strong>.<\/p>\n
      Now, run the following command to create an External Secret to sync the secret between AWS Secrets Manager and Kubernetes secret in the Cluster.<\/p>\n
      kubectl apply -f externalsecret.yaml<\/code><\/pre>\n
      Run the following command to check if the external secrets have been created.<\/p>\n
      kubectl get externalsecret -n external-secrets<\/code><\/pre>\n
      You can see the external secret is ready, and the secret is synced.<\/p>\n
      \"checking<\/figure>\n
      Run the following command to check if the secret has been created successfully.<\/p>\n
      kubectl get secret aws-secret -n external-secrets<\/code><\/pre>\n
      You will get the following output.<\/p>\n
      \"checking<\/figure>\n
      Step 7: Verify the Secret<\/h3>\n
      Now, verify if the secret in the Kubernetes secret aws-secret<\/strong> and the secret in AWS Secrets Manager<\/strong> are the same.<\/p>\n
      The secret in my Secrets Manager is given below.<\/p>\n
      \"secret<\/figure>\n
      Run the following command to get the secret aws-secret in YAML format.<\/p>\n
      kubectl get secret aws-secret -n external-secrets -o yaml<\/code><\/pre>\n
      You will get the output as below<\/p>\n
      apiVersion: v1\ndata:\n  aws-secretsmanager: dGVzdGluZyBleHRlcm5hbCBzZWNyZXRzIG1hbmFnZXI=\nimmutable: false\nkind: Secret<\/code><\/pre>\n
      Let’s decode the secret dGVzdGluZyBleHRlcm5hbCBzZWNyZXRzIG1hbmFnZXI= and see if the secret is the same as AWS Secrets Manager.<\/strong><\/p>\n
      To decode the value, run the following command.<\/p>\n
      echo 'dGVzdGluZyBleHRlcm5hbCBzZWNyZXRzIG1hbmFnZXI=' | base64 --decode<\/code><\/pre>\n
      You can see the secret is the same.<\/p>\n
      \"decoding<\/figure>\n
      Step 8: Cleanup<\/h3>\n
      If you no longer need to use the setup and want to clean it up, run the following commands.<\/p>\n
      helm uninstall external-secrets -n external-secrets\n\nkubectl delete ns external-secrets<\/code><\/pre>\n
      Conclusion<\/h2>\n
      In this blog, we explored how to set up an External Secrets Operator on AWS EKS to fetch secrets from AWS Secrets Manager and keep them in sync.<\/p>\n
      We also covered how to verify that the secrets are correct and provided commands to clean up the setup when it is no longer needed.<\/p>\n
      If you\u2019re looking for native methods to integrate Secrets Manager with EKS, check out the Secrets Store CSI Driver integration with EKS<\/a>.<\/p>\n
      \n
      Ngu\u1ed3n:<\/strong> External Secrets Operator Setup for EKS using Secrets Manager \u2014 DevOpsCube<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
      Source: https:\/\/devopscube.com\/kubernetes-external-secrets-operator\/<\/p>\n","protected":false},"author":1,"featured_media":1010,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1009","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops"],"_links":{"self":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/1009","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1009"}],"version-history":[{"count":0,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/1009\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/media\/1010"}],"wp:attachment":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1009"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1009"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1009"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}