{"id":1015,"date":"2025-01-28T06:20:09","date_gmt":"2025-01-28T06:20:09","guid":{"rendered":"https:\/\/blog.ngocha.biz\/?p=1015"},"modified":"2025-01-28T06:20:09","modified_gmt":"2025-01-28T06:20:09","slug":"vault-in-kubernetes","status":"publish","type":"post","link":"https:\/\/blog.ngocha.biz\/?p=1015","title":{"rendered":"How to Setup Vault in Kubernetes- Beginners Tutorial"},"content":{"rendered":"

This article aims to explain each of the Kubernetes vault components and step-by-step guides to set up a Vault server<\/a> in Kubernetes. Towards the end of the article, we will also discuss how an application can make use of the vault with a simple demo.<\/p>\n

As a beginner, creating components one by one while understanding the steps involved is a great way to learn about Kubernetes<\/a> and Vault. Going step by step ensures that you can focus on understanding the ‘why’ while learning the ‘how’.<\/p>\n

Creating the Vault Server in Kubernetes<\/h2>\n

As you know, Kubernetes default secret object is just base64 encoded. You need a good secret management tool and workflow to manage secret storage and retrieval for production uses cases.<\/p>\n

Hashicorp Vault<\/a> is one of the best open-source secret management tools that has good integration with Kubernetes to store and retrieve secrets.<\/p>\n

In this setup, you will learn the following.<\/p>\n

[powerkit_toc title=”Table of Contents” depth=”1″ min_count=”4″ min_characters=”1000″]<\/p>\n

Under each category, I have explained why the specific Kubernetes object is used for the vault deployment.<\/p>\n

Vault Kubernetes Manifests<\/h2>\n

All the Kubernetes YAML manifests used in this guide are hosted on Github<\/a>. Clone the repository for reference and implementation.<\/p>\n

git clone https:\/\/github.com\/techiescamp\/kubernetes-vault.git<\/code><\/pre>\n
Vault RBAC Setup<\/h2>\n
Before we get started with the setup, I would like to go through some of the basic Kubernetes objects we would be using in this vault setup.<\/p>\n
    \n
  1. ClusterRoles:<\/strong> Kubernetes ClusterRoles are entities that have been assigned certain special permissions.<\/li>\n
  2. ServiceAccounts<\/strong>: Kubernetes ServiceAccounts<\/a> are identities assigned to entities such as pods to enable their interaction with the Kubernetes APIs using the role’s permissions.<\/li>\n
  3. ClusterRoleBindings<\/strong>: ClusterRoleBindings are entities that provide roles to accounts i.e. they grant permissions to service accounts.<\/li>\n<\/ol>\n
    Vault server requires certain extra Kubernetes permissions to do its operations. Therefore, a ClusterRole is required (with the appropriate permissions) to be assigned to a ServiceAccount via a ClusterRoleBinding.<\/p>\n
    Kubernetes by default has a ClusterRole created with the required permissions i.e. ‘system:auth-delegator<\/code>‘ so it’s not required to be created again for this case. Service account and Role Binding are required to be created.<\/p>\n
    Let’s create the required RBAC for Vault.<\/p>\n
    Save the following manifest as rbac.yaml<\/code><\/p>\n
    ---\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n  name: vault\n  namespace: default\n\n---\napiVersion: rbac.authorization.k8s.io\/v1\nkind: ClusterRoleBinding\nmetadata:\n  name: vault-server-binding\nroleRef:\n  apiGroup: rbac.authorization.k8s.io\n  kind: ClusterRole\n  name: system:auth-delegator\nsubjects:\n- kind: ServiceAccount\n  name: vault\n  namespace: default\n<\/code><\/pre>\n
    Create the service account and ClusterRolebinding.<\/p>\n
    kubectl apply -f rbac.yaml<\/code><\/pre>\n
    Creating Vault ConfigMaps<\/h2>\n
    A ConfigMaps in Kubernetes lets us mount files on containers without the need to make changes to the Dockerfile or rebuilding the container image. <\/p>\n
    This feature is extremely helpful in cases where configurations have to be modified or created through files.<\/p>\n
    Vault requires a configuration file with appropriate parameters to start its servers.<\/p>\n
    Save the following manifest as configmap.yaml<\/code><\/p>\n
    apiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: vault-config\n  namespace: default\ndata:\n  extraconfig-from-values.hcl: |-\n    disable_mlock = true\n    ui = true\n    \n    listener \"tcp\" {\n      tls_disable = 1\n      address = \"[::]:8200\"\n      cluster_address = \"[::]:8201\"\n    }\n    storage \"file\" {\n      path = \"\/vault\/data\"\n    }\n<\/code><\/pre>\n
    Create the configmap<\/p>\n
    kubectl apply -f configmap.yaml\n<\/code><\/pre>\n
    It’s important to understand the idea behind the parameters inside the config map. Some of them are explained below. For a more exhaustive list of options: – refer to the official vault documentation<\/a>.<\/p>\n
      \n
    1. disable_mlock:<\/strong> Executing mlock syscall prevents memory from being swapped to<\/li>\n
    2. disk: <\/strong>This option disables the server from executing the mlock syscall.<\/li>\n
    3. ui: <\/strong>Enables the built-in web UI.<\/li>\n
    4. listener: <\/strong>Configures how Vault is listening for API requests.<\/li>\n
    5. storage: <\/strong>Configures the storage backend where Vault data is stored.<\/li>\n<\/ol>\n
      Deploy Vault Services<\/h2>\n
      Services in Kubernetes are the objects that pods use to communicate with each other. ClusterIP<\/code> type services are usually used for inter-pod communication. <\/p>\n
      There are two types of ClusterIP services<\/p>\n
        \n
      1. Headless Services<\/li>\n
      2. Services<\/li>\n<\/ol>\n
        Normal Kubernetes services act as load balancers and follow round-robin logic to distribute loads. Headless services don’t act like load balancers.<\/p>\n
        Also, normal services are assigned IPs by Kubernetes whereas Headless services are not.<\/p>\n
        For the vault server, we will create a headless service for internal usage. It will be very useful when we scale the vault to multiple replicas. <\/p>\n
        A non-headless service will be created for UI as we want to load balance requests to the replicas when accessing the UI.<\/p>\n
        Vault exposes its UI at port 8200<\/strong>.  We will use a non-headless service of type NodePort as we want to access this endpoint from outside Kubernetes Cluster.<\/p>\n
        Save the following manifest as services.yaml<\/code>. It has both service and headless service definitions.<\/p>\n
        ---\n# Service for Vault Server\napiVersion: v1\nkind: Service\nmetadata:\n  name: vault\n  namespace: default\n  labels:\n    app.kubernetes.io\/name: vault\n    app.kubernetes.io\/instance: vault\n  annotations:\nspec:\n  type: NodePort  \n  publishNotReadyAddresses: true\n  ports:\n    - name: http\n      port: 8200\n      targetPort: 8200\n      nodePort: 32000\n    - name: https-internal\n      port: 8201\n      targetPort: 8201\n  selector:\n    app.kubernetes.io\/name: vault\n    app.kubernetes.io\/instance: vault\n    component: server\n\n---\n# Headless Service\napiVersion: v1\nkind: Service\nmetadata:\n  name: vault-internal\n  namespace: default\n  labels:\n    app.kubernetes.io\/name: vault\n    app.kubernetes.io\/instance: vault\n  annotations:\nspec:\n  clusterIP: None\n  publishNotReadyAddresses: true\n  ports:\n    - name: \"http\"\n      port: 8200\n      targetPort: 8200\n    - name: https-internal\n      port: 8201\n      targetPort: 8201\n  selector:\n    app.kubernetes.io\/name: vault\n    app.kubernetes.io\/instance: vault\n    component: server\n<\/code><\/pre>\n
        Create the services.<\/p>\n
        kubectl apply -f services.yaml<\/code><\/pre>\n
        Understanding publishNotReadyAddresses:<\/h3>\n
        It’s a configuration option inside the headless services manifest file.<\/p>\n
        By default, Kubernetes includes pods under a service only when the pod is in the “ready” state.<\/p>\n
        The “publishNotReadyAddresses<\/code>” option changes this behavior by including pods that may or may not be in the ready state. You can see the list of pods in the ready state by doing “kubectl get pods”.<\/p>\n
        Need for Vault StatefulSet<\/h2>\n
        StatefulSet is the Kubernetes object used to manage stateful applications. <\/p>\n
        It’s preferred over deployments for this use case as it provides guarantees about the ordering and uniqueness of these Pods i.e. the management of volumes is better with stateful sets.<\/p>\n
        This section is critical to get a deeper understanding of the vault.<\/p>\n
        As a beginner, it is important to understand why we want to deploy a Statefulset and not Deployments. After all, our focus is on understanding the ‘why’ along with learning the ‘how’.<\/p>\n
        Why do We Need Statefulset?<\/h3>\n
        Vault is a stateful application i.e. it stores data (like configurations, secrets, metadata of vault operations) inside a volume. If the data is stored in memory, then the data will get erased once the pod restarts. <\/p>\n
        Also, Vault may have to be scaled to more than one pod in caseload increases. <\/p>\n
        All these operations have to be done in such a way that data consistency is maintained across vault pods like vault-0<\/code>, vault-1<\/code>, vault-2<\/code>.<\/p>\n
        How can we achieve this in Kubernetes? Think and then read ahead!<\/p>\n
        Vault implements continuous replication of data across all its pods. So when data is written on vault-0<\/code> it gets replicated into vault-1<\/code>. vault-2<\/code> replicates data from vault-3<\/code>. And so on…<\/p>\n
        The thing to understand here is that vault-1<\/code> needs to know where to look for vault-0<\/code>. <\/em>Otherwise, How will the replication happen?<\/p>\n
          \n
        1. How will it know from where to fetch data for the replication process?<\/li>\n
        2. How will vault-1 know where to look for vault-0?<\/li>\n
        3. How will vault-2 know where to look for vault-1?<\/li>\n<\/ol>\n
          Let’s try to answer these questions now.<\/p>\n
          In case of deployments & stateful sets, pods are always assigned a unique name that can be used to look for the pods.<\/p>\n
          In the case of deployments,<\/strong> pods are always assigned a unique name but this unique name changes after the pod are deleted & recreated<\/strong>. So it’s not useful to identify any pod.<\/p>\n
          Case of deployments:\nname of pod initially: vault-7c6c5fd47c-fkvpf \nname of pod after it gets deleted & recreated: vault-c5f7c6dfk4-7pfcv \nHere, pod name got changed.<\/code><\/pre>\n
          In the case of the stateful set <\/strong>– each pod is assigned a unique name and this unique name stays with it even if the pod is deleted <\/strong>& recreated.<\/p>\n
          Case of statefulsets:\nname of pod initially: vault-0\nname of pod after it gets deleted & recreated: vault-0\nHere, pod name remained the same.<\/code><\/pre>\n
          That’s why we want to use a stateful set here i.e. so that we can reach any pod without any discrepancies.<\/p>\n
          Beyond Vault<\/h3>\n
          These concepts of Statefulset & deployments are not unique to Vault, if you explore – you’ll find that many popular Kubernetes tools such as Elasticsearch, Postgresql use stateful sets and not deployments due to the same logic.<\/p>\n
          Deploy Vault StatefulSet<\/strong><\/h2>\n
          First, let’s create the Statefulset. I have added an explanation for the vault Statefulset as well.<\/p>\n
          Save the following manifest as statefulset.yaml<\/code><\/p>\n
          apiVersion: apps\/v1\nkind: StatefulSet\nmetadata:\n  name: vault\n  namespace: default\n  labels:\n    app.kubernetes.io\/name: vault\n    app.kubernetes.io\/instance: vault\nspec:\n  serviceName: vault-internal\n  replicas: 1\n  selector:\n    matchLabels:\n      app.kubernetes.io\/name: vault\n      app.kubernetes.io\/instance: vault\n      component: server\n  template:\n    metadata:\n      labels:\n        app.kubernetes.io\/name: vault\n        app.kubernetes.io\/instance: vault\n        component: server\n    spec:\n      serviceAccountName: vault\n      securityContext:\n        runAsNonRoot: true\n        runAsGroup: 1000\n        runAsUser: 100\n        fsGroup: 1000\n      volumes:\n        - name: config\n          configMap:\n            name: vault-config\n        - name: home\n          emptyDir: {}\n      containers:\n        - name: vault          \n          image: hashicorp\/vault:1.18.0\n          imagePullPolicy: IfNotPresent\n          command:\n          - \"\/bin\/sh\"\n          - \"-ec\"\n          args: \n          - |\n            cp \/vault\/config\/extraconfig-from-values.hcl \/tmp\/storageconfig.hcl;\n            [ -n \"${HOST_IP}\" ] && sed -Ei \"s|HOST_IP|${HOST_IP?}|g\" \/tmp\/storageconfig.hcl;\n            [ -n \"${POD_IP}\" ] && sed -Ei \"s|POD_IP|${POD_IP?}|g\" \/tmp\/storageconfig.hcl;\n            [ -n \"${HOSTNAME}\" ] && sed -Ei \"s|HOSTNAME|${HOSTNAME?}|g\" \/tmp\/storageconfig.hcl;\n            [ -n \"${API_ADDR}\" ] && sed -Ei \"s|API_ADDR|${API_ADDR?}|g\" \/tmp\/storageconfig.hcl;\n            [ -n \"${TRANSIT_ADDR}\" ] && sed -Ei \"s|TRANSIT_ADDR|${TRANSIT_ADDR?}|g\" \/tmp\/storageconfig.hcl;\n            [ -n \"${RAFT_ADDR}\" ] && sed -Ei \"s|RAFT_ADDR|${RAFT_ADDR?}|g\" \/tmp\/storageconfig.hcl;\n            \/usr\/local\/bin\/docker-entrypoint.sh vault server -config=\/tmp\/storageconfig.hcl     \n          securityContext:\n            allowPrivilegeEscalation: false\n          env:\n            - name: HOSTNAME\n              valueFrom:\n                fieldRef:\n                  fieldPath: metadata.name\n            - name: VAULT_ADDR\n              value: \"http:\/\/127.0.0.1:8200\"\n            - name: VAULT_API_ADDR\n              value: \"http:\/\/$(POD_IP):8200\"\n            - name: SKIP_CHOWN\n              value: \"true\"\n            - name: SKIP_SETCAP\n              value: \"true\"\n            - name: VAULT_CLUSTER_ADDR\n              value: \"https:\/\/$(HOSTNAME).vault-internal:8201\"\n            - name: HOME\n              value: \"\/home\/vault\"\n          volumeMounts:\n            - name: data\n              mountPath: \/vault\/data  \n            - name: config\n              mountPath: \/vault\/config\n            - name: home\n              mountPath: \/home\/vault\n          ports:\n            - containerPort: 8200\n              name: http\n            - containerPort: 8201\n              name: https-internal\n            - containerPort: 8202\n              name: http-rep\n          readinessProbe:\n            exec:\n              command: [\"\/bin\/sh\", \"-ec\", \"vault status -tls-skip-verify\"]\n            failureThreshold: 2\n            initialDelaySeconds: 5\n            periodSeconds: 5\n            successThreshold: 1\n            timeoutSeconds: 3\n  volumeClaimTemplates:\n    - metadata:\n        name: data\n      spec:\n        accessModes:\n          - ReadWriteOnce\n        resources:\n          requests:\n             storage: 1Gi\n<\/code><\/pre>\n
          Create the Statefulset.<\/p>\n
          kubectl apply -f statefulset.yaml<\/code><\/pre>\n
          The Statefulset YAML of the vault has a lot of components such as configmap mounts, security context, probes, etc.<\/p>\n
          Note:<\/strong> Vault remains in a non ready state of now. It needs to be unsealed to become in ready state.  Feel free to skip to next section to unseal vault. For production use cases, auto unseal <\/a>options should be used.<\/p><\/blockquote>\n
          Let us dive deeper and understand what each part is doing.<\/p>\n
          Configurations: <\/strong>Vault’s extra configuration file is being mounted at \/vault\/config\/extraconfig-from-values.hcl<\/code> and is then being copied to \/tmp\/storageconfig.hcl <\/code>where it is used when the process starts.<\/p>\n
          cp \/vault\/config\/extraconfig-from-values.hcl \/tmp\/storageconfig.hcl;\n\/usr\/local\/bin\/docker-entrypoint.sh vault server -config=\/tmp\/storageconfig.hcl    <\/code><\/pre>\n
          ServiceAccount: <\/strong>Since we want the vault server to have the required permissions of ‘system:auth-delegator’. A service account has been assigned to the pod.<\/p>\n
           serviceAccountName: vault<\/code><\/pre>\n
          SecurityContext:<\/strong> Running pods with privileged processes pose a security risk. This is because running a process with root on a pod is identical to running a process with root on the host node. <\/p>\n
          Ideally, pods should be run keeping in mind isolation between containers and host nodes. Therefore, here pods have been instructed to run as non-root users.<\/p>\n
          securityContext:\n    runAsNonRoot: true\n    runAsGroup: 1000\n    runAsUser: 100\n    fsGroup: 1000<\/code><\/pre>\n
          Probes:<\/strong> Probes ensure that the vault does not get stuck in a loop due to any bug and can be restarted automatically in case an unexpected error comes up.<\/p>\n
          readinessProbe:\n    exec:\n        command: [\"\/bin\/sh\", \"-ec\", \"vault status -tls-skip-verify\"]\n    failureThreshold: 2\n    initialDelaySeconds: 5\n    periodSeconds: 5\n    successThreshold: 1\n    timeoutSeconds: 3<\/code><\/pre>\n
          VolumeClaimTemplates: <\/strong>A template by which a stateful set can create volumes for replicas.<\/p>\n
          volumeClaimTemplates:\n    - metadata:\n        name: data\n      spec:\n        accessModes:\n          - ReadWriteOnce\n        resources:\n          requests:\n             storage: 1Gi<\/code><\/pre>\n
          The vault setup is complete. The next step is to unseal and initialize the vault.<\/p>\n
          Unseal & Initialise Vault<\/strong><\/h2>\n
          Initialization is the process by which Vault’s storage starts preparation to receive data. <\/p>\n
          Vault generates an in-memory master key and applies Shamir’s secret sharing algorithm to disassemble that master key into multiple keys. These keys are called “unseal keys”.<\/p>\n
          So to initialize the vault, first, we need to unseal the vault using the unseal keys.<\/p>\n
          Step 1:<\/strong> First, we are creating tokens and keys to start using the vault.<\/p>\n
          Note:  Pleaset install jq if it is not installed on your system.<\/p><\/blockquote>\n
          kubectl exec vault-0 -- vault operator init -key-shares=1 -key-threshold=1 -format=json > keys.json\n\nVAULT_UNSEAL_KEY=$(cat keys.json | jq -r \".unseal_keys_b64[]\")\necho $VAULT_UNSEAL_KEY\n\nVAULT_ROOT_KEY=$(cat keys.json | jq -r \".root_token\")\necho $VAULT_ROOT_KEY<\/code><\/pre>\n
          Step 2: <\/strong>Unseal is the state at which the vault can construct keys that are required to decrypt the data stored inside it.<\/p>\n
          kubectl exec vault-0 -- vault operator unseal $VAULT_UNSEAL_KEY<\/code><\/pre>\n
          Important Note:<\/strong> If the Vault pod restarts, it gets sealed autoamtically. You will need to unseal vault again using the above unseal command. Also, if you run more than one replicaset, you need to login to all pods and execute the unseal commands. For production use cases, there are auto-unseal <\/a>options available.<\/p><\/blockquote>\n
          Login & Access Vault UI<\/h2>\n
          You can log in to Vault using UI and CLI.<\/p>\n
          Note:<\/strong> In production setup, vault UI and login will be secured using LDAP, okta  or other secure mechanisms that follows standard security guidelines.<\/p><\/blockquote>\n
          The following command will remotely execute the vault log command as we have the VAULT_ROOT_KEY<\/code> set locally in the environment variable.<\/p>\n
          kubectl exec vault-0 -- vault login $VAULT_ROOT_KEY<\/code><\/pre>\n
          The above command will display a token. Save the token as you will need it to log in to the vault UI.<\/p>\n
          \"Kubernetes<\/figure>\n
          As we have created a service on nodeport 32000<\/code>, we will be able to access the Vault UI using any of the Node IPs on port 32000. You can use the saved token to log in to the UI.<\/p>\n
          For example,<\/p>\n
          http:\/\/33.143.55.228:32000<\/code><\/pre>\n
          \"\"<\/figure>\n
          Creating Vault Secrets<\/h2>\n
          We can create secrets using CLI as well as the UI. We will use the CLI to create secrets.<\/p>\n
          To use the vault CLI, we need to exec into the vault pod.<\/p>\n
          kubectl exec -it vault-0 -- \/bin\/sh<\/code><\/pre>\n
          Create secrets<\/h3>\n
          There are multiple secret engines (Databases, Consul<\/a>, AWS, etc). Refer to the official documentation<\/a> to know more about the supported secret engines.<\/p>\n
          Here, we are utilizing key-value engine v2<\/a>. It has advanced capabilities to keep multiple versions of the same keys. v1 can manage only a single version at a time.<\/p>\n
          Let’s enable the Key value engine.<\/p>\n
          vault secrets enable -version=2 -path=\"demo-app\" kv<\/code><\/pre>\n
          Create a secret in key-value format and list it. The id (key) is name<\/code> and secret(value)  would be devopscube<\/code>. Path is demo-app\/user01<\/code><\/p>\n
          vault kv put demo-app\/user01 name=devopscube\nvault kv get demo-app\/user01 <\/code><\/pre>\n
          \"Kubernetes<\/figure>\n
          Create a Policy<\/h3>\n
          By default, the secret path has the deny policy enabled. We need to explicitly add a policy to read\/write\/delete the secrets.<\/p>\n
          The following policy dictates that the entity be allowed the read operation for secrets stored under “demo-app<\/code>“. Execute it to create the policy<\/p>\n
          vault policy write demo-policy - <<EOH\npath \"demo-app\/*\" {\n  capabilities = [\"read\"]\n}\nEOH<\/code><\/pre>\n
          You can list and validate the policy.<\/p>\n
          vault policy list<\/code><\/pre>\n
          Enable Vault Kubernetes Authentication Method<\/h2>\n
          For Kubernetes pods to interact with Vault and get the secrets, it needs a vault token. The kubernetes auth method with the pod service account makes it easy for pods to retrieve secrets from the vault.<\/p>\n
          In this way, any pod which has been assigned the “vault<\/code>” as the service account<\/a> – will be able to read these secrets without requiring any vault token.<\/p>\n
          Let’s enable the kubernetes auth method.<\/p>\n
          vault auth enable kubernetes<\/code><\/pre>\n
          You can view and enable auth methods from the vault UI as well.<\/p>\n
          \"vault<\/figure>\n
          We have attached a service account with a ClusterRole to the vault Statefulset. The following command configures the service account token to enable the vault server to make API calls to Kubernetes using the token, Kubernetes URL and the cluster API CA certificate.<\/p>\n
          KUBERNETES_PORT_443_TCP_ADDR<\/code> is the env variable inside the pod that returns the internal API endpoint.<\/p>\n
          vault write auth\/kubernetes\/config \\\n  token_reviewer_jwt=\"$(cat \/var\/run\/secrets\/kubernetes.io\/serviceaccount\/token)\" \\\n  kubernetes_host=\"https:\/\/$KUBERNETES_PORT_443_TCP_ADDR:443\" \\\n  kubernetes_ca_cert=@\/var\/run\/secrets\/kubernetes.io\/serviceaccount\/ca.crt<\/code><\/pre>\n
          NOTE: <\/strong>Use the disable_iss_validation=true<\/code> option in the above command to disables issuer validation, this allowa Vault to skip checking the iss issuer claim in JWT tokens. <\/p>\n
          Avoid using in production environment, this is only for testing.<\/p><\/blockquote>\n
          Now we have to create a vault approle<\/strong> that binds a Kubernetes service account, namespace, and vault policies. This way vault server knows if a specific service account is authorized to read the stored secrets.<\/p>\n
          Let’s create a service account that can be used by application pods to retrieve secrets from the vault.<\/p>\n
          kubectl create serviceaccount vault-auth<\/code><\/pre>\n
          Let’s create a vault approle<\/code> named webapp<\/code> and bind a service account named vault-auth <\/code>in the default<\/code> namespace. Also, we are attaching the demo-policy<\/code> we have created which has read access to a secret.<\/p>\n
          vault write auth\/kubernetes\/role\/webapp \\\n        bound_service_account_names=vault-auth \\\n        bound_service_account_namespaces=default \\\n        policies=demo-policy \\\n        ttl=72h<\/code><\/pre>\n
          Now, any pod in the default namespace with a vault-auth<\/code> service account can request the secret under demo-policy.<\/p>\n
          Fetching Secrets Stored in Vault With Service Accounts<\/h2>\n
          The objective is that a pod should be able to fetch secrets from the vault without it being fed any vault token i.e. it should use the service account token to make the request to the vault.<\/p>\n
          \"\"<\/figure>\n
          Here is how it works.<\/p>\n
            \n
          1. A JWT token<\/a> (Service account token) from the pod is passed to the vault server.<\/li>\n
          2. Vault server requests the Kubernetes API server to get the service account and namespace attached to the JWT token.<\/li>\n
          3. The Kubernetes API server returns the namespace and service account details.<\/li>\n
          4. Vault server validates if the service account is authorized to read secrets using the attached policies.<\/li>\n
          5. After validation, the vault server returns a vault token.<\/li>\n
          6. In another API call, the vault token is passed with a secret path to retrieve the secrets.<\/li>\n<\/ol>\n
            To demonstrate this, first, we will deploy a pod named vault-client<\/code> with vault-auth<\/code> service account in the default<\/code> namespace.<\/p>\n
            Save the manifest as pod.yaml<\/code><\/p>\n
            ---\napiVersion: v1\nkind: Pod\nmetadata:\n  name: vault-client\n  namespace: default\nspec:\n  containers:\n  - image: nginx:latest\n    name: nginx\n  serviceAccountName: vault-auth<\/code><\/pre>\n
            Deploy the pod.<\/p>\n
            kubectl apply -f pod.yaml<\/code><\/pre>\n
            Now, take the exec session of the pod, so that we can make the API requests and see if it is able to authenticate to the vault server and retrieve the required secrets.<\/p>\n
            kubectl exec -it vault-client -- \/bin\/bash<\/code><\/pre>\n
            Follow the steps given below.<\/p>\n
            Step 1:<\/strong>  Make an API request to the vault server using the service account token (JWT) to acquire a client token.<\/p>\n
            First, install jq if it’s not installed on your system using the following command.<\/p>\n
            apt-get update && apt-get install -y jq<\/code><\/pre>\n
            Then, save the service account token to a variable jwt_token<\/code><\/p>\n
            jwt_token=$(cat \/var\/run\/secrets\/kubernetes.io\/serviceaccount\/token)<\/code><\/pre>\n
            Make the API call using curl. Replace http:\/\/35.193.55.248:32000<\/code> with your vault URL.<\/p>\n
            curl --request POST \\\n    --data '{\"jwt\": \"'$jwt_token'\", \"role\": \"webapp\"}' \\\n    http:\/\/35.193.55.248:32000\/v1\/auth\/kubernetes\/login | jq<\/code><\/pre>\n
            The above API request will return a JSON containing the client_token<\/code>. Using this token, secrets can be read.<\/p>\n
            Sample JSON (beautified) output:<\/p>\n
            {\n  \"request_id\": \"87440e92-cbe9-8357-059d-c4ecdfd58e84\",\n  \"lease_id\": \"\",\n  \"renewable\": false,\n  \"lease_duration\": 0,\n  \"data\": null,\n  \"wrap_info\": null,\n  \"warnings\": null,\n  \"auth\": {\n    \"client_token\": \"s.r2kKKaYnGR48CR9aWvoV8skx\",\n    \"accessor\": \"bqqCCmrxS58TiTyafH9udHm7\",\n    \"policies\": [\n      \"default\",\n      \"demo-policy\"\n    ],\n    \"token_policies\": [\n      \"default\",\n      \"demo-policy\"\n    ]\n  }\n}<\/code><\/pre>\n
            Now we will use the client_token to make an API call and fetch the stored secrets under demo-app<\/code>. Replace the token and vault URL with relevant values.<\/p>\n
            curl -H \"X-Vault-Token: s.bSRl7TNajYxWvA7WiLdTQS7Z\" \\\n     -H \"X-Vault-Namespace: vault\" \\\n     -X GET http:\/\/35.193.55.248:32000\/v1\/demo-app\/data\/user01?version=1 | jq<\/code><\/pre>\n
            The above API request will return a JSON containing the secrets under ‘data’.<\/p>\n
            Sample output (beautified)<\/p>\n
            {\n  \"request_id\": \"d2a2fc9c-6eca-9a12-a4a8-8799230e9449\",\n  \"lease_id\": \"\",\n  \"renewable\": false,\n  \"lease_duration\": 0,\n  \"data\": {\n    \"data\": {\n      \"name\": \"devopscube\"\n    }\n  }\n}<\/code><\/pre>\n
            Inject Secrets to Pods Using Vault Agent<\/h2>\n
            Using a vault injector, you can inject secrets into Kubernetes pods. We have a detailed guide on setting up a vault injector that explains the whole workflow.<\/p>\n
            Refer: Vault Agent Injector Tutorial<\/a><\/p>\n
            \"\"<\/figure>\n
            Possible Errors & Troubleshooting<\/h2>\n
            If you delete the stateful set and redeploy the vault again,  don’t run the init command again. It might throw the following error.<\/p>\n
            Error initializing: Error making API request.\n\nURL: PUT http:\/\/127.0.0.1:8200\/v1\/sys\/init\nCode: 400. Errors:<\/code><\/pre>\n
            It happens because by design when you delete a Statefulset the volumes attached to the pod won’t get deleted. This way you have a chance to copy the data.<\/p>\n
            So the PVC will have the initialized configs. So you just have to use the existing key.json and use the secret token to unseal the vault without initialization. Or, you can delete the PVC entirely and redeploy the vault with a new PVC.<\/p>\n
            Conclusion<\/h2>\n
            By now, you have learned to set up, configure and utilize vault in Kubernetes. It’s a great starting point and there are more concepts and workflows that you can explore.<\/p>\n
            When it comes to vault production implementation<\/strong>, there are many considerations and standard security practices to be followed.<\/p>\n
            Also, when used with application pods to retrieve secrets, you can use Vault injector<\/a> to inject secrets in the pod. We will publish the vault injector tutorial in the upcoming posts.<\/p>\n
            \n
            Ngu\u1ed3n:<\/strong> How to Setup Vault in Kubernetes- Beginners Tutorial \u2014 DevOpsCube<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
            Source: https:\/\/devopscube.com\/vault-in-kubernetes\/<\/p>\n","protected":false},"author":1,"featured_media":1016,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1015","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops"],"_links":{"self":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/1015","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1015"}],"version-history":[{"count":0,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/1015\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/media\/1016"}],"wp:attachment":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1015"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1015"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1015"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}