{"id":538,"date":"2025-01-27T09:50:29","date_gmt":"2025-01-27T09:50:29","guid":{"rendered":"https:\/\/blog.ngocha.biz\/?p=538"},"modified":"2025-01-27T09:50:29","modified_gmt":"2025-01-27T09:50:29","slug":"cluster-autoscaler","status":"publish","type":"post","link":"https:\/\/blog.ngocha.biz\/?p=538","title":{"rendered":"Cluster AutoScaler Setup on AWS EKS: A Comprehensive Guide"},"content":{"rendered":"

In this hands-on guide, you will to implement node autoscaling in EKS cluster using Cluster AutoScaler.<\/p>

By default, when you deploy an EKS cluster, node auto-scaling is not enabled.<\/p>

This means that even if you enable Horizontal Pod Autoscaler (HPA), pods may go into a pending state when node resources are exhausted. For pods to scale further, you need to scale the underlying nodes.<\/p>

This is where Cluster Autoscaler comes into play. It automatically adjusts the number of nodes in your cluster to meet resource demands.<\/p>

By the end of this guide, you will:<\/p>

  1. Understand what Cluster AutoScaler is.<\/li>
  2. Learn how it works behind the scenes with AWS<\/li>
  3. Set up Cluster AutoScaler on EKS with hands-on experience<\/li>
  4. Explore different scenarios to test scaling based on AutoScaler<\/li><\/ol>

    What is a Cluster AutoScaler?<\/h2>

    Cluster AutoScaler<\/a> is a tool designed to automatically scale Kubernetes cluster<\/a> nodes based on workloads. It is maintained by the Kubernetes community.<\/p>

    It supports almost all cloud platforms and managed Kubernetes services, such as EKS, AKS, GKE, etc…<\/p>

    When you deploy Cluster AutoScaler, it continuously monitors the API server for unscheduled Pods<\/strong> and automatically adds nodes to the cluster to make resources available for the pods.<\/p>

    Additionally, it scales down nodes when the cluster has more resources than needed.<\/p>

    As you may know, cloud-based Kubernetes implementations typically include node groups<\/strong> to manage worker nodes efficiently.<\/p>

    \ud83d\udca1<\/div>
    A Node Group<\/strong><\/b> is a set of worker nodes within a Kubernetes cluster that share the same configuration, such as instance type, networking, and scaling policies<\/div><\/div>

    If there are multiple node groups present, the Cluster AutoScaler scales nodes using the node groups that match the specified expander strategy<\/code><\/strong> on the deployment.<\/p>

    \ud83d\udca1<\/div>
    The Expander Strategy in Kubernetes Cluster Autoscaler determines which node group to scale up when additional resources are needed but multiple node groups are eligible for expansion.<\/div><\/div>

    There are a total of six expander strategies available, they are:<\/p>\n\n

      \n
    1. least-waste<\/strong> – Select the node group that leaves the least amount of CPU and memory used after sca<\/span>ling.<\/li>\n\n\n
    2. random<\/strong> – This is the default expander when no expander is specified, and it is <\/span>used when there is no problem scaling any node type.<\/li>\n\n\n
    3. most-pods<\/strong> – This expander scales the node group, which can schedule most pods.<\/li>\n\n\n
    4. least-nodes<\/strong> – Select this to scale the node group, which can schedule pods with minimum nodes.<\/li>\n\n\n
    5. price<\/strong> – Scales the node group whose cost is low, check here<\/a> for more details.<\/li>\n\n\n
    6. priority<\/strong> – Select the node group that was assigned by the user in the configuration file<\/a>.<\/li>\n<\/ol>\n\n
      \ud83d\udca1<\/div>
      You can set the expander strategy using the --expander<\/code> flag, which we will explore in detail in the hands-on section<\/div><\/div>

      We can deploy Cluster AutoScaler using two methods:<\/p>

      1. Auto-Discovery method (<\/strong>Recommended)<\/strong> – Automatically discovers every node groups ASGs with required tags and scales them if needed.<\/li>
      2. Manual method<\/strong> – You have to specify the node groups ASG’s min capacity, maximum capacity, and name.<\/li><\/ol>

        How does Kubernetes Cluster AutoScaler work?<\/h2>

        The workflow diagram of AWS EKS Cluster AutoScaler is given below.<\/p>

        \"workflow
        Click the Image to view in HD<\/span><\/figcaption><\/figure>

        Here is how the cluster AutoScaler works.<\/p>

        1. A manifest is applied to create a deployment on the cluster.<\/li>
        2. The API server tells the Scheduler to assign the deployment pods to the nodes.<\/li>
        3. The Pods are scheduled on nodes until resources are exhausted. Any remaining Pods that cannot be scheduled<\/strong> due to insufficient resources go into a Pending state<\/strong>.<\/li>
        4. The API server updates the pending status<\/strong> of these Pods along with the reason (e.g., insufficient CPU, memory).<\/li>
        5. The Cluster AutoScaler, which continuously monitors the API server, notices the pods are in a pending state because of resource unavailability.<\/li>
        6. The Autoscaler analyzes the resource requirements and selects the most suitable node group based on the configured expander strategy.<\/strong><\/li>
        7. Then, it gets the ASG associated with the node group and uses AWS APIs to request the ASG to scale nodes.<\/li>
        8. Once the ASG creates the required nodes, the Scheduler schedules the pods on the new node.<\/li>
        9. If workloads decrease (e.g., a job finishes or a deployment is scaled down), some nodes may no longer be needed.<\/li>
        10. A node is eligible for removal if it has been underutilized for a set time (default: 10 minutes). Once a node is identified for removal, the Cluster Autoscaler requests that AWS Auto Scaling Group (ASG) terminate it.<\/li><\/ol>

          Setup Prerequisites<\/strong><\/h2>

          The prerequisites required for this setup are listed below.<\/p>

          1. EKS Cluster<\/a><\/li>
          2. AWS CLI<\/a><\/li>
          3. Kubectl<\/a><\/li>
          4. eksctl<\/li>
          5. Permission to create IAM Role<\/a> and Policy<\/li>
          6. Pod Identity agent plugin is enabled on the cluster<\/li><\/ol>

            Setup Cluster AutoScaler on EKS Cluster<\/h2>

            Let’s set up a Cluster AutoScaler on the EKS cluster, we will use the auto-discovery<\/code> method for this setup.<\/p>

            For the auto-discovery method to work, ASGs must have the following tags.<\/p>

            1. k8s.io\/cluster-autoscaler\/enabled<\/li>
            2. k8s.io\/cluster-autoscaler\/<cluster-name><\/li><\/ol>

              Cluster AutoScaler uses these to find the ASGs automatically.<\/p>

              \ud83d\udca1<\/div>
              The above tags are applied as default to the ASG when you create the node group using eksctl<\/a>.<\/div><\/div>

              These tags might not apply when you create a node group using Terraform or a CLI command, make sure the node groups ASG has these tags.<\/p>

              To check if the node groups ASG have the mentioned tag, run the following command to get the names of all the ASG in your AWS.<\/p>

              aws autoscaling describe-auto-scaling-groups --query \"AutoScalingGroups[*].AutoScalingGroupName\" --output table<\/code><\/pre>
              Then, run the following command to check the tags assigned to the specific ASG.<\/p>
              aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names  <asg-name> --query \"AutoScalingGroups[*].Tags\" --output table<\/code><\/pre>
              Update the ASG name in the above command that you want to check the tags; the node groups ASG will have the node group name.<\/p>
              For example, if your node group name is ng-spo<\/code>t, then your ASG name will be eks-ng-spot-62ca5663-d8f9-a974-10c3-e0ca52223c7c<\/code>.<\/p>
              Now, follow the steps below one by one to set up Cluster AutoScaler on the EKS cluster.<\/p>
              Step 1: Create an IAM Policy<\/h3>
              Let’s start with creating an IAM policy for the Cluster AutoScaler, which assigns permission to scale nodes and other required permissions.<\/p>
              First, run the following command to create a JSON file with the required permissions.<\/p>
              cat <<EoF > ca-policy.json\n{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Action\": [\n                \"autoscaling:DescribeAutoScalingGroups\",\n                \"autoscaling:DescribeAutoScalingInstances\",\n                \"autoscaling:DescribeLaunchConfigurations\",\n                \"autoscaling:DescribeTags\",\n                \"autoscaling:SetDesiredCapacity\",\n                \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n                \"ec2:DescribeLaunchTemplateVersions\"\n            ],\n            \"Resource\": \"*\",\n            \"Effect\": \"Allow\"\n        }\n    ]\n}\nEoF<\/code><\/pre>
              Then, run the following command to create the IAM policy with the permission listed on ca-policy.json<\/code>.<\/p>
              aws iam create-policy   \\\n  --policy-name ca-policy \\\n  --policy-document file:\/\/ca-policy.json<\/code><\/pre>
              Now, run the following command to save the ARN<\/code> of the policy as a variable<\/code>, which will be helpful in the next step.<\/p>
              export POLICY_ARN=$(aws iam list-policies --query \"Policies[?PolicyName=='ca-policy'].Arn\" --output text)<\/code><\/pre>
              Run the following command to check if the ARN<\/a> is saved as a variable.<\/p>
              echo $POLICY_ARN<\/code><\/pre>
              If it shows the ARN, move on to the next step.<\/p>
              Step 2: Create an IAM Role<\/h3>
              Once the policy is created, create an IAM role and attach the policy to the role.<\/p>
              Start by creating a JSON file that contains the trust policy for the role.<\/p>
              cat <<EoF > trust-policy.json\n{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n      {\n        \"Effect\": \"Allow\",\n        \"Principal\": {\n          \"Service\": \"pods.eks.amazonaws.com\"\n        },\n        \"Action\": [\n          \"sts:AssumeRole\",\n          \"sts:TagSession\"\n        ]\n      }\n    ]\n}\nEoF<\/code><\/pre>
              Then, run the following command to create the IAM role with the role trust policy on trust-policy.json<\/code>.<\/p>
              aws iam create-role \\\n    --role-name  ca-role \\\n    --assume-role-policy-document file:\/\/trust-policy.json<\/code><\/pre>
              Now, run the following command to attach the policy to the role.<\/p>
              aws iam attach-role-policy \\\n    --role-name ca-role \\\n    --policy-arn $POLICY_ARN<\/code><\/pre>
              Once the role creation and policy attachment are completed, run the following command to save the ARN of the role as a variable.<\/p>
              export ROLE_ARN=$(aws iam get-role --role-name ca-role --query \"Role.Arn\" --output text)<\/code><\/pre>
              Run the following command to check if the ARN is saved as a variable.<\/p>
              echo $ROLE_ARN<\/code><\/pre>
              If it shows the ARN, move on to the next step.<\/p>
              Step 3: Download and Modify Cluster AutoScaler YAML<\/h3>
              Now, download the Cluster AutoScaler deployment YAML<\/a> and modify it.<\/p>
              Run the following command to download the YAML file.<\/p>
              wget https:\/\/raw.githubusercontent.com\/kubernetes\/autoscaler\/master\/cluster-autoscaler\/cloudprovider\/aws\/examples\/cluster-autoscaler-autodiscover.yaml<\/code><\/pre>
              Modify the following in the manifest file:<\/p>
              1. In the deployment part, change the container image version<\/a> to the same version as your EKS cluster version. For example, if your cluster version is 1.30.8, specify the container version as v1.30.0.<\/li>
              2. Specify your cluster name in the command section --node-group-auto-discovery=asg:tag=k8s.io\/cluster-autoscaler\/enabled,k8s.io\/cluster-autoscaler\/<YOUR CLUSTER NAME><\/strong><\/code>.<\/li><\/ol>
                The modified deployment part will look like this:<\/p>
                apiVersion: apps\/v1\nkind: Deployment\nmetadata:\n  name: cluster-autoscaler\n  namespace: kube-system\n  labels:\n    app: cluster-autoscaler\nspec:\n  replicas: 1\n  selector:\n    matchLabels:\n      app: cluster-autoscaler\n  template:\n    metadata:\n      labels:\n        app: cluster-autoscaler\n      annotations:\n        prometheus.io\/scrape: 'true'\n        prometheus.io\/port: '8085'\n    spec:\n      priorityClassName: system-cluster-critical\n      securityContext:\n        runAsNonRoot: true\n        runAsUser: 65534\n        fsGroup: 65534\n        seccompProfile:\n          type: RuntimeDefault\n      serviceAccountName: cluster-autoscaler\n      containers:\n        - image: registry.k8s.io\/autoscaling\/cluster-autoscaler:v1.30.0\n          name: cluster-autoscaler\n          resources:\n            limits:\n              cpu: 100m\n              memory: 600Mi\n            requests:\n              cpu: 100m\n              memory: 600Mi\n          command:\n            - .\/cluster-autoscaler\n            - --v=4\n            - --stderrthreshold=info\n            - --cloud-provider=aws\n            - --skip-nodes-with-local-storage=false\n            - --expander=least-waste\n            - --node-group-auto-discovery=asg:tag=k8s.io\/cluster-autoscaler\/enabled,k8s.io\/cluster-autoscaler\/eks-spot-cluster\n          volumeMounts:\n            - name: ssl-certs\n              mountPath: \/etc\/ssl\/certs\/ca-certificates.crt # \/etc\/ssl\/certs\/ca-bundle.crt for Amazon Linux Worker Nodes\n              readOnly: true\n          imagePullPolicy: \"Always\"\n          securityContext:\n            allowPrivilegeEscalation: false\n            capabilities:\n              drop:\n                - ALL\n            readOnlyRootFilesystem: true\n      volumes:\n        - name: ssl-certs\n          hostPath:\n            path: \"\/etc\/ssl\/certs\/ca-bundle.crt\"<\/code><\/pre>
                You can see I have changed the container version based on my cluster version and specified my cluster name in the command section.<\/p>
                You can also change the expander<\/code> command to random<\/code>, most-pods<\/code>, least-waste<\/code>, priority<\/code> as per your requirements.<\/p>
                If you want to run the Cluster AutoScaler in manual mode, remove the command:

                –node-group-auto-discovery=asg:tag=k8s.io\/cluster-autoscaler\/enabled,k8s.io\/cluster-autoscaler\/eks-spot-cluster<\/strong> from the above manifest file and use the:

                –nodes=1:4:eks-ng-spot-16ca48b9-1524-ecf0-3c0d-572a204ffa86<\/strong> to specify the nodes groups ASG manually.

                The above command structure is –nodes=<ASG-min>:<ASG-max>:<ASG name><\/strong>, in the command you have to specify the node groups ASG’s min capacity, maximum capacity and it’s name.<\/blockquote>
                Some additional commands that are enabled by default can also be customized:<\/p>\n\n
                  \n
                1. –scale-down-delay-after-add<\/strong> – This command prevents the nodes from scaling down until the specified time, by default, it is 10 minutes.<\/li>\n\n\n
                2. –scale-down-delay-after-delete<\/strong> – This command prevents the nodes from scaling down one after another, the default time to scale down nodes one by one is 10 seconds.<\/span><\/li>\n\n\n
                3. –scale-down-unneeded-time<\/strong> – This command tells how long a node can run underutilized before scaling down, and the default time is 10 minutes.<\/li>\n\n\n
                4. –scan-interval<\/strong> – Time gap taken by the Cluster AutoScaler pods scrape data from the API server, by default, it scrapes data every 10 seco<\/span>nds.<\/li>\n<\/ol>\n\n
                  Make the mentioned changes and run the following command to deploy Cluster AutoScaler and other resources required for the Cluster AutoScaler.<\/p>
                  kubectl apply -f cluster-autoscaler-autodiscover.yaml<\/code><\/pre>
                  Once the deployment is up, run the following command to annotate the deployment.<\/p>
                  kubectl -n kube-system annotate deployment.apps\/cluster-autoscaler cluster-autoscaler.kubernetes.io\/safe-to-evict=\"false\"<\/code><\/pre>
                  This annotation will prevent the Cluster AutoScaler pods from eviction during scaling.<\/p>
                  Step 4: Assign the IAM Role to the Service Account<\/h3>
                  The next step is to assign the IAM role to the Cluster AutoScalers service account using Pod Identity<\/code> to provide scaling permission.<\/p>
                  Before assigning the role, check if Pod identity is enabled on your cluster by running the following command.<\/p>
                  aws eks list-addons --cluster-name <CLUSTER NAME><\/code><\/pre>
                  Specify your cluster name in the above command.<\/p>
                  If Pod Identity is enabled on your cluster, you can see it in the output as shown below.<\/p>
                  \"verifying<\/figure>
                  If not listed, it means Pod Identity is not enabled on your cluster. Run the following command to enable Pod Identity on your cluster.<\/p>
                  aws eks create-addon --cluster-name <CLUSTER NAME> --addon-name eks-pod-identity-agent<\/code><\/pre>
                  Once enabled, run the following command to assign the IAM role to the Cluster AutoScaler’s service account using Pod Identity.<\/p>
                  eksctl create podidentityassociation \\\n    --cluster <CLUSTER NAME> \\\n    --namespace kube-system \\\n    --service-account-name cluster-autoscaler \\\n    --role-arn $ROLE_ARN<\/code><\/pre>
                  cluster-autoscaler<\/code> is the Cluster AutoScaler’s service account.<\/p>
                  Then, restart the deployment to make the Cluster AutoScaler pods use the role.<\/p>
                  kubectl rollout restart deploy cluster-autoscaler -n kube-system<\/code><\/pre>
                  Testing Cluster AutoScaler<\/h2>
                  The Cluster AutoScaler setup is ready. Let’s check if it’s working properly.<\/p>
                  To check, create a deploy.yaml<\/code> file and copy the below content:<\/p>
                  apiVersion: apps\/v1\nkind: Deployment\nmetadata:\n  name: nginx-app\nspec:\n  replicas: 4\n  selector:\n    matchLabels:\n      app: nginx-app\n  template:\n    metadata:\n      labels:\n        app: nginx-app\n    spec:\n      containers:\n      - name: app\n        image: nginx\n        resources:\n          requests:\n            memory: \"1Gi\"\n            cpu: \"500m\"<\/code><\/pre>
                  This manifest file will create a deployment with 4<\/code> replicas and set the resource request to 1Gi Memory and 500m CPU<\/code>.<\/p>
                  Currently, my cluster has 1 <\/code>node of type t3.medium, which has 1 CPU and 2GB of Memory<\/code>. Set the resource request based on your node type, which makes the nodes scale.<\/p>
                  Apply the manifest file using the following command.<\/p>
                  kubectl apply -f deploy.yaml<\/code><\/pre>
                  List the pods using the below command<\/p>
                  kubectl get po<\/code><\/pre>
                  You can see that two pods are still in a pending state because of insufficient resources.<\/p>
                  \"listing<\/figure>
                  Now, the total resource limit has exceeded the node capacity, which triggers the Cluster AutoScaler to trigger nodes based on the requirements.<\/p>
                  \"listing<\/figure>
                  You can see the scale-up is triggered, and a new node is created.<\/p>
                  The trigger will happen within 10-30 seconds, and the node will be up and running within 1 minute.<\/p>
                  \"listing<\/figure>
                  You can see that a new node is created as per the resource requirements, and all the pods are up and running.<\/p>
                  Now, delete the deployment using the following command to see the scale-down process.<\/p>
                  kubectl delete -f deploy.yaml<\/code><\/pre>
                  The unused nodes will be terminated after 10 minutes, it is the default node scale-down time.<\/p>
                  \"listing<\/figure>
                  Common Issues and Troubleshooting<\/h2>
                  Given below are some of the common issues when using Cluster AutoScaler and its troubleshooting.<\/p>
                  Check Logs<\/h3>
                  Always start the troubleshooting by checking the Cluster AutoScaler logs.<\/p>
                  Run the following command to get the logs.<\/p>
                  kubectl logs deployment\/cluster-autoscaler -n kube-system<\/code><\/pre>
                  Cluster AutoScaler does not detect Node Group Nodes<\/h3>
                  Let’s say you have multiple node groups, and the Cluster AutoScaler is running, but the Cluster AutoScaler does not detect the node group nodes.<\/p>
                  The following things may be the issue:<\/p>
                  1. The Cluster AutoScaler doesn’t have the required permission.<\/li>
                  2. The ASGs of the node groups have incorrect tags.<\/li>
                  3. Only in auto-discovery mode the node groups will be detected by the Cluster AutoScaler automatically. If you are using manual mode, you have to specify each node group using the --nodes<\/code> flag.<\/li><\/ol>
                    Pod Stuck in Pending State<\/h3>
                    If your pod has been stuck in a pending state for more than 10 minutes, and the nodes are not scaling up even though the Cluster Autoscaler is running.<\/p>
                    This may be caused by various reasons:<\/p>
                    1. The Cluster AutoScaler doesn’t have the required permission to trigger scaling.<\/li>
                    2. The node group size limit has been reached.<\/li>
                    3. The pods may have taints to deploy in specific nodes.<\/li><\/ol>
                      Nodes not Scaling Down<\/h3>
                      If your nodes are underutilized and still not scaling down, this may be caused by:<\/p>
                      1. Node groups minimum node limit has been reached.<\/li>
                      2. A node might have pods that cannot be evicted.<\/li><\/ol>
                        Cluster AutoScaler Pod gets Evicted<\/h3>
                        If your Cluster AutoScaler pod is getting evicted, you have to add the cluster-autoscaler.kubernetes.io\/safe-to-evict=\"false\"<\/code> annotation to your Cluster AutoScaler deployment.<\/p>
                        Run the following command to add the annotation to the Cluster AutoScaler deployment.<\/p>
                        kubectl -n kube-system annotate deployment.apps\/cluster-autoscaler cluster-autoscaler.kubernetes.io\/safe-to-evict=\"false\"<\/code><\/pre>
                        Then, restart the deployment to apply the changes.<\/p>
                        kubectl rollout restart deploy cluster-autoscaler -n kube-system<\/code><\/pre>
                        Best Practises<\/h2>
                        Given below are some of the best practices for Cluster AutoScaler:<\/p>
                        1. Always specify resource requests and limits for your pods so that the Cluster AutoScaler can scale based on the requirements.<\/li>
                        2. You can use taints and tolerations to schedule some pods on specific nodes.<\/li>
                        3. Use the scale-down commands to adjust the scale-down time based on your workload. (eg. –scale-down-unneeded-time=2m).<\/li>
                        4. Use HPA with Cluster AutoScaler, which makes sure HPA has enough nodes to scale pods.<\/li><\/ol>
                          Conclusion<\/h2>
                          In this guide, you learned about Kubernetes Cluster Autoscaler, its functionality, and how to set it up on an Amazon EKS cluster. <\/p>
                          You also explored testing the setup, customization options, best practices, and troubleshooting common issues.<\/p>
                          For more advanced scaling strategies, especially for workloads requiring different instance types and smarter scaling decisions, consider exploring EKS Karpenter.<\/p>\n
                          Ngu\u1ed3n:<\/strong> Cluster AutoScaler Setup on AWS EKS: A Comprehensive Guide \u2014 DevOpsCube<\/a><\/p>","protected":false},"excerpt":{"rendered":"
                          Source: https:\/\/devopscube.com\/cluster-autoscaler\/<\/p>\n","protected":false},"author":1,"featured_media":539,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-538","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops"],"_links":{"self":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/538","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=538"}],"version-history":[{"count":0,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/538\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/media\/539"}],"wp:attachment":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=538"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=538"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=538"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}