{"id":570,"date":"2025-05-02T15:38:34","date_gmt":"2025-05-02T15:38:34","guid":{"rendered":"https:\/\/blog.ngocha.biz\/?p=570"},"modified":"2025-05-02T15:38:34","modified_gmt":"2025-05-02T15:38:34","slug":"argocd-metadata-annotations-too-long","status":"publish","type":"post","link":"https:\/\/blog.ngocha.biz\/?p=570","title":{"rendered":"Metadata annotations Too long: must have at most 262144 bytes"},"content":{"rendered":"<p>In this blog I have added the solution for ArgoCD <code>Metadata.annotations: Too long<\/code> Error.<\/p>\n<h2 id=\"issueannotation-exceeding-limit\">Issue &#8211; Annotation Exceeding Limit<\/h2>\n<p>When I tried to deploy the <a href=\"https:\/\/devopscube.com\/setup-prometheus-helm-chart\/\" rel=\"noreferrer\">Prometheus-stack helm chart<\/a> using <a href=\"https:\/\/devopscube.com\/argo-cd-ultimate-guide\/\" rel=\"noreferrer\">ArgoCD<\/a>, I faced the following error.<\/p>\n<pre><code class=\"language-bash\">metadata.annotations: Too long: must have at most 262144 bytes<\/code><\/pre>\n<p>This issue occurs because the size of the value of the annotation exceeds the <a href=\"https:\/\/devopscube.com\/kubernetes-tutorials-beginners\/\" rel=\"noreferrer\"><strong>Kubernetes<\/strong><\/a><strong> 256\u202fKiB (262\u202f144\u202fbytes) limit. <\/strong>(Refer the <a href=\"https:\/\/github.com\/kubernetes\/kubernetes\/blob\/master\/staging\/src\/k8s.io\/apimachinery\/pkg\/api\/validation\/objectmeta.go?ref=devopscube.com#L44-L67\" rel=\"noreferrer\">code here<\/a>)<\/p>\n<p>That is a very good limit for storing many annotation. But here is why it exceeds in ArgoCD.<\/p>\n<p>When ArgoCD applies the YAML, it stores a copy inside the object itself under <code>kubectl.kubernetes.io\/last-applied-configuration<\/code> annotation. That hidden copy counts toward the 256\u202fKB limit on <code>metadata.annotations<\/code><\/p>\n<p>\u202fSo, if the object itself is large (big CRDs, Prometheus, Grafana\u2011dashboard ConfigMaps, etc.), the annotation pushes the total size over 256\u202fKiB and the API server rejects the object.<\/p>\n<p>You won&#8217;t face this issue when you try to deploy the helm chart directly using <code>helm install<\/code> or <code>kubectl apply<\/code><\/p>\n<h2 id=\"solutionusing-serversideapply-ssa\">Solution &#8211; Using ServerSideApply (SSA)<\/h2>\n<p>To solve this issue, you can use the <code>ServerSideApply<\/code> option available on ArgoCD.<\/p>\n<p>This option tells Argo CD to let the <a href=\"https:\/\/devopscube.com\/kubernetes-architecture-explained\/#1-kube-apiserver\" rel=\"noreferrer\">Kubernetes API server<\/a> take care of the apply instead of storing the configuration in annotations. Meaning, SSA offloads the &#8220;apply&#8221; logic and state tracking (moving it from annotations to <code>managedFields<\/code>) to the Kubernetes API server itself<\/p>\n<p>If you are using the <code>declarative method<\/code> for deploying the application, you can specify the <code>ServerSideApply<\/code> option under the <code>syncPolicy<\/code> as given in the example below.<\/p>\n<pre><code class=\"language-yaml\">apiVersion: argoproj.io\/v1alpha1\nkind: Application\nmetadata:\n  name: prometheus\n  namespace: argocd\nspec:\n  destination:\n    name: in-cluster\n    namespace: monitoring\n  project: default\n  source:\n    repoURL: '&lt;your-repo&gt;'\n    targetRevision: HEAD\n    path: kube-prometheus-stack\n    helm:\n      valueFiles:\n      - values.yaml\n  syncPolicy:\n    syncOptions:\n    - CreateNamespace=true\n    - ServerSideApply=true <\/code><\/pre>\n<p>If you are deploying using <code>ArgoCD UI<\/code>, enable the <code>Server-Side Apply<\/code> option under the <code>SYNC OPTIONS<\/code> during the configuration.<\/p>\n<figure class=\"kg-card kg-image-card\"><img decoding=\"async\" src=\"https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/2025\/05\/image.png\" class=\"kg-image\" alt=\"\" loading=\"lazy\" width=\"761\" height=\"220\" srcset=\"https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/size\/w600\/2025\/05\/image.png 600w, https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/2025\/05\/image.png 761w\" sizes=\"auto, (min-width: 720px) 720px\"><\/figure>\n<div class=\"kg-card kg-callout-card kg-callout-card-blue\">\n<div class=\"kg-callout-emoji\">\ud83d\udca1<\/div>\n<div class=\"kg-callout-text\"><a href=\"https:\/\/kubernetes.io\/docs\/reference\/using-api\/server-side-apply\/?ref=devopscube.com\" rel=\"noreferrer\">Server-Side Apply<\/a> (SSA) is a native feature of Kubernetes.<\/p>\n<p>Traditionally, <code spellcheck=\"false\" style=\"white-space: pre-wrap;\">kubectl apply<\/code> uses client-side logic. This means your local <code spellcheck=\"false\" style=\"white-space: pre-wrap;\">kubectl<\/code> fetches the current state of a resource from the cluster, compares it to the desired state defined in your YAML file, calculates the differences locally, and then sends an update to the API server.<\/p>\n<p>With SSA, that entire process shifts to the server. You send the complete desired YAML to the API server, and it takes care of comparing the current and desired states. It also keeps track of which client owns which fields in a resource &#8211; a concept known as <b><strong style=\"white-space: pre-wrap;\">field ownership<\/strong><\/b>. In our case, ArgoCD is the owner managing these fields.<\/div>\n<\/div>\n<hr>\n<p><strong>Ngu\u1ed3n:<\/strong> <a href=\"https:\/\/devopscube.com\/argocd-metadata-annotations-too-long\/\" target=\"_blank\" rel=\"noopener noreferrer\">Metadata annotations Too long: must have at most 262144 bytes \u2014 DevOpsCube<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Source: https:\/\/devopscube.com\/argocd-metadata-annotations-too-long\/<\/p>\n","protected":false},"author":1,"featured_media":571,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-570","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops"],"_links":{"self":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/570","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=570"}],"version-history":[{"count":0,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/570\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/media\/571"}],"wp:attachment":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=570"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=570"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=570"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}