{"id":854,"date":"2022-01-27T02:46:00","date_gmt":"2022-01-27T02:46:00","guid":{"rendered":"https:\/\/blog.ngocha.biz\/?p=854"},"modified":"2022-01-27T02:46:00","modified_gmt":"2022-01-27T02:46:00","slug":"alert-manager-kubernetes-guide","status":"publish","type":"post","link":"https:\/\/blog.ngocha.biz\/?p=854","title":{"rendered":"Setting Up Alert Manager on Kubernetes – Beginners Guide"},"content":{"rendered":"

AlertManager is an open-source alerting system that works with the Prometheus Monitoring system. This blog is part of the Prometheus Kubernetes tutorial series.<\/p>\n

In our previous posts, we have looked at the following.<\/p>\n

    \n
  1. Setup Prometheus on Kubernetes<\/a><\/li>\n
  2. Setup Kube State Metrics<\/a><\/li>\n<\/ol>\n

    In this guide, I will cover the Alert Manager setup and its integration with Prometheus.<\/p>\n

    \n
    Note:<\/strong><\/b> In this guide, all the Alert Manager Kubernetes objects will be created inside a namespace called monitoring<\/strong><\/b>. If you use a different namespace, you can replace it in the YAML files.<\/div>\n<\/div>\n

    Alertmanager on Kubernetes<\/h2>\n

    Alert Manager setup has the following key configurations.<\/p>\n

      \n
    1. A config map for AlertManager configuration<\/li>\n
    2. A config Map for AlertManager alert templates<\/li>\n
    3. Alert Manager Kubernetes Deployment<\/a><\/li>\n
    4. Alert Manager service to access the web UI.<\/li>\n<\/ol>\n

      Important Setup Notes<\/h2>\n

      You should have a working Prometheus setup up and running.<\/p>\n

      Prometheus should have the correct alert manager service endpoint in its config.yaml<\/code> as shown below to send the alert to Alert Manager.<\/p>\n

      \n
      Note:<\/strong><\/b> If you are following my tutorial on Prometheus Setup On Kubernetes<\/a>, you don’t have to add the following configuration because it is part of the Prometheus configmap<\/a>.<\/div>\n<\/div>\n
      alerting:\n   alertmanagers:\n      - scheme: http\n        static_configs:\n        - targets:\n          - \"alertmanager.monitoring.svc:9093\"<\/code><\/pre>\n
      All the alerting rules have to be present on Prometheus config based on your needs. It should be created as part of the Prometheus config map<\/a> with a file named prometheus.rules<\/strong><\/code> and added to the config.yaml<\/code> in the following way.<\/p>\n
      rule_files:\n      - \/etc\/prometheus\/prometheus.rules<\/code><\/pre>\n
      Alert manager alerts can be written based on the metrics you receive on Prometheus.<\/p>\n
      For receiving emails for alerts, you need to have a valid SMTP host<\/strong> in the alert manager config.yaml<\/code> (smarthost parameter). You can customize the email template as per your needs in the Alert Template config map. We have given the generic template in this guide.<\/p>\n
      Let’s get started with the setup.<\/p>\n
      Alertmanager Kubernetes Manifests<\/h2>\n
      All the Kubernetes manifests used in this tutorial can be found in this Github link.<\/p>\n
      Clone the Github repository using the following command.<\/p>\n
      git clone https:\/\/github.com\/bibinwilson\/kubernetes-alert-manager.git<\/code><\/pre>\n
      Config Map for Alert Manager Configuration<\/h2>\n
      Alert Manager reads its configuration from a config.yaml <\/strong>file. It contains the configuration of alert template path, email, and other alert receiving configurations.<\/p>\n
      In this setup, we are using email and slack webhook receivers. You can have a look at all the supported alert receivers from here<\/a>.<\/p>\n
      Create a file named AlertManagerConfigmap.yaml<\/code><\/strong> and copy the following contents.<\/p>\n
      kind: ConfigMap\napiVersion: v1\nmetadata:\n  name: alertmanager-config\n  namespace: monitoring\ndata:\n  config.yml: |-\n    global:\n    templates:\n    - '\/etc\/alertmanager\/*.tmpl'\n    route:\n      receiver: alert-emailer\n      group_by: ['alertname', 'priority']\n      group_wait: 10s\n      repeat_interval: 30m\n      routes:\n        - receiver: slack_demo\n        # Send severity=slack alerts to slack.\n          match:\n            severity: slack\n          group_wait: 10s\n          repeat_interval: 1m\n \n    receivers:\n    - name: alert-emailer\n      email_configs:\n      - to: demo@devopscube.com\n        send_resolved: false\n        from: from-email@email.com\n        smarthost: smtp.eample.com:25\n        require_tls: false\n    - name: slack_demo\n      slack_configs:\n      - api_url: https:\/\/hooks.slack.com\/services\/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sz4Qk\n        channel: '#devopscube-demo'<\/code><\/pre>\n
      Let’s create the config map using kubectl.<\/p>\n
      kubectl create -f AlertManagerConfigmap.yaml<\/code><\/pre>\n
      Config Map for Alert Template<\/h2>\n
      We need alert templates for all the receivers we use (email, Slack, etc). Alert manager will dynamically substitute the values and deliver alerts to the receivers based on the template. You can customize these templates based on your needs.<\/p>\n
      Create a file named AlertTemplateConfigMap.yaml<\/code><\/strong> and copy the contents from this file link ==> Alert Manager Template YAML<\/a><\/p>\n
      Create the configmap using kubectl.<\/p>\n
      kubectl create -f AlertTemplateConfigMap.yaml<\/code><\/pre>\n
      Create a Deployment<\/h2>\n
      In this deployment, we will mount the two config maps we created.<\/p>\n
      Create a file called Deployment.yaml<\/code><\/strong> with the following contents.<\/p>\n
      apiVersion: apps\/v1\nkind: Deployment\nmetadata:\n  name: alertmanager\n  namespace: monitoring\nspec:\n  replicas: 1\n  selector:\n    matchLabels:\n      app: alertmanager\n  template:\n    metadata:\n      name: alertmanager\n      labels:\n        app: alertmanager\n    spec:\n      containers:\n      - name: alertmanager\n        image: prom\/alertmanager:latest\n        args:\n          - \"--config.file=\/etc\/alertmanager\/config.yml\"\n          - \"--storage.path=\/alertmanager\"\n        ports:\n        - name: alertmanager\n          containerPort: 9093\n        resources:\n            requests:\n              cpu: 500m\n              memory: 500M\n            limits:\n              cpu: 1\n              memory: 1Gi\n        volumeMounts:\n        - name: config-volume\n          mountPath: \/etc\/alertmanager\n        - name: templates-volume\n          mountPath: \/etc\/alertmanager-templates\n        - name: alertmanager\n          mountPath: \/alertmanager\n      volumes:\n      - name: config-volume\n        configMap:\n          name: alertmanager-config\n      - name: templates-volume\n        configMap:\n          name: alertmanager-templates\n      - name: alertmanager\n        emptyDir: {}<\/code><\/pre>\n
      Create the alert manager deployment using kubectl.<\/p>\n
      kubectl create -f Deployment.yaml<\/code><\/pre>\n
      Create the Alert Manager Service Endpoint<\/h2>\n
      We need to expose the alert manager using NodePort or Load Balancer just to access the Web UI. Prometheus will talk to the alert manager using the internal service endpoint.<\/p>\n
      Create a Service.yaml<\/code><\/strong> file with the following contents.<\/p>\n
      apiVersion: v1\nkind: Service\nmetadata:\n  name: alertmanager\n  namespace: monitoring\n  annotations:\n      prometheus.io\/scrape: 'true'\n      prometheus.io\/port:   '9093'\nspec:\n  selector: \n    app: alertmanager\n  type: NodePort  \n  ports:\n    - port: 9093\n      targetPort: 9093\n      nodePort: 31000<\/code><\/pre>\n
      Create the service using kubectl.<\/p>\n
      kubectl create -f Service.yaml<\/code><\/pre>\n
      Now, you will be able to access Alert Manager on Node Port 31000<\/code>. For example,<\/p>\n
      http:\/\/35.114.150.153:31000<\/code><\/pre>\n
      \"kubernetes<\/figure>\n
      \n
      Ngu\u1ed3n:<\/strong> Setting Up Alert Manager on Kubernetes – Beginners Guide \u2014 DevOpsCube<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
      Source: https:\/\/devopscube.com\/alert-manager-kubernetes-guide\/<\/p>\n","protected":false},"author":1,"featured_media":855,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-854","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops"],"_links":{"self":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/854","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=854"}],"version-history":[{"count":0,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/854\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/media\/855"}],"wp:attachment":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=854"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=854"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=854"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}