{"id":880,"date":"2024-07-17T01:41:00","date_gmt":"2024-07-17T01:41:00","guid":{"rendered":"https:\/\/blog.ngocha.biz\/?p=880"},"modified":"2024-07-17T01:41:00","modified_gmt":"2024-07-17T01:41:00","slug":"setup-highly-available-jenkins","status":"publish","type":"post","link":"https:\/\/blog.ngocha.biz\/?p=880","title":{"rendered":"How To Setup Highly Available Jenkins"},"content":{"rendered":"

This blog will discuss setting up a Highly available Jenkins<\/strong> using the open-source version.<\/p>\n

Before getting into the high availability discussion, you first need to understand how Jenkins manages its data and the high-level architecture.<\/p>\n

If you are not aware of the architecture, I highly suggest reading Jenkins architecture <\/a>post.<\/p>\n

Types of Jenkins HA Setup<\/h2>\n

There is no direct way of setting up Jenkins in High availability mode due to how Jenkins manages its data.<\/p>\n

Jenkins manages its data in a flat-file (xml)<\/strong>, and you can have only one instance of the master node that can read the Jenkins data.<\/p>\n

Because of which, running multiple replicas of the Jenkins master in an active-active<\/strong> configuration (where all replicas are actively processing jobs) is not officially supported<\/p>\n

For example, if you try to configure multiple master nodes with a shared volume, you will have inconsistent reads and writes that lead to unstable Jenkins.<\/p>\n

There are two ways you can have Jenkins in HA mode.<\/p>\n

    \n
  1. Jenkins active-actice\/active-passive setup: <\/strong>Only enterprise Jenkin<\/a>s comes with a supported plugin to have this setup. But most organizations use open-source versions, and his option is out of scope. Also, there is an option with HAproxy, however, I think it’s a bit of administrative overhead.<\/li>\n
  2. Jenkins HA setup in autoscaling Group: <\/strong>Having Jenkins in the autoscaling group is a workaround to have a highly available Jenkins. It is not a 100% HA solution; however, for any reason if your Jenkins server crasher, another instance will come up within a short period. This will work on both VM and Kubernetes based Jenkins setup<\/li>\n<\/ol>\n

    Jenkins HA Setup With Autoscaling Group<\/strong><\/h2>\n

    You can call this method a poor man’s solution for Jenkins HA.<\/strong> Because, when an instance goes down, there are a few minutes of downtime for another instance to come up (vm + java service startup time). But it works \ud83d\ude42<\/p>\n

    This method will work on any private\/public cloud platform with the option of autoscaling.<\/p>\n

    This method is similar to deploying Jenkins on Kubernetes<\/a>, where if the Jenkins pod goes down, another pod will come up with the same data.<\/p>\n

    Instead of a pod, it’s a VM., so the startup time takes more time than spinning up a new pod.<\/p>\n

    Let’s look at the high-level architecture of the Jenkins HA setup using an autoscaling group.<\/p>\n

    \"Jenkins
    Click to view in HD<\/span><\/figcaption><\/figure>\n

    Here the concept is pretty simple.<\/p>\n

      \n
    1. Jenkins is deployed in an autoscaling group with min and max one count. It ensures we have only one instance of Jenkins running all the time.<\/li>\n
    2. Then, we have a dedicated external disk to hold the Jenkins data. It could be a separate volume or an NFS share. The autoscaling image startup script (AMI user data) gracefully attaches this disk when it comes up.<\/li>\n
    3. If the active Jenkins instance goes down, the autoscaling policies will bring up another instance while terminative the inactive one. In this process, the data disk gets detached from the terminating instance gracefully (in AWS using lifecycle hooks) and gets attached to the new instance, preserving the previous Jenkins state and data.<\/li>\n
    4. All the existing jobs will fail during downtime or continue when a new instance comes up.<\/li>\n<\/ol>\n

      This setup also makes the upgrade and patching process so easy.<\/p>\n

      HA Setup<\/strong> on Kubernetes<\/h2>\n

      When deployed in Kubernetes, It should be deployed via StatefulSet. Also, wee need to keep the StatefulSet replica as one and the Jenkins data directory should be mounted to the a Persistent Volume<\/strong> to store Jenkins data (e.g., job history, plugins, configurations).<\/p>\n

      This ensures data persists even if the Jenkins master pod is rescheduled or restarted.<\/p>\n

      Horizontal Scaling For Agents<\/h2>\n

      A good news is, you can scale Jenkins agents horizontally.<\/p>\n

      There are several ways to scale Jenkins agents. In this method, the agents are dynamically provisioned and terminated once the build is completed.<\/p>\n

      For example,<\/p>\n

        \n
      1. Scaling Agents with Kubernetes pods<\/a><\/li>\n
      2. Scaling Agents with Docker<\/a><\/li>\n
      3. Scaling Agents with ec2 instances.<\/li>\n<\/ol>\n

        Jenkins HA Setup<\/h2>\n

        If you want to try this setup on the AWS cloud, you can use my Documentation.<\/p>\n

        I have documented the whole setup for AWS here -> Jenkins HA Setup Using AWS Autoscaling Group<\/a><\/p>\n

        The IaC<\/a> code for the setup is present in the DevOps Projects<\/a> Githup Repo<\/p>\n

        https:\/\/github.com\/techiescamp\/devops-projects<\/code><\/pre>\n
        Do try out this setup and see if it works for you.<\/p>\n
        Also, you should backup jenkins data<\/a> periodically using plugins and disk snapshots. You can easily restore a working version if you have all the data backup.<\/p>\n
        If you are new to Jenkins, check out our Jenkins tutorial for beginner<\/a>s.<\/p>\n
        \n
        Ngu\u1ed3n:<\/strong> How To Setup Highly Available Jenkins \u2014 DevOpsCube<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
        Source: https:\/\/devopscube.com\/setup-highly-available-jenkins\/<\/p>\n","protected":false},"author":1,"featured_media":881,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-880","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops"],"_links":{"self":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/880","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=880"}],"version-history":[{"count":0,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/880\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/media\/881"}],"wp:attachment":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=880"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=880"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=880"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}