{"id":940,"date":"2023-08-28T01:47:00","date_gmt":"2023-08-28T01:47:00","guid":{"rendered":"https:\/\/blog.ngocha.biz\/?p=940"},"modified":"2023-08-28T01:47:00","modified_gmt":"2023-08-28T01:47:00","slug":"service-discovery-example","status":"publish","type":"post","link":"https:\/\/blog.ngocha.biz\/?p=940","title":{"rendered":"Consul Service Discovery Example [Practical Guide]"},"content":{"rendered":"<p>In this blog, you will learn about the consul <a href=\"https:\/\/devopscube.com\/service-discovery-explained\/\" rel=\"noreferrer noopener\">service discovery<\/a> concept practically by setting up Nginx service discovery using Hashicorp Consul.<\/p><p><a href=\"https:\/\/devopscube.com\/service-discovery-explained\/\">Service discovery<\/a> is a concept that every DevOps engineer should know. Primarily in microservices, service discovery plays a key role.<\/p><p>Also, to <a href=\"https:\/\/devopscube.com\/learn-kubernetes-complete-roadmap\/\">learn Kubernetes<\/a>, knowledge of service discovery is a prerequisite because the <a href=\"https:\/\/devopscube.com\/kubernetes-architecture-explained\/\">Kubernetes architecture<\/a> includes a service discovery component called etcd.<\/p><h2 id=\"setup-prerequisites\">Setup Prerequisites<\/h2><p>Before starting the setup, you will need the following:<\/p>\n<!--kg-card-begin: html-->\n<ol class=\"is-style-cnvs-list-styled\">\n<li>You need four Ubuntu servers. You can use <a href=\"https:\/\/devopscube.com\/vagrant-tutorial-beginners\/\" data-type=\"post\" data-id=\"7566\">Vagrant<\/a> or cloud servers for the setup.\n<ul class=\"is-style-default wp-block-list\">\n<li>Consul Server<\/li>\n\n\n<li>Load Balancer<\/li>\n\n\n<li>Two Backends for Load Balancer<\/li>\n<\/ul>\n<\/li>\n\n\n<li>If you are using cloud servers, allow the required ports in the security group. Check this link for the <a href=\"https:\/\/developer.hashicorp.com\/consul\/docs\/install\/ports?ref=devopscube.com#required-ports\" data-type=\"URL\" data-id=\"https:\/\/developer.hashicorp.com\/consul\/docs\/install\/ports#required-ports\" target=\"_blank\" rel=\"noreferrer noopener\">required ports for Consul<\/a>.<\/li>\n<\/ol>\n<!--kg-card-end: html-->\n<h2 id=\"setup-architecture\">Setup Architecture<\/h2><figure class=\"kg-card kg-image-card\"><img decoding=\"async\" src=\"https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/2025\/03\/image-13-29.png\" class=\"kg-image\" alt=\"Nginx Service Discovery Using Consul\" loading=\"lazy\" width=\"2000\" height=\"1654\" srcset=\"https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/size\/w600\/2025\/03\/image-13-29.png 600w, https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/size\/w1000\/2025\/03\/image-13-29.png 1000w, https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/size\/w1600\/2025\/03\/image-13-29.png 1600w, https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/2025\/03\/image-13-29.png 2147w\" sizes=\"auto, (min-width: 720px) 720px\"><\/figure><h2 id=\"setup-consul-server\">Setup Consul Server<\/h2><p>We will be running a single-node Consul server for this setup. For production use cases, the consul should run a <a href=\"https:\/\/devopscube.com\/setup-consul-cluster-guide\/\">cluster<\/a> with a minimum of three nodes.<\/p><p><strong>Step 1:<\/strong> Log in to the consul server node and update the apt cache.<\/p><pre><code>sudo apt-get update -y<\/code><\/pre><p><strong>Step 2: <\/strong>Go to the <a href=\"https:\/\/developer.hashicorp.com\/consul\/downloads?ref=devopscube.com\" rel=\"noreferrer noopener\">downloads page<\/a> and get the installation command or execute the following commands to install consul.<\/p><pre><code>wget -O- https:\/\/apt.releases.hashicorp.com\/gpg | gpg --dearmor | sudo tee \/usr\/share\/keyrings\/hashicorp-archive-keyring.gpg\n\necho \"deb [signed-by=\/usr\/share\/keyrings\/hashicorp-archive-keyring.gpg] https:\/\/apt.releases.hashicorp.com $(lsb_release -cs) main\" | sudo tee \/etc\/apt\/sources.list.d\/hashicorp.list\n\nsudo apt update &amp;&amp; sudo apt install consul<\/code><\/pre><p><strong>Step 3:<\/strong> Verify the installation by checking the consul version.<\/p><pre><code>consul --version<\/code><\/pre><p><strong>Step 4:<\/strong> All the consul server configs are present in the <code>\/etc\/consul.d<\/code> folder. Let&#8217;s move the default <strong><code>consul.hcl<\/code><\/strong> config file to a file named <strong><code>consul.hcl.back<\/code><\/strong><\/p><pre><code>sudo mv \/etc\/consul.d\/consul.hcl \/etc\/consul.d\/consul.hcl.back<\/code><\/pre><p><strong>Step 5:<\/strong> Create a new file named consul.hcl<\/p><pre><code>sudo vi \/etc\/consul.d\/consul.hcl<\/code><\/pre><p>Add the following content and save the file. You can create your own encrypted key using the command <code>consul keygen<\/code> and replace the highlighted one.<\/p><pre><code>\"bind_addr\" = \"0.0.0.0\"\n\"client_addr\" = \"0.0.0.0\"\n\"data_dir\" = \"\/var\/consul\"\n\"encrypt\" = \"ZENZNrsXU336Uma+S4XUj9sxvICj32N7XdEzrbYbRpY=\"\n\"datacenter\" = \"dc1\"\n\"ui\" = true\n\"server\" = true\n\"log_level\" = \"INFO\"<\/code><\/pre><p>Here is the explanation for the configuration.<\/p><ol><li><strong>bind_addr<\/strong>: This specifies the IP address bound to all available network interfaces.<\/li><li><strong>client_addr<\/strong>: This specifies the IP address on which the Consul API should listen.<\/li><li><strong>data_dir<\/strong>: This specifies the directory in which the Consul should store its data.<\/li><li><strong>encrypt<\/strong>: This specifies the encryption key to use to encrypt communications between Consul agents. This key is used to encrypt all network traffic between Consul agents and must be kept secret.<\/li><li><strong>datacenter<\/strong>: This specifies the datacenter name for this Consul agent. A data center is a logical grouping of consul agent nodes within a network.<\/li><li><strong>ui<\/strong>: This enables the built-in Consul UI, which provides a web-based interface for interacting with Consul.<\/li><li><strong>server<\/strong>: This specifies that this server should run as a Consul server. Because the consul agent is also of the same binary. In agents, we specify the server as false.<\/li><li><strong>log_level<\/strong>: This sets the logging level for the Consul agent. In this case, only logs with a level of &#8220;INFO&#8221; or higher will be displayed.<\/li><\/ol><p><strong>Step 6: <\/strong>Start the consul server in the background using the following command. We are using the <code>-dev<\/code> flag to specify that we are running a single server in dev mode.<\/p><pre><code>sudo nohup consul agent -dev -config-dir \/etc\/consul.d\/ &amp;<\/code><\/pre><p>You can verify the consul server status using the following command.<\/p><pre><code> consul members<\/code><\/pre><p>Step 6: If you visit the &lt;server IP:8500&gt; you should be able to access the consul dashboard.<\/p><pre><code> http:\/\/&lt;server-ip-address-server&gt;:8500<\/code><\/pre><figure class=\"kg-card kg-image-card\"><img decoding=\"async\" src=\"https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/2025\/03\/image-14-39.png\" class=\"kg-image\" alt=\"Consul server UI\" loading=\"lazy\" width=\"1019\" height=\"521\" srcset=\"https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/size\/w600\/2025\/03\/image-14-39.png 600w, https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/size\/w1000\/2025\/03\/image-14-39.png 1000w, https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/2025\/03\/image-14-39.png 1019w\" sizes=\"auto, (min-width: 720px) 720px\"><\/figure><h2 id=\"set-up-backend-servers\">Set up Backend Servers<\/h2><p>Now that we have the consul server, we can configure the Nignx backend servers and register the node and Nginx service to the consul server (Service registry).<\/p><p>We need to set up Nginx and the consul agent in all the backend nodes. The consul agent is responsible for connecting to the consul server and registering the server details and Nginx service details.<\/p><p>Do the below configuration in<strong> both backend servers<\/strong>.<\/p><p><strong>Step 1:<\/strong> Log in to the backend instances and update the package information.<\/p><pre><code>sudo apt-get update -y<\/code><\/pre><p><strong>Step 2:<\/strong> Install Nginx in both instances by running the command.<\/p><pre><code>sudo apt install nginx -y<\/code><\/pre><p><strong>Step 2:<\/strong> Once Nginx is installed, we need to go to the default HTML location and edit the <strong><code>index.html<\/code> <\/strong>file in both servers to differentiate the servers.<\/p><p>Use the following command to go to the directory and edit the file.<\/p><pre><code>cd \/var\/www\/html\n\nsudo vi index.html<\/code><\/pre><p>Copy the below HTML file to the index.html file. On the second server, replace <strong>SERVER-01<\/strong> with <strong>SERVER-0<\/strong>2 in the HTML file to differentiate between the two backend servers.<\/p><pre><code>&lt;!DOCTYPE html&gt;\n&lt;html&gt;\n&lt;head&gt;\n\t&lt;title&gt;Backend Server &lt;\/title&gt;\n&lt;\/head&gt;\n&lt;body&gt;\n\t&lt;h1&gt;This is Backend SERVER-01&lt;\/h1&gt;\n&lt;\/body&gt;\n&lt;\/html&gt;\n<\/code><\/pre><p><strong>Step 3: <\/strong>Install consul as an agent in the servers. To install consul run the following commands.<\/p><pre><code>wget -O- https:\/\/apt.releases.hashicorp.com\/gpg | gpg --dearmor | sudo tee \/usr\/share\/keyrings\/hashicorp-archive-keyring.gpg\n\necho \"deb [signed-by=\/usr\/share\/keyrings\/hashicorp-archive-keyring.gpg] https:\/\/apt.releases.hashicorp.com $(lsb_release -cs) main\" | sudo tee \/etc\/apt\/sources.list.d\/hashicorp.list\n\nsudo apt update &amp;&amp; sudo apt install consul<\/code><\/pre><p>Add check if it is installed properly by running the following command.<\/p><pre><code>consul -version<\/code><\/pre><p><strong>Step 4:<\/strong> We need to replace the default consul configuration file <strong><code>config.hcl<\/code><\/strong> inside the location <strong><code>\/etc\/consul.d<\/code><\/strong> with our <strong><code>consul.hcl<\/code><\/strong> file.<\/p><p>Rename the default file and create a new file using the following commands.<\/p><pre><code>sudo mv \/etc\/consul.d\/consul.hcl \/etc\/consul.d\/consul.hcl.back\n\nsudo vi \/etc\/consul.d\/consul.hcl<\/code><\/pre><p>Add the following contents to the file. If you have used a different encrypt key in the server, you need to replace it with your <strong><code>encrypt<\/code><\/strong> key. Also, replace <strong><code>34.222.141.217<\/code><\/strong> with your consul server IP address.<\/p><pre><code>\"server\" = false\n\"datacenter\" = \"dc1\"\n\"data_dir\" = \"\/var\/consul\"\n\"encrypt\" = \"ZENZNrsXU336Uma+S4XUj9sxvICj32N7XdEzrbYbRpY=\"\n\"log_level\" = \"INFO\"\n\"enable_script_checks\" = true\n\"enable_syslog\" = true\n\"leave_on_terminate\" = true\n\"start_join\" = [\"34.222.141.217\"]<\/code><\/pre><p>The above configuration is used by the consul agent to register the node information to the consul server.<\/p><p><strong>Step 5: <\/strong>Now we need to create a <strong><code>backend.hcl<\/code><\/strong> config file in  <strong><code>\/etc\/consul.d<\/code><\/strong> directory to register the Nginx service and health check URLs with the consul server. So that the consul server will constantly monitor the Nginx service health.<\/p><pre><code>sudo vi \/etc\/consul.d\/backend.hcl<\/code><\/pre><p>Add the following contents to the file and save it.<\/p><pre><code>\"service\" = {\n  \"Name\" = \"backend\"\n  \"Port\" = 80\n  \"check\" = {\n    \"args\" = [\"curl\", \"localhost\"]\n    \"interval\" = \"3s\"\n  }\n}<\/code><\/pre><p>This file registers your backend servers to the consul server.<\/p><p>Now you can validate the configs by running the following command.<\/p><pre><code>consul validate \/etc\/consul.d<\/code><\/pre><p><strong>Step 6:<\/strong> After completing every configuration start the consul agent using the command.<\/p><pre><code>sudo nohup consul agent -config-dir \/etc\/consul.d\/ &amp;<\/code><\/pre><p>Now check if is working by visiting your consul UI. If the backend shows in your UI as shown in the gif image, it means the backend has registered itself successfully.<\/p><figure class=\"kg-card kg-image-card\"><img decoding=\"async\" src=\"https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/2025\/03\/consul-1.gif\" class=\"kg-image\" alt=\"Consul registered backends in UI.\" loading=\"lazy\" width=\"971\" height=\"640\" srcset=\"https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/size\/w600\/2025\/03\/consul-1.gif 600w, https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/2025\/03\/consul-1.gif 971w\" sizes=\"auto, (min-width: 720px) 720px\"><\/figure><h2 id=\"setup-load-balancer\">Setup Load-Balancer<\/h2><p>The next step is to configure the Load Balancer that updates its backend server details from the consul server (service registry).<\/p><blockquote>To fetch the backend server details we will use <strong>consul-template<\/strong> binary. It is responsible for making API calls to the consul server and getting the backend server details. It then substitutes the values using a template and generates the <strong><code>loadbalancer.conf<\/code><\/strong> file used by Nginx.<\/blockquote><p><strong>Step 1:<\/strong> Log in to the load-balancer server. Update the package information and install unzip.<\/p><pre><code>sudo apt-get update -y\nsudo apt-get install unzip -y<\/code><\/pre><p><strong>Step 2:<\/strong> Install Nginx using the following command.<\/p><pre><code>sudo apt install nginx -y<\/code><\/pre><p><strong>Step 2:<\/strong> Now download the <strong>consul-template<\/strong> binary using the command given below.<\/p><pre><code>sudo curl -L  https:\/\/releases.hashicorp.com\/consul-template\/0.30.0\/consul-template_0.30.0_linux_amd64.zip -o \/opt\/consul-template.zip\n\nsudo unzip \/opt\/consul-template.zip -d  \/usr\/local\/bin\/<\/code><\/pre><p>Verify the installation by checking the version.<\/p><pre><code>consul-template --version<\/code><\/pre><p><strong>Step 3:<\/strong> Create a file named <code>load-balancer.conf.ctmpl<\/code> inside the location<code> \/etc\/nginx\/conf.d<\/code><\/p><pre><code>sudo vi \/etc\/nginx\/conf.d\/load-balancer.conf.ctmpl<\/code><\/pre><p>Copy the following contents to the file. Here the block highlighted in bold is the go template that iterates over the <code>backend<\/code>  service in the consul server and gets all the IP addresses of registered backends.<\/p><pre><code>upstream backend {\n {{- range service \"backend\" }} \n  server {{ .Address }}:{{ .Port }}; \n {{- end }} \n}\n\nserver {\n   listen 80;\n\n   location \/ {\n      proxy_pass http:\/\/backend;\n   }\n}<\/code><\/pre><p><strong>Step 4: <\/strong>Now create a <strong><code>consul-template.hcl<\/code><\/strong> file in the same location. consul-template uses this config file to get details about the consul server IP, and the destination to copy the substituted <strong><code>load-balancer.conf<\/code><\/strong> file.<\/p><pre><code>sudo vi \/etc\/nginx\/conf.d\/consul-template.hcl<\/code><\/pre><p>Add the following contents to the file. Replace <strong>34.222.141.217<\/strong> with your consul server IP. The source file is the consul template file with the go template and the destination file is the rendered Nginx configuration from the template.<\/p><pre><code>consul {\n address = \"34.222.141.217:8500\"\n\n retry {\n   enabled  = true\n   attempts = 12\n   backoff  = \"250ms\"\n }\n}\ntemplate {\n source      = \"\/etc\/nginx\/conf.d\/load-balancer.conf.ctmpl\"\n destination = \"\/etc\/nginx\/conf.d\/load-balancer.conf\"\n perms       = 0600\n command = \"service nginx reload\"\n}<\/code><\/pre><p><strong>Step 5:<\/strong> Now disable the default server configuration by deleting the default file, using the command.<\/p><pre><code> sudo rm \/etc\/nginx\/sites-enabled\/default<\/code><\/pre><p>We need to remove this file to avoid conflict with the server configuration because this file contains the default server configuration.<\/p><p>Restart Nginx to make the changes, and run the following command to restart Nginx.<\/p><pre><code>sudo systemctl restart nginx<\/code><\/pre><p><strong>Step 6:<\/strong> After finishing the configurations start the consul agent template using the command. Consul Template starts a daemon that continuously monitors the Consul server key\/value store for changes.<\/p><pre><code>sudo nohup consul-template -config=\/etc\/nginx\/conf.d\/consul-template.hcl &amp;<\/code><\/pre><p>After successful execution, you will find a <strong><code>load-balancer.conf <\/code><\/strong>file with backend values updated as shown below. The server IPs were retrieved by the consul template from the consul server.<\/p><pre><code>upstream backend {\n  server 172.31.30.119:80;\n  server 172.31.31.173:80;\n}\n\nserver {\n   listen 80;\n\n   location \/ {\n      proxy_pass http:\/\/backend;\n   }\n}<\/code><\/pre><p>Now if you try accessing the load balancer IP, it will show the custom HTML of the backend server. If you refresh the page, it will show the HTML page of the second server as the default algorithm is round-robin.<\/p><h2 id=\"testing-service-discovery\">Testing Service Discovery<\/h2><p>Now that everything is set up and running, test it by watching what happens when you stop one of your backend servers.<\/p><p>Stop one of the backend servers. The Consul server will mark the stopped backend as unhealthy and the health check will fail.<\/p><figure class=\"kg-card kg-image-card\"><img decoding=\"async\" src=\"https:\/\/storage.ghost.io\/c\/5f\/2f\/5f2f4d20-2abf-4534-8d40-7aa233aedd43\/content\/images\/2025\/03\/image-15-28.png\" class=\"kg-image\" alt=\"\" loading=\"lazy\" width=\"588\" height=\"396\"><\/figure><p>The Consul template sends an immediate signal that one of the servers is down, and it automatically creates a new configuration file with only one server.<\/p><p>The consul template running in the load-balancer node notices the failed backend as it continuously monitors the key values in the consul server.<\/p><p>It then updates  \/etc\/nginx\/conf.d\/load-balancer.conf with just one backend IP and gracefully reloads the Nginx service.<\/p><p>This workflow ensures that only health backends are always updated in the Nginx <strong><code>load-balancer.conf<\/code><\/strong> file.<\/p><h2 id=\"automating-the-setup\">Automating the Setup<\/h2><p>The automation (<a href=\"https:\/\/devopscube.com\/infrastructure-as-code-configuration-management\/\">IaC<\/a>) code for this project using Terraform and Ansible on AWS is hosted in the <a href=\"https:\/\/github.com\/techiescamp\/devops-projects\/tree\/main\/02-consul-sevice-discovery?ref=devopscube.com\" rel=\"noreferrer\">DevOps Projects Github repository<\/a>.<\/p><p>You can use that code to automate the whole setup explained in his guide.<\/p>\n<hr><p><strong>Ngu\u1ed3n:<\/strong> <a href=\"https:\/\/devopscube.com\/service-discovery-example\/\" target=\"_blank\" rel=\"noopener noreferrer\">Consul Service Discovery Example [Practical Guide] \u2014 DevOpsCube<\/a><\/p>","protected":false},"excerpt":{"rendered":"<p>Source: https:\/\/devopscube.com\/service-discovery-example\/<\/p>\n","protected":false},"author":1,"featured_media":941,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-940","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops"],"_links":{"self":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/940","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=940"}],"version-history":[{"count":0,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/940\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/media\/941"}],"wp:attachment":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=940"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=940"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=940"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}