{"id":697,"date":"2016-10-06T12:51:34","date_gmt":"2016-10-06T12:51:34","guid":{"rendered":"https:\/\/blog.ngocha.biz\/?p=697"},"modified":"2016-10-06T12:51:34","modified_gmt":"2016-10-06T12:51:34","slug":"replicated-glusterfs-cluster-aws-ec2","status":"publish","type":"post","link":"https:\/\/blog.ngocha.biz\/?p=697","title":{"rendered":"How to Setup a Replicated GlusterFS Cluster on AWS EC2"},"content":{"rendered":"

GlusterFS<\/a> is one of the best open source distributed file systems. If you want a highly available distributed file system for your applications, GlusterFs is one of the good options.<\/p>\n

Now:<\/strong> AWS offers a managed scalable file storage called Elastic File System<\/a>. If you don’t want the administrative overhead of glusterFS clusters, you can give EFS a try.<\/p>\n

GlusterFS cluster on AWS ec2<\/h3>\n

This guide covers all the necessary steps to setup a GlusterFs cluster using ec2 instances and extra EBS volumes. Here we are setting up a two node cluster, however, you can increase the node count based on your needs.<\/p>\n

Instance Configurations<\/h3>\n

1. Create two instances with extra EBS volumes.<\/p>\n

2. Make sure the instances can talk to each other.<\/p>\n

3. Login to the servers and set the hostnames as node1 and node2 using the following commands.<\/p>\n

hostnamectl set-hostname node1\nhostnamectl set-hostname node2<\/code><\/pre>\n
Run bash<\/code> on the terminal for the hostname to show up.<\/p>\n
4. Make an entry in the \/etc\/hosts<\/code> file with the IP and hostname of both servers as shown below. The IP\u2019s should be resolvable using the hostnames. Change the IP in the following to your server IP\u2019s.<\/p>\n
172.31.21.201 node1\n172.31.21.202 node2<\/code><\/pre>\n
Instance Port Configuration<\/h3>\n
You need to open the following ports int he ec2 security groups as well the server firewall if it is enabled.<\/p>\n
111\n24007 - GlusterFS Daemon. 24008 - GlusterFS Management 38465 to 38467 - GlusterFS NFS service 49152 to n - Depends on number of bricks.<\/code><\/pre>\n
Create Mount Points for EBS Volumes<\/h3>\n
You need to do the following in both the ec2 instances. 1. Format the volume to xfs. <\/p>\n
sudo mkfs -t xfs \/dev\/xvdb<\/code><\/pre>\n
xvdb<\/code> is the name of the EBS volume. You can list the available devices using lsblk<\/code> command.<\/p>\n
2. Create a mount directory named \/gshare<\/code> and mount the formatted volume.<\/p>\n
sudo mkdir \/gshare\nsudo mount \/share \/dev\/xvdb<\/code><\/pre>\n
3. Add the mount to \/etc\/fstab<\/code><\/p>\n
\/dev\/xvdb    \/gshare   xfs   defaults,nofail     0<\/code><\/pre>\n
GlusterFS Installation<\/h3>\n
Perform the following steps on both the servers.<\/p>\n
    \n
  1. Create a GlusterFs repo.<\/li>\n<\/ol>\n
    sudo vi \/etc\/yum.repos.d\/Gluster.repo<\/code><\/pre>\n
    Copy the following to the repo file.<\/p>\n
    [gluster38]\nname=Gluster 3.8\nbaseurl=http:\/\/mirror.centos.org\/centos\/7\/storage\/$basearch\/gluster-3.8\/\ngpgcheck=0\nenabled=1<\/code><\/pre>\n
    2. Install GlusterFS server.<\/p>\n
    sudo yum install glusterfs-server -y<\/code><\/pre>\n
    3. Start and verify the glusterd service.<\/p>\n
    sudo systemctl  start glusterd\nsudo systemctl  status glusterd<\/code><\/pre>\n
    GlusterFs Configuration<\/h3>\n
    1. From node1 execute the following command to create a trusted storage pool with node2. <\/p>\n
    sudo gluster peer probe node2<\/code><\/pre>\n
    After successful execution, you would get peer probe: success.<\/code> as output.<\/p>\n
    2. Check the peer status using the following command.<\/p>\n
    [ec2-user@node1 ~]$ sudo gluster peer status Number of Peers: 1 Hostname: node2 Uuid: 47ee8304-36ea-4b95-9214-4854bc98b737 State: Peer in Cluster (Connected) [ec2-user@node1 ~]$<\/code><\/pre>\n
    3. Create a data<\/code> directory on gshare<\/code> mount on both the servers.<\/p>\n
    sudo mkdir \/gshare\/data<\/code><\/pre>\n
      \n
    1. Create a GlusterFS HA shared volume.<\/li>\n<\/ol>\n
      sudo gluster volume create gdata replica 2 node1:\/gshare\/data node2:\/gshare\/data<\/code><\/pre>\n
      5. Start the gdata<\/code> volume.<\/p>\n
      sudo gluster volume start gdata<\/code><\/pre>\n
      6. By default NFS is disabled. If you want NFS functionality for glusterFs volume, you can enable it using the following command.<\/p>\n
      sudo gluster volume set gdata nfs.disable off<\/code><\/pre>\n
      7. Set the volume permissions for gdata<\/code> volume for client access. Here I am using 172.*<\/code> CIDR. You need to replace it based on your network range.<\/p>\n
      sudo gluster volume set gdata auth.allow \"172.*\"<\/code><\/pre>\n
      8. To get all the info about the volume, execute the following command.<\/p>\n
      sudo gluster volume info gdata<\/code><\/pre>\n
      GlusterFS Client Setup<\/h3>\n
      1. Enable fuse<\/code> kernel module. <\/p>\n
      sudo modprobe fuse2. <\/code><\/pre>\n
        \n
      1. Install all the glusterFS client dependencies. <\/li>\n<\/ol>\n
        sudo yum install fuse fuse-libs openib libibverbs -y<\/code><\/pre>\n
        3. Install the GlusterFS client. <\/p>\n
        sudo yum install  glusterfs-client -y<\/code><\/pre>\n
        GlusterFS Client Configuration<\/h3>\n
        The data will get replicated only if you are writing from a GlusterFS client. You can mount the GlusterFS volume to any number of clients. We highly recommend you to map the gluster nodes to a domain name and use it with the clients for mounting.<\/p>\n
        Note<\/strong> A client machine is not part of the glusterFS cluster. It is the machine in which you want to mount the replicated volume.<\/p>\n
          \n
        1. Create a client mount directory.<\/li>\n<\/ol>\n
          sudo mkdir \/gfdata<\/code><\/pre>\n
          2. Mount gfdata<\/code> directory to the glusterFS replicated volume.<\/p>\n
          sudo mount -t glusterfs node1:\/gdata \/gfdata<\/code><\/pre>\n
          Troubleshooting GlusterFs<\/h3>\n
          1. You can view all the logs of gluserFS server on the following directory.<\/p>\n
           \/var\/log\/glusterfs<\/code><\/pre>\n
          To monitor logs in real time you can use tail -f<\/code> along with the path to log file. <\/p>\n
          2. All glusterfs client logs are saved in the following with the volume name. <\/p>\n
          \/var\/log\/glusterfs\/<\/code><\/pre>\n
          \n
          Ngu\u1ed3n:<\/strong> How to Setup a Replicated GlusterFS Cluster on AWS EC2 \u2014 DevOpsCube<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
          Source: https:\/\/devopscube.com\/replicated-glusterfs-cluster-aws-ec2\/<\/p>\n","protected":false},"author":1,"featured_media":698,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-697","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops"],"_links":{"self":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/697","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=697"}],"version-history":[{"count":0,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/posts\/697\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=\/wp\/v2\/media\/698"}],"wp:attachment":[{"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=697"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=697"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ngocha.biz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=697"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}