Menu

Install and Configure GlusterFS(CDN) on Linux
GlusterFS is a scale-out network-attached storage file system. It has found applications including cloud computing, streaming media services, and content delivery networks. GlusterFS was developed originally by Gluster, Inc. and then by Red Hat, Inc., as a result of Red Hat acquiring Gluster in 2011.


In June 2012, Red Hat Storage Server was announced as a commercially supported integration of GlusterFS with Red Hat Enterprise Linux.Red Hat bought Inktank Storage in April 2014, which is the company behind the Ceph distributed file system, and re-branded GlusterFS-based Red Hat Storage Server to "Red Hat Gluster Storage"

Requirements:
Here, we are going to configure GlusterFS volume with two replicas. Make sure you have two 64bit systems (either virtual or physical) with 1GB of memory, and one spare hard disk on each system.

gluster1.hackthesec.local|192.168.12.16|Ubuntu 16.04|1GG|/dev/sdb (5GB)--Node 1

gluster2.hackthesec.local|192.168.12.17|Debian 8|1GB|/dev/sdb (5GB)--Node 2

client.hackthesec.local|192.168.12.8|CentOS 7 / Debian|NA|NA|Client Machine

Configure DNS:

GlusterFS components use DNS for name resolutions, so configure either DNS or set up a hosts entry. If you do not have a DNS on your environment, modify /etc/hosts file and update it accordingly.
sudo vi /etc/hosts

192.168.12.16  gluster1.hackthesec.local  gluster1
192.168.12.17  gluster2.hackthesec.local  gluster2
192.168.12.20 client.hackthesec.local client
Add GlusterFS Repository:

Before proceeding to the installation, we need to configure GlusterFS repository on both storage nodes. Follow the instruction to add the repository to your system.

Debian:

Install support package for https transactions.
sudo apt-get install -y lsb-release
sudo apt-get install -y apt-transport-https
Add the public key for GlusterFS repository.
wget -O - http://download.gluster.org/pub/gluster/glusterfs/LATEST/rsa.pub | sudo apt-key add -
echo deb https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/$(lsb_release -sc)/apt $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/gluster.list
Ubuntu 16.04 / 14.04:
sudo apt-get install -y software-properties-common
sudo add-apt-repository ppa:gluster/glusterfs-3.8
Install GlusterFS:
Once you have added the repository on your systems, we are good to go for the installation of GlusterFS.


Update the repository cache.
sudo apt-get update
Install GlusterFS package using the following command.
sudo apt-get install -y glusterfs-server
Start the glusterfs-server service on all gluster nodes.
sudo service glusterfs-server start
Verify that the glusterfs service is running fine.
hackthesec@gluster1:~$ sudo service glusterfs-server status

● glusterfs-server.service - LSB: GlusterFS server
   Loaded: loaded (/etc/init.d/glusterfs-server; bad; vendor preset: enabled)
   Active: active (running) since Sat 2016-09-24 21:47:20 IST; 1min 6s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1570 ExecStop=/etc/init.d/glusterfs-server stop (code=exited, status=0/SUCCESS)
  Process: 1664 ExecStart=/etc/init.d/glusterfs-server start (code=exited, status=0/SUCCESS)
    Tasks: 7 (limit: 512)
   Memory: 25.5M
      CPU: 3.552s
   CGroup: /system.slice/glusterfs-server.service
           └─1673 /usr/sbin/glusterd -p /var/run/glusterd.pid

Sep 24 21:47:16 gluster1.hackthesec.local systemd[1]: Starting LSB: GlusterFS server...
Sep 24 21:47:16 gluster1.hackthesec.local glusterfs-server[1664]:  * Starting glusterd service glusterd
Sep 24 21:47:20 gluster1.hackthesec.local glusterfs-server[1664]:    ...done.
Sep 24 21:47:20 gluster1.hackthesec.local systemd[1]: Started LSB: GlusterFS server.
Configure Firewall:

You would need to either disable the firewall or configure the firewall to allow all connections within a cluster.
sudo iptables -I INPUT -p all -s <ip-address> -j ACCEPT
Add Storage:
Assuming that you have one spare hard disk on your machine, /dev/sdb is the one I will use here for a brick. Create a single partition on the spare hard disk shown like below.


You would need to perform the below steps on both nodes.
sudo iptables -I INPUT -p all -s <ip-address> -j ACCEPT
Add Storage:

Assuming that you have one spare hard disk on your machine, /dev/sdb is the one I will use here for a brick. Create a single partition on the spare hard disk shown like below.

You would need to perform the below steps on both nodes.
sudo fdisk /dev/sdb
Format and Mount the created partition with the filesystem of your choice.
sudo mkfs.ext4 /dev/sdb1
sudo mkdir -p /data/gluster
sudo mount /dev/sdb1 /data/gluster
echo "/dev/sdb1 /data/gluster ext4 defaults 0 0" | sudo tee --append /etc/fstab
Configure GlusterFS on Ubuntu 16.04:

Before creating a volume, we need to create trusted storage pool by adding gluster2.hackthesec.local. You can run GlusterFS configuration commands on any one server in the cluster will execute the same command on all other servers.

Here I will run all GlusterFS commands in gluster1.hackthesec.local node.
hackthesec@gluster1:~$ sudo gluster peer probe gluster2.hackthesec.local
peer probe: success.

hackthesec@gluster1:~$ sudo gluster peer status

Number of Peers: 1

Hostname: gluster2.hackthesec.local
Uuid: 51470928-dfa8-42e1-a221-d7bbcb8c13bd
State: Peer in Cluster (Connected)

hackthesec@gluster1:~$ sudo gluster pool list

UUID                                    Hostname                State
51470928-dfa8-42e1-a221-d7bbcb8c13bd    gluster2.hackthesec.local  Connected
dc7c1639-d21c-4adf-b28f-5150229e6980    localhost               Connected
Setup GlusterFS Volume:

Create a brick (directory) called "gvol0" in the mounted file system on both nodes.
sudo mkdir -p /data/gluster/gvol0
Since we are going to use replicated volume, so create the volume named "gvol0" with two replicas.
hackthesec@gluster1:~$ sudo gluster volume create gvol0 replica 2 gluster1.hackthesec.local:/data/gluster/gvol0 gluster2.hackthesec.local:/data/gluster/gvol0
volume create: gvol0: success: please start the volume to access data
Start the volume.
hackthesec@gluster1:~$ sudo gluster volume start gvol0
volume start: gvol0: success
Check the status of the created volume.
hackthesec@gluster1:~$ sudo gluster volume info gvol0

Volume Name: gvol0
Type: Replicate
Volume ID: ca102e4b-6cd1-4d9d-9c5a-03b882c76da0
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: gluster1.hackthesec.local:/data/gluster/gvol0
Brick2: gluster2.hackthesec.local:/data/gluster/gvol0
Options Reconfigured:
transport.address-family: inet
performance.readdir-ahead: on
nfs.disable: on
Setup GlusterFS Client:

Install glusterfs-client package to support the mounting of GlusterFS filesystems. Run all commands as root user.
$ su -

### CentOS / RHEL ###
yum install -y glusterfs-client

### Ubuntu / Debian ###
apt-get install -y glusterfs-client
Create a directory to mount the GlusterFS filesystem.
mkdir -p /mnt/glusterfs
Now, mount the GlusterFS filesystem to /mnt/glusterfs using the following command.
mount -t glusterfs gluster1.hackthesec.local:/gvol0 /mnt/glusterfs
You can also use gluster2.hackthesec.local instead of gluster1.hackthesec.com in the above command.

Verify the mounted GlusterFS filesystem.
[root@client ~]# df -hP /mnt/glusterfs
Filesystem                     Size  Used Avail Use% Mounted on
gluster1.hackthesec.local:/gvol0  4.8G   11M  4.6G   1% /mnt/glusterfs
You can also use below command to verify the GlusterFS filesystem.
[root@client ~]# cat /proc/mounts
rootfs / rootfs rw 0 0
sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=490448k,nr_inodes=122612,mode=755 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0
devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0
tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpuacct,cpu 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
cgroup /sys/fs/cgroup/net_cls cgroup rw,nosuid,nodev,noexec,relatime,net_cls 0 0
configfs /sys/kernel/config configfs rw,relatime 0 0
/dev/mapper/centos-root / xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0
selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=25,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0
/dev/mapper/centos-home /home xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0
/dev/sda1 /boot xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0
tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=100136k,mode=700 0 0
gluster1.hackthesec.local:/gvol0 /mnt/glusterfs fuse.glusterfs rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072 0 0
fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
Add below entry to /etc/fstab for automatically mounting during system boot.
gluster1.hackthesec.local:/gvol0 /mnt/glusterfs glusterfs  defaults,_netdev 0 0
Test GlusterFS Replication and High-Availability:

GlusterFS Server Side:

To check the replication, mount the created GlusterFS volume on the same storage node.
hackthesec@gluster1:~$ sudo mount -t glusterfs gluster1.hackthesec.local:/gvol0 /mnt
hackthesec@gluster2:~$ sudo mount -t glusterfs gluster2.hackthesec.local:/gvol0 /mnt
Data inside the /mnt directory of both nodes will always be same (replication).

GlusterFS Client Side:
Let’s create some files on the mounted filesystem on the client.hackthesec.local.
touch /mnt/glusterfs/file1
touch /mnt/glusterfs/file2


[root@client ~]# ll /mnt/glusterfs/
total 0
-rw-r--r--. 1 root root 0 Sep 26 11:22 file1
-rw-r--r--. 1 root root 0 Sep 26 11:22 file2
Test the both GlusterFS nodes whether they have same data inside /mnt.

hackthesec@gluster1:~$ ll /mnt/
total 12
drwxr-xr-x  4 root root 4096 Sep 26 20:52 ./
drwxr-xr-x 24 root root 4096 Sep 25 21:57 ../
-rw-r--r--  1 root root    0 Sep 26 20:52 file1
-rw-r--r--  1 root root    0 Sep 26 20:52 file2


hackthesec@gluster2:~$ ls -al /mnt/
total 12
drwxr-xr-x  4 root root 4096 Sep 26 06:30 .
drwxr-xr-x 23 root root 4096 Sep 24 08:39 ..
-rw-r--r--  1 root root    0 Sep 26  2016 file1
-rw-r--r--  1 root root    0 Sep 26  2016 file2
As you know, we have mounted GlusterFS volume from gluster1.hackthesec.local on client.hackthesec.local, now it is the time to test the high-availability of the volume by shutting down the node.
hackthesec@gluster1:~$ sudo poweroff
Now test the availability of the files, you would see files that we created recently even though the node is down.
[root@client ~]# ll /mnt/glusterfs/
total 0
-rw-r--r--. 1 root root 0 Sep 26 11:22 file1
-rw-r--r--. 1 root root 0 Sep 26 11:22 file2
Create some more files on the GlusterFS filesystem to check the replication.
touch /mnt/glusterfs/file3
touch /mnt/glusterfs/file4
Verify the files count.
[root@client ~]# ll /mnt/glusterfs/
total 0
-rw-r--r--. 1 root root 0 Sep 26 11:22 file1
-rw-r--r--. 1 root root 0 Sep 26 11:22 file2
-rw-r--r--. 1 root root 0 Sep 26 11:26 file3
-rw-r--r--. 1 root root 0 Sep 26 11:26 file4
Since the gluster1 is down, all your data’s are now written on gluster2.hackthesec.local due to High-Availability. Now power on the node1 (gluster1.hackthesec.local).

Check the /mnt of the gluster1.hackthesec.local; you should see all four files in the directory, this confirms the replication is working as expected.
hackthesec@gluster1:~$ sudo mount -t glusterfs gluster1.hackthesec.local:/gvol0 /mnt
[sudo] password for hackthesec:

hackthesec@gluster1:~$ ll /mnt/
total 12
drwxr-xr-x  4 root root 4096 Sep 26 20:59 ./
drwxr-xr-x 24 root root 4096 Sep 25 21:57 ../
-rw-r--r--  1 root root    0 Sep 26 20:52 file1
-rw-r--r--  1 root root    0 Sep 26 20:52 file2
-rw-r--r--  1 root root    0 Sep 26 20:56 file3
-rw-r--r--  1 root root    0 Sep 26 20:56 file4
Thanks for watching...
www.hackthesec.co.in

About Author:


I am a Linux Administrator and Security Expert with this site i can help lot's of people about linux knowladge and as per security expert i also intersted about hacking related news.TwitterFacebook

Next
Newer Post
Previous
Older Post

31 comments:

  1. Greetings! Quick question that's totally off topic.
    Do you know how to make your site mobile friendly? My site looks weird when viewing from my iphone 4.
    I'm trying to find a theme or plugin that might
    be able to correct this issue. If you have any suggestions, please share.
    Thank you!

    Feel free to surf to my web page - fashion nails designs

    ReplyDelete
  2. Hello, Neat post. There's a problem witth your website in internet explorer,
    may test this? IE still is the marketplace chief and a big compknent off other
    peple will pass over your wonderful writing because of this problem.


    My web site; 4 piece towel bale (http://www.ebay.co.uk/)

    ReplyDelete
  3. You will also require the best oil in order to
    cook with because oils need different cooking times as well as the results vary depending,
    which oil you use. Colorful fruits are perfect to be seen Pavlovas, ideas being sliced
    banana, strawberries, kiwi fruit and passion fruit.
    For finest results when managing, use parchment paper for
    moving to freezer.

    ReplyDelete
  4. Ꭲhanks a bunch for sharing this with aⅼl
    folks you гeally recognise wһat you are talking about!
    Bookmarked. Ꮲlease also discuss wіth mʏ site =).

    We could hаve a hyperlink trde agreement among us

    ReplyDelete
  5. Have you еver tһoսght abօut writing an ebook
    or guest authoring οn ߋther websites? І have a blog centered on the ѕame ideas yоu discuss
    and woսld really like to have you share somе stories/іnformation.
    Ι know my visitors wouⅼd value уour woгk.
    If you ɑre even remotely intеrested, feeel frwe tо shoot me an е-mail.

    ReplyDelete
  6. Ꮇy spouse аnd Ι stumbled ߋveг heгe coming frоm
    ɑ different page and thouցht I may as ᴡell check
    things օut. I likke wһat Ӏ sеe sso і am juѕt folⅼowіng yoս.
    Ꮮook forward to finding ߋut аbout yоur web paցe yet ɑgain.

    ReplyDelete
  7. Heyy there gгeat blog! Doеѕ running a blog such as this require a massive amount ԝork?
    I'ѵe no understanding ߋf coding buut Ι was holing to start
    mу own blog in the nesar future. Anyhow, if yoᥙ
    have аny rdcommendations or tips fⲟr neᴡ bloog owners pleasе share.
    I understand tһis iss off subject howevr І just had to ɑsk.
    Cheers!

    ReplyDelete
  8. Every weekend i used to visit this web page, because i wish for enjoyment, as
    this this site conations actually fastidious funny data too.

    ReplyDelete
  9. When someone writes an article he/she retains the thought of a user in his/her mind that how a user can understand
    it. Therefore that's why this article is perfect.

    Thanks!

    ReplyDelete
  10. Marvelous, what a webpage it is! This website presents useful information to us, keep it up.

    ReplyDelete
  11. Excellent, what a blog it is! This weblog provides helpful data to us, keep it up.

    ReplyDelete
  12. I used to be looking for information like this and you've
    done a great job with your presentation here. I'd just add if you are going to buy
    twitter followers, or almost any social multimedia marketing, the best We've
    found is sociaboost. contendo. I tried the typical options and always
    got burnt, these guys treat my site right.

    ReplyDelete
  13. Saved as a favorite, I like your site!

    ReplyDelete
  14. I always used to study paragraph in news papers but now as
    I am a user of net thus from now I am using net for articles or reviews,
    thanks to web.

    ReplyDelete
  15. Howdy! This is my first visit to your blog! We are a collection of volunteers and
    starting a new project in a community in the same niche. Your blog provided us valuable information to work on. You have done a marvellous job!

    ReplyDelete
  16. excellent points altogether, you simply gained a
    emblem new reader. What could you suggest in regards to your post that you just made a
    few days ago? Any certain?

    ReplyDelete
  17. Verry soon this site will be famous amid all bog people,
    due to it's good articles

    ReplyDelete
  18. It's hard to come by knowledgeable people in this particular subject, but you seem like you know what you're talking about!
    Thanks

    ReplyDelete
  19. Very nice article, totally what I wanted to
    find.

    ReplyDelete
  20. Thank you for the good writeup. It in fact was a amusement account it.

    Look advanced to more added agreeable from you!
    However, how can we communicate?

    ReplyDelete
  21. This article gives clear idea in support of the new viewer of
    blogging, that in facht how to do blogging.

    ReplyDelete
  22. Thank you for sharing your info. I truly appreciate your efforts and I will
    be waiting for your next write ups thank you once again.

    ReplyDelete
  23. I think the admin of this site is truly working hard in support of his wweb page, for the reason that here every data is quality based stuff.

    ReplyDelete
  24. Wow, fantastic blog layout! How long have you been blogging for?

    you make blogging look easy. The overall look of your
    site is great, as well as the content!

    ReplyDelete
  25. I'm curious to find out what blog platform you have
    been working with? I'm having some minor security problems with my latest
    website and I would like to find something more risk-free. Do you have any recommendations?

    ReplyDelete
  26. It's an amazing paragraph designed for all the internet users; they will get benefit from
    it I am sure.

    ReplyDelete
  27. Thanks for sharing your info. I really appreciate your efforts and
    I will be waiting for your further write ups thank you once again.

    ReplyDelete
  28. Thanks for sharing your thoughts on decent aerobic.
    Regards

    ReplyDelete
  29. Thanks for sharing your info. I truly appreciate your
    efforts and I will be waiting for your next post thanks once again.

    ReplyDelete
  30. Wow that was strange. I just wrote an very long comment
    but after I clicked submit my comment didn't show up.
    Grrrr... well I'm not writing all that over again. Anyhow,
    just wanted to say superb blog!

    ReplyDelete
  31. hello!,I really like your writing so much!

    proportion we keep up a correspondence more approximately your
    post on AOL? I require an expert on this area to
    unravel my problem. May be that is you! Looking ahead to peer you.

    ReplyDelete

 
Top