Assumes you installed debian, docker, etc as per the list here
- I will have one gluster volume i will call
glusterfs-vol1
- I will install glusterfs on my docker nodes (best practice is to have seperate dedicated VMs)
- I have 3 nodes in my cluster (docker01, docker02, docker03)
- I will have one brick per node (brick1, brick2, brick3)
- the volume will be dispered - more on volume types
Add a VHD to each of your docker host VMs - for example a 100GB Volume If using Hyper-V this can be done without rebooting (add it as new SCSI VHD and the VM OS will detect it instantly)
Perform these steps on every node.
sudo lsblk
to confirm device node (should be sdb but could be different if you diverged from any of gists)
sudo fdisk /dev/sdb
(then g
, then n
and accept defaults and lastly w
to write out changes)
sudo mkfs.xfs /dev/sdb1
(this formats the new paritions with XFS)
sudo mkdir /mnt/glusterfs
(this is where you will mount the new parition)
sudo mount /dev/sdb1 /mnt/glusterfs
(this mounts and is used in next steps, don't miss).
sudo mkdir /mnt/glusterfs/vol1-brick1
this created the folder where brick1 will be stored
sudo ls -al /dev/disk/by-uuid/
this gets you the UUID for the partition you created earlier
edit fstab (be careful) with sudo nano /etc/fstab
add the followling line as the last line in fstab, use the UUID you got in the last step, do not use the one here.
UUID=c78ed594-ef62-445d-9486-10938f49b603 /mnt/glusterfs xfs defaults 0 0
sudo findmnt --verify
you should see no errors related to /dev/sdb or /dev/sb1 (you may see errors about CD-ROM those can be ignored) if you get ANY errors do not proceed until you have checked your previous work
sudo mkdir /mnt/glusterfs/vol1-brick2
this created the folder where brick2 will be stored
sudo ls -al /dev/disk/by-uuid/
this gets you the UUID for the partition you created earlier
edit fstab (be careful) with sudo nano /etc/fstab
add the followling line as the last line in fstab, use the UUID you got in the last step, do not use the one here.
UUID=c78ed594-ef62-445d-9486-10938f49b603 /mnt/glusterfs xfs defaults 0 0
sudo findmnt --verify
you should see no errors related to /dev/sdb or /dev/sb1 (you may see errors about CD-ROM those can be ignored) if you get ANY errors do not proceed until you have checked your previous work
sudo mkdir /mnt/glusterfs/vol1-brick3
this created the folder where brick3 will be stored
sudo ls -al /dev/disk/by-uuid/
this gets you the UUID for the partition you created earlier
edit fstab (be careful) with sudo nano /etc/fstab
add the followling line as the last line in fstab, use the UUID you got in the last step, do not use the one here.
UUID=c78ed594-ef62-445d-9486-10938f49b603 /mnt/glusterfs xfs defaults 0 0
sudo findmnt --verify
you should see no errors related to /dev/sdb or /dev/sdb1 (you may see errors about CD-ROM those can be ignored) if you get ANY errors do not proceed until you have checked your previous work
sudo apt-get install glusterfs-server
sudo systemctl start glusterd
sudo systemctl enable glusterd
sudo -s
gluster peer probe docker02.alexbal.com; gluster peer probe docker03.alexbal.com;
gluster pool list
gluster volume create gluster-vol1 disperse 3 redundancy 1 docker01.yourdomain.com:/mnt/glusterfs/vol1-brick1 docker02.yourdomain.com:/mnt/glusterfs/vol1-brick2 docker03.yourdomain.com:/mnt/glusterfs/vol1-brick3
gluster volume start gluster-vol1
gluster volume info gluster-vol1
sudo mkdir /mnt/gluster-vol1
make the mount point
sudo nano /etc/fstab
edit the fstab
add the following as the last line in the fstab
localhost:/gluster-vol1 /mnt/gluster-vol1 glusterfs defaults,_netdev,noauto,x-systemd.automount 0 0
exit and save
sudo mount -a
should mount with no errors
sudo df /mnt/gluster-vol1/
should return details about the gluster file system (size etc)
To test create a file using touch sudo touch /mnt/gluster-vol1/hello-world-txt
now check for that file in the same path on the other 2 nodes. If you did everything correctly you now have a replicating and redundant file system!
Note: findmnt --verify
cant be used for this mount as it doesn't support checking glusterfs
added
,noauto,x-systemd.automount
guidance to the fstab section to fix mounting sometimes not working correctly because things were not setup when needed to start, this may require more x-systemd mount directives if this doesn't fix (but seems to fix so far)