Following:
- http://gluster.readthedocs.io/en/latest/Administrator%20Guide/SSL/
- https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3.1/html/Administration_Guide/chap-Network_Encryption.html
I use GlusterFS 3.9 here.
3 server machines, 1 client machine, all Ubuntu 16.04. I used the cheapest DigitalOcean instances to test this.
We assume that the servers have IPs 1.1.1.1, 2.2.2.2, and 3.3.3.3, and the client has 9.9.9.9 (replace these with your real IPs).
For various files, I use the tutorial prefix; it would make sense for you to replace that by nameofyourcompany or nameforyoursetup.
From the directory where you keep your CA cert and private key (there are lots of guides how to make one):
touch tutorial-gluster-server-privkey.pem
chmod 600 tutorial-gluster-server-privkey.pem
openssl req -newkey rsa:2048 -subj "/C=GB/ST=England/L=London/O=Tutorial Ltd/OU=Internal/CN=tutorial-gluster-server/emailAddress=admin@example.com" -out tutorial-gluster-server.csr -keyout tutorial-gluster-server-privkey.pem -nodes
touch tutorial-gluster-client-privkey.pem
chmod 600 tutorial-gluster-client-privkey.pem
openssl req -newkey rsa:2048 -subj "/C=GB/ST=England/L=London/O=Tutorial Ltd/OU=Internal/CN=tutorial-gluster-client/emailAddress=admin@example.com" -out tutorial-gluster-client.csr -keyout tutorial-gluster-client-privkey.pem -nodes
openssl ca -batch -config tutorial-ca.conf -notext -in tutorial-gluster-server.csr -out tutorial-gluster-server-cert.pem
openssl ca -batch -config tutorial-ca.conf -notext -in tutorial-gluster-client.csr -out tutorial-gluster-client-cert.pem
scp tutorial-root-ca-cert.pem root@1.1.1.1:/etc/ssl/glusterfs.ca
scp tutorial-root-ca-cert.pem root@2.2.2.2:/etc/ssl/glusterfs.ca
scp tutorial-root-ca-cert.pem root@3.3.3.3:/etc/ssl/glusterfs.ca
scp tutorial-gluster-server-privkey.pem root@1.1.1.1:/etc/ssl/glusterfs.key
scp tutorial-gluster-server-privkey.pem root@2.2.2.2:/etc/ssl/glusterfs.key
scp tutorial-gluster-server-privkey.pem root@3.3.3.3:/etc/ssl/glusterfs.key
scp tutorial-gluster-server-cert.pem root@1.1.1.1:/etc/ssl/glusterfs.pem
scp tutorial-gluster-server-cert.pem root@2.2.2.2:/etc/ssl/glusterfs.pem
scp tutorial-gluster-server-cert.pem root@3.3.3.3:/etc/ssl/glusterfs.pem
scp tutorial-root-ca-cert.pem root@9.9.9.9:/etc/ssl/glusterfs.ca
scp tutorial-gluster-client-privkey.pem root@9.9.9.9:/etc/ssl/glusterfs.key
scp tutorial-gluster-client-cert.pem root@9.9.9.9:/etc/ssl/glusterfs.pem
On all glusterfs servers:
add-apt-repository ppa:gluster/glusterfs-3.9
apt-get update
apt-get install -y glusterfs-server
On all glusterfs clients:
add-apt-repository ppa:gluster/glusterfs-3.9
apt-get update
apt-get install -y glusterfs-client
On the glusterfs servers:
touch /var/lib/glusterd/secure-access
service glusterfs-server restart
On the glusterfs clients:
mkdir -p /var/lib/glusterd
touch /var/lib/glusterd/secure-access
On first glusterfs server:
gluster peer probe 2.2.2.2
gluster peer probe 3.3.3.3
On all glusterfs servers (we later have to use force because we create the brick on the root volume for testing):
mkdir -p /var/gluster/brick1/gv0
On one glusterfs server:
gluster volume create gv0 replica 3 1.1.1.1:/var/gluster/brick1/gv0 2.2.2.2:/var/gluster/brick1/gv0 3.3.3.3:/var/gluster/brick1/gv0 force
We do not start the volume yet, as we first want to enable SSL.
On one glusterfs server:
gluster volume set gv0 client.ssl on
gluster volume set gv0 server.ssl on
gluster volume set gv0 auth.ssl-allow 'tutorial-gluster-server,tutorial-gluster-client'
On one glusterfs server:
gluster volume start gv0
On a glusterfs client, mount the volume:
mount -t glusterfs 1.1.1.1:/gv0 /mnt/
Now the volume should be mounted over an encrypted connection.
On one glusterfs server, we now remove tutorial-gluster-client from the ssl-allow entry; then the client should no longer be able to mount it:
gluster volume stop gv0
gluster volume set gv0 auth.ssl-allow 'tutorial-gluster-server'
gluster volume start gv0
On a glusterfs client, test it:
umount /mnt/
mount -t glusterfs 1.1.1.1:/gv0 /mnt/
should print
Mount failed. Please check the log file for more details.
and /var/log/glusterfs/mnt.log should contain something like
0-gv0-client-0: failed to set the volume [Permission denied]
To allow it again, on one clusterfs server:
gluster volume stop gv0
gluster volume set gv0 auth.ssl-allow 'tutorial-gluster-server,tutorial-gluster-client'
gluster volume start gv0
and on the client we changed:
umount /mnt/
mount -t glusterfs 1.1.1.1:/gv0 /mnt/