Quick three-node Lustre set-up on CentOS 6

Three-Node-LustreFS-Cluster-Quickstart.md

Mostly following the guide found here:

https://wiki.hpdd.intel.com/display/PUB/Create+and+Mount+a+Lustre+Filesystem

• Downgrade kernel if needed:
  • yum remove kernel-2.6.32-<whatever>*
• Enable EPEL repo:
  • yum install epel-release
• Enable ZFS on Linux repo:
  • yum localinstall --nogpgcheck http://archive.zfsonlinux.org/epel/zfs-release.el6.noarch.rpm
• Enable lustre repos:

[lustre-server]
name=CentOS-$releasever - Lustre
baseurl=https://downloads.hpdd.intel.com/public/lustre/latest-feature-release/el6/server/
gpgcheck=0

[e2fsprogs]
name=CentOS-$releasever - Ldiskfs
baseurl=https://downloads.hpdd.intel.com/public/e2fsprogs/latest/el6/RPMS
gpgcheck=0

[lustre-client]
name=CentOS-$releasever - Lustre
baseurl=https://downloads.hpdd.intel.com/public/lustre/latest-feature-release/el6/client/
gpgcheck=0

• On all of the MDS, OSS and client machines:
  • Upgrade e2fsprogs: yum upgrade e2fsprogs
  • Install the lustre-tests package: yum install lustre-tests
  • Create the following file /etc/modprobe.d/lnet.conf:
    • options lnet networks=tcp0(eth1)
    • Adjust the networks parameter as required for the type of interconnect and network interface to be used.
• On the MGS and OSS only, create the file /etc/sysconfig/modules/lnet.modules to auto-load the lnet kernel module on boot:

#!/bin/sh

if [ ! -c /dev/lnet ] ; then
    exec /sbin/modprobe lnet >/dev/null 2>&1
fi

• On the MGS/MDT/MDS:
  • Intialise a disk or partition to use for lustre.
  • Create a lustre MDT:
    • mkfs.lustre --fsname=whatevs --mgs --mdt --index=0 /dev/sdX
  • Create a mount point and mount the lustre FS:
    • mkdir /mnt/mdt && mount -t lustre /dev/sdX /mnt/mdt
• On the OST/OSS:
  • Intialise a disk or partition to use for lustre.
  • Create a lustre OST:
    • mkfs.lustre --ost --fsname=whatevs --mgsnode=192.168.N.N@tcp0 --index=0 /dev/sdX
    • Adjust the --mgsnode parameter for the address and protocol used for the MGS.
  • Create a mount point and mount the lustre FS:
    • mkdir /ostoss_mount && mount -t lustre /dev/sdX /ostoss_mount
• On the client:
  • Load the lustre kernel module: modprobe lustre.
  • Create the file /etc/sysconfig/modules/lustre.modules to load the lustre module on boot:

#!/bin/sh

/sbin/lsmod | /bin/grep lustre 1>/dev/null 2>&1
if [ ! $? ] ; then
   /sbin/modprobe lustre >/dev/null 2>&1
fi

• Create a mount point: mkdir /mnt/lustre.
  • Mount the lustre FS:
    • mount -t lustre 192.168.N.N@tcp0:/whatevs /mnt/lustre