Skip to content

Instantly share code, notes, and snippets.

@maxbelyanin

maxbelyanin/man.md

Last active January 21, 2016 07:08
Show Gist options
  • Save maxbelyanin/e11e15041758a8137581 to your computer and use it in GitHub Desktop.
Save maxbelyanin/e11e15041758a8137581 to your computer and use it in GitHub Desktop.
Download ZIP
Xen 6.5 software Raid – installation
Raw
man.md

1.Install XenServer 6.0.2 on /dev/sda and do NOT configure any local storage

2./dev/sda should containt three partitions, please verify with the following command:

sgdisk -p /dev/sda

3.The first partition is used for XenServer installation, the second one is used for backups during XenServer upgrades. If third partition is not created? create it with following command:

sgdisk --new=3:16779264:3907029134 /dev/sda &&
sgdisk --typecode=3:fd00

4.Create filesystem (FS) for sda3.

mkfs.ext3 /dev/sda3

4.Reboot

reboot

5.Prepare /dev/sdb removing GPT data structure.

sgdisk --zap-all /dev/sdb

6.Convert disc from GPT to MBR format.

sgdisk --mbrtogpt --clear /dev/sdb

7.Clone partition table from /dev/sda to /dev/sdb.

/dev/sdb have have three partitions: /dev/sda1, /dev/sda2, /dev/sda3.

sgdisk -R /dev/sdb /dev/sda

8.Set type code for each partition. fd00 means Linux RAID.

sgdisk --typecode=1:fd00 /dev/sdb &&
sgdisk --typecode=2:fd00 /dev/sdb &&
sgdisk --typecode=3:fd00 /dev/sdb

9.Add RAID module to the Kernel because it is not loaded by default.

modprobe md_mod

10.Create new array with device metadata: two RAID1 partitions.

Create md0 (root), md1 (swap) y md2 (storage).

mdadm --create /dev/md0 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb1 missing &&
mdadm --create /dev/md1 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb2 missing &&
mdadm --create /dev/md2 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb3 missing

11.Create filesystem (FS) for root.

mkfs.ext3 /dev/md0

12.Mount root FS.

mount /dev/md0 /mnt

13.Copy all files from root to /mnt.

cp -xR --preserve=all / /mnt

14.Generate RAID configuration in mdadm.conf file.

mdadm --detail --scan > /mnt/etc/mdadm.conf

15.Update fstab file to establish new RAID device.

Is made changing LABEL tag by /dev/md0.

sed -i 's/LABEL=[a-zA-Z\-]*/\/dev\/md0/' /mnt/etc/fstab

16.Mount /dev directory on /mnt/dev allowing can be accessible from both places.

Mount filesystem with type sysfs on /mnt/sys.

Mount filesystem with type proc on /mnt/proc.

mount --bind /dev /mnt/dev &&
mount -t sysfs none /mnt/sys &&
mount -t proc none /mnt/proc

17.Change root to /mnt to execute /sbin/extlinux command to init the system by /boot.

chroot /mnt /sbin/extlinux --install /boot

18.Copy output file to the partition /dev/sdb

dd if=/mnt/usr/share/syslinux/gptmbr.bin of=/dev/sdb

19.Change root position to /mnt.

chroot /mnt

20.Create filesystem image.

mkinitrd -v -f --theme=/usr/share/splash --without-multipath /boot/initrd-`uname -r`.img `uname -r`

21.Return to root init position. exit

22.Update extlinux.conf file to establish the new RAID device.

Is made changing LABEL tag by /dev/md0.

sed -i 's/LABEL=[a-zA-Z\-]*/\/dev\/md0/' /mnt/boot/extlinux.conf

Move to /mnt

Execute extlinux for init by boot/ in raid mode.

cd /mnt
extlinux --raid -i boot/

# Set /dev/sdb as system boot.
``` shell
sgdisk /dev/sdb --attributes=1:set:2

# Move to root folder and unmount dev, sys, proc y mnt.
``` shell
cd
umount /mnt/dev
umount /mnt/sys
umount /mnt/proc
umount /mnt

# Sinchronize disc data with ram memory and reboot the system.
``` shell
sync
reboot
===========================================================================================
1.Install XenServer 6.0.2 on /dev/sda and do NOT configure any local storage

2.Create file /etc/sysconfig/modules/raid.modules with needed modules
``` shell
echo "modprobe raid10" > /etc/sysconfig/modules/raid.modules

3.Make this file executable

chmod +x /etc/sysconfig/modules/raid.modules

4.Reboot

reboot

5./dev/sda should containt three partitions, please verify with the following command:

sgdisk -p /dev/sda

6.The first partition is used for XenServer installation, the second one is used for backups during XenServer upgrades. If third partition is not created? create it with following command:

sgdisk --new=3:16779264:3907029134 /dev/sda &&
sgdisk --typecode=3:fd00

7.Reboot

reboot

8.Now we are going to use /dev/sdb /dev/sdc /dev/sdd as the mirror disk. Clear the partition table.

sgdisk --zap-all /dev/sdb &&
sgdisk --zap-all /dev/sdc &&
sgdisk --zap-all /dev/sdd

9.Install a GPT table on /dev/sdb /dev/sdc /dev/sdd

sgdisk --mbrtogpt --clear /dev/sdb &&
sgdisk --mbrtogpt --clear /dev/sdc &&
sgdisk --mbrtogpt --clear /dev/sdd

10.Create partitions on /dev/sdb /dev/sdc /dev/sdd. Please note that the following commands are dependent on your installation. Copy the start and the last sectors from the /dev/sda (output of sgdisk -p /dev/sda)

sgdisk --new=1:2048:8388641 /dev/sdb &&
sgdisk --typecode=1:fd00 /dev/sdb &&
sgdisk --attributes=1:set:2 /dev/sdb &&
sgdisk --new=2:8390656:16777249 /dev/sdb &&
sgdisk --typecode=2:fd00 /dev/sdb &&
sgdisk --new=3:16779264:3907029134 /dev/sdb &&
sgdisk --typecode=3:fd00 /dev/sdb &&
sgdisk --new=1:2048:8388641 /dev/sdc &&
sgdisk --typecode=1:fd00 /dev/sdc &&
sgdisk --attributes=1:set:2 /dev/sdc &&
sgdisk --new=2:8390656:16777249 /dev/sdc &&
sgdisk --typecode=2:fd00 /dev/sdc &&
sgdisk --new=3:16779264:3907029134 /dev/sdc &&
sgdisk --typecode=3:fd00 /dev/sdc &&
sgdisk --new=1:2048:8388641 /dev/sdd &&
sgdisk --typecode=1:fd00 /dev/sdd &&
sgdisk --attributes=1:set:2 /dev/sdd &&
sgdisk --new=2:8390656:16777249 /dev/sdd &&
sgdisk --typecode=2:fd00 /dev/sdd &&
sgdisk --new=3:16779264:3907029134 /dev/sdd &&
sgdisk --typecode=3:fd00 /dev/sdd

11.Reboot

reboot

12.Create RAID devices mknod /dev/md0 b 9 0 && mknod /dev/md1 b 9 1 && mknod /dev/md2 b 9 2 &&

mdadm --create /dev/md0 --level=1 --raid-devices=4 --metadata=0.90 missing /dev/sdb1 /dev/sdc1 /dev/sdd1 &&
mdadm --create /dev/md1 --level=1 --raid-devices=4 --metadata=0.90 missing /dev/sdb2 /dev/sdc2 /dev/sdd2 &&
mdadm --create /dev/md2 --level=10 --raid-devices=4 --metadata=0.90 missing /dev/sdb3 /dev/sdc3 /dev/sdd3

13.Create bitmaps for each RAID device. Bitmaps slightly impact throughput but significantly reduce the rebuilt time when the array fails. mdadm --grow /dev/md0 -b internal && mdadm --grow /dev/md1 -b internal && mdadm --grow /dev/md2 -b internal 14.Format the root disk and mount it at /mnt and copy the root filesystem to the RAID array (please be patient this step may take a while).

mkfs.ext3 /dev/md0 &&
mount /dev/md0 /mnt &&
cp -vxpR / /mnt

15.Change the root filesystem in /mnt/etc/fstab to /dev/md0.

sed -i 's/LABEL=[a-zA-Z\-]*/\/dev\/md0/' /mnt/etc/fstab

LABEL=root-jehrcjvh / ext3 defaults 1 1 16.Copy RAID settings to /etc/mdadm.conf (for mkinitrd) and to /mnt/etc/mdadm.conf

mdadm --detail --scan >> /mnt/etc/mdadm.conf &&
cp /mnt/etc/mdadm.conf /etc/mdadm.conf

17.Create new boot image

mkdir /mnt/root/initrd-raid &&
mkinitrd -v --fstab=/mnt/etc/fstab /mnt/root/initrd-raid/initrd-`uname -r`-raid.img `uname -r`

18.Copy new boot image to /mnt/boot and change boot menu

rm /mnt/boot/initrd-3.10-xen.img &&
cp /mnt/root/initrd-raid/initrd-`uname -r`-raid.img /mnt/boot/ &&
cd /mnt/boot &&
ln -s initrd-`uname -r`-raid.img initrd-3.10-xen.img

19.Replace in /mnt/boot/extlinux.conf line “root=LABEL=root-…” to “root=/dev/md0” at all menu items

sed -i 's/LABEL=[a-zA-Z\-]*/\/dev\/md0/' /mnt/boot/extlinux.conf &&
sed -i 's/LABEL=[a-zA-Z\-]*/\/dev\/md0/' /boot/extlinux.conf

20.Set MBR for GPT to /dev/sdb

cat /mnt/usr/share/syslinux/gptmbr.bin > /dev/sdb &&
cd /mnt &&
extlinux  --raid -i boot/

21.Unmount the new root and reboot. Important: Remember to use the boot menu of your BIOS to boot from the second hard disk this time! umount /mnt/proc && umount /mnt/sys && umount /mnt/dev && umount /mnt &&

reboot

22.XenServer is up again, include /dev/sda in the array

sgdisk --attributes=1:set:2 /dev/sda &&
sgdisk --typecode=1:fd00 /dev/sda &&
sgdisk --typecode=2:fd00 /dev/sda &&
sgdisk --typecode=3:fd00 /dev/sda &&
mdadm -a /dev/md0 /dev/sda1 &&
mdadm -a /dev/md1 /dev/sda2 &&
mdadm -a /dev/md2 /dev/sda3

23.The array needs to complete its initial build/synchronisation. That will take a while.

watch --interval=1 cat /proc/mdstat

24.Add /dev/md2 as a local SR to XenServer.

xe sr-create content-type=user device-config:device=/dev/md2 name-label="Local Storage" shared=false type=lvm

type=ext is required if you turned on thin provisioning in the installer. Otherwise use type=lvm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment