Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 73 You must be signed in to star a gist
  • Fork 19 You must be signed in to fork a gist
  • Save fevangelou/2f7aa0d9b5cb42d783302727665bf80a to your computer and use it in GitHub Desktop.
Save fevangelou/2f7aa0d9b5cb42d783302727665bf80a to your computer and use it in GitHub Desktop.
Software RAID 1 setup on Ubuntu Server 20.04 (or newer)

Looking to create a Software RAID 1 setup for your 2-disk server on Ubuntu Server 20.04?

Screen Shot 2020-06-05 at 20 55 31

Let's start with the basics: the official guide by Ubuntu (https://ubuntu.com/server/docs/install/storage) is outdated/wrong. And as of March 2021 it's possible that there's a bug as well with how the bios_grub partitions are created when using multiple disks.

Now on to the solution:

  • Select "Custom storage layout" when you reach the storage configuration step of the installer.
  • If the disks have existing partitions, click on each disk under AVAILABLE DEVICES and then select REFORMAT. This will (temporarily) wipe out the partitions.
  • Now select the 1st disk to add as "boot" disk (same menu that had REFORMAT in).
  • Do the same with the 2nd disk. [Note on March 2021: If this does not work for you in the newest installer of 20.04.2, just skip that part. In the end the 1st disk will have 4 partitions (bios, /boot, swap and /) and the 2nd just 3 (/boot, swap and /), which doesn't really matter. If you want to have the same number of partitions on each disk, go ahead and create a 1 MB partition on the 2nd disk, before doing the 3 others. This way, blocks will be sync'ed between the 2 disks. It's possible that there's a bug in how the bios_grub partitions are created in the installer when using multiple disks as this behaviour is not consistent with version 20.04.1 or the current Ubuntu Server docs as referenced here.]
  • You should now see two 1.000M bios_grub partitions created under USED DEVICES. These small partitions will be used by GRUB for booting the server. [Note on March 2021: the setup works with one bios_grub partition as well].
  • The trick to setup a softRAID array is to create partitions for /boot, swap and / on each disk, but WITHOUT formatting them (and as such, there won't be a mount point for now).
  • So go ahead and "Add GPT Partition" on the 1st disk, give it a 1G size and choose to leave it unformatted. Do the same for the 2nd disk. These will be the /boot partitions for the softRAID array. Under each disk on AVAILABLE DEVICES you will now see "partition 2".
  • Now we'll prepare the swap partitions. "Add GPT Partition" on the 1st disk, give it the same or half the size of your RAM (e.g. let's say 16G cause we have 16G of actual RAM - but it's really up to you to decide that) and choose to leave it unformatted. Do the same for the 2nd disk. Under each disk on AVAILABLE DEVICES you will now see "partition 3".
  • Now we'll prepare the / partitions. "Add GPT Partition" on the 1st disk, do not set a size (so it uses all available) and choose to leave it unformatted as with all the other partitions you created so far. Do the same for the 2nd disk. Under each disk on AVAILABLE DEVICES you will now see "partition 4".
  • Now click on "Create software RAID (md)" under AVAILABLE DEVICES. We'll create the first softRAID partition (md0) by selecting the two "partition 2" entries (one from each disk). Click "Save".
  • Repeat the process for md1 and select the two "partition 3" entries. Hit "Save".
  • Repeat the process for md2 and select the two "partition 4" entries. Hit "Save".
  • We now have 3 pairs of AVAILABLE DEVICES which will now format as the actual softRAID partitions. So select md0 and then "Add GPT Partition", format as EXT4 and mount on /boot.
  • Select md1 and then "Add GPT Partition", format as SWAP.
  • Select md2 and then "Add GPT Partition", format as EXT4 and mount on /.
  • All these mdX softRAID partitions will now appear under USED DEVICES and you are ready to proceed with Ubuntu's installation.
  • At the very bottom, you should now see "Done" enabled so hit it and proceed.

Enjoy :)

Discuss here: https://gist.github.com/fevangelou/2f7aa0d9b5cb42d783302727665bf80a

@laimaretto
Copy link

laimaretto commented Mar 17, 2021

This does not work for me. As soon as I set "Use as boot device" for the first and second drives, the option to configure softRAID goes gray and it becomes unavailable.

On the other hand, if I do not set "boot device" at a disk level, I can configure the md0 device + LVM, but I end up with a message "select a boot disk" and I can't move on from there ...

Even if I go and create an empty partition on the first drive without formating, the installer creates on its own a /boot/efi partition and makes the sofRAID option unavailable, because if go and select softRaid, only the second drive appears as an option ...

image

It seems rather difficutl to setup softRAID + LVM with 20.04...

@ngwebb1
Copy link

ngwebb1 commented Mar 30, 2021

Hi there. I have an M.2 SSD and two 3TB HDDs. I'll def be using this guide for setting up my cloud server, but I have some questions. What should I do if I want Ubuntu Server to be on the SSD and use the two HDDs in RAID 1?

Would it be best to use the entire SSD for the Ubuntu Server installation?
Should the /boot partition be on the SSD, the RAID 1 array of the two HDDS, or both?
Where does the swap partition go?

@fevangelou
Copy link
Author

@BlueStarZero Install Ubuntu on the SSD only, ignore the HDDs. After the installation is setup, create a SoftRAID 1 array for the HDDs using "mdadm".

@ngwebb1
Copy link

ngwebb1 commented Mar 31, 2021

@BlueStarZero Install Ubuntu on the SSD only, ignore the HDDs. After the installation is setup, create a SoftRAID 1 array for the HDDs using "mdadm".

I was researching this very thing last night and thought about doing mdadm for RAID 1. Thanks for confirming my thoughts. :)

@remylrt
Copy link

remylrt commented Apr 5, 2021

Thank you so much for this guide ! That was exactly what I was looking for 🙏

@n3kt0n
Copy link

n3kt0n commented Apr 13, 2021

with the slightest (obvious) modifications, this guide works perfectly for installing Ubuntu 20.04 on RAID 0, where I/O is the priority

@norsemangrey
Copy link

@fevangelou This guide seems to be BIOS orientated. Will this also work for a UEFI enabled system?

@fevangelou
Copy link
Author

@norsemangrey Legacy BIOS or UEFI should not matter.

@Jayto1
Copy link

Jayto1 commented May 22, 2021

Thank you so much for making this guide. This is my first time using Ubuntu (2nd time using Linux) and 1st time using Raid. I configured RAID 1 on 2 480GB SSD's. Thanks again.

@jeff231li
Copy link

Thank you for creating this guide! I was able to set up a RAID 1 for a server machine and wouldn't be possible without your guide.

@dracoBoreus
Copy link

Ty! It was a great help!

@Yan963
Copy link

Yan963 commented Oct 7, 2021

Ok, so, I though with that kind of guide, I couldn't fail setting up my raid, but apparently, I missed something.

I configured my partitions like this :
UbuntuRaid

Installation went fine, but at reboot, I get the error "No boot device available or operating system detected".

Any idea of what I did wrong?

@fevangelou
Copy link
Author

@Yan963 You probably didn't set the boot drive (that small -few MBs- EFI partition for the installer to make). The option is inside the submenu at the first list (File System Summary) if I recall correctly.

@Yan963
Copy link

Yan963 commented Oct 8, 2021

I did set both drives as boot drive, which added a 512MBs ESP partition on each, is that what you mean?

@fevangelou
Copy link
Author

fevangelou commented Oct 8, 2021 via email

@yorickdowne
Copy link

Smashing. I wouldn't have figured those specific steps out, or not in time. You saved me a lot of effort, this worked beautifully.

@vipthomps
Copy link

This was a huge help. thank you! I just did this with 20.04.4 installer iso

@ngmgithub
Copy link

ngmgithub commented Jul 7, 2022

Anyone have any luck scripting this process into an Automated Server Install?

Ref: Automated Server Installs

@ryanjgillies
Copy link

Has anyone tried this approach with Server 22.04.1?

I've gone through these steps line by line (with and without the March 2020 comments) but not been able to boot after, I just end up straight back in bios. I tried a suggestion from another thread suggesting creating a single softRAID partition (md0) and then partitioning that into /boot, swap and / but no luck either.

I've read yet another suggestion that /boot is better kept out of RAID on a USB stick but haven't tried it yet, anyone have any thoughts on that?

@yorickdowne
Copy link

USB sticks are notorious for failing, don't do that. The TrueNAS crowd learned that the hard way.

@ryanjgillies
Copy link

Any alternatives? Is it fair to say that it is best for /boot to be kept separate to the RAID partition? If so what's the best solution?

@ryanjgillies
Copy link

I decided to go back to Ubuntu Server 20.04.4 and try again but I had the same issue. Continually returning to BIOS after install.

Instead I switched to the Ubuntu Legacy Server ISO installer as opposed to the live version. The prompts differ but the process is largely the same. This resolved my issues completely and I now have a fully functioning Ubuntu Server install.

@77369155
Copy link

This is the first time i am setting up a Server and this part worked perfectly. Thank you so much =)
I used popsicle and "ubuntu-22.04.1-live-server-amd64"
right after this guide, the 2 drives started syncing, if i am not mistaking. I tried to shut the server down and it responded:
"operation inhibited by disk manager (mdraid-sync-job)"
Is there any way i can check the progress, to estimate the remaining time?
I was able to shut it down 20 hours later. I was patient, kept it running, letting it do it's thing and hoped that it doesn't take months or years.

@alanivey
Copy link

Works for 22.04 IoT, thanks!

@volehuy1998
Copy link

ProLiant DL360 Gen10
iLO 5 - 2.42 Apr 05 2021
ubuntu-20.04.4-live-server-amd64.iso

Need it. Thank you so much

@bstempien
Copy link

Worked great for me!

@hitechelp
Copy link

Well done - went off without a hitch on 22.04(3) using 2 1Tb drives, 1ssd and 1hdd. Tested by unplugging each drive, one at a time obviously, then rebooting. Boots either way. Now, what's the preferred way to re-sync a new drive after a failure?

@fevangelou
Copy link
Author

fevangelou commented Jan 28, 2024

@hitechelp Hetzner has a concise guide on swapping faulty SoftRAID drives, check it out: https://docs.hetzner.com/robot/dedicated-server/raid/exchanging-hard-disks-in-a-software-raid/ - this obviously applies to any hardware, local or remote.

@msteele51
Copy link

In this model, the EFI partition is still not synced between the disks though, right?

@auvipy
Copy link

auvipy commented Feb 13, 2024

In this model, the EFI partition is still not synced between the disks though, right?

that is seems to be the case for me atleast. and the USB is creating problem to choose another ssd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment