-
-
Save joemiller/6049831 to your computer and use it in GitHub Desktop.
#!/bin/bash | |
# | |
# this script will attempt to detect any ephemeral drives on an EC2 node and create a RAID-0 stripe | |
# mounted at /mnt. It should be run early on the first boot of the system. | |
# | |
# Beware, This script is NOT fully idempotent. | |
# | |
METADATA_URL_BASE="http://169.254.169.254/2012-01-12" | |
yum -y -d0 install mdadm curl | |
# Configure Raid - take into account xvdb or sdb | |
root_drive=`df -h | grep -v grep | awk 'NR==2{print $1}'` | |
if [ "$root_drive" == "/dev/xvda1" ]; then | |
echo "Detected 'xvd' drive naming scheme (root: $root_drive)" | |
DRIVE_SCHEME='xvd' | |
else | |
echo "Detected 'sd' drive naming scheme (root: $root_drive)" | |
DRIVE_SCHEME='sd' | |
fi | |
# figure out how many ephemerals we have by querying the metadata API, and then: | |
# - convert the drive name returned from the API to the hosts DRIVE_SCHEME, if necessary | |
# - verify a matching device is available in /dev/ | |
drives="" | |
ephemeral_count=0 | |
ephemerals=$(curl --silent $METADATA_URL_BASE/meta-data/block-device-mapping/ | grep ephemeral) | |
for e in $ephemerals; do | |
echo "Probing $e .." | |
device_name=$(curl --silent $METADATA_URL_BASE/meta-data/block-device-mapping/$e) | |
# might have to convert 'sdb' -> 'xvdb' | |
device_name=$(echo $device_name | sed "s/sd/$DRIVE_SCHEME/") | |
device_path="/dev/$device_name" | |
# test that the device actually exists since you can request more ephemeral drives than are available | |
# for an instance type and the meta-data API will happily tell you it exists when it really does not. | |
if [ -b $device_path ]; then | |
echo "Detected ephemeral disk: $device_path" | |
drives="$drives $device_path" | |
ephemeral_count=$((ephemeral_count + 1 )) | |
else | |
echo "Ephemeral disk $e, $device_path is not present. skipping" | |
fi | |
done | |
if [ "$ephemeral_count" = 0 ]; then | |
echo "No ephemeral disk detected. exiting" | |
exit 0 | |
fi | |
# ephemeral0 is typically mounted for us already. umount it here | |
umount /mnt | |
# overwrite first few blocks in case there is a filesystem, otherwise mdadm will prompt for input | |
for drive in $drives; do | |
dd if=/dev/zero of=$drive bs=4096 count=1024 | |
done | |
partprobe | |
mdadm --create --verbose /dev/md0 --level=0 -c256 --raid-devices=$ephemeral_count $drives | |
echo DEVICE $drives | tee /etc/mdadm.conf | |
mdadm --detail --scan | tee -a /etc/mdadm.conf | |
blockdev --setra 65536 /dev/md0 | |
mkfs -t ext3 /dev/md0 | |
mount -t ext3 -o noatime /dev/md0 /mnt | |
# Remove xvdb/sdb from fstab | |
chmod 777 /etc/fstab | |
sed -i "/${DRIVE_SCHEME}b/d" /etc/fstab | |
# Make raid appear on reboot | |
echo "/dev/md0 /mnt ext3 noatime 0 0" | tee -a /etc/fstab |
Thanks! Was getting 'resource busy' at create RAID step but your umounting of the cloud-config mounted ephemeral drive solved my problem.
If you trust the docs, there's a much easier way to find all the ephemeral volumes:
find /dev -regextype posix-egrep -regex '.*/(sd|xvd)[b-e]'
On Ubuntu 14.04
root_drive=
df -h | grep -v grep | awk 'NR==2{print $1}'`
was pulling the first drive, not necessarily the root drive, for me until I changed it to
root_drive=
df -h | grep -P '^.*/$' | awk 'NF==6{print $1}'`.
On Ubuntu 16.04 I am using this to get the ephemeral volumes:
root@ip-10-228-67-215:~# lsblk -I 202 -d -n --output NAME,MAJ:MIN | grep -v "202:0" | awk '{print $1}'
xvdf
xvdg
Note that Ubuntu 16.04+ uses systemd and it does not support the nobootwait
mount option; use nofail
instead.
Great script. Saved me some time. thanks! - I did add some code for this to run efficiently as a startup script... details below. Cheers! - Joseph P.
PS: The correct startup log on EC2 (As of this date) is: "/var/log/boot.log"
Additions:
- created as 'startup' script on redhat / centos
- added code to verify script is running on startup
- added code so that this script will not execute on reboot if the raid disk is already mounted. (In case of a shutdown it will typically execute as ephemeral disks are wiped out.)
- added code to prevent /etc/fstab mount point entry from being written multiple times.
additional code
####### To use as startup script ########
### 1. Copy script to directory: /opt/aws/
# mkdir -p /opt/aws/
# cp 'myscript.sh' /opt/aws/
# chmod 755 /opt/aws/myscript.sh
#
### 2: Add script path to 'rc.local' to execute at startup:
# echo "/opt/aws/myscript.sh" | tee -a /etc/rc.d/rc.local
#
### Intended use:
# EC2 reboot: raid volume and data will be persistent on "/dev/md0"
# EC2 shutdown: all ephemeral storage is wiped. This script will initialize all instance stores and mount raid disk on boot.
#
# checksum to verify script is executing at reboot
DATE=$(date +'%F %H:%M:%S')
DIR=/tmp
echo "Current date and time: $DATE" > $DIR/ephem_bootscript_lastrun.txt
# check if Raid 0 disk is mounted | IF mounted then exit - if not continue
if grep '/dev/md0' /etc/mtab > /dev/null 2>&1; then
echo "'/dev/md0' is mounted...exiting" && exit
else
echo "'/dev/md0' not mounted"
echo "Continuing..."
fi
# Make raid volume mount on reboot (Prevent writing entry to /etc/fstab multiple times)
MP=/yourmountpoint
MOUNT_PATH="/dev/md0 ${MP} ext3 noatime 0 0"
FILE="/etc/fstab"
grep -qF "$MOUNT_PATH" "$FILE" || echo "$MOUNT_PATH" >> "$FILE"
I think the issue of the drive not being ready at boot is because there is a difference in the state of the ephemeral drives on a reboot versus a stop and then start. You get new unformatted drives when using a stop then restart but you get the same drives on a reboot. I'm trying to figure out how to detect that on boot and run the mkfs if that is the case.