Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Find, format, and mount an AWS Ephemeral NVMe disk within ec2 in user data
#!/usr/bin/env bash
###
## This mounts a (single) ephemral NVMe drive in an EC2 server.
## It's meant to be run once, within user-data
## For EBS drives (non-ephemeral storage), see: https://gist.github.com/jalaziz/c22c8464cb602bc2b8d0a339b013a9c4
#
# Install the "nvme" command
# See: https://github.com/linux-nvme/nvme-cli
sudo apt-get install -y nvme-cli
# Create a mount point (directory)
sudo mkdir -p /some/mount
# Find ephemeral storage (assumes a single ephemeral disk)
# and format it (assumes this is run on first-boot in user-data, so the disk is not formatted)
EPHEMERAL_DISK=$(sudo nvme list | grep 'Amazon EC2 NVMe Instance Storage' | awk '{ print $1 }')
sudo mkfs.ext4 $EPHEMERAL_DISK
sudo mount -t ext4 $EPHEMERAL_DISK /some/mount
### For some crazy reason, add ephemeral disk mount to /etc/fstab
## even tho you lose data in stop/starts of ec2 (I believe you keep the data via regular reboots?)
#
# Find the mounted drive UUID so we can mount by UUID
EPHEMERAL_UUID=$(sudo blkid -s UUID -o value $EPHEMERAL_DISK)
echo "UUID=$EPHEMERAL_UUID /opt/nomad ext4 defaults 0 0" | sudo tee -a /etc/fstab
@fideloper
Copy link
Author

fideloper commented Oct 18, 2020

The Issue

NVMe drives in AWS have a few fun factors:

  1. AWS EC2 instances have you mount drives at device names such as /dev/sda1, but within the EC2 instance, you'll only see device names such as /dev/nvme0n1.
  2. Drive re-ordering means
    1. The drive names (e.g. /dev/nvme0n1) can change during reboot
    2. Disk drives will be named inconsistently (root drives vs secondary drives). For example, a new server's secondary drive might be /dev/nvme0n1 or /dev/nvme1n1.

This means we need a programmatic way to decipher which drive is the root drive vs a secondary drive to correctly mount secondary EBS disks or ephemeral storage.

Ephemeral NVMe Drives

The above script will find, format, and mount an AWS Ephemeral NVMe disk within ec2.

It's meant to be run within a user-data script.

I've tried this on Ubuntu 18.04 and 20.04.

EBS NVMe Drives

For EBS drives (non-ephemeral storage), you'll want to use this gist as a guide to help you:

  1. Create symlinks in /dev/foo to the name of the drives you give them when created within AWS (instead of the drive names you see in the server, such as /dev/nvme0n1)
  2. Use the symlinks, which give you known device names, to format/mount the EBS drives as needed

The 70-ec2-nvme-devices.rules file in the gist above goes into the /usr/lib/udev/rules.d directory (possibly /etc/udev/rules.d), and the ebsnvme-id command goes in /sbin/ebsnvme-id.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment