Skip to content

Instantly share code, notes, and snippets.

@kig
Last active November 26, 2023 02:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kig/edfba72e7de681542ec2939ede2f1199 to your computer and use it in GitHub Desktop.
Save kig/edfba72e7de681542ec2939ede2f1199 to your computer and use it in GitHub Desktop.
NMVeOF setup scripts
#!/bin/bash
# Adapted from https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/managing_storage_devices/configuring-nvme-over-fabrics-using-nvme-rdma_managing-storage-devices
# Exports an NVMe device over Fabrics so that it can be used by remote hosts.
#
# Usage: nvmeof_controller.sh DEVICE SERVER_IP SERVER_PORT TRANSPORT SUBSYSTEM_NAME NAMESPACE PORT
# Example: nvmeof_controller.sh /dev/nvme1n1 192.168.0.100 4420 tcp my_nvme 10 1
#
# Now you can import the device on a remote host.
#
# nvme discover -t TRANSPORT -a SERVER_IP -s SERVER_PORT
# nvme connect -t TRANSPORT -a SERVER_IP -s SERVER_PORT -n SUBSYSTEM_NAME
#
# For example:
#
# nvme connect -t tcp -a 192.168.0.100 -s 4420 -n my_nvme
#
# Find the new device name with nvme list
# Partitions on the device get imported as well.
#
# mount /dev/nvme1n1p1 /mnt
#
set -e
set -x
DEVICE=$1
SERVER_IP=$2
TRSVCID=$3
TRANSPORT=$4
SUBSYSTEM_NAME=$5
NAMESPACE=$6
PORT=$7
modprobe nvmet-rdma
mkdir /sys/kernel/config/nvmet/subsystems/"${SUBSYSTEM_NAME}"
cd /sys/kernel/config/nvmet/subsystems/"${SUBSYSTEM_NAME}"
echo 1 > attr_allow_any_host
mkdir namespaces/"${NAMESPACE}"
cd namespaces/"${NAMESPACE}"
echo -n "${DEVICE}" > device_path
echo 1 > enable
mkdir /sys/kernel/config/nvmet/ports/"${PORT}"
cd /sys/kernel/config/nvmet/ports/"${PORT}"
echo -n "${SERVER_IP}" > addr_traddr
echo rdma > addr_trtype
echo "${TRSVCID}" > addr_trsvcid
echo ipv4 > addr_adrfam
ln -s /sys/kernel/config/nvmet/subsystems/"${SUBSYSTEM_NAME}" /sys/kernel/config/nvmet/ports/"${PORT}"/subsystems/"${SUBSYSTEM_NAME}"
dmesg | grep "nvmet" | tail -5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment