Skip to content

Instantly share code, notes, and snippets.

@tweakoz
Last active June 20, 2024 14:27
Show Gist options
  • Save tweakoz/a68407be79e3cea85c3cb0744c9d4505 to your computer and use it in GitHub Desktop.
Save tweakoz/a68407be79e3cea85c3cb0744c9d4505 to your computer and use it in GitHub Desktop.
Mellanox IPOIB setup Debian/Ubuntu/Netplan/Systemd
################################
# install mlnxofed driver
################################
download: https://www.mellanox.com/products/infiniband-drivers/linux/mlnx_ofed
################################
# put HCA's in IPOIB mode
################################
mst start # start mellanox software tools
mst status # get device id's
mlxconfig -d /dev/mst/mtxxxx_pciconf0 set LINK_TYPE_P1=1 # port 1 to IPOIB
mlxconfig -d /dev/mst/mtxxxx_pciconf0 set LINK_TYPE_P2=1 # port 2 to IPOIB
################################
# netplan
################################
ip link # find IB interfaces
# add devices to netplan
################################
# systemd
################################
# create a systemd service file to launch opensmd
# assume here we name it "/etc/systemd/system/yo_opensmd.service"
[Unit]
Description=OpenSMD(infiniband)
ConditionPathExists=/etc/init.d/opensmd
[Service]
Type=forking
ExecStart=/etc/init.d/opensmd start
ExecStop=/etc/init.d/opensmd stop
[Install]
WantedBy=multi-user.target
# start and enable service
systemctl start yo_opensmd
systemctl enable yo_opensmd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment