Skip to content

Instantly share code, notes, and snippets.

@1duo

1duo/centos.install.mellanox.gpudirect.md

Last active April 4, 2018 00:36
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save 1duo/666d749ac7bf24ac4cc4f67984756edf to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save 1duo/666d749ac7bf24ac4cc4f67984756edf to your computer and use it in GitHub Desktop.
Download ZIP
Install Mellanox OFED GPUDirect RDMA on CentOS 7.
Raw
centos.install.mellanox.gpudirect.md

Assueme NVIDIA Driver and CUDA already successfully installed.

  • Install Dependencies
sudo yum install -y tcl tk
  • Install MLNX_OFED v2.1-x.x.x or later

Download: www.mellanox.com -> Products -> Software - > InfiniBand/VPI Drivers -> Linux SW/Drivers

tar xvzf MLNX_OFED_LINUX-4.1-1.0.2.0-rhel7.3-x86_64.tgz
sudo ./mlnxofedinstall

Reboot the system for the changes to take effect.

  • Install Plugin module to enable GPUDirect RDMA

Download: www.mellanox.com -> Products -> Software - > InfiniBand/VPI Drivers -> GPUDirect RDMA

tar xvzf nvidia-peer-memory_1.0.5.tar.gz
cd nvidia-peer-memory-1.0
./build_module.sh
sudo rpmbuild --rebuild /tmp/nvidia_peer_memory-1.0-5.src.rpm
sudo rpm -ivh /root/rpmbuild/RPMS/x86_64/nvidia_peer_memory-1.0-5.x86_64.rpm
  • Validate installtion
sudo service nv_peer_mem start
sudo service nv_peer_mem status

References:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment