Building mxnet with CUDA 8 GPU support in R - Windows Instructions (Nov 19, 2016)

readme.md

Building mxnet with CUDA 8 GPU support in R - Windows Instructions (Nov 19, 2016)

The goal here is to build mxnet.io on Windows. This was tested on Windows 10. Overall, the procedure is to compile mxnet to generate an up-to-date libmxnet.dll. Once we have that, we will build and install the R package containing all dependent libraries, including the newly-built libmxnet.dll.

This guide was adapted from the primary setup documentation http://mxnet.io/get_started/setup.html which seems out-of-date.

Dependencies

• Visual Studio 2013 (tested with Community Edition)
• cmake - from here
• OpenBLAS v0.2.19 - win64 binary package called OpenBLAS-v0.2.19-Win64-int32.zip from here
• OpenCV v3.1.0 - opencv-3.1.0.exe from here
• CUDA v8 - cuda_8.0.44_win10.exe from here
• CuDNN v5.1 - cudnn-8.0-windows10-x64-v5.1.zip from here. Note you need to register with nVidia to download.
• R and RTools - tested R v3.3.2 (R-3.3.2-win.exe from here) and RTools v3.4 (Rtools34.exe from here)

Steps to Install Dependencies

Assume that packages will be saved to C:\Development\. Feel free to choose another place, but update the below paths accordingly.

1. Install Visual Studio 2013

2. Unzip cmake to C:\Development\cmake\. You should now have:

   C:\Development\cmake\bin\cmake-gui.exe

3. Unzip OpenBlas to C:\Development\OpenBLAS-v0.2.19-Win64-int32\. You should now have the following directories:

   C:\Development\OpenBLAS-v0.2.19-Win64-int32\bin C:\Development\OpenBLAS-v0.2.19-Win64-int32\lib C:\Development\OpenBLAS-v0.2.19-Win64-int32\include

4. Unzip OpenCV to C:\Development\opencv-3.1.0\. You should now have the following directories:

   C:\Development\opencv-3.1.0\build C:\Development\opencv-3.1.0\sources

5. Run the CUDA v8 installation. You should now have the following directories:

   C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin

6. Unzip CuDNN v5.1 to C:\Development\cudnn-8.0-windows10-x64-v5.1\. You should now have directories called:

   C:\Development\cudnn-8.0-windows10-x64-v5.1\cuda\bin C:\Development\cudnn-8.0-windows10-x64-v5.1\cuda\include C:\Development\cudnn-8.0-windows10-x64-v5.1\cuda\lib

7. Install R and RTools

Building mxnet

To build mxnet, we'll need to use cmake to configure an appropriate Visual Studio solution (.sln) file. Once that is complete, we can open the solution file and build a release version of mxnet in Visual Studio 2013.

1. Clone the latest mxnet source code to C:\Development\mxnet\ using git clone --recursive https://github.com/dmlc/mxnet.git to ensure the submodules are fetched. You should have at least the following directories and they should not be empty:

   C:\Development\mxnet\cmake C:\Development\mxnet\R-package C:\Development\mxnet\dmlc-core C:\Development\mxnet\mshadow C:\Development\mxnet\ps-lite

2. Run the cmake GUI (cmake-gui.exe from above)

   • In the Where is the source code: field, enter the mxnet source code directory which should be C:\Development\mxnet\.

   • In the Where to build the binaries: field, enter C:\Development\mxnet\build\.

   • Make sure "Grouped" is checked, then click the configure button towards the bottom of the GUI.

   • A dialog will pop up and you should change the generator to be Visual Studio 12 2013 Win64, then click Finish.

   • After running for a bit, there will be some red lines in the cmake interface that need to be fixed. The general procedure is to update the RED lines and click configure again. Here are somethings that need to be checked/fixed:

     • Make sure that under the USE, all of USE_CUDA, USE_CUDNN, USE_MXNET_LIBNAMING, USE_OPENCV, and USE_OPENMP are all checked and the remainder are unchecked.
     • OpenCV_DIR and OpenCV_CONFIG_PATH should be C:/Development/opencv-3.1.0/build.
     • CUDNN_INCLUDE should be C:/Development/cudnn-8.0-windows10-x64-v5.1/cuda/include.
     • CUDNN_LIBRARY should be C:/Development/cudnn-8.0-windows10-x64-v5.1/cuda/lib/x64/cudnn.lib
     • CUDA_TOOLKIT_ROOT_DIR should be C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0
     • OpenBLAS_INCLUDE_DIR should be C:/Development/OpenBLAS-v0.2.19/include
     • OpenBLAS_LIB should be C:/Development/OpenBLAS-v0.2.19/lib/libopenblas.dll.a

   • Once the last Configure run completes with Configuring done in th log window, click Generate.

   • The log window should say Generating done.

At this point we can proceed to build mxnet libraries in Visual studio.

3. Go to C:\Development\mxnet\build\ and double-click to open the mxnet.sln file. This will open Visual Studio 2013.

4. Make sure the toggles on the toolbar read Release and x64, then click to the Build > Build Solution.

5. Wait for the compilation and linking procedures to finish (took approximately 15 minutes on my laptop). When complete you should have the following file available:

   C:\Development\mxnet\build\Release\libmxnet.dll

Building the mxnet R package

The mxnet source code contains a directory called C:\Development\mxnet\R-package\. This is the base directory used to compile and build an R package which can then be installed on any windows machine. This package will contain all necessary DLLs to run the mxnet libraries, so many DLLs will need to be copied to the right place.

Some of the DLLs will be the ones installed already (i.e. from CUDA, CuDNN, OpenBLAS, etc.) but there are some that we will copy from the primary mxnet R package that doesn't contain up-to-date GPU support.

1. Go to the main release page and download the latest windows build release for GPU: 20160531_win10_x64_gpu.7z.

2. Unzip this to C:\Development\20160531_win10_x64_gpu\ and you should have the following directories:

   C:\Development\20160531_win10_x64_gpu\3rdparty C:\Development\20160531_win10_x64_gpu\include

3. Go to C:\Development\mxnet\R-package\ and make the following directories:

   C:\Development\mxnet\R-package\inst\include
   C:\Development\mxnet\R-package\inst\libs\x64\

4. Copy the three directories named dmlc, mshadow, mxnet in C:\Development\20160531_win10_x64_gpu\include to C:\Development\mxnet\R-package\inst\include\.

5. Copy C:\Development\20160531_win10_x64_gpu\3rdparty\gnuwin\unzip32.dll to C:\Development\mxnet\R-package\inst\libs\x64\.

6. From the C:\Development\20160531_win10_x64_gpu\3rdparty\openblas\bin directory, copy libgcc_s_seh-1.dll, libgfortran-3.dll, and libquadmath-0.dll to C:\Development\mxnet\R-package\inst\libs\x64\.

7. From the C:\Users\flam\Downloads\20160531_win10_x64_gpu\3rdparty\vc\ directory, copy vcomp120.dll to C:\Development\mxnet\R-package\inst\libs\x64\.

8. From the C:\Development\OpenBLAS-v0.2.19-Win64-int32\bin directory, copy libopenblas.dll to C:\Development\mxnet\R-package\inst\libs\x64\.

9. From the C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin directory, copy cublas64_80.dll, cudart64_80.dll, curand64_80.dll, nvrtc64_80.dll to C:\Development\mxnet\R-package\inst\libs\x64\.

10. From the C:\Development\cudnn-8.0-windows10-x64-v5.1\cuda\bin directory, copy cudnn64_5.dll to C:\Development\mxnet\R-package\inst\libs\x64\.

11. From the C:\Development\opencv-3.1.0\build\x64\vc12\bin directory, copy opencv_world310.dll to C:\Development\mxnet\R-package\inst\libs\x64\.

12. Finally copy the locally built mxnet library C:\Development\mxnet\build\Release\libmxnet.dll to C:\Development\mxnet\R-package\inst\libs\x64\. At this point there should be 13 dlls in that directory.

We're now ready to build and install the R package.

13. Open up a command prompt and cd to C:\Development\mxnet\.

14. If your R.exe isn't on the path, add it or just reference it directly and run:

    "C:\Program Files\R\R-3.3.2\bin\R.exe" CMD INSTALL --build --no-multiarch R-package

15. Once it has completed building the package, you should have a file called C:\Development\mxnet\mxnet_0.7.zip and the following message:

    packaged installation of 'mxnet' as mxnet_0.7.zip

    • DONE (mxnet)

That mxnet_0.7.zip can now be distributed to any other Windows nodes that need the mxnet R package by just using install.packages() and referencing the mxnet_0.7.zip as the local package.

Test out the R package

If you run require(mxnet), this should load properly, but you can also run a more complex test using instructions from the main page: http://mxnet.io/tutorials/r/mnistCompetition.html.

I am very appreciate with your install tutorial. and...
I have two question about making the 'libmxnet.dll' in order to construct GPU usable environment.
Because of my limited computer hardware and software background, I would like to ask some question to you.
Could you mind if you comment my problems?

1. "Download the latest mxnet.io source from Github or use git to clone to C:\Development\mxnet"
   I am wondering whether you regard 'this source file' as 'https://github.com/dmlc/mxnet'

When I try to click 'Clone or download' and downloading done, some folders(dmlc-core, mshadow, ps-lite) empty.
So I have to download this inner file from
https://github.com/dmlc/dmlc-core/tree/39007ac49b6087339dc3104324cb4e0de47f1c5f
https://github.com/dmlc/mshadow/tree/f67e112dcfe7e054cb7866d289d8c826808bd359
https://github.com/dmlc/ps-lite/tree/4a060e4e8aa40c3a931a0f8af9211279e012f8a2
and combine previous mxnet folder.
I wonder if I'm doing well

2. "Run the cmake GUI (cmake-gui.exe from above)"
   When I tried to check your list that need to be fixed, cmake showed not finding about python 2.x, python 3.x location.
   I ignored these issues, and made libmxnet.dll file finally.
   Is it okay ignore these python issues?

@JHBeak16 sorry for the late reply. I did a recursive clone git clone --recursive https://github.com/dmlc/mxnet.git so it pulls it all the submodules at the same time. I'll update the gist to note that.

As for the python issues, I'm not sure. I had Anaconda installed, so I didn't have those issues.

Thanks for posting, this was super helpful in getting it to compile!

Hi,
I'm having an issue while running point 14, when i run CMD INSTALL --build --no-multiarch R-package, i'm getting this message

I've done all the previus steps without any error, can you help me ?

javier, I'm getting the same issue, did you find a solution?

No, I haven't, i check the history of commits on the R-package repo and i found a suspicious commit, where they delete the NAMESPACE file, now is autogenerated, but generating this file didn't work for me either.

I'm waiting for the next release to try again, and be able to speed up my models.

https://github.com/dmlc/mxnet/commit/a210bd6f4995069b4d3e8dc47eb551ce5587a690#diff-63f5b071c6f25fbe78a468efde1055c9

I created the NAMESPACE file manually and R installed mxnet successfully.

What is your error messages?

kuent@KBLADE C:\dev\mxnet\buildLibrary\mxnet

R CMD INSTALL --build --no-multiarch R-package

• installing to library 'C:/Users/kuent/OneDrive/Dokumente/R/win-library/3.4'
• installing source package 'mxnet' ...
  ** libs
  make: Nothing to be done for `all'.
  installing to C:/Users/kuent/OneDrive/Dokumente/R/win-library/3.4/mxnet/libs/x64
  ** R
  ** demo
  ** inst
  ** preparing package for lazy loading
  ** help
  No man pages found in package 'mxnet'
  *** installing help indices
  ** building package indices
  ** installing vignettes
  ** testing if installed package can be loaded
• MD5 sums
  packaged installation of 'mxnet' as mxnet_0.10.0.zip
• DONE (mxnet)