Largely based on the Tensorflow 1.8 gist, Tensorflow 1.6 gist, and Tensorflow 1.7 gist for xcode, this should hopefully simplify things a bit.
- NVIDIA Web-Drivers 387.10.10.10.40.105 for 10.13.6
- CUDA-Drivers 396.148
- CUDA 9.2 Toolkit (9.2.148 + patch 1)
- cuDNN 7.2.1.38 (latest for macOS)
- NCCL 2.2.13_1 (latest for macOS)
- Python 3.6.5
- XCode 9.2
- bazel stable 0.15.2 (latest on HomeBrew)
- Tensorflow 1.8 Source Code
For package management, ignore if you have your own python, wget or you want to download manually.
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install wgetbrew install pythonMore explanations on Python's site https://docs.python-guide.org/starting/install3/osx/.
One important thing is to install the web driver version and not the native one which is also provided by NVIDIA.
Download and install from https://www.nvidia.com/download/driverResults.aspx/136062/en-us
Download and install from https://www.nvidia.com/object/macosx-cuda-396.148-driver.html
NOTE : Is installed into
/usr/local/cuda
-
Download and install from https://download.developer.apple.com/Developer_Tools/Xcode_9.2/Xcode_9.2.xip.
Or Find
XCode 9.2on https://developer.apple.com/download/more/ -
Unarchive and rename
XCode.apptoXcode9.2.appin case you want to build and use it next time.
If you have Homebrew installed, run :
brew install bazelOr Download the installer https://github.com/bazelbuild/bazel/releases/download/0.15.2/bazel-0.15.2-installer-darwin-x86_64.sh and run :
chmod 755 bazel-0.15.2-installer-darwin-x86_64.sh
./bazel-0.15.2-installer-darwin-x86_64.sh- Download the
CUDA Toolkithttps://developer.nvidia.com/cuda-downloads?target_os=MacOSX&target_arch=x86_64&target_version=1013&target_type=dmglocal from NVIDIA.
There should be 2 packages to download, the core cuda_9.2.148_mac.dmg and a
patch cuda_9.2.148.1_mac.dmg.
- Install them both in the order with the
samplesoption (we will need them later).
NOTE : Is installed into
/Developer/NVIDIA/CUDA-9.2
-
Download
NCCL 2.2.13 O/S agnostic and CUDA 9.2from https://developer.nvidia.com/nccl/nccl-download. -
Unarchive it manualy
tar -xvzf ./nccl_2.2.13-1+cuda9.2_x86_64.txz- And move it to a permanent place e.g.
/usr/local/nccl
sudo mkdir -p /usr/local/nccl
cd nccl_2.2.13-1+cuda9.2_x86_64
sudo mv * /usr/local/nccl
sudo mkdir -p /usr/local/include/third_party/nccl
sudo ln -s /usr/local/nccl/include/nccl.h /usr/local/include/third_party/ncclEdit ~/.bash_profile and add the following according to your existing configuration
export PATH=/usr/local/bin:/usr/local/sbin:$PATH
export CUDA_HOME=/usr/local/cuda
export CUDA_TOOLKIT_HOME=/Developer/NVIDIA/CUDA-9.2
export DYLD_LIBRARY_PATH=$CUDA_TOOLKIT_HOME/lib:$CUDA_HOME/lib:$CUDA_HOME/extras/CUPTI/lib:$DYLD_LIBRARY_PATH
export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
export PATH=$DYLD_LIBRARY_PATH:$CUDA_TOOLKIT_HOME/bin:$PATHWe want to compile some CUDA samples to check if the GPU is correctly recognized and supported.
cd /Developer/NVIDIA/CUDA-9.2/samples
sudo chown -R $(whoami) *
sudo make -C 1_Utilities/deviceQuery
./bin/x86_64/darwin/release/deviceQueryIt should return something similar.
CUDA Device Query (Runtime API) version (CUDART static linking)
Detected 1 CUDA Capable device(s)
Device 0: "GeForce GT 750M"
CUDA Driver Version / Runtime Version 9.2 / 9.2
CUDA Capability Major/Minor version number: 3.0
Total amount of global memory: 2048 MBytes (2147024896 bytes)
( 2) Multiprocessors, (192) CUDA Cores/MP: 384 CUDA Cores
GPU Max Clock rate: 926 MHz (0.93 GHz)
Memory Clock rate: 2508 Mhz
Memory Bus Width: 128-bit
L2 Cache Size: 262144 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 1 copy engine(s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (UVA): Yes
Device supports Compute Preemption: No
Supports Cooperative Kernel Launch: No
Supports MultiDevice Co-op Kernel Launch: No
Device PCI Domain ID / Bus ID / location ID: 0 / 1 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 9.2, CUDA Runtime Version = 9.2, NumDevs = 1
Result = PASS
If not already done, register at https://developer.nvidia.com/cudnn Download cuDNN 7.0.5
Change into your download directory and follow the post installation steps.
tar -xzvf cudnn-9.1-osx-x64-v7-ga.tgz
sudo cp cuda/include/cudnn.h /usr/local/cuda/include
sudo cp cuda/lib/libcudnn* /usr/local/cuda/lib
sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib/libcudnn*cd /tmp
git clone https://github.com/tensorflow/tensorflow
cd tensorflow
git checkout v1.8.0Apply the following patch to fix a couple build issues
NOTE : Command line #2 that adds a new line at the end of the patch is very important!
wget https://gist.github.com/jeanjerome/b3b722bb1632e67251f42b19fcafb65d#file-xtensorflow18macos-patch
sed -i '' -e '$a\' xtensorflow18macos-patch
git apply xtensorflow18macos.patchYou need the following informations to proceed the configuration :
- Python path e.g.
/usr/local/bin/python3, - CUDA SDK version
9.2, - cuDNN version
7.2, - CUDA compute capabilities e.g.
3.0for my poor NVIDIA GeForce GT 750M.
Pay attension to CUDA compute capabilities, you might want to find your own according to the output of the previous deviceQuery script.
So now run
./configureYou should see a series of prompts
Extracting Bazel installation...
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.15.2-homebrew installed.
Please specify the location of python. [Default is /usr/bin/python]: /usr/local/bin/python3
Found possible Python library paths:
/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
Please input the desired Python library path to use. Default is [/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages]
Do you wish to build TensorFlow with Google Cloud Platform support? [Y/n]: n
No Google Cloud Platform support will be enabled for TensorFlow.
Do you wish to build TensorFlow with Hadoop File System support? [Y/n]: n
No Hadoop File System support will be enabled for TensorFlow.
Do you wish to build TensorFlow with Amazon S3 File System support? [Y/n]: n
No Amazon S3 File System support will be enabled for TensorFlow.
Do you wish to build TensorFlow with Apache Kafka Platform support? [Y/n]: n
No Apache Kafka Platform support will be enabled for TensorFlow.
Do you wish to build TensorFlow with XLA JIT support? [y/N]:
No XLA JIT support will be enabled for TensorFlow.
Do you wish to build TensorFlow with GDR support? [y/N]:
No GDR support will be enabled for TensorFlow.
Do you wish to build TensorFlow with VERBS support? [y/N]:
No VERBS support will be enabled for TensorFlow.
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]:
No OpenCL SYCL support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.
Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 9.0]: 9.2
Please specify the location where CUDA 9.2 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7.0]: 7.2
Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 3.5,5.2]3.0
Do you want to use clang as CUDA compiler? [y/N]:
nvcc will be used as CUDA compiler.
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
Do you wish to build TensorFlow with MPI support? [y/N]:
No MPI support will be enabled for TensorFlow.
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:
Not configuring the WORKSPACE for Android builds.
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See tools/bazel.rc for more details.
--config=mkl # Build with MKL support.
--config=monolithic # Config for mostly static monolithic build.
Configuration finishedTakes a bit more than 1.5 hour on my machine.
bazel clean
bazel build --config=cuda --config=opt --action_env PATH --action_env LD_LIBRARY_PATH --action_env DYLD_LIBRARY_PATH //tensorflow/tools/pip_package:build_pip_packagebazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
ls /tmp/tensorflow_pkg
tensorflow-1.8.0-cp36-cp36m-macosx_10_13_x86_64.whlIf you want to use virtualenv or something, now is the time. Or just:
pip install /tmp/tensorflow_pkg/tensorflow-1.8.0-cp36-cp36m-macosx_10_13_x86_64.whlFiles in /tmp would be cleaned after reboot.
cp /tmp/tensorflow_pkg/*.whl ~/It's useful to leave the .whl file lying around in case you want to install it for another environment.
I am using virtualenv (and you should do) so activate it
source ./bin/activate
pythonPython 3.6.5 (default, Jun 17 2018, 12:13:06)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.Session()
2018-08-18 18:56:16.456756: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:859] OS X does not support NUMA - returning NUMA node zero
2018-08-18 18:56:16.457496: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties:
name: GeForce GT 750M major: 3 minor: 0 memoryClockRate(GHz): 0.9255
pciBusID: 0000:01:00.0
totalMemory: 2.00GiB freeMemory: 1.51GiB
2018-08-18 18:56:16.457532: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-08-18 18:56:17.305242: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-08-18 18:56:17.305278: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929] 0
2018-08-18 18:56:17.305288: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0: N
2018-08-18 18:56:17.305963: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1280 MB memory) -> physical GPU (device: 0, name: GeForce GT 750M, pci bus id: 0000:01:00.0, compute capability: 3.0)
<tensorflow.python.client.session.Session object at 0x1035d9c50>Experience the new Autokeras framework. It runs on tensorflow, with Python 3. Lucky we are !
pip install autokeras
wget https://gist.github.com/jeanjerome/b3b722bb1632e67251f42b19fcafb65d/raw/mnist_test.py
python mnist_test.pyUsing TensorFlow backend.
2018-08-18 19:22:52.374988: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:859] OS X does not support NUMA - returning NUMA node zero
2018-08-18 19:22:52.375213: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties:
name: GeForce GT 750M major: 3 minor: 0 memoryClockRate(GHz): 0.9255
pciBusID: 0000:01:00.0
totalMemory: 2.00GiB freeMemory: 1.50GiB
2018-08-18 19:22:52.375236: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-08-18 19:22:52.670486: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-08-18 19:22:52.670521: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929] 0
2018-08-18 19:22:52.670525: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0: N
2018-08-18 19:22:52.670607: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1157 MB memory) -> physical GPU (device: 0, name: GeForce GT 750M, pci bus id: 0000:01:00.0, compute capability: 3.0)
Initializing search.
Initialization finished.
Training model 0
Using TensorFlow backend.
2018-08-18 19:22:56.373450: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:859] OS X does not support NUMA - returning NUMA node zero
2018-08-18 19:22:56.373636: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties:
name: GeForce GT 750M major: 3 minor: 0 memoryClockRate(GHz): 0.9255
pciBusID: 0000:01:00.0
totalMemory: 2.00GiB freeMemory: 198.35MiB
2018-08-18 19:22:56.373662: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-08-18 19:22:56.680835: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-08-18 19:22:56.680870: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929] 0
2018-08-18 19:22:56.680875: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0: N
2018-08-18 19:22:56.680956: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 161 MB memory) -> physical GPU (device: 0, name: GeForce GT 750M, pci bus id: 0000:01:00.0, compute capability: 3.0)
...........................................
Epoch 1: loss 2.106443405151367, metric_value 0.9856
...........................................
Epoch 2: loss 1.945971131324768, metric_value 0.984
...........................................
Epoch 3: loss 1.328037977218628, metric_value 0.9886
...........................................
Epoch 4: loss 1.2873637676239014, metric_value 0.99
With my GPU, each epoch lasts 02:28 min.
With the CPU only, that lasts 02:35 min.
Well, I couldn't be more pleased with my GeForce GT 750M. But those who have a NVIDIA GeForce GTX 10xx with eGPU should experiment a sharp acceleration.
Tested on a MacBook Pro (15-inch, mi-2014) 10.13.6, 2.5 GHz Intel Core i7 and NVIDIA GeForce GT 750M :(