TensorFlow-GPU on a dedicated remote PC, using a Mac for development

00-readme.md

My random notes on the adventure of developing with TensorFlow on a Mac, running calculations on a NVIDIA GPU in a separate PC. Your miles may vary.

cygwin-method.md

The Cygwin method

Install Cygwin

Install cygwin with these extra packages:

• openssh >= 6.1-p
• cygrunsrv >= 1.40-2
• nano
• rsync

Run cygwin terminal as administrator:

ssh-host-config
# answer yes to all, answer default ([]) for "Enter the value of CYGWIN for the daemon"

Allow port 22 in Windows firewall: (instructions originate from this gist)

• Control Panel
  • System and Security
    • Windows Firewall
      • On the left select "Advanced Settings"
      • Select "Inbound Rules"
      • "New Rule..."
        • Port
        • TCP: Specific ports: 22
        • Allow the connection
        • Domain, [x] Private, [x] Public

Try it by opening a new Cygwin terminal:

ssh localhost

On another machine you want to connect from, add your public key of that machine: (run on your other machine, e.g. mac)

ssh-copy-id USER@HOSTNAME_OR_ADDRESS_OF_WINDOWS_MACHINE

Install Miniconda

https://docs.conda.io/en/latest/miniconda.html

• During installation, check the "expose conda in PATH" even though it has dangerous red text.

Create a Miniconda environment and install tensorflow-gpu

Open a Cygwin terminal as administrator, then:

conda create --name .conda_venv_tf
/cygdrive/c/ProgramData/Miniconda3/Scripts/activate.bat .conda_venv_tf
conda install tensorflow-gpu
python - <<_PY_
import tensorflow as tf
print(tf.config.list_physical_devices('GPU'))
_PY_

Notes

• Cygwin doesn't have a CLI package manager, but the installation program is one! For example, to install nano run: /cygdrive/c/Users/$USER/Downloads/setup-x86_64.exe -q -P nano,rsync

wsl-method.md

TF with CUDA on Windows 10 via WSL using MiniConda

Setup for using a Windows PC with an NVIDIA GPU to run tensorflow

Prerequisites:

• A dedicated Windows computer
• An NVIDIA GPU (on the Windows computer) that meets the requiresments of TensorFlow
• Familiarity with Linux and Python

1. Install Ubuntu in WSL with SSH

Install a recent Ubuntu in WSL from for example the Microsoft Store. When done, open a WSL terminal and...

Setup openssh-server:

sudo apt install openssh-server
sudo nano /etc/ssh/sshd_config
# set PasswordAuthentication yes
sudo service ssh --full-restart

On the main machine, transfer your public key to the WSL machine:

ssh-copy-id USER@HOSTNAME_OR_ADDRESS_OF_WINDOWS_MACHINE

Disable PasswordAuthentication in ssh server on WLS machine:

sudo nano /etc/ssh/sshd_config
# set PasswordAuthentication no
sudo service ssh --full-restart

You can now do the rest from your main machine

2. Install Miniconda

With an ssh session to your GPU machine...

• wget the sh script at https://docs.conda.io/en/latest/miniconda.html
• install miniconda using the downloaded bash script

Create a conda environment and install tensorflow-gpu:

conda create --name .conda_venv_tf
conda activate .conda_venv_tf
conda install tensorflow-gpu

3. Install nodejs

curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
sudo apt-get install -y nodejs