I've been using a GPU workstation with an RTX 4090 for almost a year now, and it's been one of the best decisions I've made. With a personal GPU server, you no longer need to rely on cloud-based GPU instances from services like RunPod or Vast.ai every time you want to run a job or try new models. The best part? No stress about recurring GPU instance costs! :-)
However, I rarely work directly on my workstation. Instead, I prefer the flexibility of accessing the GPU remotely using my MacBook, whether I'm working from different locations within my home, from a co-working space, or a cozy cafe in another part of town.
In this blog, I will walk you through the steps to configure a personal GPU Ubuntu server.
For this guide, I assume you already have a workstation running Ubuntu with a GPU and it is connected to your local network
Let's start by setting up local access, which will allow you to ssh into your GPU server when you're on the same home Wi-Fi network. This is ideal for a work-from-home (WFH) setup where your workstation is running in a corner of your living space.
First, we need to install an SSH (Secure Shell) server. This will allow you to securely access your GPU machine remotely. Open a terminal on your Ubuntu machine and run the following commands:
sudo apt update &&
sudo apt install openssh-server
This command updates your package lists and installs the OpenSSH server.
Next, enable the SSH service using this command:
sudo systemctl enable --now ssh
You can verify if the service is enabled by running:
sudo systemctl status ssh
Look for a line starting with Active: active (running) for ssh.service. This indicates that the SSH service is up and running.
Note: The OpenSSH server starts running on boot by default.
To allow SSH connections through the system firewall, you need to open the appropriate port. Ubuntu's default firewall, UFW (Uncomplicated Firewall), makes this process straightforward:
sudo ufw allow ssh
This command adds an exception to your firewall rules, permitting incoming SSH connections. You can check the SSH status with:
sudo ufw status
You should see the output similar to:
To Action From
-- ------ ----
22/tcp ALLOW Anywhere
22/tcp(v6) ALLOW Anywhere (v6)
Now that your GPU server is set up, it's time to test the connection. From your laptop (which should be on the same local network as your GPU machine), open a terminal and use the following command:
ssh user@local-ip-address
Replace user with your Ubuntu user and local-ip-address with the IP address of your GPU machine on the local network.
- To find your username on the workstation, you can use the
whoamicommand. - To find your local IP address, use one of these methods on your workstation:
- Run
hostname -Iand use the first address listed. - Use
ip addr show | grep -winet for more detailed network information. - How to find my IP address on Ubuntu Linux is a great blog on it. It explains multiple commands like
ip addr show | grep -w inetornetworkctl statusto get the local IP address.
- Run
Your local IP address typically starts with 192.168.
Note: If your router dynamically changes the local IP address of your workstation, it's best to log into your router and assign a fixed local IP address to ensure consistent access.
If everything is configured correctly, you'll be prompted to enter your password, after which you'll gain remote access to your GPU server.
It is recommended to set up key-based authentication for better security and convenience purposes. This allows you to connect to your remote server without entering a password each time.
- It is quite common to setup ssh key-based authentication.
- For detailed instructions on setting up SSH keys, refer to the DigitalOcean guide on Setting up SSH keys on Ubuntu 20.04.
While local access is great for working within your home network, sometimes you need to access your GPU workstation from outside your local network, such as from co-working spaces or a cozy cafe.
One simple and secure way to achieve this is by using ngrok.
ngrok helps creates secure tunnels from public endpoints to locally running services. It allows you to expose your personal server to the internet, enabling remote access from anywhere without complex network configurations.
Here's how to set it up:
First, you need to install ngrok on your GPU workstation. Open a terminal and run this command:
snap install ngrok- For more installation options, see https://dashboard.ngrok.com/get-started/setup/linux.
Visit ngrok's website and sign up for a free account if you haven't already.
After signing up, you'll receive an auth token. On your GPU workstation, run:
ngrok config add-authtoken YOUR_AUTH_TOKEN
You can get the config file path and edit using ngrok config check and vim <path>, respectively.
Now, you can create a secure tunnel to your SSH service:
ngrok tcp 22
This command will display a URL that looks like tcp://X.tcp.ngrok.io:PORT. Note down this URL.
From any external laptop, you can now SSH into your GPU workstation using:
ssh -p YYYY user@X.tcp.ngrok.io
Replace PORT with the port number and X with the subdomain from the ngrok URL. Replace user with your Ubuntu username.
The above steps ensure that you can remotely access the workstation from external network. However, no one is going to manually start the ngrok every time before heading out.
To ensure ngrok starts automatically when your workstation boots:
sudo vim /etc/systemd/system/ngrok.service
Add the following content:
[Unit]
Description=start ngrok tunnel on startup
After=network.target
[Service]
ExecStart=/snap/bin/ngrok tcp 22
Restart=on-failure
User=<your_username>
[Install]
WantedBy=multi-user.target
Replace <your_username> with your Ubuntu username. Save the file and exit the editor.
sudo systemctl enable ngrok.service
sudo systemctl start ngrok.service
Now ngrok will automatically start and create a tunnel when your workstation boots.
Note: With a free account, ngrok assigns a new port (YYYY) each time your workstation boots. You can get the new port from the ngrok dashboard.
For a dedicated TCP endpoint port that doesn't change on reboot, you need a paid ngrok personal account ($10/month).
- Once you have a paid account, reserve a TCP endpoint at https://dashboard.ngrok.com/cloud-edge/tcp-addresses.
Add the following content:
[Unit]
Description=start ngrok tunnel on startup
After=network.target
[Service]
ExecStart=/snap/bin/ngrok tcp --region=<region> --remote-addr=<remote-address> 22
Restart=on-failure
User=<your_username>
[Install]
WantedBy=multi-user.target
Replace <region>, <remote-address>, and <your_username> with the appropriate values from your reserved TCP endpoint config.
With this setup, your SSH remote endpoint will remain the same even if the system reboots.