ssh port fowarding

1_forward.md

We have two machines: master and agent. The goal is for agent to ssh into master, allocating a local port on master which forwards to a port on agent.

In this example, we'll use ssh - but that's arbitrary. The "master" is lightning and the "agent" is fabulinus.

First: open the ssh connection:

sauer@fabulinus:~$ ssh -Nf -R 10022:localhost:22 lightning

This will fork ssh into the background (-f) and not run a program (-N). Thus, it's just forwarding the port and nothing else.

sauer@fabulinus:~$ ps -C ssh -o pid,cmd
    PID CMD
3808164 ssh -Nf -R 10022:localhost:22 lightning

Now, on lightning, verify that the connection is listening:

sauer@lightning:~$ sudo lsof -P -c ssh | grep -i 10022
sshd    7554 sauer   11u  IPv6            1534266      0t0     TCP localhost:10022 (LISTEN)
sshd    7554 sauer   12u  IPv4            1534267      0t0     TCP localhost:10022 (LISTEN)

There are two open ports - one ipv4 and one ipv6, both on the loopback interface. I also put an entry at the end of /etc/hosts to allow connectiions by name:

sauer@lightning:~$ grep fake /etc/hosts
127.0.0.1      fake_fabulinus

I put the entry at the end, because reverse-resolution works top-don in /etc/hosts, so the exiting localhost entry in there will still reverse resolve 127.0.0.1 to localhost, as expected. But the hostname will resolve to the IP. You can have multiple hosts on a line, but just adding this to the end f not present is marginally easier to script than editing the line.

So, I should be able to ssh now.

sauer@lightning:~$ ssh fake_fabulinus -p 10022 hostname
The authenticity of host '[fake_fabulinus]:10022 ([127.0.0.1]:10022)' can't be established.
ECDSA key fingerprint is SHA256:I1Z9Y2biEz7EkEHztIU2ioJiZ5y8xsK/IeCaizl7CMk.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[fake_fabulinus]:10022' (ECDSA) to the list of known hosts.
sauer@fake_fabulinus's password: 
fabulinus

This could be repeated with multiple ports for multiple hosts. The ssh port forward command can also have multiple -R options added to forward multiple ports.

2_automation.md

To automate this, I like to use ssh config and systemctl.

First, for the user creating the forwarded connections, edit ~/.ssh/config:

Host master_portforward
  User remoteusername
  Hostname master
  HostKeyAlias master
  ExitOnForwardFailure yes
  #LogLevel FATAL
  LogLevel VERBOSE
  Compression yes
  PreferredAuthentications publickey
  # ssh
  RemoteForward localhost:10022   localhost:22
  # imap
  RemoteForward localhost:10143  localhost:143
  # if the ports on master should be open on master's public IP
  # GatewayPorts yes

In that example, two ports are forwarded from master to agent - ssh and imap. The Hostname and HostKeyAlias are there so it's still possible to ssh master as usual. However, using the fake hostname as in ssh master_portforward trips this part of the ssh config and loads the forwards, etc. It also uses verbose logging for journald, and exits when forwarding fails (normally that just prints a warning).

Now, the automation. That's just this service file:

[Unit]
Description=master ssh tunnel
After=sshd.service
Wants=dbmail-imapd.service sshd.service
# disable restart interval
StartLimitIntervalSec=0
# maybe try 500 times?
StartLimitBurst=500

[Service]
ExecStart=/usr/bin/ssh -N master_portforward
Restart=always
User=port_forward_user

[Install]
WantedBy=multi-user.target

It's configured to wait until after the imap and ssh servers are started (since that's what's being forwarded). It also starts after sshd, mostly just because that's easier than figuring out what the network service is called. Note that the command run is ssh -N fake_hostname. The -N suppresses runnign a command / allocating a terminal, and leaving the -f off from above means it's easy for systemctl to notice if ssh goes away and restart it.

Just sudo systemctl edit --full --force master_forward.service, paste that in, adjust the User and Wants lines, and enjoy. Using systemctl edit automatically runs the dameon reload that everyone forgets, and saves you from having to figure out the file location. :) To adjust the list of ports, just edit the .ssh/config and systemctl restart the service.