meetmatt/openstack-ssh-issue.md

## openstack-ssh-issue.md

      
    Raw
  

              openstack-ssh-issue.md
            
          
    Setup

Host: Ubuntu 20.04

OVS bridge: br-ex

Openstack security group: allow all ip4.

Single network interface: 192.168.1.100, gateway 192.168.1.1 (home router).

The IP from the physical interface is moved to br-ex, physical nic added as port (that way I can allocate floating IPs from the home router's network 192.168.1.1/24).

Openstack: two networks, private (10.10.0.0/24), public (192.168.1.1/24).

Virual router (10.10.0.1 / 192.168.1.212) provides NAT to private network.

On the openstack host there's a manually added routing table: 10.10.0.0/24 via 192.168.1.212.

Instance: 10.10.0.135, Ubuntu 20.04 cloud image, default cloud config, except that I enabled root password to debug.

Instances have internet, host can ping instances, instances can ping host, router's gateway IP is also pingable (192.168.1.212).

I can telnet from host to instance's 2222 port, but ssh connection fails with the following errors:
on the client (host):
user@linux:~$ ssh -vvv 10.10.0.165 -p 2222 -i ~/.ssh/id_rsa
OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug2: resolve_canonicalize: hostname 10.10.0.165 is address
debug2: ssh_connect_direct
debug1: Connecting to 10.10.0.135 [10.10.0.165] port 2222.
debug1: Connection established.
debug1: identity file /home/user/.ssh/id_rsa type 0
debug1: identity file /home/user/.ssh/id_rsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.2
kex_exchange_identification: read: Connection reset by peer

TCP dump on the client, host:

22:35:30.325816 IP linux.43346 > 10.10.0.165.2222: Flags [S], seq 3223694581, win 64240, options [mss 1460,sackOK,TS val 1819428723 ecr 0,nop,wscale 7], length 0
22:35:30.330191 IP linux.43346 > 10.10.0.165.2222: Flags [.], ack 958130231, win 502, options [nop,nop,TS val 1819428728 ecr 4186453357], length 0
22:35:30.330336 IP linux.43346 > 10.10.0.165.2222: Flags [P.], seq 0:41, ack 1, win 502, options [nop,nop,TS val 1819428728 ecr 4186453357], length 41
22:35:30.337408 IP linux.43346 > 192.168.1.212.2222: Flags [R], seq 3223694623, win 0, length 0
22:35:30.539379 IP linux.43346 > 10.10.0.165.2222: Flags [P.], seq 0:41, ack 1, win 502, options [nop,nop,TS val 1819428937 ecr 4186453357], length 41

Logs from the instance (sshd -d -p 2222)

debug1: sshd version OpenSSH_8.2, OpenSSL 1.1.1f  31 Mar 2020
debug1: private host key #0: ssh-rsa SHA256:eoIbj8/sd4pOqJ+KoOa0m4L45XLbZQeKTJST5SEg+l4
debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:vmGFEFwwqiyXC16NfSub1At2wF8628KJePuOdBUU8XQ
debug1: private host key #2: ssh-ed25519 SHA256:80+iGAzYQFnaMrxHNlJhyz1W57okgMGJ4dnPljZ+Q9Y
debug1: rexec_argv[0]='/usr/sbin/sshd'
debug1: rexec_argv[1]='-d'
debug1: rexec_argv[2]='-p'
debug1: rexec_argv[3]='2222'
debug1: Set /proc/self/oom_score_adj from 0 to -1000
debug1: Bind to port 2222 on 0.0.0.0.
Server listening on 0.0.0.0 port 2222.
debug1: Bind to port 2222 on ::.
Server listening on :: port 2222.
debug1: Server will not fork when running in debugging mode.
debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8
debug1: sshd version OpenSSH_8.2, OpenSSL 1.1.1f  31 Mar 2020
debug1: private host key #0: ssh-rsa SHA256:eoIbj8/sd4pOqJ+KoOa0m4L45XLbZQeKTJST5SEg+l4
debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:vmGFEFwwqiyXC16NfSub1At2wF8628KJePuOdBUU8XQ
debug1: private host key #2: ssh-ed25519 SHA256:80+iGAzYQFnaMrxHNlJhyz1W57okgMGJ4dnPljZ+Q9Y
debug1: inetd sockets after dupping: 3, 3
debug1: getpeername failed: Transport endpoint is not connected
debug1: ssh_remote_port failed

TCP dump on the instance:

# tcpdump port 2222
14:32:28.041983 IP 192.168.1.100.57982 > test.2222: Flags [S], seq 3892836825, win 64240, options [mss 1460,sackOK,TS val 2597138009 ecr 0,nop,wscale 7], length 0
14:32:28.044350 IP test.2222 > 192.168.1.100.57982: Flags [S.], seq 1250637171, ack 3892836826, win 65160, options [mss 1460,sackOK,TS val 2370174790 ecr 2597138009,nop,wscale 7], length 0
14:32:28.045111 IP 192.168.1.100.57982 > test.2222: Flags [.], ack 1, win 502, options [nop,nop,TS val 2597138013 ecr 2370174790], length 0
14:32:28.049972 IP 192.168.1.100.57982 > test.2222: Flags [P.], seq 1:42, ack 1, win 502, options [nop,nop,TS val 2597138017 ecr 2370174790], length 41
14:32:28.050087 IP test.2222 > 192.168.1.100.57982: Flags [.], ack 42, win 509, options [nop,nop,TS val 2370174796 ecr 2597138017], length 0
14:32:28.050699 IP 192.168.1.100.57982 > test.2222: Flags [R], seq 3892836867, win 0, length 0
14:32:28.256557 IP 192.168.1.100.57982 > test.2222: Flags [P.], seq 1:42, ack 1, win 502, options [nop,nop,TS val 2597138224 ecr 2370174790], length 41
14:32:28.257220 IP test.2222 > 192.168.1.100.57982: Flags [R], seq 1250637172, win 0, length 0

Btw, I can ssh from the instance to the host easily.