onetun, smoltcp, wireguard

onetun-architecture.md

Tcp

tcp_proxy_server

Starts an ordinary TcpListener which is bound to port_forward.source. After accepting a connection, the socket is passed to handle_tcp_proxy_connection function along with virtual_port, port_forward and wg.

handle_tcp_proxy_connection function

This function:

• prepares channels
• create a TcpVirtualInterface, and
• starts virtual_interface.poll_loop(), which is the heavy-lifter controls communication b/w wg and the virtual TCP stack.

virtual_interface.poll_loop() function

virtual_interface.poll_loop()

1. Creates a VirtualIpDevice.
2. Creates an Interface that wraps the device (i.e., Interface<VirtualIpDevice>)
3. Creates a server socket (prepares tx/rx buffer, etc.)
   • The server socket starts listening on destination.ip()
4. Creates a client socket (prepares tx/rx buffer, etc.)
5. Runs a loop
   • virtual_interface.poll(socket_set)
   • Client socket is treated more carefully. Looks at the TCP state of the client_socket:
     • Closed? Established? Take corresponding actions
     • can_recv()? receive and push data by data_to_real_client_tx.send()
     • can_send()?
       • read data from data_to_virtual_server_rx if avaialble
       • client_socket.send_slice(..)

• Waits until virtual_client_ready_rx receives signal
• Runs a loop. tokio::select (Note socket is an ordinary tokio TcpStream).
  1. socket.readable()?
     • Read from the socket
     • Send data to data_to_virtual_server_tx
  2. data_to_real_client_rx.recv()?
     • socket.try_write(data)

TcpVirtualInterface

A TcpVirtualInterface has access to three channels:

• virtual_client_ready_tx
• data_to_real_client_tx
• data_to_virtual_server_rx

data_to_real_client_tx.send() is called by the poll_loop. The virtual client forwards data to the real client.

Smoltcp

Devices

If you want use smoltcp stack you should implment the following traits:

• Device
  • fn receive() -> (RxToken, TxToken)
  • fn transmite() -> (TxToken)
  • fn capabilities(): ethernet or ip? what's the MTU?
• RxToken::consume(f)
• TxToken::consume(f)

Interfaces

You add options to the InterfaceBuidler (i.e., InterfaceBuilder::new(device).ip_addrs(...).routes(...)) and call .finalize() to finally get a interface.

The most important methods for an interface is the poll(sockets) function.

poll(sockets): calls socket_ingress() and socket_egress(), which calls receive() + consume() and transmit() + socket.dispatch(), respectively. dispatch() is a function that takes a txtoken and a packet and consumes

Sockets

When you want to create a socket, initialize rx/tx ring buffers (e.g., TcpSocketBuffer; the data will be exchanged by the of either ethernet packet or IP packet format, based on the device capabilities()).

• socket.send(): You enqueue L7 data to the tx_buffer
• socket.recv(): You dequeue L7 data from rx_buffer

Question: then who enqueues packet to rx_buffer, and who dequeues packets from the tx_buffer? Answer:

1. socket_ingress() calls process_ip() and then process_tcp(). Inside process_tcp(), there are actions that enqueue_unallocated() and write_unallocated() to the rx_buffer.
2. Acknowledged octets (?) are dequeued (in particular, dequeue_allocated()) from the tx_buffer.

Example usage

1. Impl device, rxtoken, txtoken
2. Create Interface wrapper around device
3. Create TcpRxbuffer and TcpTxbuffer and create the TcpSocket
4. Run a loop: iface.poll() and take actions.
5. Done?

OneTun

Wg tasks

The wg opens a UDP listening port (that communicates w/ UDP port on the peer wg endpoint).

• although we don't create a wg interface for userspace wg (i.g., wg0 TUN), the peer will think that we are running a wireguard endpoint, and it will send packets to the host UDP packet.

wg.routine_task()

Handles handshake, keep-alive, etc.

wg.consume_task()

Receives encrypted packets from the WireGuard endpoint, decapsulates them, and dispatches newly received IP packets.

1. Runs a loop

route_tcp_segment() function will look up wg.virtual_port_ip_tx and if it finds a match, it will return RouteResult::Dispatch(virtual_port, <proto>).

ip_sink::run_ip_sink_interface(wg)

tunnel::port_forward(pf, source_peer_ip, tcp_port_pool, udp_port_pool, wg)

VirtualIpDevice

Two arguments

1. wg:
2. ip_dispatch_rx:

wg.register_virtual_interface(virtual_port, ip_dispatch_tx)

A wg maintains a mapping from virtual_port to ip_dispatch_tx.

route_tcp_segment(segment: &[u8]) :

    /// Makes a decision on the handling of an incoming TCP segment.
    fn route_tcp_segment(&self, segment: &[u8]) -> RouteResult {
        TcpPacket::new_checked(segment)
            .ok()
            .map(|tcp| {
                if self
                    .virtual_port_ip_tx
                    .get(&VirtualPort(tcp.dst_port(), PortProtocol::Tcp))
                    .is_some()
                {
                    RouteResult::Dispatch(VirtualPort(tcp.dst_port(), PortProtocol::Tcp))
                } else { /* ... */ }
            })
            .unwrap_or(RouteResult::Drop)
    }

Question: how does wg endpoint forwards the connection to the remote wg endpoint server? Or how does it know that it has to forward the connection?