Skip to content

Instantly share code, notes, and snippets.

@nrdmn
Last active October 14, 2024 15:48
Show Gist options
  • Save nrdmn/7971be650919b112343b1cb2757a3fe6 to your computer and use it in GitHub Desktop.
Save nrdmn/7971be650919b112343b1cb2757a3fe6 to your computer and use it in GitHub Desktop.
vsock notes

vsock notes

about vsocks

Vsocks are a means of providing socket communication (either stream or datagram) directly between VMs and their host operating system. The host and each VM have a 32 bit CID (Context IDentifier) and may connect or bind to a 32 bit port number. Ports < 1024 are privileged ports.

special values

name value description
VMADDR_PORT_ANY -1 Bind to a random available port.
VMADDR_CID_ANY -1 Bind to any CID. This seems to work inside VMs only.
VMADDR_CID_HYPERVISOR 0 The hypervisor's CID.
VMADDR_CID_RESERVED 1 Reserved; this must not be used.
VMADDR_CID_HOST 2 The host's CID.

how to launch a vm with a vsock device

This creates a VM with a vsock device with CID 123.

qemu-system-x86_64 -device vhost-vsock-pci,guest-cid=123

vsock server on host

c

#include <sys/socket.h>
#include <linux/vm_sockets.h>
#include <string.h>
#include <stdio.h>

int main()
{
	int s = socket(AF_VSOCK, SOCK_STREAM, 0);

	struct sockaddr_vm addr;
	memset(&addr, 0, sizeof(struct sockaddr_vm));
	addr.svm_family = AF_VSOCK;
	addr.svm_port = 9999;
	addr.svm_cid = VMADDR_CID_HOST;

	bind(s, &addr, sizeof(struct sockaddr_vm));

	listen(s, 0);

	struct sockaddr_vm peer_addr;
	socklen_t peer_addr_size = sizeof(struct sockaddr_vm);
	int peer_fd = accept(s, &peer_addr, &peer_addr_size);

	char buf[64];
	size_t msg_len;
	while ((msg_len = recv(peer_fd, &buf, 64, 0)) > 0) {
		printf("Received %lu bytes: %.*s\n", msg_len, msg_len, buf);
	}

	return 0;
}

python

#!/usr/bin/env python3

import socket

CID = socket.VMADDR_CID_HOST
PORT = 9999

s = socket.socket(socket.AF_VSOCK, socket.SOCK_STREAM)
s.bind((CID, PORT))
s.listen()
(conn, (remote_cid, remote_port)) = s.accept()

print(f"Connection opened by cid={remote_cid} port={remote_port}")

while True:
    buf = conn.recv(64)
    if not buf:
        break

    print(f"Received bytes: {buf}")

vsock client in vm

c

#include <sys/socket.h>
#include <linux/vm_sockets.h>
#include <string.h>

int main()
{
	int s = socket(AF_VSOCK, SOCK_STREAM, 0);

	struct sockaddr_vm addr;
	memset(&addr, 0, sizeof(struct sockaddr_vm));
	addr.svm_family = AF_VSOCK;
	addr.svm_port = 9999;
	addr.svm_cid = VMADDR_CID_HOST;

	connect(s, &addr, sizeof(struct sockaddr_vm));

	send(s, "Hello, world!", 13, 0);

	close(s);

	return 0;
}

python

#!/usr/bin/env python3

import socket

CID = socket.VMADDR_CID_HOST
PORT = 9999

s = socket.socket(socket.AF_VSOCK, socket.SOCK_STREAM)
s.connect((CID, PORT))
s.sendall(b"Hello, world!")
s.close()

wireshark

ip link add type vsockmon
ip link set vsockmon0 up
wireshark -k -i vsockmon0
ip link set vsockmon0 down
ip link del vsockmon0

see also

@stefano-garzarella
Copy link

Nice notes!

Just an update, starting from Linux v5.6 we also support loopback communication (mainly for CI and developing) and we used the CID = 1 for this. So VMADDR_CID_RESERVED is now VMADDR_CID_LOCAL and can be used for local communication.

More details here: https://stefano-garzarella.github.io/posts/2020-02-20-vsock-nested-vms-loopback/

@tristone13th
Copy link

Thanks for the example, I have a question. if the guest wants to be the server, how does it know the CID it should bind to?

@stefano-garzarella
Copy link

@tristone13th you can bind VMADDR_CID_ANY (as INADDR_ANY (0.0.0.0) for TCP) , or you can get the guest CID using ioctl(IOCTL_VM_SOCKETS_GET_LOCAL_CID) like in this script: https://gist.github.com/stefano-garzarella/b4971c2bed3c632b48c5e2a823c1cbe1

@tristone13th
Copy link

@tristone13th you can bind VMADDR_CID_ANY (as INADDR_ANY (0.0.0.0) for TCP) , or you can get the guest CID using ioctl(IOCTL_VM_SOCKETS_GET_LOCAL_CID) like in this script: stefano-garzarella/b4971c2bed3c632b48c5e2a823c1cbe1

Is there any difference between these 2? I assume 1 VM can only get 1 CID, so maybe these 2 have identical effects? Point out if I'm wrong.

@stefano-garzarella
Copy link

@tristone13th right if you are not using vsock with a nested VM.

In a nested VM environment (L0 -> L1 (cid = 4) -> L2 (cid = 5)), L1 has CID = 4 when talking with L0 and CID = 2 (VMADDR_CID_HOST) when talking with L2.
So if in L1 we bind a socket to VMADDR_CID_ANY, it can receive connections for both L0 and L2, if we bind to 4 it can receive connections only from L0, if we bind to VMADDR_CID_HOST only from L2.

Anyway, with VMADDR_CID_ANY the application can always use the sockaddr parameter of the accept() to deny connections.

@tristone13th
Copy link

@stefano-garzarella Get it, many thanks!

@jeffrey4l
Copy link

is there any support in a windows guest?

@kanpov
Copy link

kanpov commented Sep 29, 2024

Maybe also mention seqpacket vsock?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment