vsock notes

vsock-notes.md

vsock notes

about vsocks

Vsocks are a means of providing socket communication (either stream or datagram) directly between VMs and their host operating system. The host and each VM have a 32 bit CID (Context IDentifier) and may connect or bind to a 32 bit port number. Ports < 1024 are privileged ports.

• Vsock communication between VMs does not seem to work.
• Only one vsock device per VM is supported.

special values

name	value	description
VMADDR_PORT_ANY	-1	Bind to a random available port.
VMADDR_CID_ANY	-1	Bind to any CID. This seems to work inside VMs only.
VMADDR_CID_HYPERVISOR	0	The hypervisor's CID.
VMADDR_CID_RESERVED	1	Reserved; this must not be used.
VMADDR_CID_HOST	2	The host's CID.

how to launch a vm with a vsock device

This creates a VM with a vsock device with CID 123.

qemu-system-x86_64 -device vhost-vsock-pci,guest-cid=123

vsock server on host

c

#include <sys/socket.h>
#include <linux/vm_sockets.h>
#include <string.h>
#include <stdio.h>

int main()
{
	int s = socket(AF_VSOCK, SOCK_STREAM, 0);

	struct sockaddr_vm addr;
	memset(&addr, 0, sizeof(struct sockaddr_vm));
	addr.svm_family = AF_VSOCK;
	addr.svm_port = 9999;
	addr.svm_cid = VMADDR_CID_HOST;

	bind(s, &addr, sizeof(struct sockaddr_vm));

	listen(s, 0);

	struct sockaddr_vm peer_addr;
	socklen_t peer_addr_size = sizeof(struct sockaddr_vm);
	int peer_fd = accept(s, &peer_addr, &peer_addr_size);

	char buf[64];
	size_t msg_len;
	while ((msg_len = recv(peer_fd, &buf, 64, 0)) > 0) {
		printf("Received %lu bytes: %.*s\n", msg_len, msg_len, buf);
	}

	return 0;
}

python

#!/usr/bin/env python3

import socket

CID = socket.VMADDR_CID_HOST
PORT = 9999

s = socket.socket(socket.AF_VSOCK, socket.SOCK_STREAM)
s.bind((CID, PORT))
s.listen()
(conn, (remote_cid, remote_port)) = s.accept()

print(f"Connection opened by cid={remote_cid} port={remote_port}")

while True:
    buf = conn.recv(64)
    if not buf:
        break

    print(f"Received bytes: {buf}")

vsock client in vm

c

#include <sys/socket.h>
#include <linux/vm_sockets.h>
#include <string.h>

int main()
{
	int s = socket(AF_VSOCK, SOCK_STREAM, 0);

	struct sockaddr_vm addr;
	memset(&addr, 0, sizeof(struct sockaddr_vm));
	addr.svm_family = AF_VSOCK;
	addr.svm_port = 9999;
	addr.svm_cid = VMADDR_CID_HOST;

	connect(s, &addr, sizeof(struct sockaddr_vm));

	send(s, "Hello, world!", 13, 0);

	close(s);

	return 0;
}

python

#!/usr/bin/env python3

import socket

CID = socket.VMADDR_CID_HOST
PORT = 9999

s = socket.socket(socket.AF_VSOCK, socket.SOCK_STREAM)
s.connect((CID, PORT))
s.sendall(b"Hello, world!")
s.close()

wireshark

ip link add type vsockmon
ip link set vsockmon0 up
wireshark -k -i vsockmon0
ip link set vsockmon0 down
ip link del vsockmon0

see also

• https://stefano-garzarella.github.io/posts/2019-11-08-kvmforum-2019-vsock/
• http://man7.org/linux/man-pages/man7/vsock.7.html
• https://wiki.qemu.org/Features/VirtioVsock
• https://gist.github.com/mcastelino/9a57d00ccf245b98de2129f0efe39857
• https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/vm_sockets.h
• https://libvirt.org/formatdomain.html#vsock

@tristone13th right if you are not using vsock with a nested VM.

In a nested VM environment (L0 -> L1 (cid = 4) -> L2 (cid = 5)), L1 has CID = 4 when talking with L0 and CID = 2 (VMADDR_CID_HOST) when talking with L2.
So if in L1 we bind a socket to VMADDR_CID_ANY, it can receive connections for both L0 and L2, if we bind to 4 it can receive connections only from L0, if we bind to VMADDR_CID_HOST only from L2.

Anyway, with VMADDR_CID_ANY the application can always use the sockaddr parameter of the accept() to deny connections.

@stefano-garzarella Get it, many thanks!