Virtualization and networking blog

A missing piece for AF_VSOCK in the Linux kernel has been network namespace support. We discussed it as a future challenge during the KVM Forum 2019 talk and it was mentioned in several conference discussions since then.

I started working on namespace support back in 2019, but never had the chance to complete it. Last year, Bobby Eshleman (Meta) restarted the effort and drove it through 16 revisions of the patch series. Daniel Berrangé, Michael S. Tsirkin, Paolo Abeni, and I contributed with reviews and suggestions that shaped the current user API. The result has been merged into net-next and will be available in Linux 7.0.

A vDPA device is a type of device that follows the virtio specification for its datapath but has a vendor-specific control path.

vDPA devices can be both physically located on the hardware or emulated by software.

A small vDPA parent driver in the host kernel is required only for the control path. The main advantage is the unified software stack for all vDPA devices:

• vhost interface (vhost-vdpa) for userspace or guest virtio driver, like a VM running in QEMU
• virtio interface (virtio-vdpa) for bare-metal or containerized applications running in the host
• management interface (vdpa netlink) for instantiating devices and configuring virtio parameters

SOCAT is a CLI utility which enables the concatenation of two sockets together. It establishes two bidirectional byte streams and transfers data between them.

socat supports several address types (e.g. TCP, UDP, UNIX domain sockets, etc.) to construct the streams. The latest version 1.7.4, released earlier this year [2021-01-04], supports also AF_VSOCK addresses:

• VSOCK-LISTEN:<port>

  • Listen on port and accepts a VSOCK connection.

• VSOCK-CONNECT:<cid>:<port>

  • Establishes a VSOCK stream connection to the specified cid and port.

FOSDEM 2021

If you are interested on VSOCK, I’ll talk witn Andra Paraschiv (AWS) about it at FOSDEM 2021. The talk is titled Leveraging virtio-vsock in the cloud and containers and it’s scheduled for Saturday, February 6th 2021 at 11:30 AM (CET).

We will show cool VSOCK use cases and some demos about developing, debugging, and measuring the VSOCK performance, including socat demos.

During the last KVM Forum 2019, we discussed some next steps and several requests came from the audience.

In the last months, we worked on that and recent Linux releases contain interesting new features that we will describe in this blog post:

• Nested VMs support, available in Linux 5.5
• Local communication support, available in Linux 5.6

DevConf.CZ 2020

These updates and an introduction to AF_VSOCK were presented at DevConf.CZ 2020 during the “VSOCK: VM↔host socket with minimal configuration” talk. Slides and recording are available.

In order to save memory and boot time, QEMU 4.2 and later versions are able to mmap(2) the kernel and initrd specified with -kernel and -initrd parameters. This approach allows us to avoid reading and copying them into a buffer, saving memory and time.

The memory pages that contain kernel and initrd are shared between multiple VMs using the same kernel and initrd images. So, when many VMs are launched we can save memory by sharing pages, and save time by avoiding reading them each time from the disk.

Slides and recording are available for the “virtio-vsock in QEMU, Firecracker and Linux: Status, Performance and Challenges” talk that Andra Paraschiv and I presented at KVM Forum 2019. This was the 13th edition of the KVM Forum conference. It took place in Lyon, France in October 2019.

We talked about the current status and future works of VSOCK drivers in Linux and how Firecracker and QEMU provides the virtio-vsock device.

The stefano-garzarella/qemu-boot-time repository contains a Python perf-script and (Linux, QEMU, SeaBIOS) patches to measure the boot time of a Linux VM with QEMU/KVM.

Using I/O writes, we can trace events to measure the time consumed during the boot phase by the different components:

• firmware
  • SeaBIOS
  • qboot
• optionrom
  • linuxboot [bzImage]
  • pvh [vmlinux + PVH entry point]
• Linux kernel

QEMU 4.0 is now able to boot directly into the uncompressed Linux x86_64 kernel binary with minimal firmware involvement using the PVH entry point defined in the x86/HVM direct boot ABI. (CONFIG_PVH=y must be enabled in the Linux config file).

The x86/HVM direct boot ABI was initially developed for Xen guests, but with latest changes in both QEMU and Linux, QEMU is able to use that same entry point for booting KVM guests.

The iperf-vsock repository contains few patches to add the support of VSOCK address family to iperf3. In this way iperf3 can be used to measure the performance between guest and host using VSOCK sockets.

The VSOCK address family facilitates communication between virtual machines and the host they are running on.

To test VSOCK sockets (only Linux), you must use the new option --vsock on both server and client. Other iperf3 options (e.g. -t, -l, -P, -R, --bidir) are well supported by VSOCK tests.