Vsock on sgarzare's blog

AF_VSOCK: network namespace support is here

sgarzare@redhat.com (Stefano Garzarella) — Wed, 11 Feb 2026 18:00:00 +0100

A missing piece for AF_VSOCK in the Linux kernel has been network namespace support. We discussed it as a future challenge during the KVM Forum 2019 talk and it was mentioned in several conference discussions since then.

I started working on namespace support back in 2019, but never had the chance to complete it. Last year, Bobby Eshleman (Meta) restarted the effort and drove it through 16 revisions of the patch series. Daniel Berrangé, Michael S. Tsirkin, Paolo Abeni, and I contributed with reviews and suggestions that shaped the current user API. The result has been merged into net-next and will be available in Linux 7.0.

Updated on 2026-04-17 to cover the write-once behavior of child_ns_mode, merged after the initial publication of this post.

Background

Network namespaces are a fundamental building block for containers in Linux. They provide isolation of the network stack, so each namespace has its own interfaces, routing tables, and sockets.

Before Linux 7.0, AF_VSOCK was not namespace-aware. All vsock sockets lived in the same global space, regardless of the network namespace they were created in. This caused two problems:

No isolation: a VM started inside a network namespace (or container) was reachable via vsock from any other namespace on the host, breaking the isolation that containers expect.
No CID reuse: since CIDs were global, two VMs in different namespaces could not use the same CID, even if they were completely isolated from each other at the network level.

Design

The new implementation introduces two modes, configured per network namespace:

global: CIDs are shared across namespaces. This is the original behavior and the default, so existing setups continue to work without any change.
local: namespaces are completely isolated. Sockets in a local-mode namespace can only communicate with other sockets in the same namespace.

Two sysctl knobs are available since Linux 7.0:

/proc/sys/net/vsock/child_ns_mode: the parent namespace uses this to set the mode that new child namespaces will inherit. Accepts global or local. This sysctl is write-once: after the first write, it becomes immutable.
/proc/sys/net/vsock/ns_mode: read-only, shows the mode of the current namespace. The mode is immutable after namespace creation.

This design ensures backward compatibility: the default is global, matching the previous behavior. Namespace isolation is opt-in.

Each namespace gets its mode from the parent’s child_ns_mode at creation time. Once set, the namespace’s ns_mode is immutable: every socket and VM in that namespace follows it.

The child_ns_mode sysctl is also write-once: the first write locks the value and any subsequent write of a different value returns -EBUSY. This guarantees that a namespace manager can set the value once and be certain it won’t change before creating its namespaces, preventing races where two administrator processes set conflicting modes and one ends up with a namespace in the wrong mode.

Supported vsock transports

This series adds namespace support to two transports:

vhost-vsock: host-to-guest (H2G) transport, emulates the virtio-vsock device for KVM guests
vsock-loopback: local transport, useful for testing and debugging without running VMs

The missing transports are the guest-to-host (G2H) ones (virtio, hyperv, vmci). These run in the guest as device drivers, and we currently don’t have a way to assign a vsock device to a specific namespace, since vsock devices are not standard network devices. For now, G2H transports operate in global mode, so they are reachable from any global namespace, but not from local namespaces. This means that sockets in a local namespace cannot communicate with the host through these transports. We plan to work on that in the future.

Examples

Loopback

In the following examples, the commands without a namespace prefix run in the initial network namespace (init_netns), which is the default namespace where all processes start. The init_netns is always in global mode.

These examples use the vsock loopback device for local communication, without any VM involved.

Make sure the vsock_loopback kernel module is loaded:

$ sudo modprobe vsock_loopback

Namespace isolation with unshare

Global mode (default)

By default, child_ns_mode is set to global. This is the same behavior as before Linux 7.0: vsock sockets are shared across namespaces.

A listener started in a new namespace is reachable from the init_netns using the loopback CID (VMADDR_CID_LOCAL = 1):

$ unshare --user --net nc --vsock -l 1234 &
$ nc --vsock 1 1234
# reachable - global mode, no isolation

Local mode

Setting child_ns_mode to local enables isolation. Since this sysctl is write-once, we use a nested unshare to avoid locking the init_netns mode:

$ unshare --user --map-root-user --net bash -c \
    'echo local > /proc/sys/net/vsock/child_ns_mode && \
     unshare --net nc --vsock -l 1234' &
$ nc --vsock 1 1234
Ncat: Connection reset by peer.

Namespace isolation with ip netns

The same can be done with ip netns, which requires root (or CAP_SYS_ADMIN).

First, create a global namespace and check its mode:

$ sudo ip netns add vsock_ns_global
$ sudo ip netns exec vsock_ns_global cat /proc/sys/net/vsock/ns_mode
global

A listener in the global namespace is reachable from the init_netns:

$ sudo ip netns exec vsock_ns_global nc --vsock -l 1234 &
$ nc --vsock 1 1234
# reachable - global mode, no isolation

Now create a local namespace. Since child_ns_mode is write-once, we use unshare to create a parent namespace and set its mode to local before creating the child:

$ sudo unshare --net bash -c \
    'echo local > /proc/sys/net/vsock/child_ns_mode && \
     ip netns add vsock_ns_local'
$ sudo ip netns exec vsock_ns_local cat /proc/sys/net/vsock/ns_mode
local

A listener in the local namespace is not reachable from the init_netns:

$ sudo ip netns exec vsock_ns_local nc --vsock -l 1234 &
$ nc --vsock 1 1234
Ncat: Connection reset by peer.

But communication within the same namespace still works:

$ sudo ip netns exec vsock_ns_local nc --vsock 1 1234
# reachable - same namespace

VMs with QEMU

The vhost-vsock H2G transport exposes the /dev/vhost-vsock device, which QEMU opens at VM startup to emulate the virtio-vsock device for the guest. Since namespace support applies to this transport, VMs inherit the namespace mode as well.

In the following examples, we reuse the vsock_ns_global and vsock_ns_local namespaces created in the previous section.

Global mode

With global mode, the VM started in a global namespace is reachable from any other global namespace, including the init_netns:

$ sudo ip netns exec vsock_ns_global \
    qemu-system-x86_64 -m 1G -M q35,accel=kvm \
    -drive file=guest.qcow2,if=virtio,snapshot=on \
    -device vhost-vsock-pci,guest-cid=42

# start a listener in the guest (global namespace)
guest_global$ nc --vsock -l 1234

# from the init_netns (global) - reachable
$ nc --vsock 42 1234

Local mode

With local mode, the VM is only reachable from within the same namespace:

$ sudo ip netns exec vsock_ns_local \
    qemu-system-x86_64 -m 1G -M q35,accel=kvm \
    -drive file=guest.qcow2,if=virtio,snapshot=on \
    -device vhost-vsock-pci,guest-cid=42

# start a listener in the guest (local namespace)
guest_local$ nc --vsock -l 1234

# from the init_netns (global) - isolated
$ nc --vsock 42 1234
Ncat: Connection reset by peer.

# from the same namespace - reachable
$ sudo ip netns exec vsock_ns_local nc --vsock 42 1234

Guest-to-host (G2H) behavior

As mentioned in the Supported vsock transports section, the G2H virtio transport does not support namespaces yet. The virtio-vsock device in the guest always operates in global mode, so only sockets in global namespaces can communicate with the host.

Using the VM started in vsock_ns_global, a listener in the guest’s init_netns is reachable from the host:

# start a listener in the guest
guest_global$ nc --vsock -l 1234

# from the host - reachable
$ nc --vsock 42 1234

But a listener started in a local namespace inside the guest is not reachable from the host:

# create a local namespace in the guest and start a listener
guest_global$ unshare --user --map-root-user --net bash -c \
    'echo local > /proc/sys/net/vsock/child_ns_mode && \
     unshare --net nc --vsock -l 1234' &

# from the host - isolated
$ nc --vsock 42 1234
Ncat: Connection reset by peer.

CID reuse

Note that we used the same CID (42) in both examples without turning off the first VM. This is possible because the second VM is in a local namespace, so its CID space is isolated. With global mode, QEMU would fail to start the second VM because the CID is already in use.

Patches

SOCAT now supports AF_VSOCK

sgarzare@redhat.com (Stefano Garzarella) — Fri, 22 Jan 2021 15:16:04 +0100

SOCAT is a CLI utility which enables the concatenation of two sockets together. It establishes two bidirectional byte streams and transfers data between them.

socat supports several address types (e.g. TCP, UDP, UNIX domain sockets, etc.) to construct the streams. The latest version 1.7.4, released earlier this year [2021-01-04], supports also AF_VSOCK addresses:

VSOCK-LISTEN:<port>
- Listen on port and accepts a VSOCK connection.
VSOCK-CONNECT:<cid>:<port>
- Establishes a VSOCK stream connection to the specified cid and port.

FOSDEM 2021

If you are interested on VSOCK, I’ll talk witn Andra Paraschiv (AWS) about it at FOSDEM 2021. The talk is titled Leveraging virtio-vsock in the cloud and containers and it’s scheduled for Saturday, February 6th 2021 at 11:30 AM (CET).

We will show cool VSOCK use cases and some demos about developing, debugging, and measuring the VSOCK performance, including socat demos.

Examples

socat could be very useful for concatenating and redirecting sockets. In this section we will see some examples.

Examples below refer to a guest with CID 42 that we created using virt-builder and virt-install .

VM setup

virt-builder is able to download the installer and create the disk image with Fedora 33 or other distros. It is also able to set the root password and inject the ssh public key, simplifying the creation of guest disk image:

VM_IMAGE="vsockguest_f33.qcow2"

host$ virt-builder --root-password=password:mypassword \
        --ssh-inject root:file:/home/user/.ssh/id_rsa.pub \
        --output=${VM_IMAGE} \
        --format=qcow2 --size 10G --selinux-relabel \
        --update fedora-33

Once the disk image is ready, we create our VM with virt-install. We can specify the VM settings like the number of vCPUs, the amount of RAM, and the CID assigned to the VM [42]:

host$ virt-install --name vsockguest \
        --ram 2048 --vcpus 2 --os-variant fedora33 \
        --import --disk path=${VM_IMAGE},bus=virtio \
        --graphics none --vsock cid.address=42

After the creation of the VM, we will remain attached to the console and we can detach from it by pressing ctrl-].

We can reattach to the console in this way:

host$ virsh console vsockguest

If the VM is turned off, we can boot it and attach directly to the console in this way:

host$ virsh start --console vsockguest

ncat like

It’s possible to use socat like ncat, transferring stdin and stdout via VSOCK.

Guest listening

In this example we start socat in the guest listening on port 1234:

guest$ socat - VSOCK-LISTEN:1234

Then we connect from the host using the CID 42 assigned to the VM:

host$ socat - VSOCK-CONNECT:42:1234

At this point we can exchange characters between guest and host, since stdin and stdout are linked through the VSOCK socket.

Host listening

In this example we do the opposite, starting socat in the host listening on port 1234:

host$ socat - VSOCK-LISTEN:1234

Then, in the guest, we connect to the host using the well defined CID 2. It’s always used to reach the host:

guest$ socat - VSOCK-CONNECT:2:1234

ssh over VSOCK

The coolest feature of socat is to concatenate sockets of different address families, so in this example we redirect ssh traffic through VSOCK socket exposed by the VM.

This example could be useful if the VM doesn’t have any NIC attached and we want to provide some network connectivity, like the ssh access.

First of all, in the guest we start socat linking the VSOCK socket listening on port 22, to a TCP socket which will connect to the local TCP port 22 where the ssh server is listening:

guest$ socat VSOCK-LISTEN:22,reuseaddr,fork TCP:localhost:22

On the host we can use socat as ProxyCommand of ssh to forward the connection through the AF_VSOCK socket:

host$ ssh -o "ProxyCommand socat - VSOCK-CONNECT:42:22" root@localhost

socat redirects all the traffic and allow us to use ssh over VSOCK to reach the guest.

Connecting sibling VMs

Another scenario where socat can be very useful is connecting two sibling VMs running on the same host.

Currently this is not supported by vhost-vsock, so we can use socat to concatenate the two VMs.

Let’s see an example: suppose we launch another VM with CID = 24 on the same host. Now we want to use ncat in the VMs to communicate with each other.

Guest 42 will listen on port 1234 and guest 24 will initiate the connection. But before we do this we need to set up the bridge in the host with socat to allow the two VMs to communicate:

host$ socat VSOCK-LISTEN:1234 VSOCK-CONNECT:42:1234

At this point we can launch ncat in the VMs and communicate:

guest_42$ nc --vsock -l 1234

Note: as destination CID we have to use the host’s well-known CID (2) because we need to connect to the socat bridge running in the host, that will redirect packets correctly between the two machines:

guest_24$ nc --vsock 2 1234

AF_VSOCK: nested VMs and loopback support available

sgarzare@redhat.com (Stefano Garzarella) — Tue, 25 Feb 2020 20:30:21 +0100

During the last KVM Forum 2019, we discussed some next steps and several requests came from the audience.

In the last months, we worked on that and recent Linux releases contain interesting new features that we will describe in this blog post:

Nested VMs support, available in Linux 5.5
Local communication support, available in Linux 5.6

DevConf.CZ 2020

These updates and an introduction to AF_VSOCK were presented at DevConf.CZ 2020 during the “VSOCK: VM↔host socket with minimal configuration” talk. Slides and recording are available.

Nested VMs

Before Linux 5.5, the AF_VSOCK core supported only one transport loaded at run time. That was a limit for nested VMs, because we need multiple transports loaded together.

Types of transport

Under the AF_VSOCK core, that provides the socket interface to the user space applications, we have several transports that implement the communication channel between guest and host.

vsock transports

These transports depend on the hypervisor and we can put them in two groups:

H2G (host to guest) transports: they run in the host and usually they provide the device emulation; currently we have vhost and vmci transports.
G2H (guest to host) transports: they run in the guest and usually they are device drivers; currently we have virtio, vmci, and hyperv transports.

Multi-transports

In a nested VM environment, we need to load both G2H and H2G transports together in the L1 guest, for this reason, we implemented the multi-transports support to use vsock through nested VMs.

vsock and nested VMs

Starting from Linux 5.5, the AF_VSOCK can handle two types of transports loaded together at runtime:

H2G transport, to communicate with the guest
G2H transport, to communicate with the host.

So in the QEMU/KVM environment, the L1 guest will load both virtio-transport, to communicate with L0, and vhost-transport to communicate with L2.

Local Communication

Another feature recently added is the possibility to communicate locally on the same host. This feature, suggested by Richard WM Jones, can be very useful for testing and debugging applications that use AF_VSOCK without running VMs.

Linux 5.6 introduces a new transport called vsock-loopback, and a new well know CID for local communication: VMADDR_CID_LOCAL (1). It’s a special CID to direct packets to the same host that generated them.

vsock loopback

Other CIDs can be used for the same purpose, but it’s preferable to use VMADDR_CID_LOCAL:

Local Guest CID
- if G2H is loaded (e.g. running in a VM)
VMADDR_CID_HOST (2)
- if H2G is loaded and G2H is not loaded (e.g. running on L0). If G2H is also loaded, then VMADDR_CID_HOST is used to reach the host

Richard recently used the vsock local communication to implement a regression test test for nbdkit/libnbd vsock support, using the new VMADDR_CID_LOCAL.

Example

# Listening on port 1234 using ncat(1)
l0$ nc --vsock -l 1234

# Connecting to the local host using VMADDR_CID_LOCAL (1)
l0$ nc --vsock 1 1234

Patches

KVM Forum 2019: virtio-vsock in QEMU, Firecracker and Linux

sgarzare@redhat.com (Stefano Garzarella) — Sat, 09 Nov 2019 18:45:25 +0100

Slides and recording are available for the “virtio-vsock in QEMU, Firecracker and Linux: Status, Performance and Challenges” talk that Andra Paraschiv and I presented at KVM Forum 2019. This was the 13th edition of the KVM Forum conference. It took place in Lyon, France in October 2019.

We talked about the current status and future works of VSOCK drivers in Linux and how Firecracker and QEMU provides the virtio-vsock device.

Summary

Initially, Andra gave an overview of VSOCK, she described the state of the art, and the key features:

it is very simple to configure, the host assigns an unique CID (Context-ID) to each guest, and no configuration is needed inside the guest;
it provides AF_VSOCK address family, allowing user space application in the host and guest to communicate using standard POSIX Socket API (e.g. bind, listen, accept, connect, send, recv, etc.)

Andra also described common use cases for VSOCK, such as guest agents (clipboard sharing, remote console, etc.), network applications using SOCK_STREAM, and services provided by the hypervisor to the guests.

Going into the implementation details, Andra explained how the device in the guest communicates with the vhost backend in the host, exchanging data and events (i.e. ioeventfd, irqfd).

Firecracker

Focusing on Firecracker, Andra gave a brief overview on this new VMM (Virtual Machine Monitor) written in Rust and she explained why, in the v0.18.0 release, they switched from the experimental vhost-vsock implementation to a vhost-less solution:

focus on security impact
less dependency on host kernel features

This change required a device emulation in Firecracker, that implements virtio-vsock device model over MMIO. The device is exposed in the host using UDS (Unix Domain Sockets).

Andra described how Firecracker maps the VSOCK ports on the uds_path specified in the VM configuration:

Host-Initiated Connections
- Guest: create an AF_VSOCK socket and listen() on PORT
- Host: connect() to AF_UNIX at uds_path
- Host: send() “CONNECT PORT\n”
- Guest: accept() the new connection
Guest-Initiated Connections
- Host: create and listen() on an AF_UNIX socket at uds_path_PORT
- Guest: create an AF_VSOCK socket and connect() to HOST_CID and PORT
- Host: accept() the new connection

Finally, she showed the performance of this solution, running iperf-vsock benchmark, varying the size of the buffer used in Firecracker to transfer packets between the virtio-vsock device and the UNIX domain socket. The throughput on the guest to host path reaches 10 Gbps.

QEMU

In the second part of the talk, I described the QEMU implementation. QEMU provides the virtio-vsock device using the vhost-vsock kernel module.

The vsock device in QEMU handles only:

configuration
- user or management tool can configure the guest CID
live-migration
- connected SOCK_STREAM sockets become disconnected. Applications must handle a connection reset error and should reconnect.
- guest CID can be not available in the new host because can be assigned to another VM. In this case the guest is notified about the CID change.

The vhost-vsock kernel module handles the communication with the guest, providing in-kernel virtio device emulation, to have very high performance and to interface directly to the host socket layer. In this way, also host application can directly use POSIX Socket API to communicate with the guest. So, guest and host applications can be switched between them, changing only the destination CID.

virtio-vsock Linux drivers

After that, I told the story of VSOCK in the Linux tree, started in 2013 when the first implementation was merged, and the changes in the last year.

These changes mainly regard fixes, but for the virtio/vhost transports we also improved the performance with two simple changes released with Linux v5.4:

reducing the number of credit update messages exchanged
increasing the size of packets queued in the virtio-vsock device from 4 KB up to 64 KB, the maximum packet size handled by virtio-vsock devices.

With these changes we are able to reach ~40 Gbps in the Guest -> Host path, because the guest can now send up to 64 KB packets directly to the host; for the Host -> Guest path, we reached ~25 Gbps, because the host is still using 4 KB buffer preallocated by the guest.

Tools and languages that support VSOCK

In the last few years, several applications, tools, and languages started to support VSOCK and I listed them to update the audience:

Tools:
- wireshark >= 2.40 [2017-07-19]
- iproute2 >= 4.15 [2018-01-28]
  - ss
- tcpdump
  - merged in master [2019-04-16]
- nmap >= 7.80 [2019-08-10]
  - ncat
- nbd
  - nbdkit >= 1.15.5 [2019-10-19]
  - libnbd >= 1.1.6 [2019-10-19]
- iperf-vsock
  - iperf3 fork
Languages:
- C
  - glibc >= 2.18 [2013-08-10]
- Python
  - python >= 3.7 alpha 1 [2017-09-19]
- Golang
  - https://github.com/mdlayher/vsock
- Rust
  - libc crate >= 0.2.59 [2019-07-08]
    - struct sockaddr_vm
    - VMADDR_* macros
  - nix crate >= 0.15.0 [2019-08-10]
    - VSOCK supported in the socket API (nix::sys::socket)

Next steps

Concluding, I went through the next challenges that we are going to face:

multi-transport to use VSOCK in a nested VM environment. because we are limited by the fact that the current implementation can handle only one transport loaded at run time, so, we can’t load virtio_transport and vhost_transport together in the L1 guest. I already sent some patches upstream [RFC, v1], but they are still in progress.
network namespace support to create independent addressing domains with VSOCK socket. This could be useful for partitioning VMs in different domains or, in a nested VM environment, to isolate host applications from guest applications bound to the same port.
virtio-net as a transport for the virtio-vsock to avoid to re-implement features already done in virtio-net, such as mergeable buffers, page allocation, small packet handling.

From the audience

Other points to be addressed came from the comments we received from the audience:

loopback device could be very useful for developers to test applications that use VSOCK socket. The current implementation support loopback only in the guest, but it would be better to support it also in the host, adding VMADDR_CID_LOCAL special address.
VM to VM communication was asked by several people. Introducing it in the VSOCK core could complicate the protocol, the addressing and could require some sort of firewall. For now we do not have in mind to do it, but I developed a simple user space application to solve this issue: vsock-bridge. In order to improve the performance of this solution, we will consider the possibility to add sendfile(2) or MSG_ZEROCOPY support to the AF_VSOCK core.
virtio-vsock windows drivers is not planned to be addressed, but contributions are welcome. Other virtio windows drivers are available in the vm-guest-drivers-windows repository.

Stay tuned!

iperf3-vsock: how to measure VSOCK performance

sgarzare@redhat.com (Stefano Garzarella) — Thu, 22 Aug 2019 17:52:06 +0200

The iperf-vsock repository contains few patches to add the support of VSOCK address family to iperf3. In this way iperf3 can be used to measure the performance between guest and host using VSOCK sockets.

The VSOCK address family facilitates communication between virtual machines and the host they are running on.

To test VSOCK sockets (only Linux), you must use the new option --vsock on both server and client. Other iperf3 options (e.g. -t, -l, -P, -R, --bidir) are well supported by VSOCK tests.

Prerequisites

Linux host kernel >= 4.8
Linux guest kernel >= 4.8
QEMU >= 2.8

Build iperf3-vsock from source

Clone repository

git clone https://github.com/stefano-garzarella/iperf-vsock
cd iperf-vsock

Building

mkdir build
cd build
../configure
make

(Note: If configure fails, try running ./bootstrap.sh first)

Example with Fedora 30 (host and guest):

Host: start the VM

GUEST_CID=3
sudo modprobe vhost_vsock
sudo qemu-system-x86_64 -m 1G -smp 2 -cpu host -M accel=kvm	\
     -drive if=virtio,file=/path/to/fedora.img,format=qcow2     \
     -device vhost-vsock-pci,guest-cid=${GUEST_CID}

Guest: start iperf server

# SELinux can block you, so you can write a policy or temporally disable it
sudo setenforce 0
iperf-vsock/build/src/iperf3 --vsock -s

Host: start iperf client

iperf-vsock/build/src/iperf3 --vsock -c ${GUEST_CID}

Output

Connecting to host 3, port 5201
[  5] local 2 port 4008596529 connected to 3 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  1.30 GBytes  11.2 Gbits/sec
[  5]   1.00-2.00   sec  1.67 GBytes  14.3 Gbits/sec
[  5]   2.00-3.00   sec  1.57 GBytes  13.5 Gbits/sec
[  5]   3.00-4.00   sec  1.49 GBytes  12.8 Gbits/sec
[  5]   4.00-5.00   sec   971 MBytes  8.15 Gbits/sec
[  5]   5.00-6.00   sec  1.01 GBytes  8.71 Gbits/sec
[  5]   6.00-7.00   sec  1.44 GBytes  12.3 Gbits/sec
[  5]   7.00-8.00   sec  1.62 GBytes  13.9 Gbits/sec
[  5]   8.00-9.00   sec  1.61 GBytes  13.8 Gbits/sec
[  5]   9.00-10.00  sec  1.63 GBytes  14.0 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  14.3 GBytes  12.3 Gbits/sec                  sender
[  5]   0.00-10.00  sec  14.3 GBytes  12.3 Gbits/sec                  receiver

iperf Done.