Add vhost-user support with RNG device implementation by dorindabassey · Pull Request #527 · containers/libkrun

dorindabassey · 2026-02-04T10:34:26Z

This PR adds vhost-user frontend support to libkrun, enabling virtio devices to run in separate processes using rust-vmm's vhost-device backends for improved isolation and flexibility. The RNG frontend is implemented as the initial use case, and is designed to easily support additional devices (sound, GPU, can, etc).

src/devices/src/virtio/vhost_user/device.rs

mtjhrc

I had a look at the tests, and honestly I don't see much value in them, we don't have any arbitrary coverage percent metric for merging stuff, so I would just remove most of them.

The rest of the code looks good, but I've only had a quick look so far, when this was still a draft, I'll try running this and have another look later.

Having vhost-user support in libkrun seems pretty cool, thanks!

src/devices/src/virtio/vhost_user/rng.rs

src/devices/src/virtio/vhost_user/device.rs

tylerfanelli · 2026-02-06T18:54:50Z

Not really a review, just a comment:

@dorindabassey This is some great work and a huge addition to the project.

@slp passt uses vhost-user to provide near-native network performance in user mode for QEMU. Perhaps we should consider doing the same here, especially as the main network driver for the v2 API.

sbrivio-rh · 2026-02-06T20:22:16Z

@slp passt uses vhost-user to provide near-native network performance in user mode for QEMU. Perhaps we should consider doing the same here, especially as the main network driver for the v2 API.

Right, that would be great to have! It's mostly about throughput and latency, but also, perhaps, one day, to implement live migration of TCP connections, as it's only available via vhost-user interface.

I'm not sure how much effort networking support would take on top of this pull request, and whether it's beyond its scope or not, but let me share a couple of pointers just in case. The part of API that's implemented by passt is this:

https://passt.top/passt/tree/vhost_user.c?id=af7b81b5408da8c56bb22dd11679f2b4024a45c8#n1128

And there's a bit of documentation about how it can be used with QEMU here:

https://www.qemu.org/docs/master/system/devices/net.html#using-passt-as-the-user-mode-network-stack

Let me know if you have any question! I'll also tag @vivier, the author of the vhost-user implementation in passt, for good measure.

tylerfanelli · 2026-02-06T20:35:32Z

I'm not sure how much effort networking support would take on top of this pull request, and whether it's beyond its scope or not

It's fine for this PR to stand as-is. This can be used as a base for another series adding the network implementation.

include/libkrun.h

mtjhrc · 2026-02-09T09:00:19Z

passt uses vhost-user to provide near-native network performance in user mode for QEMU. Perhaps we should consider doing the same here, especially as the main network driver for the v2 API.

I'm not sure how much effort networking support would take on top of this pull request, and whether it's beyond its scope or not

Adding support for virtio-net over vhost doesn't require any changes to any of our networking code. Our virtio-net is not attached by default.

All the user would have to do to enable it is:

Disable the implicit vsock device to get rid of TSI (krun_disable_implicit_vsock). Optionally add a vsock device back (krun_add_vsock) to use vsock for other purpose
Use something like krun_add_vhost_user_device to attach the net device

src/devices/src/virtio/vhost_user/rng.rs

include/libkrun.h

src/vmm/src/builder.rs

src/devices/src/virtio/vhost_user/device.rs

src/libkrun/src/lib.rs

include/libkrun.h

src/devices/src/virtio/vhost_user/device.rs

src/vmm/src/builder.rs

mtjhrc · 2026-02-18T12:33:02Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces significant new functionality by adding vhost-user frontend support, with an initial implementation for the RNG device. The changes are well-structured across the C FFI, device layer, and VMM builder. A key aspect is the correct implementation of file-backed guest memory using memfd, which is crucial for vhost-user. My review includes suggestions to improve robustness, clarify the API, and clean up the example code.

examples/chroot_vm.c

src/devices/src/virtio/vhost_user/device.rs

src/libkrun/src/lib.rs

src/vmm/src/builder.rs

mtjhrc · 2026-02-18T13:02:18Z

src/vmm/src/builder.rs

+                .iter()
+                .any(|dev| dev.device_type == VIRTIO_ID_RNG);
+
+            if !vm_resources.disable_implicit_rng && !has_vhost_user_rng {


The idea I mentioned with krun_disable_implicit_rng was that it would replace this automatic disablement of the internal device. I'm not saying that the automatically disabling of the built-in device is necessarily bad though - in fact it will often be more user friendly.

However, having the user explicitly call krun_disable_implicit_rng has one big advantage though, that this will definitely be consistent with future devices - imagine in the future we could have vhost alternatives for basically every device, but adding more auto detection rules here will not be backwards compatible, potentially causing great confusion with regards to what is in fact enabled.

To be clear, I am not saying this has to necessarily be changed, It's just an observation.

Yeah, good point about future compatibility. For now I think the auto-detection provides a better user experience, but I'll keep this in mind if we add vhost-user support for other device types in the future.

Implement vhost-user support for connecting to external virtio device backends running in separate processes. Add vhost-user feature flag, vhost dependency, and krun_add_vhost_user_device() generalized API for adding vhost-user devices. Signed-off-by: Dorinda Bassey <dbassey@redhat.com>

Add memfd-backed memory region creation to enable memory sharing with vhost-user backends via FD passing. When vhost-user is enabled, all guest RAM regions are created with memfd backing instead of anonymous mmap. This lays the groundwork for vhost-user device support while maintaining backward compatibility such that the VM boots normally with standard memory when vhost-user is not configured. Signed-off-by: Dorinda Bassey <dbassey@redhat.com>

Implement generic vhost-user device wrapper with connection, feature negotiation, and Guest physical address(GPA) to Virtual address(VA) translation. Supports protocol feature negotiation (CONFIG, MQ) and forwards backend interrupts to guest. Signed-off-by: Dorinda Bassey <dbassey@redhat.com>

Implements a vhost-user RNG device. The VMM now switches between the standard RNG device and vhost-user RNG depending on whether a socket path is configured via krun_add_vhost_user_device() This allows us to use the RNG device from the rust-vmm vhost-device running in a separate process for better isolation and flexibility. Signed-off-by: Dorinda Bassey <dbassey@redhat.com>

Adds --vhost-user-rng command line option to specify a vhost-user RNG backend socket path. When provided, the VM uses the external vhost-user RNG device instead of the built-in virtio-rng implementation. Example usage: ./examples/chroot_vm \ --vhost-user-rng=/tmp/vhost-rng.sock0 \ / /bin/sh -c "head -c 32 /dev/hwrng | xxd" Signed-off-by: Dorinda Bassey <dbassey@redhat.com>

mtjhrc

I left some further review comments.

After some simple fixes, this actually works great for for virtio-net over vhost using passt.

Here are my the necessary minimum changes to fix this for passt + example:
dorindabassey/libkrun@vhost-user-support...mtjhrc:libkrun:vhost-user-net

The fixes are included in the changes requested to this PR. I'll create a separate PR for the example (and maybe an integration test too).

mtjhrc · 2026-02-25T13:23:34Z

src/libkrun/src/lib.rs

+    const VIRTIO_ID_RNG: u32 = 4;
+    const VIRTIO_ID_SND: u32 = 25;
+    const VIRTIO_ID_CAN: u32 = 36;
+
+    if device_type != VIRTIO_ID_RNG && device_type != VIRTIO_ID_SND && device_type != VIRTIO_ID_CAN
+    {
+        return -libc::EINVAL;
+    }


I think we should remove this check.

Like I mentioned in other comment somewhere, I don't think there is a reason for limiting the device types user can attach. In fact the implementation is already generic enough for many device types and even future devices that don't exist in the spec yet. It is the job of the guest to understand the virtio device(s), if the guest OS doesn't understand the device type it can just ignore the device or crash if there is some mismatch, from the libkrun side we don't really care.

mtjhrc · 2026-02-25T15:05:55Z

src/devices/src/virtio/vhost_user/device.rs

+            frontend
+                .set_features(backend_features)
+                .map_err(|e| io::Error::new(ErrorKind::Other, e))?;


Please remove this call here, this directly breaks passt and in general is incorrect.

Configuring the features here is redundant since the features are also configured in activate(). Apart from being redundant the value of backend_features is also incorrect here (if you trace the logic backend_features == VHOST_USER_F_PROTOCOL_FEATURES at this point - the virtio features are stripped).

mtjhrc · 2026-02-25T15:37:50Z

src/devices/src/virtio/vhost_user/device.rs

+        // Separate backend-only features from virtio features
+        let backend_features = avail_features & VHOST_USER_F_PROTOCOL_FEATURES;
+        let our_avail_features = avail_features & !VHOST_USER_F_PROTOCOL_FEATURES;
+
+        // Determine actual queue count - may require protocol feature negotiation
+        if backend_features & VHOST_USER_F_PROTOCOL_FEATURES != 0 {


VHOST_USER_F_PROTOCOL_FEATURES is not a mask, but a single bit. To illustrate you are actually doing:

if avail_features & VHOST_USER_F_PROTOCOL_FEATURES & VHOST_USER_F_PROTOCOL_FEATURES != 0) { ...

Personally I would just alias the variable or make the avail_features mutable and do something like this (but feel free to write it a bit differently, just remove the confusing and redundant bit operation):

Suggested change

// Separate backend-only features from virtio features

let backend_features = avail_features & VHOST_USER_F_PROTOCOL_FEATURES;

let our_avail_features = avail_features & !VHOST_USER_F_PROTOCOL_FEATURES;

// Determine actual queue count - may require protocol feature negotiation

if backend_features & VHOST_USER_F_PROTOCOL_FEATURES != 0 {

// Strip the vhost specific bit to leave only standard virtio features

let has_backend_features = avail_features & VHOST_USER_F_PROTOCOL_FEATURES != 0;

let avail_features = avail_features & !VHOST_USER_F_PROTOCOL_FEATURES;

if has_backend_features != {

Then also change the field name of the struct into has_backend_features: bool (you are only ever storing the single bit there anyway).

mtjhrc · 2026-02-25T15:41:11Z

src/devices/src/virtio/vhost_user/device.rs

+        let backend_features = avail_features & VHOST_USER_F_PROTOCOL_FEATURES;
+        let our_avail_features = avail_features & !VHOST_USER_F_PROTOCOL_FEATURES;
+
+        // Determine actual queue count - may require protocol feature negotiation


This comment seems misplaced, maybe it was meant to be above line 140:
``let actual_num_queues = if num_queues == 0 {`

mtjhrc · 2026-02-25T15:50:05Z

src/devices/src/virtio/vhost_user/device.rs

+        // NOTE: Do NOT use EFD_NONBLOCK here - the monitoring thread needs to block
+        let vring_call_event = EventFd::new(0)?; // Blocking eventfd
+
+        let has_protocol_features = backend_feature_bits & (1 << 30) != 0;


the magic hardcoded 1 << 30 constant here is in fact VHOST_USER_F_PROTOCOL_FEATURES.

mtjhrc · 2026-02-25T16:41:51Z

src/devices/src/virtio/vhost_user/device.rs

+
+        // Connect to the vhost-user backend
+        let stream = UnixStream::connect(socket_path)?;
+        let mut frontend = Frontend::from_stream(stream, 1);


You have hardcoded only 1 queue here. If you attempt to use attempt to configure a device with more than 1 queue (and without MQ feature!), the configuration methods would fail.

You can simply do this:

Suggested change

let mut frontend = Frontend::from_stream(stream, 1);

// NOTE: `num_queues` could be 0 here, but this is actually fine

// because if `VhostUserProtocolFeatures::MQ` is supported the negotioted

// value will be used automatically by Frontend

let mut frontend = Frontend::from_stream(stream, num_queues as u64);

sbrivio-rh · 2026-02-25T17:52:55Z

After some simple fixes, this actually works great for for virtio-net over vhost using passt.

Crazy stuff, thanks to both of you! @vivier have you seen this? We should iperf3 it soon.

I'll try to find some time to play with this within a couple of weeks / months (surely too late for any concrete review / practical support here though).

Completed brainstorming session. Design includes: - VhostUserFs device wrapping generic VhostUserDevice (PR containers#527) - DAX window via FUSE_SETUPMAPPING/REMOVEMAPPING (no custom vhost-user msgs) - Per-file DAX via FUSE_ATTR_DAX - Custom DEVICE_STATE protocol for snapshot/restore of daemon state - Minimal test daemon with synthetic in-memory filesystem - 8 implementation phases Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…of C API and RNG This commit integrates PR containers#527 infrastructure by cherry-picking three commits: - 1900376: Feature flags, Cargo.toml changes, C API, module setup - 635abe7: Memfd-backed guest memory in builder.rs - e5510dc: Core VhostUserDevice implementation Included: - VhostUserDevice generic wrapper (~450 lines) for vhost-user socket connection, feature negotiation, memory sharing, vring setup, and interrupt forwarding - Memfd-backed guest RAM enabling fd-passing to vhost-user daemons - vhost-user feature flag in devices, vmm, and libkrun crates - VhostUserDeviceConfig struct for generic device configuration Stripped (not needed for Phase 1): - krun_add_vhost_user_device() C API function - krun_disable_implicit_rng() C API function - disable_implicit_rng field from VmResources - All header file additions - RNG device specialization logic The vhost-user infrastructure is now ready for the FS device specialization in subsequent phases.

mtjhrc reviewed Feb 4, 2026

View reviewed changes

src/devices/src/virtio/vhost_user/device.rs Outdated Show resolved Hide resolved

dorindabassey force-pushed the vhost-user-support branch from 6600bb5 to f9fe527 Compare February 5, 2026 15:57

dorindabassey marked this pull request as ready for review February 5, 2026 15:59

dorindabassey requested review from MatiasVara, jakecorrenti, slp and tylerfanelli as code owners February 5, 2026 15:59

mtjhrc requested changes Feb 5, 2026

View reviewed changes

dorindabassey force-pushed the vhost-user-support branch from f9fe527 to 7d18fa0 Compare February 6, 2026 13:33

tylerfanelli requested changes Feb 6, 2026

View reviewed changes

include/libkrun.h Outdated Show resolved Hide resolved

dorindabassey force-pushed the vhost-user-support branch 3 times, most recently from 9cfb1ea to fe4e9ba Compare February 10, 2026 11:28

mtjhrc reviewed Feb 10, 2026

View reviewed changes

src/devices/src/virtio/vhost_user/rng.rs Outdated Show resolved Hide resolved

mtjhrc reviewed Feb 10, 2026

View reviewed changes

include/libkrun.h Outdated Show resolved Hide resolved

mtjhrc reviewed Feb 10, 2026

View reviewed changes

src/vmm/src/builder.rs Outdated Show resolved Hide resolved

mtjhrc reviewed Feb 10, 2026

View reviewed changes

src/devices/src/virtio/vhost_user/device.rs Outdated Show resolved Hide resolved

dorindabassey force-pushed the vhost-user-support branch 2 times, most recently from b3bfb2e to 3feb780 Compare February 12, 2026 11:59

mtjhrc requested changes Feb 12, 2026

View reviewed changes

mtjhrc reviewed Feb 12, 2026

View reviewed changes

src/vmm/src/builder.rs Show resolved Hide resolved

dorindabassey force-pushed the vhost-user-support branch 2 times, most recently from e5c413f to 5b0b670 Compare February 16, 2026 16:03

dorindabassey marked this pull request as draft February 17, 2026 09:54

dorindabassey force-pushed the vhost-user-support branch from 5b0b670 to 731cc43 Compare February 18, 2026 11:45

dorindabassey marked this pull request as ready for review February 18, 2026 11:45

dorindabassey force-pushed the vhost-user-support branch from 731cc43 to ab4d9e5 Compare February 18, 2026 11:57

gemini-code-assist bot reviewed Feb 18, 2026

View reviewed changes

examples/chroot_vm.c Outdated Show resolved Hide resolved

src/devices/src/virtio/vhost_user/device.rs Show resolved Hide resolved

src/libkrun/src/lib.rs Outdated Show resolved Hide resolved

src/vmm/src/builder.rs Outdated Show resolved Hide resolved

mtjhrc reviewed Feb 18, 2026

View reviewed changes

dorindabassey force-pushed the vhost-user-support branch from ab4d9e5 to d21847a Compare February 19, 2026 09:40

Dorinda Bassey added 5 commits February 19, 2026 13:27

dorindabassey force-pushed the vhost-user-support branch from d21847a to aef9b7a Compare February 19, 2026 12:27

dorindabassey requested review from mtjhrc and tylerfanelli February 25, 2026 14:59

mtjhrc requested changes Feb 25, 2026

View reviewed changes

-        let mut frontend = Frontend::from_stream(stream, 1);
+        // NOTE: `num_queues` could be 0 here, but this is actually fine
+        // because if `VhostUserProtocolFeatures::MQ` is supported the negotioted
+        // value will be used automatically by Frontend
+        let mut frontend = Frontend::from_stream(stream, num_queues as u64);

Conversation

dorindabassey commented Feb 4, 2026

Uh oh!

Uh oh!

mtjhrc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tylerfanelli commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sbrivio-rh commented Feb 6, 2026

Uh oh!

tylerfanelli commented Feb 6, 2026

Uh oh!

Uh oh!

mtjhrc commented Feb 9, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mtjhrc commented Feb 18, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mtjhrc Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dorindabassey Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

mtjhrc left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mtjhrc Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

mtjhrc Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

mtjhrc Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

mtjhrc Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

mtjhrc Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

mtjhrc Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sbrivio-rh commented Feb 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

tylerfanelli commented Feb 6, 2026 •

edited

Loading

mtjhrc Feb 18, 2026 •

edited

Loading

mtjhrc left a comment •

edited

Loading

mtjhrc Feb 25, 2026 •

edited

Loading