[Rust-VMM] Virtio on Xen with Rust
Viresh Kumar
viresh.kumar at linaro.org
Tue Apr 19 01:11:10 UTC 2022
+rust-vmm at lists.opendev.org
On Thu, 14 Apr 2022 at 14:54, Viresh Kumar <viresh.kumar at linaro.org> wrote:
>
> +xen-devel
>
> On 14-04-22, 14:45, Viresh Kumar wrote:
> > Hello,
> >
> > We verified our hypervisor-agnostic Rust based vhost-user backends with Qemu
> > based setup earlier, and there was growing concern if they were truly
> > hypervisor-agnostic.
> >
> > In order to prove that, we decided to give it a try with Xen, a type-1
> > bare-metal hypervisor.
> >
> > We are happy to announce that we were able to make progress on that front and
> > have a working setup where we can test our existing Rust based backends, like
> > I2C, GPIO, RNG (though only I2C is tested as of now) over Xen.
> >
> > Key components:
> > --------------
> >
> > - Xen: https://github.com/vireshk/xen
> >
> > Xen requires MMIO and device specific support in order to populate the
> > required devices at the guest. This tree contains four patches on the top of
> > mainline Xen, two from Oleksandr (mmio/disk) and two from me (I2C).
> >
> > - libxen-sys: https://github.com/vireshk/libxen-sys
> >
> > We currently depend on the userspace tools/libraries provided by Xen, like
> > xendevicemodel, xenevtchn, xenforeignmemory, etc. This crates provides Rust
> > wrappers over those calls, generated automatically with help of bindgen
> > utility in Rust, that allow us to use the installed Xen libraries. Though we
> > plan to replace this with Rust based "oxerun" (find below) in longer run.
> >
> > - oxerun (WIP): https://gitlab.com/mathieupoirier/oxerun/-/tree/xen-ioctls
> >
> > This is Rust based implementations for Ioctl and hypercalls to Xen. This is WIP
> > and should eventually replace "libxen-sys" crate entirely (which are C based
> > implementation of the same).
> >
> > - vhost-device: https://github.com/vireshk/vhost-device
> >
> > These are Rust based vhost-user backends, maintained inside the rust-vmm
> > project. This already contain support for I2C and RNG, while GPIO is under
> > review. These are not required to be modified based on hypervisor and are
> > truly hypervisor-agnostic.
> >
> > Ideally the backends are hypervisor agnostic, as explained earlier, but
> > because of the way Xen maps the guest memory currently, we need a minor update
> > for the backends to work. Xen maps the memory via a kernel file
> > /dev/xen/privcmd, which needs calls to mmap() followed by an ioctl() to make
> > it work. For this a hack has been added to one of the rust-vmm crates,
> > vm-virtio, which is used by vhost-user.
> >
> > https://github.com/vireshk/vm-memory/commit/54b56c4dd7293428edbd7731c4dbe5739a288abd
> >
> > The update to vm-memory is responsible to do ioctl() after the already present
> > mmap().
> >
> > - vhost-user-master (WIP): https://github.com/vireshk/vhost-user-master
> >
> > This implements the master side interface of the vhost protocol, and is like
> > the vhost-user-backend (https://github.com/rust-vmm/vhost-user-backend) crate
> > maintained inside the rust-vmm project, which provides similar infrastructure
> > for the backends to use. This shall be hypervisor independent and provide APIs
> > for the hypervisor specific implementations. This will eventually be
> > maintained inside the rust-vmm project and used by all Rust based hypervisors.
> >
> > - xen-vhost-master (WIP): https://github.com/vireshk/xen-vhost-master
> >
> > This is the Xen specific implementation and uses the APIs provided by
> > "vhost-user-master", "oxerun" and "libxen-sys" crates for its functioning.
> >
> > This is designed based on the EPAM's "virtio-disk" repository
> > (https://github.com/xen-troops/virtio-disk/) and is pretty much similar to it.
> >
> > One can see the analogy as:
> >
> > Virtio-disk == "Xen-vhost-master" + "vhost-user-master" + "oxerun" + "libxen-sys" + "vhost-device".
> >
> >
> >
> > Test setup:
> > ----------
> >
> > 1. Build Xen:
> >
> > $ ./configure --libdir=/usr/lib --build=x86_64-unknown-linux-gnu --host=aarch64-linux-gnu --disable-docs --disable-golang --disable-ocamltools --with-system-qemu=/root/qemu/build/i386-softmmu/qemu-system-i386;
> > $ make -j9 debball CROSS_COMPILE=aarch64-linux-gnu- XEN_TARGET_ARCH=arm64
> >
> > 2. Run Xen via Qemu on X86 machine:
> >
> > $ qemu-system-aarch64 -machine virt,virtualization=on -cpu cortex-a57 -serial mon:stdio \
> > -device virtio-net-pci,netdev=net0 -netdev user,id=net0,hostfwd=tcp::8022-:22 \
> > -device virtio-scsi-pci -drive file=/home/vireshk/virtio/debian-bullseye-arm64.qcow2,index=0,id=hd0,if=none,format=qcow2 -device scsi-hd,drive=hd0 \
> > -display none -m 8192 -smp 8 -kernel /home/vireshk/virtio/xen/xen \
> > -append "dom0_mem=5G,max:5G dom0_max_vcpus=7 loglvl=all guest_loglvl=all" \
> > -device guest-loader,addr=0x46000000,kernel=/home/vireshk/kernel/barm64/arch/arm64/boot/Image,bootargs="root=/dev/sda2 console=hvc0 earlyprintk=xen" \
> > -device ds1338,address=0x20 # This is required to create a virtual I2C based RTC device on Dom0.
> >
> > This should get Dom0 up and running.
> >
> > 3. Build rust crates:
> >
> > $ cd /root/
> > $ git clone https://github.com/vireshk/xen-vhost-master
> > $ cd xen-vhost-master
> > $ cargo build
> >
> > $ cd ../
> > $ git clone https://github.com/vireshk/vhost-device
> > $ cd vhost-device
> > $ cargo build
> >
> > 4. Setup I2C based RTC device
> >
> > $ echo ds1338 0x20 > /sys/bus/i2c/devices/i2c-0/new_device; echo 0-0020 > /sys/bus/i2c/devices/0-0020/driver/unbind
> >
> > 5. Lets run everything now
> >
> > # Start the I2C backend in one terminal (open new terminal with "ssh
> > # root at localhost -p8022"). This tells the I2C backend to hook up to
> > # "/root/vi2c.sock0" socket and wait for the master to start transacting.
> > $ /root/vhost-device/target/debug/vhost-device-i2c -s /root/vi2c.sock -c 1 -l 0:32
> >
> > # Start the xen-vhost-master in another terminal. This provides the path of
> > # the socket to the master side and the device to look from Xen, which is I2C
> > # here.
> > $ /root/xen-vhost-master/target/debug/xen-vhost-master --socket-path /root/vi2c.sock0 --name i2c
> >
> > # Start guest in another terminal, i2c_domu.conf is attached. The guest kernel
> > # should have Virtio related config options enabled, along with i2c-virtio
> > # driver.
> > $ xl create -c i2c_domu.conf
> >
> > # The guest should boot fine now. Once the guest is up, you can create the I2C
> > # RTC device and use it. Following will create /dev/rtc0 in the guest, which
> > # you can configure with 'hwclock' utility.
> >
> > $ echo ds1338 0x20 > /sys/bus/i2c/devices/i2c-0/new_device
> >
> >
> > Hope this helps.
> >
> > --
> > viresh
>
> i2c_domu.conf
>
> > kernel="/root/Image"
> > memory=512
> > vcpus=2
> > command="console=hvc0 earlycon=xenboot"
> > name="domu"
> > i2c = [ "virtio=true, irq=1, base=1" ]
>
> --
> viresh
More information about the Rust-vmm
mailing list