From fandree at amazon.com Wed Feb 3 14:10:00 2021 From: fandree at amazon.com (Florescu, Andreea) Date: Wed, 3 Feb 2021 14:10:00 +0000 Subject: [Rust-VMM] rust-vmm plans for 2021 & meeting community members Message-ID: <1612361400264.27289@amazon.com> Hey everyone, Last week myself and Laura (lauralg@) had a meeting with a few people from Sartura [1] in which we discussed about where we are with the project, and potential paths for collaboration. They are interested in using vmm-reference [2] and rust-vmm for their internal projects, and so we were thinking that it would be a good idea to also talk about the future plans that people in this community have. We've been asked a couple of times for a road map for this project, and I believe it would be nice to start working on it, and have it publicly available. Sartura is interested in presenting a POC by the end of next week, and so I was thinking we can schedule a meeting in which they present their POC, and then we can also discuss about the road map for 2021. The target date for the meeting is 15 February 2021. The folks at Sartura are working on the CEST timezone, so we should try to find an hour where they can join as well. I'll be scheduling the meeting such that most people can join. Please add your best fit here: [3]. Alternatively, we can also schedule 2 meetings to have more coverage if people are interested. Thanks, Andreea [1] https://www.sartura.hr/ [2] https://github.com/rust-vmm/vmm-reference [3] https://doodle.com/poll/e8swc9fvrdntzna2? Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fandree at amazon.com Thu Feb 11 15:00:09 2021 From: fandree at amazon.com (Florescu, Andreea) Date: Thu, 11 Feb 2021 15:00:09 +0000 Subject: [Rust-VMM] rust-vmm plans for 2021 & meeting community members In-Reply-To: <1612361400264.27289@amazon.com> References: <1612361400264.27289@amazon.com> Message-ID: <1613055608716.74192@amazon.com> Hey everyone, It looks like the winning hour for the meeting is 09:00 AM GMT +1. Thanks everyone for responding.? I'll shortly send a meeting invite as well. Thanks, Andreea ________________________________ From: Florescu, Andreea Sent: Wednesday, February 3, 2021 4:10 PM To: rust-vmm at lists.opendev.org Cc: Loghin, Laura Subject: rust-vmm plans for 2021 & meeting community members Hey everyone, Last week myself and Laura (lauralg@) had a meeting with a few people from Sartura [1] in which we discussed about where we are with the project, and potential paths for collaboration. They are interested in using vmm-reference [2] and rust-vmm for their internal projects, and so we were thinking that it would be a good idea to also talk about the future plans that people in this community have. We've been asked a couple of times for a road map for this project, and I believe it would be nice to start working on it, and have it publicly available. Sartura is interested in presenting a POC by the end of next week, and so I was thinking we can schedule a meeting in which they present their POC, and then we can also discuss about the road map for 2021. The target date for the meeting is 15 February 2021. The folks at Sartura are working on the CEST timezone, so we should try to find an hour where they can join as well. I'll be scheduling the meeting such that most people can join. Please add your best fit here: [3]. Alternatively, we can also schedule 2 meetings to have more coverage if people are interested. Thanks, Andreea [1] https://www.sartura.hr/ [2] https://github.com/rust-vmm/vmm-reference [3] https://doodle.com/poll/e8swc9fvrdntzna2? Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fandree at amazon.com Thu Feb 11 15:07:33 2021 From: fandree at amazon.com (Florescu, Andreea) Date: Thu, 11 Feb 2021 15:07:33 +0000 Subject: [Rust-VMM] rust-vmm 2021 road map discussion Message-ID: Meeting agenda: 1. Discuss about the rust-vmm road map for 2021 2. Sartura will present a demo using rust-vmm/vmm-reference. It might be a stretch to fit the road map discussions in one hour, so we can look at the high level plans, and then schedule separate meetings to dive deep into certain topics. See you soon! Andreea Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 2758 bytes Desc: not available URL: From fandree at amazon.com Thu Feb 11 15:08:32 2021 From: fandree at amazon.com (Florescu, Andreea) Date: Thu, 11 Feb 2021 15:08:32 +0000 Subject: [Rust-VMM] rust-vmm 2021 road map discussion Message-ID: <469b4581e74543929f8c131ac0a0e428@EX13D10EUB003.ant.amazon.com> Meeting agenda: 1. Discuss about the rust-vmm road map for 2021 2. Sartura will present a demo using rust-vmm/vmm-reference. It might be a stretch to fit the road map discussions in one hour, so we can look at the high level plans, and then schedule separate meetings to dive deep into certain topics. See you soon! Andreea ==============Conference Bridge Information============== You have been invited to an online meeting, powered by Amazon Chime. Chime meeting ID: 3058950859 Join via Chime clients (manually): Select 'Meetings > Join a Meeting', and enter 3058950859 Join via Chime clients (auto-call): If you invite auto-call as attendee, Chime will call you when the meeting starts, select 'Answer' Join via browser screen share: https://chime.aws/3058950859 Join via phone (US): +1-929-432-4463,,,3058950859# Join via phone (US toll-free): +1-855-552-4463,,,3058950859# International dial-in: https://chime.aws/dialinnumbers/ In-room video system: Ext: 62000, Meeting PIN: 3058950859# ================================================= Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 3701 bytes Desc: not available URL: From fandree at amazon.com Thu Feb 11 15:14:43 2021 From: fandree at amazon.com (Florescu, Andreea) Date: Thu, 11 Feb 2021 15:14:43 +0000 Subject: [Rust-VMM] rust-vmm 2021 road map discussion Message-ID: <21a8f44b478b4d6d8119624e3180742f@EX13D10EUB003.ant.amazon.com> Meeting agenda: 1. Discuss about the rust-vmm road map for 2021 2. Sartura will present a demo using rust-vmm/vmm-reference. It might be a stretch to fit the road map discussions in one hour, so we can look at the high level plans, and then schedule separate meetings to dive deep into certain topics. See you soon! Andreea ==============Conference Bridge Information============== You have been invited to an online meeting, powered by Amazon Chime. Chime meeting ID: 3058950859 Join via Chime clients (manually): Select 'Meetings > Join a Meeting', and enter 3058950859 Join via Chime clients (auto-call): If you invite auto-call as attendee, Chime will call you when the meeting starts, select 'Answer' Join via browser screen share: https://chime.aws/3058950859 Join via phone (US): +1-929-432-4463,,,3058950859# Join via phone (US toll-free): +1-855-552-4463,,,3058950859# International dial-in: https://chime.aws/dialinnumbers/ In-room video system: Ext: 62000, Meeting PIN: 3058950859# ================================================= Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 3701 bytes Desc: not available URL: From fandree at amazon.com Fri Feb 12 13:22:43 2021 From: fandree at amazon.com (Florescu, Andreea) Date: Fri, 12 Feb 2021 13:22:43 +0000 Subject: [Rust-VMM] Call for Google Summer of Code 2021 project ideas In-Reply-To: References: Message-ID: <1613136163375.99584@amazon.com> Hey Stefan, Thanks for taking care of organizing GSoC, and for allowing rust-vmm to also participate under the QEMU umbrella! I am a bit unsure of how can we propose projects related to rust-vmm. We did a bit of brainstorming in our team, and we came up with 3 project ideas. I'll just paste them below, but please let me know if we were supposed to propose them some other way. === Implement the Virtio Console device in Rust === '''Summary:''' Implement the basic emulation for the Virtio Console device in Rust Implement the basic functionality (excluding the optional features: VIRTIO_CONSOLE_F_SIZE, VIRTIO_CONSOLE_F_MULTIPORT, or VIRTIO_CONSOLE_F_EMERG_WRITE) of the Virtio Console Device, using the Virtio building blocks (queue implementations, VirtioDevice traits) defined in rust-vmm/vm-virtio. The virtio console device uses one virtio queue for transmitting data, and one virtio queue for receiving data. The implementation can be extended to also support a subset of the previously mentioned optional features. '''Links:''' * About rust-vmm: https://github.com/rust-vmm/community * rust-vmm/vm-virtio: https://github.com/rust-vmm/vm-virtio * virtio-console spec: https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-2550003 '''Details:''' * Skill level: intermediate * Language: Rust * Mentor: iul at amazon.com * Suggested by: fandree at amazon.com === Mocking framework for Virtio Queues === '''Summary:''' Implement a mocking framework for virtio queues Paravirtualized devices (such as those defined by the Virtio standard) are used to provide high performance device emulation. Virtio drivers from a guest VM communicate with the device model using an efficient mechanism based on queues stored in a shared memory area that operate based on a protocol and message format defined by the standard. Various implementations of devices and other virtualization building blocks require mocking the contents that a driver would place into a Virtio queue for validation, testing, and evaluation purposes. This project aims to lay the foundations of a reusable framework for mocking the driver side of Virtio queue operation, that can be consumed by rust-vmm crates and other projects. At the basic level, this means providing a flexible and easy to use interface for users to set up the underlying memory areas and populate contents (as the driver would do) for the basic split queue format in a generic manner. This can further be extended for the packed format and with device-specific mocking capabilities. '''Links:''' * About rust-vmm: https://github.com/rust-vmm/community * Virtio queue spec: https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-230005 Issue in rust-vmm about reusing the mocking logic: rust-vmm/vm-virtio: https://github.com/rust-vmm/vm-virtio '''Details:''' * Skill level: intermediate * Language: Rust * Mentor: aagch at amazon.com * Suggested by: aagch at amazon.com === Local running rust-vmm-ci === '''Summary:''' Run the rust-vmm-ci locally The rust-vmm-ci provides automation for uniformely running the tests on all rust-vmm repositories. It is built on top of Buildkite, and only allows running the tests in the Buildkite context. To run the same tests as in the CI locally, users need to manually copy the Buildkite pipeline steps. The scope of this project is to make it possible for the same tests to easily run locally. This project makes it easier to contribute to all rust-vmm repositories. In order for that to be possible, the following steps are required: - the Buildlkite pipeline is autogenerated from code instead of being a static list of tests to run. This also allows us to uniformely use the same container version for running all the tests (instead of manually modifying each step in the pipeline) - the code for autogenerating the Buildkite pipeline is reused for generating a Python script which can be run locally '''Links:''' * rust-vmm-ci: https://github.com/rust-vmm/rust-vmm-ci * Buildkite pipeline that currently runs the tests: https://github.com/rust-vmm/rust-vmm-ci/blob/master/.buildkite/pipeline.yml * About rust-vmm: https://github.com/rust-vmm/community * Buildkite documentation: https://buildkite.com/docs/tutorials/getting-started '''Details:''' * Skill level: intermediate * Language: Python * Mentor: fandree at amazon.com * Suggested by: fandree at amazon.com ?Thanks again! Andreea ________________________________ From: Stefan Hajnoczi Sent: Monday, January 11, 2021 1:47 PM To: qemu-devel; kvm; rust-vmm at lists.opendev.org; Alex Bennée; Alexander Graf; Alberto Garcia; David Hildenbrand; Eduardo Habkost; Igor Mammedov; John Snow; Julia Suvorova; Gerd Hoffmann; Kevin Wolf; Laurent Vivier; Marc-André Lureau; Aleksandar Markovic; Sergio Lopez; Stefano Garzarella; Paolo Bonzini; Philippe Mathieu-Daudé Subject: [EXTERNAL] [Rust-VMM] Call for Google Summer of Code 2021 project ideas CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Dear QEMU, KVM, and rust-vmm community, QEMU will apply for Google Summer of Code (https://summerofcode.withgoogle.com/) again this year. This internship program offers paid, 10-week, remote work internships for contributing to open source. QEMU can act as an umbrella organization for KVM kernel and rust-vmm projects too. Please post project ideas on the QEMU wiki before February 14th: https://wiki.qemu.org/Google_Summer_of_Code_2021 What's new this year: * The number of internship hours has been halved to 175 hours over 10 weeks. Project ideas must be smaller to fit and students will have more flexibility with their working hours. * Eligibility has been expanded to include "licensed coding school or similar type of program". Good project ideas are suitable for 175 hours (10 weeks half-day) work by a competent programmer who is not yet familiar with the codebase. In addition, they are: * Well-defined - the scope is clear * Self-contained - there are few dependencies * Uncontroversial - they are acceptable to the community * Incremental - they produce deliverables along the way Feel free to post ideas even if you are unable to mentor the project. It doesn't hurt to share the idea! I will review project ideas and keep you up-to-date on QEMU's acceptance into GSoC. For more background on QEMU internships, check out this video: https://www.youtube.com/watch?v=xNVCX7YMUL8 Stefan _______________________________________________ Rust-vmm mailing list Rust-vmm at lists.opendev.org http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005. -------------- next part -------------- An HTML attachment was scrubbed... URL: From slp at redhat.com Fri Feb 12 13:51:56 2021 From: slp at redhat.com (Sergio Lopez) Date: Fri, 12 Feb 2021 14:51:56 +0100 Subject: [Rust-VMM] Call for Google Summer of Code 2021 project ideas In-Reply-To: <1613136163375.99584@amazon.com> References: <1613136163375.99584@amazon.com> Message-ID: <20210212135105.2llsgdiagk2tzu2m@mhamilton> On Fri, Feb 12, 2021 at 01:22:43PM +0000, Florescu, Andreea wrote: > Hey Stefan, > > > Thanks for taking care of organizing GSoC, and for allowing rust-vmm to also participate under the QEMU umbrella! > > I am a bit unsure of how can we propose projects related to rust-vmm. > > We did a bit of brainstorming in our team, and we came up with 3 project ideas. > > I'll just paste them below, but please let me know if we were supposed to propose them some other way. > > > === Implement the Virtio Console device in Rust === > > '''Summary:''' Implement the basic emulation for the Virtio Console device in Rust > > Implement the basic functionality (excluding the optional features: > VIRTIO_CONSOLE_F_SIZE, VIRTIO_CONSOLE_F_MULTIPORT, or VIRTIO_CONSOLE_F_EMERG_WRITE) > of the Virtio Console Device, using the Virtio building blocks (queue implementations, > VirtioDevice traits) defined in rust-vmm/vm-virtio. The virtio console device uses > one virtio queue for transmitting data, and one virtio queue for receiving data. > The implementation can be extended to also support a subset of the previously > mentioned optional features. FWIW, libkrun already has support for virtio-console with the basic functionality and VIRTIO_CONSOLE_F_SIZE, and this code could be easily borrowed for implementing it in some rust-vmm crate: https://github.com/containers/libkrun/blob/main/src/devices/src/virtio/console Sergio. > '''Links:''' > * About rust-vmm: https://github.com/rust-vmm/community > * rust-vmm/vm-virtio: https://github.com/rust-vmm/vm-virtio > * virtio-console spec: https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-2550003 > > '''Details:''' > * Skill level: intermediate > * Language: Rust > * Mentor: iul at amazon.com > * Suggested by: fandree at amazon.com > > > === Mocking framework for Virtio Queues === > > '''Summary:''' Implement a mocking framework for virtio queues > > Paravirtualized devices (such as those defined by the Virtio standard) are used > to provide high performance device emulation. Virtio drivers from a guest VM > communicate with the device model using an efficient mechanism based on queues > stored in a shared memory area that operate based on a protocol and message format > defined by the standard. Various implementations of devices and other > virtualization building blocks require mocking the contents that a driver would > place into a Virtio queue for validation, testing, and evaluation purposes. > > This project aims to lay the foundations of a reusable framework for mocking the > driver side of Virtio queue operation, that can be consumed by rust-vmm crates and > other projects. At the basic level, this means providing a flexible and easy to > use interface for users to set up the underlying memory areas and populate contents > (as the driver would do) for the basic split queue format in a generic manner. This > can further be extended for the packed format and with device-specific mocking > capabilities. > > '''Links:''' > * About rust-vmm: https://github.com/rust-vmm/community > * Virtio queue spec: https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-230005 > Issue in rust-vmm about reusing the mocking logic: rust-vmm/vm-virtio: https://github.com/rust-vmm/vm-virtio > > '''Details:''' > * Skill level: intermediate > * Language: Rust > * Mentor: aagch at amazon.com > * Suggested by: aagch at amazon.com > > > === Local running rust-vmm-ci === > > '''Summary:''' Run the rust-vmm-ci locally > > The rust-vmm-ci provides automation for uniformely running the tests on > all rust-vmm repositories. It is built on top of Buildkite, and only allows > running the tests in the Buildkite context. To run the same tests as in the CI > locally, users need to manually copy the Buildkite pipeline steps. > > The scope of this project is to make it possible for the same tests to easily run > locally. This project makes it easier to contribute to all rust-vmm repositories. > > In order for that to be possible, the following steps are required: > - the Buildlkite pipeline is autogenerated from code instead of being a static > list of tests to run. This also allows us to uniformely use the same container > version for running all the tests (instead of manually modifying each step in > the pipeline) > - the code for autogenerating the Buildkite pipeline is reused for generating > a Python script which can be run locally > > > '''Links:''' > * rust-vmm-ci: https://github.com/rust-vmm/rust-vmm-ci > * Buildkite pipeline that currently runs the tests: https://github.com/rust-vmm/rust-vmm-ci/blob/master/.buildkite/pipeline.yml > * About rust-vmm: https://github.com/rust-vmm/community > * Buildkite documentation: https://buildkite.com/docs/tutorials/getting-started > > '''Details:''' > * Skill level: intermediate > * Language: Python > * Mentor: fandree at amazon.com > * Suggested by: fandree at amazon.com > > > ?Thanks again! > > Andreea > > ________________________________ > From: Stefan Hajnoczi > Sent: Monday, January 11, 2021 1:47 PM > To: qemu-devel; kvm; rust-vmm at lists.opendev.org; Alex Bennée; Alexander Graf; Alberto Garcia; David Hildenbrand; Eduardo Habkost; Igor Mammedov; John Snow; Julia Suvorova; Gerd Hoffmann; Kevin Wolf; Laurent Vivier; Marc-André Lureau; Aleksandar Markovic; Sergio Lopez; Stefano Garzarella; Paolo Bonzini; Philippe Mathieu-Daudé > Subject: [EXTERNAL] [Rust-VMM] Call for Google Summer of Code 2021 project ideas > > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. > > > > Dear QEMU, KVM, and rust-vmm community, > QEMU will apply for Google Summer of Code > (https://summerofcode.withgoogle.com/) again this year. This internship > program offers paid, 10-week, remote work internships for > contributing to open source. QEMU can act as an umbrella organization > for KVM kernel and rust-vmm projects too. > > Please post project ideas on the QEMU wiki before February 14th: > https://wiki.qemu.org/Google_Summer_of_Code_2021 > > What's new this year: > * The number of internship hours has been halved to 175 hours over > 10 weeks. Project ideas must be smaller to fit and students will have > more flexibility with their working hours. > * Eligibility has been expanded to include "licensed coding school or > similar type of program". > > Good project ideas are suitable for 175 hours (10 weeks half-day) work by a > competent programmer who is not yet familiar with the codebase. In > addition, they are: > * Well-defined - the scope is clear > * Self-contained - there are few dependencies > * Uncontroversial - they are acceptable to the community > * Incremental - they produce deliverables along the way > > Feel free to post ideas even if you are unable to mentor the project. > It doesn't hurt to share the idea! > > I will review project ideas and keep you up-to-date on QEMU's > acceptance into GSoC. > > For more background on QEMU internships, check out this video: > https://www.youtube.com/watch?v=xNVCX7YMUL8 > > Stefan > > _______________________________________________ > Rust-vmm mailing list > Rust-vmm at lists.opendev.org > http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm > > > > Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From alex.bennee at linaro.org Fri Feb 12 14:46:09 2021 From: alex.bennee at linaro.org (Alex =?utf-8?Q?Benn=C3=A9e?=) Date: Fri, 12 Feb 2021 14:46:09 +0000 Subject: [Rust-VMM] Call for Google Summer of Code 2021 project ideas In-Reply-To: <20210212135105.2llsgdiagk2tzu2m@mhamilton> References: <1613136163375.99584@amazon.com> <20210212135105.2llsgdiagk2tzu2m@mhamilton> Message-ID: <874kihz662.fsf@linaro.org> Sergio Lopez writes: > On Fri, Feb 12, 2021 at 01:22:43PM +0000, Florescu, Andreea wrote: >> Hey Stefan, >> >> >> Thanks for taking care of organizing GSoC, and for allowing rust-vmm to also participate under the QEMU umbrella! >> >> I am a bit unsure of how can we propose projects related to rust-vmm. >> >> We did a bit of brainstorming in our team, and we came up with 3 project ideas. >> >> I'll just paste them below, but please let me know if we were supposed to propose them some other way. >> >> >> === Implement the Virtio Console device in Rust === >> >> '''Summary:''' Implement the basic emulation for the Virtio Console device in Rust >> >> Implement the basic functionality (excluding the optional features: >> VIRTIO_CONSOLE_F_SIZE, VIRTIO_CONSOLE_F_MULTIPORT, or VIRTIO_CONSOLE_F_EMERG_WRITE) >> of the Virtio Console Device, using the Virtio building blocks (queue implementations, >> VirtioDevice traits) defined in rust-vmm/vm-virtio. The virtio console device uses >> one virtio queue for transmitting data, and one virtio queue for receiving data. >> The implementation can be extended to also support a subset of the previously >> mentioned optional features. > > FWIW, libkrun already has support for virtio-console with the basic > functionality and VIRTIO_CONSOLE_F_SIZE, and this code could be easily > borrowed for implementing it in some rust-vmm crate: > > https://github.com/containers/libkrun/blob/main/src/devices/src/virtio/console Nevertheless we (Project Stratos) are quite interested in seeing stand-alone VirtIO backends in Rust and experimenting with how easy it is to plug them into different hypervisors (e.g. via KVM/vhost-user and Xen/IOREQ). VirtIO console seems like a simple enough device to experiment with. Alas my Rust is still very much at the newbie stage but I'll happily help with patch review and general VirtIO questions. >> '''Links:''' >> * About rust-vmm: https://github.com/rust-vmm/community >> * rust-vmm/vm-virtio: https://github.com/rust-vmm/vm-virtio >> * virtio-console spec: https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-2550003 >> >> '''Details:''' >> * Skill level: intermediate >> * Language: Rust >> * Mentor: iul at amazon.com >> * Suggested by: fandree at amazon.com I've already done some GSoC mentoring for QEMU and nowadays the program does suggest having backup mentors to help fill in the gaps when the primary mentor is unavailable. -- Alex Bennée From fandree at amazon.com Mon Feb 15 10:13:45 2021 From: fandree at amazon.com (Florescu, Andreea) Date: Mon, 15 Feb 2021 10:13:45 +0000 Subject: [Rust-VMM] rust-vmm 2021 road map discussion In-Reply-To: <8b81ba41cae54864b66f9b5808c61137@EX13D10EUB003.ant.amazon.com> References: <8b81ba41cae54864b66f9b5808c61137@EX13D10EUB003.ant.amazon.com> Message-ID: <1613384025462.64389@amazon.com> Hey everyone, Thanks for attending the meeting! Below there are some action items & meeting minutes that I took. There is no tl;dr sorry about that :) Action Items: * fandree@ to create a poll for setting up regular sync meetings * aagch@ fandree@ lauralg@ to create issues with the AWS-side of the road map * fandree@ (or delegate to someone else) to create a 2021 Road Map in rust-vmm * sameo@ to create issues for PCI * fandree@ sameo@ sync to create issues for vm-vcpu * Sartura will submit their changes to vmm-reference as a PR * Sartura & Sergio Lopez to sync on how they can reuse the C bindings (currently in libkrun) * Amy/Sergio/Sebastien open issue in rust-vmm/community to discuss about vfio-user Road Map 2021: * vm-device: * work on publishing it * vm-virtio * Sartura interested in: * net * block * the PRs are opened in vmm-reference, we’ll discuss priorities inside the team (AWS), and come up with a timeline; Sartura is interested in picking up the PRs and merging them. * vm-vcpu * Vm & Vcpu traits * default implementation for booting a Linux * Sartura is interested in booting Windows as well * Hypervisor abstractions * AWS to sync with Cloud Hypervisor and find the best way of contributing this * vm-memory: * simplify the interface after PRs in flux settle down * PCI * sameo@ will add issue an issue (if one does not exist this) to * vm-superio: * there are no concerns to consume it in Cloud Hypervisor * the crate provides fixes for CVE: * https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-27173 * this needs to be improved as it is cumbersome to use; AWS will provide a better abstraction * we are working on a fix for TX FIFO: * the guest is interrupted too often because of a miss in the implementation: * instead of sending the THRE interrupt for every 64 bytes that the device receives, we’re sending the interrupt for each byte * this incurs delays in the guest & makes the serial console slower than it needs to be * PR from community member: https://github.com/rust-vmm/vm-superio/pull/28 * merged fix for THRE: * when writing to out is not possible, the serial console indefinitely blocks (because we are not triggering THRE anymore) * when using named pipes bytes can be lost in the fraction of time between the time when the pipe gets full up until it gets cleared * when using ring buffers, the output is not full, so this issue is easier to address * vfio-user: * can use the rust-vmm/vfio-ioctls crate: https://github.com/rust-vmm/vfio-ioctls * adding support for this has been discussed at Red Hat: * concerns: the spec is not final, waiting for it to stabilize before proposing a crate * Red Hat is willing to add support for it in rust-vmm * Testing & fuzzing: * add fuzzing (linux-loader, virtio devices, vm-memory) * proposed as GSoC projects: * ability to run tests locally using rust-vmm-ci * mocking utility frameworks for virtio queues * Misc: * Transition crates to workspaces where appropriate to simplify development (i.e. kvm, virtio) * components are still published to crates.io individually * Add threat model to existing components * reference-vmm : * polish it and make it modular * add Arm support * as more functionality gets added to rust-vmm, we can showcase how multiple VMM configurations can be quickly built * Sartura is interested in providing improvements to the serial console: * configuring named pipes for stdin/stdout * Community Improvements: * Process for reporting security vulnerabilities * Improve contributing documentation * Provide a public road map for existing repositories & for the project * Setup regular sync meetings * interested in participating: * Sartura * Intel * Google * AWS​ ________________________________ From: Florescu, Andreea Sent: Thursday, February 11, 2021 5:07 PM To: rust-vmm at lists.opendev.org; Agache, Alexandru; Loghin, Laura; Iordache, Alexandra; Ochescu, Catalin; Barbu, Iulian; borna.blazevic at sartura.hr; Luka Perkov; goran.medic at sartura.hr; meet at chime.aws Cc: compute-capsule-feed at amazon.com; Weiss, Radu; Brendan Burns; Paolo Bonzini; juraj.vijtiuk at sartura.hr; Wise, Bob; saket.sinha89 at gmail.com; Liguori, Anthony; Dima, Alin; Gheorghe, Alexandru-Cosmin; Cihodaru, Alexandru-Ciprian; Boeuf, Sebastien; Popa, Diana-Maria; Gowdar, Meena; leo1.blazevic at gmail.com Subject: rust-vmm 2021 road map discussion When: Monday, February 15, 2021 10:00 AM-11:00 AM. Where: Meeting agenda: 1. Discuss about the rust-vmm road map for 2021 2. Sartura will present a demo using rust-vmm/vmm-reference. It might be a stretch to fit the road map discussions in one hour, so we can look at the high level plans, and then schedule separate meetings to dive deep into certain topics. See you soon! Andreea ==============Conference Bridge Information============== You have been invited to an online meeting, powered by Amazon Chime. Chime meeting ID: 3058950859 Join via Chime clients (manually): Select 'Meetings > Join a Meeting', and enter 3058950859 Join via Chime clients (auto-call): If you invite auto-call as attendee, Chime will call you when the meeting starts, select 'Answer' Join via browser screen share: https://chime.aws/3058950859 Join via phone (US): +1-929-432-4463,,,3058950859# Join via phone (US toll-free): +1-855-552-4463,,,3058950859# International dial-in: https://chime.aws/dialinnumbers/ In-room video system: Ext: 62000, Meeting PIN: 3058950859# ================================================= Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefanha at gmail.com Wed Feb 17 11:20:53 2021 From: stefanha at gmail.com (Stefan Hajnoczi) Date: Wed, 17 Feb 2021 11:20:53 +0000 Subject: [Rust-VMM] Call for Google Summer of Code 2021 project ideas In-Reply-To: <1613136163375.99584@amazon.com> References: <1613136163375.99584@amazon.com> Message-ID: Thanks, I have published the rust-vmm project ideas on the wiki: https://wiki.qemu.org/Google_Summer_of_Code_2021 Please see Sergio's reply about virtio-consoile is libkrun. Maybe it affects the project idea? Stefan From fandree at amazon.com Thu Feb 18 11:49:52 2021 From: fandree at amazon.com (Andreea Florescu) Date: Thu, 18 Feb 2021 13:49:52 +0200 Subject: [Rust-VMM] Call for Google Summer of Code 2021 project ideas In-Reply-To: References: <1613136163375.99584@amazon.com> Message-ID: On 2/17/21 1:20 PM, Stefan Hajnoczi wrote: > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. > > > > Thanks, I have published the rust-vmm project ideas on the wiki: > https://wiki.qemu.org/Google_Summer_of_Code_2021 Thanks, Stefan! > > Please see Sergio's reply about virtio-consoile is libkrun. Maybe it > affects the project idea? I synced offline with Sergio. It seems I've misread his comment. The code is already available in libkrun, and to port it to rust-vmm will likely take just a couple of days. We explored the option of also requesting to implement VIRTIO_CONSOLE_F_MULTIPORT to make it an appropriate GSoC project, but we decided this is not a good idea since it doesn't look like that feature is useful for the projects consuming rust-vmm. It also adds complexity, and we would need to maintain that as well. Since it would still be nice to have virtio-console in rust-vmm, I'll just open an issue in vm-virtio and label it with "help wanted" so people can pick it up. Can we remove the virtio-console project from the list of GSoC ideas? Also, iul at amazon.com will like to help with mentoring the GSoC projects. Can he be added as a co-mentor of: "Mocking framework for Virtio Queues"? Sorry for the ping-pong, and thanks again for everything! Andreea Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005. -------------- next part -------------- A non-text attachment was scrubbed... Name: 0x9B51563C1FA36782.asc Type: application/pgp-keys Size: 2460 bytes Desc: not available URL: From stefanha at gmail.com Thu Feb 18 17:43:52 2021 From: stefanha at gmail.com (Stefan Hajnoczi) Date: Thu, 18 Feb 2021 17:43:52 +0000 Subject: [Rust-VMM] Call for Google Summer of Code 2021 project ideas In-Reply-To: References: <1613136163375.99584@amazon.com> Message-ID: On Thu, Feb 18, 2021 at 11:50 AM Andreea Florescu wrote: > On 2/17/21 1:20 PM, Stefan Hajnoczi wrote: > > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. > > > > > > > > Thanks, I have published the rust-vmm project ideas on the wiki: > > https://wiki.qemu.org/Google_Summer_of_Code_2021 > Thanks, Stefan! > > > > Please see Sergio's reply about virtio-consoile is libkrun. Maybe it > > affects the project idea? > I synced offline with Sergio. It seems I've misread his comment. > The code is already available in libkrun, and to port it to rust-vmm > will likely take just a couple of days. We explored the option of also > requesting to implement VIRTIO_CONSOLE_F_MULTIPORT to make it an > appropriate GSoC project, but we decided this is not a good idea since > it doesn't look like that feature is useful for the projects consuming > rust-vmm. It also adds complexity, and we would need to maintain that as > well. > > Since it would still be nice to have virtio-console in rust-vmm, I'll > just open an issue in vm-virtio and label it with "help wanted" so > people can pick it up. > Can we remove the virtio-console project from the list of GSoC ideas? Done. > Also, iul at amazon.com will like to help with mentoring the GSoC projects. > Can he be added as a co-mentor of: "Mocking framework for Virtio Queues"? Done. Thanks for getting the rust-vmm mentors together! I'm looking forward to more Rust virtualization projects :). Stefan From alex.bennee at linaro.org Fri Feb 19 16:04:34 2021 From: alex.bennee at linaro.org (Alex =?utf-8?Q?Benn=C3=A9e?=) Date: Fri, 19 Feb 2021 16:04:34 +0000 Subject: [Rust-VMM] vhost reply_ack negotiation (a.k.a differences in vhost-user behaviour with libvhost-user and vhost-user-backend.rs) Message-ID: <8735xskm7j.fsf@linaro.org> Hi, I finally got a chance to get down into the guts of vhost-user while attempting to port my original C RPMB daemon to Rust using the vhost-user-backend and related crates. I ended up with this hang during negotiation: startup vhost_user_write req:1 flags:0x1 vhost_user_read_start vhost_user_read req:1 flags:0x5 vhost_user_backend_init: we got 170000000 vhost_user_write req:15 flags:0x1 vhost_user_read_start vhost_user_read req:15 flags:0x5 vhost_user_set_protocol_features: 2008 vhost_user_write req:16 flags:0x1 vhost_user_write req:3 flags:0x1 vhost_user_write req:1 flags:0x1 vhost_user_read_start vhost_user_read req:1 flags:0x5 vhost_user_write req:13 flags:0x1 kernel initialises device virtio_rpmb virtio1: init done! vhost_user_write req:13 flags:0x1 vhost_dev_set_features: 130000000 vhost_user_set_features: 130000000 vhost_user_write req:2 flags:0x1 vhost_user_write req:5 flags:0x9 vhost_user_read_start The proximate cause is the vhost crate handling: MasterReq::SET_MEM_TABLE => { let res = self.set_mem_table(&hdr, size, &buf, rfds); self.send_ack_message(&hdr, res)?; } which gates on the replay_ack_enabled flag: fn send_ack_message( &mut self, req: &VhostUserMsgHeader, res: Result<()>, ) -> Result<()> { if dbg!(self.reply_ack_enabled) { let hdr = self.new_reply_header::(req, 0)?; let val = match res { Ok(_) => 0, Err(_) => 1, }; let msg = VhostUserU64::new(val); self.main_sock.send_message(&hdr, &msg, None)?; } Ok(()) } which is only set when we have all the appropriate acknowledged flags: fn update_reply_ack_flag(&mut self) { let vflag = VhostUserVirtioFeatures::PROTOCOL_FEATURES.bits(); let pflag = VhostUserProtocolFeatures::REPLY_ACK; if (self.virtio_features & vflag) != 0 && (self.acked_virtio_features & vflag) != 0 && self.protocol_features.contains(pflag) && (self.acked_protocol_features & pflag.bits()) != 0 { self.reply_ack_enabled = true; } else { self.reply_ack_enabled = false; } } which from above you can see QEMU helpfully dropped those bits in the reply. It does however work in the C/libvhost version: virtio_rpmb virtio1: init done! vhost_user_write req:13 flags:0x1 vhost_dev_set_features: 130000000 vhost_user_set_features: 130000000 vhost_user_write req:2 flags:0x1 vhost_user_write req:37 flags:0x9 vhost_user_read_start vhost_user_read req:37 flags:0x5 vhost_user_write req:8 flags:0x1 vhost_user_write req:10 flags:0x1 vhost_user_write req:9 flags:0x1 vhost_user_write req:12 flags:0x1 vhost_user_write req:13 flags:0x1 albeit with a slightly different message sequence (VHOST_USER_ADD_MEM_REG instead of VHOST_USER_SET_MEM_TABLE). Reading the C code you can see why: need_reply = vmsg.flags & VHOST_USER_NEED_REPLY_MASK; reply_requested = vu_process_message(dev, &vmsg); if (!reply_requested && need_reply) { vmsg_set_reply_u64(&vmsg, 0); reply_requested = 1; } So regardless of what may have been negotiated it will always reply with something if the master requested it do so. This points us at the specification which reads: - Bit 3 is the need_reply flag - see :ref:`REPLY_ACK ` for details. which says in VHOST_USER_PROTOCOL_F_REPLY_ACK that this bit should only be honoured when the feature has been negotiated. Which brings us to a series of questions: - Should QEMU have preserved VhostUserVirtioFeatures::PROTOCOL_FEATURES when doing the eventual VHOST_USER_SET_FEATURES reply? - Is vhost.rs being to strict or libvhost-user too lax in interpreting the negotiated features before processing the ``need_reply`` [Bit 3] field of the messages? - are VHOST_USER_SET_MEM_TABLE to VHOST_USER_SET_INFLIGHT_FD included in the "list of the ones that do" require replies or do they only reply when REPLY_ACK has been negotiated as the ambiguous "seealso::" box out seems to imply? Currently I have some hacks in: https://github.com/stsquad/vhost/tree/my-hacks which gets my daemon booting up to the point we actually need to do a transaction. However I won't submit a PR until I've worked out exactly where the problems are. -- Alex Bennée From fandree at amazon.com Mon Feb 22 09:24:58 2021 From: fandree at amazon.com (Florescu, Andreea) Date: Mon, 22 Feb 2021 09:24:58 +0000 Subject: [Rust-VMM] Enabling automatic dependency updates Message-ID: <1613985898163.37078@amazon.com> Hey everyone, I've enabled automatic dependency updates for all rust-vmm repositories. These are performed by Dependabot [1]. The main reason for that is making sure we uniformly test all the rust-vmm crates using the latest version of the rust-vmm-ci (which is pulled as a git submodule). Initially I've set it up such that is also updates the Rust dependencies, but it seems that is not very wise for some repositories where we need to use strict dependencies (for example we need to use a strict version of critcmp =0.3.0 so that we can properly parse the output of benchmarks). I've since disabled the Rust updated and now dependabot only updates git submodules. You should expect to see quite a few PRs from dependabot-preview GitHub user. They will **all** fail at least one test: the commit message one. The reason for the failure is tracked in the rust-vmm-ci [2] repo. Thanks, Andreea [1] https://dependabot.com/ [2] https://github.com/rust-vmm/rust-vmm-ci/issues/61 Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dgilbert at redhat.com Mon Feb 22 13:06:07 2021 From: dgilbert at redhat.com (Dr. David Alan Gilbert) Date: Mon, 22 Feb 2021 13:06:07 +0000 Subject: [Rust-VMM] vhost reply_ack negotiation (a.k.a differences in vhost-user behaviour with libvhost-user and vhost-user-backend.rs) In-Reply-To: <8735xskm7j.fsf@linaro.org> References: <8735xskm7j.fsf@linaro.org> Message-ID: * Alex Bennée (alex.bennee at linaro.org) wrote: > Hi, > > I finally got a chance to get down into the guts of vhost-user while > attempting to port my original C RPMB daemon to Rust using the > vhost-user-backend and related crates. I ended up with this hang during > negotiation: > > startup > > vhost_user_write req:1 flags:0x1 > vhost_user_read_start > vhost_user_read req:1 flags:0x5 > vhost_user_backend_init: we got 170000000 > vhost_user_write req:15 flags:0x1 > vhost_user_read_start > vhost_user_read req:15 flags:0x5 > vhost_user_set_protocol_features: 2008 > vhost_user_write req:16 flags:0x1 > vhost_user_write req:3 flags:0x1 > vhost_user_write req:1 flags:0x1 > vhost_user_read_start > vhost_user_read req:1 flags:0x5 > vhost_user_write req:13 flags:0x1 > > kernel initialises device > > virtio_rpmb virtio1: init done! > vhost_user_write req:13 flags:0x1 > vhost_dev_set_features: 130000000 > vhost_user_set_features: 130000000 > vhost_user_write req:2 flags:0x1 > vhost_user_write req:5 flags:0x9 > vhost_user_read_start > > The proximate cause is the vhost crate handling: > > MasterReq::SET_MEM_TABLE => { > let res = self.set_mem_table(&hdr, size, &buf, rfds); > self.send_ack_message(&hdr, res)?; > } > > which gates on the replay_ack_enabled flag: > > fn send_ack_message( > &mut self, > req: &VhostUserMsgHeader, > res: Result<()>, > ) -> Result<()> { > if dbg!(self.reply_ack_enabled) { > let hdr = self.new_reply_header::(req, 0)?; > let val = match res { > Ok(_) => 0, > Err(_) => 1, > }; > let msg = VhostUserU64::new(val); > self.main_sock.send_message(&hdr, &msg, None)?; > } > Ok(()) > } > > which is only set when we have all the appropriate acknowledged flags: > > fn update_reply_ack_flag(&mut self) { > let vflag = VhostUserVirtioFeatures::PROTOCOL_FEATURES.bits(); > let pflag = VhostUserProtocolFeatures::REPLY_ACK; > if (self.virtio_features & vflag) != 0 > && (self.acked_virtio_features & vflag) != 0 > && self.protocol_features.contains(pflag) > && (self.acked_protocol_features & pflag.bits()) != 0 > { > self.reply_ack_enabled = true; > } else { > self.reply_ack_enabled = false; > } > } > > which from above you can see QEMU helpfully dropped those bits in the > reply. It does however work in the C/libvhost version: > > virtio_rpmb virtio1: init done! > vhost_user_write req:13 flags:0x1 > vhost_dev_set_features: 130000000 > vhost_user_set_features: 130000000 > vhost_user_write req:2 flags:0x1 > vhost_user_write req:37 flags:0x9 > vhost_user_read_start > vhost_user_read req:37 flags:0x5 > vhost_user_write req:8 flags:0x1 > vhost_user_write req:10 flags:0x1 > vhost_user_write req:9 flags:0x1 > vhost_user_write req:12 flags:0x1 > vhost_user_write req:13 flags:0x1 > > albeit with a slightly different message sequence > (VHOST_USER_ADD_MEM_REG instead of VHOST_USER_SET_MEM_TABLE). Reading > the C code you can see why: > > need_reply = vmsg.flags & VHOST_USER_NEED_REPLY_MASK; > > reply_requested = vu_process_message(dev, &vmsg); > if (!reply_requested && need_reply) { > vmsg_set_reply_u64(&vmsg, 0); > reply_requested = 1; > } > > So regardless of what may have been negotiated it will always reply with > something if the master requested it do so. This points us at the > specification which reads: > > - Bit 3 is the need_reply flag - see :ref:`REPLY_ACK ` for > details. > > which says in VHOST_USER_PROTOCOL_F_REPLY_ACK that this bit should only > be honoured when the feature has been negotiated. Which brings us to a > series of questions: > > - Should QEMU have preserved VhostUserVirtioFeatures::PROTOCOL_FEATURES > when doing the eventual VHOST_USER_SET_FEATURES reply? > > - Is vhost.rs being to strict or libvhost-user too lax in interpreting > the negotiated features before processing the ``need_reply`` [Bit 3] > field of the messages? I think vhost.rs is being correctly strict - but there would be no harm in it flagging that you'd hit an inconsistency if it finds a need_reply without the feature. > - are VHOST_USER_SET_MEM_TABLE to VHOST_USER_SET_INFLIGHT_FD included > in the "list of the ones that do" require replies or do they only > reply when REPLY_ACK has been negotiated as the ambiguous "seealso::" > box out seems to imply? set_mem_table gives a reply when postcopy is enabled (and then qemu replies to the reply!) but otherwise doesn't. (Note there's an issue opened for .rs to support ADD_MEM_REGION since it's a lot better than SET_MEM_TABLE which has a fixed size table that's small). Dave > Currently I have some hacks in: > > https://github.com/stsquad/vhost/tree/my-hacks > > which gets my daemon booting up to the point we actually need to do a > transaction. However I won't submit a PR until I've worked out exactly > where the problems are. > > -- > Alex Bennée > -- Dr. David Alan Gilbert / dgilbert at redhat.com / Manchester, UK From alex.bennee at linaro.org Mon Feb 22 13:21:04 2021 From: alex.bennee at linaro.org (Alex =?utf-8?Q?Benn=C3=A9e?=) Date: Mon, 22 Feb 2021 13:21:04 +0000 Subject: [Rust-VMM] vhost reply_ack negotiation (a.k.a differences in vhost-user behaviour with libvhost-user and vhost-user-backend.rs) In-Reply-To: References: <8735xskm7j.fsf@linaro.org> Message-ID: <871rd86xrf.fsf@linaro.org> Dr. David Alan Gilbert writes: > * Alex Bennée (alex.bennee at linaro.org) wrote: >> Hi, >> >> I finally got a chance to get down into the guts of vhost-user while >> attempting to port my original C RPMB daemon to Rust using the >> vhost-user-backend and related crates. I ended up with this hang during >> negotiation: >> >> startup >> >> vhost_user_write req:1 flags:0x1 >> vhost_user_read_start >> vhost_user_read req:1 flags:0x5 >> vhost_user_backend_init: we got 170000000 GET_FEATURES >> vhost_user_write req:15 flags:0x1 >> vhost_user_read_start >> vhost_user_read req:15 flags:0x5 >> vhost_user_set_protocol_features: 2008 >> vhost_user_write req:16 flags:0x1 >> vhost_user_write req:3 flags:0x1 >> vhost_user_write req:1 flags:0x1 >> vhost_user_read_start >> vhost_user_read req:1 flags:0x5 >> vhost_user_write req:13 flags:0x1 >> >> kernel initialises device >> >> virtio_rpmb virtio1: init done! >> vhost_user_write req:13 flags:0x1 >> vhost_dev_set_features: 130000000 >> vhost_user_set_features: 130000000 SET_FEATURES >> vhost_user_write req:2 flags:0x1 >> vhost_user_write req:5 flags:0x9 >> vhost_user_read_start >> >> >> - Should QEMU have preserved VhostUserVirtioFeatures::PROTOCOL_FEATURES >> when doing the eventual VHOST_USER_SET_FEATURES reply? >> >> - Is vhost.rs being to strict or libvhost-user too lax in interpreting >> the negotiated features before processing the ``need_reply`` [Bit 3] >> field of the messages? > > I think vhost.rs is being correctly strict - but there would be no harm > in it flagging that you'd hit an inconsistency if it finds a need_reply > without the feature. But the feature should have been negotiated. So unless the slave can assume it is enabled because it asked I think QEMU is in the wrong by not preserving the feature bits in it's SET_FEATURES reply. We just gets away with it with libvhostuser being willing to reply anyway. > >> - are VHOST_USER_SET_MEM_TABLE to VHOST_USER_SET_INFLIGHT_FD included >> in the "list of the ones that do" require replies or do they only >> reply when REPLY_ACK has been negotiated as the ambiguous "seealso::" >> box out seems to imply? > > set_mem_table gives a reply when postcopy is enabled (and then qemu > replies to the reply!) but otherwise doesn't. > (Note there's an issue opened for .rs to support ADD_MEM_REGION > since it's a lot better than SET_MEM_TABLE which has a fixed size table > that's small). Thanks for the heads up. > > Dave > >> Currently I have some hacks in: >> >> https://github.com/stsquad/vhost/tree/my-hacks >> >> which gets my daemon booting up to the point we actually need to do a >> transaction. However I won't submit a PR until I've worked out exactly >> where the problems are. >> >> -- >> Alex Bennée >> -- Alex Bennée From fandree at amazon.com Mon Feb 22 13:24:30 2021 From: fandree at amazon.com (Florescu, Andreea) Date: Mon, 22 Feb 2021 13:24:30 +0000 Subject: [Rust-VMM] [Action Required] rust-vmm sync meetings Message-ID: <1614000270456.88753@amazon.com> Hey folks, In the last week sync meeting we discussed about recreating the bi-weekly rust-vmm sync meeting. I created a poll to see when most people can join: https://doodle.com/poll/nrrsivwytgc5zy9n?utm_source=poll&utm_medium=link ? The poll will close on 2021-03-01 so please choose your preferred hour by then. I'll schedule recurring meetings (every 2 weeks, on Monday) starting with ?2021-03-08. Thanks, Andreea Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dgilbert at redhat.com Mon Feb 22 13:27:22 2021 From: dgilbert at redhat.com (Dr. David Alan Gilbert) Date: Mon, 22 Feb 2021 13:27:22 +0000 Subject: [Rust-VMM] vhost reply_ack negotiation (a.k.a differences in vhost-user behaviour with libvhost-user and vhost-user-backend.rs) In-Reply-To: <871rd86xrf.fsf@linaro.org> References: <8735xskm7j.fsf@linaro.org> <871rd86xrf.fsf@linaro.org> Message-ID: * Alex Bennée (alex.bennee at linaro.org) wrote: > > Dr. David Alan Gilbert writes: > > > * Alex Bennée (alex.bennee at linaro.org) wrote: > >> Hi, > >> > >> I finally got a chance to get down into the guts of vhost-user while > >> attempting to port my original C RPMB daemon to Rust using the > >> vhost-user-backend and related crates. I ended up with this hang during > >> negotiation: > >> > >> startup > >> > >> vhost_user_write req:1 flags:0x1 > >> vhost_user_read_start > >> vhost_user_read req:1 flags:0x5 > >> vhost_user_backend_init: we got 170000000 > > GET_FEATURES > > >> vhost_user_write req:15 flags:0x1 > >> vhost_user_read_start > >> vhost_user_read req:15 flags:0x5 > >> vhost_user_set_protocol_features: 2008 > >> vhost_user_write req:16 flags:0x1 > >> vhost_user_write req:3 flags:0x1 > >> vhost_user_write req:1 flags:0x1 > >> vhost_user_read_start > >> vhost_user_read req:1 flags:0x5 > >> vhost_user_write req:13 flags:0x1 > >> > >> kernel initialises device > >> > >> virtio_rpmb virtio1: init done! > >> vhost_user_write req:13 flags:0x1 > >> vhost_dev_set_features: 130000000 > >> vhost_user_set_features: 130000000 > > SET_FEATURES > > >> vhost_user_write req:2 flags:0x1 > >> vhost_user_write req:5 flags:0x9 > >> vhost_user_read_start > >> > > >> > >> - Should QEMU have preserved VhostUserVirtioFeatures::PROTOCOL_FEATURES > >> when doing the eventual VHOST_USER_SET_FEATURES reply? > >> > >> - Is vhost.rs being to strict or libvhost-user too lax in interpreting > >> the negotiated features before processing the ``need_reply`` [Bit 3] > >> field of the messages? > > > > I think vhost.rs is being correctly strict - but there would be no harm > > in it flagging that you'd hit an inconsistency if it finds a need_reply > > without the feature. > > But the feature should have been negotiated. So unless the slave can > assume it is enabled because it asked I think QEMU is in the wrong by > not preserving the feature bits in it's SET_FEATURES reply. We just gets > away with it with libvhostuser being willing to reply anyway. Oh I wasn't trying to reply to that bit; I can never remember how the vhost/virtio feature bit negotiation works... Dave > > > >> - are VHOST_USER_SET_MEM_TABLE to VHOST_USER_SET_INFLIGHT_FD included > >> in the "list of the ones that do" require replies or do they only > >> reply when REPLY_ACK has been negotiated as the ambiguous "seealso::" > >> box out seems to imply? > > > > set_mem_table gives a reply when postcopy is enabled (and then qemu > > replies to the reply!) but otherwise doesn't. > > (Note there's an issue opened for .rs to support ADD_MEM_REGION > > since it's a lot better than SET_MEM_TABLE which has a fixed size table > > that's small). > > Thanks for the heads up. > > > > > Dave > > > >> Currently I have some hacks in: > >> > >> https://github.com/stsquad/vhost/tree/my-hacks > >> > >> which gets my daemon booting up to the point we actually need to do a > >> transaction. However I won't submit a PR until I've worked out exactly > >> where the problems are. > >> > >> -- > >> Alex Bennée > >> > > > -- > Alex Bennée > -- Dr. David Alan Gilbert / dgilbert at redhat.com / Manchester, UK From liuj97 at gmail.com Tue Feb 23 10:23:08 2021 From: liuj97 at gmail.com (Jiang Liu) Date: Tue, 23 Feb 2021 18:23:08 +0800 Subject: [Rust-VMM] vhost reply_ack negotiation (a.k.a differences in vhost-user behaviour with libvhost-user and vhost-user-backend.rs) In-Reply-To: References: <8735xskm7j.fsf@linaro.org> <871rd86xrf.fsf@linaro.org> Message-ID: I just found this thread in my email junk box:( I do have found some bugs in the vhost_rs crate, related to handle the need_reply flag. But that bug only affects virtio-fs fs_map operations. Please refer to: https://github.com/rust-vmm/vhost/pull/19 https://github.com/rust-vmm/vhost/pull/19/commits/a2c5a4f50e45ae1ab78622dda9a3f861bd43a17e Thanks, Gerry > On Feb 22, 2021, at 9:27 PM, Dr. David Alan Gilbert wrote: > > * Alex Bennée (alex.bennee at linaro.org) wrote: >> >> Dr. David Alan Gilbert writes: >> >>> * Alex Bennée (alex.bennee at linaro.org) wrote: >>>> Hi, >>>> >>>> I finally got a chance to get down into the guts of vhost-user while >>>> attempting to port my original C RPMB daemon to Rust using the >>>> vhost-user-backend and related crates. I ended up with this hang during >>>> negotiation: >>>> >>>> startup >>>> >>>> vhost_user_write req:1 flags:0x1 >>>> vhost_user_read_start >>>> vhost_user_read req:1 flags:0x5 >>>> vhost_user_backend_init: we got 170000000 >> >> GET_FEATURES >> >>>> vhost_user_write req:15 flags:0x1 >>>> vhost_user_read_start >>>> vhost_user_read req:15 flags:0x5 >>>> vhost_user_set_protocol_features: 2008 >>>> vhost_user_write req:16 flags:0x1 >>>> vhost_user_write req:3 flags:0x1 >>>> vhost_user_write req:1 flags:0x1 >>>> vhost_user_read_start >>>> vhost_user_read req:1 flags:0x5 >>>> vhost_user_write req:13 flags:0x1 >>>> >>>> kernel initialises device >>>> >>>> virtio_rpmb virtio1: init done! >>>> vhost_user_write req:13 flags:0x1 >>>> vhost_dev_set_features: 130000000 >>>> vhost_user_set_features: 130000000 >> >> SET_FEATURES >> >>>> vhost_user_write req:2 flags:0x1 >>>> vhost_user_write req:5 flags:0x9 >>>> vhost_user_read_start >>>> >> >>>> >>>> - Should QEMU have preserved VhostUserVirtioFeatures::PROTOCOL_FEATURES >>>> when doing the eventual VHOST_USER_SET_FEATURES reply? >>>> >>>> - Is vhost.rs being to strict or libvhost-user too lax in interpreting >>>> the negotiated features before processing the ``need_reply`` [Bit 3] >>>> field of the messages? >>> >>> I think vhost.rs is being correctly strict - but there would be no harm >>> in it flagging that you'd hit an inconsistency if it finds a need_reply >>> without the feature. >> >> But the feature should have been negotiated. So unless the slave can >> assume it is enabled because it asked I think QEMU is in the wrong by >> not preserving the feature bits in it's SET_FEATURES reply. We just gets >> away with it with libvhostuser being willing to reply anyway. > > Oh I wasn't trying to reply to that bit; I can never remember how the > vhost/virtio feature bit negotiation works... > > Dave > >>> >>>> - are VHOST_USER_SET_MEM_TABLE to VHOST_USER_SET_INFLIGHT_FD included >>>> in the "list of the ones that do" require replies or do they only >>>> reply when REPLY_ACK has been negotiated as the ambiguous "seealso::" >>>> box out seems to imply? >>> >>> set_mem_table gives a reply when postcopy is enabled (and then qemu >>> replies to the reply!) but otherwise doesn't. >>> (Note there's an issue opened for .rs to support ADD_MEM_REGION >>> since it's a lot better than SET_MEM_TABLE which has a fixed size table >>> that's small). >> >> Thanks for the heads up. >> >>> >>> Dave >>> >>>> Currently I have some hacks in: >>>> >>>> https://github.com/stsquad/vhost/tree/my-hacks >>>> >>>> which gets my daemon booting up to the point we actually need to do a >>>> transaction. However I won't submit a PR until I've worked out exactly >>>> where the problems are. >>>> >>>> -- >>>> Alex Bennée >>>> >> >> >> -- >> Alex Bennée >> > -- > Dr. David Alan Gilbert / dgilbert at redhat.com / Manchester, UK > > > _______________________________________________ > Rust-vmm mailing list > Rust-vmm at lists.opendev.org > http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm From mst at redhat.com Tue Feb 23 11:44:55 2021 From: mst at redhat.com (Michael S. Tsirkin) Date: Tue, 23 Feb 2021 06:44:55 -0500 Subject: [Rust-VMM] vhost reply_ack negotiation (a.k.a differences in vhost-user behaviour with libvhost-user and vhost-user-backend.rs) In-Reply-To: <8735xskm7j.fsf@linaro.org> References: <8735xskm7j.fsf@linaro.org> Message-ID: <20210223064312-mutt-send-email-mst@kernel.org> Cc: Raphael On Fri, Feb 19, 2021 at 04:04:34PM +0000, Alex Bennée wrote: > Hi, > > I finally got a chance to get down into the guts of vhost-user while > attempting to port my original C RPMB daemon to Rust using the > vhost-user-backend and related crates. I ended up with this hang during > negotiation: > > startup > > vhost_user_write req:1 flags:0x1 > vhost_user_read_start > vhost_user_read req:1 flags:0x5 > vhost_user_backend_init: we got 170000000 > vhost_user_write req:15 flags:0x1 > vhost_user_read_start > vhost_user_read req:15 flags:0x5 > vhost_user_set_protocol_features: 2008 > vhost_user_write req:16 flags:0x1 > vhost_user_write req:3 flags:0x1 > vhost_user_write req:1 flags:0x1 > vhost_user_read_start > vhost_user_read req:1 flags:0x5 > vhost_user_write req:13 flags:0x1 > > kernel initialises device > > virtio_rpmb virtio1: init done! > vhost_user_write req:13 flags:0x1 > vhost_dev_set_features: 130000000 > vhost_user_set_features: 130000000 > vhost_user_write req:2 flags:0x1 > vhost_user_write req:5 flags:0x9 > vhost_user_read_start > > The proximate cause is the vhost crate handling: > > MasterReq::SET_MEM_TABLE => { > let res = self.set_mem_table(&hdr, size, &buf, rfds); > self.send_ack_message(&hdr, res)?; > } > > which gates on the replay_ack_enabled flag: > > fn send_ack_message( > &mut self, > req: &VhostUserMsgHeader, > res: Result<()>, > ) -> Result<()> { > if dbg!(self.reply_ack_enabled) { > let hdr = self.new_reply_header::(req, 0)?; > let val = match res { > Ok(_) => 0, > Err(_) => 1, > }; > let msg = VhostUserU64::new(val); > self.main_sock.send_message(&hdr, &msg, None)?; > } > Ok(()) > } > > which is only set when we have all the appropriate acknowledged flags: > > fn update_reply_ack_flag(&mut self) { > let vflag = VhostUserVirtioFeatures::PROTOCOL_FEATURES.bits(); > let pflag = VhostUserProtocolFeatures::REPLY_ACK; > if (self.virtio_features & vflag) != 0 > && (self.acked_virtio_features & vflag) != 0 > && self.protocol_features.contains(pflag) > && (self.acked_protocol_features & pflag.bits()) != 0 > { > self.reply_ack_enabled = true; > } else { > self.reply_ack_enabled = false; > } > } > > which from above you can see QEMU helpfully dropped those bits in the > reply. It does however work in the C/libvhost version: > > virtio_rpmb virtio1: init done! > vhost_user_write req:13 flags:0x1 > vhost_dev_set_features: 130000000 > vhost_user_set_features: 130000000 > vhost_user_write req:2 flags:0x1 > vhost_user_write req:37 flags:0x9 > vhost_user_read_start > vhost_user_read req:37 flags:0x5 > vhost_user_write req:8 flags:0x1 > vhost_user_write req:10 flags:0x1 > vhost_user_write req:9 flags:0x1 > vhost_user_write req:12 flags:0x1 > vhost_user_write req:13 flags:0x1 > > albeit with a slightly different message sequence > (VHOST_USER_ADD_MEM_REG instead of VHOST_USER_SET_MEM_TABLE). Reading > the C code you can see why: > > need_reply = vmsg.flags & VHOST_USER_NEED_REPLY_MASK; > > reply_requested = vu_process_message(dev, &vmsg); > if (!reply_requested && need_reply) { > vmsg_set_reply_u64(&vmsg, 0); > reply_requested = 1; > } > > So regardless of what may have been negotiated it will always reply with > something if the master requested it do so. This points us at the > specification which reads: > > - Bit 3 is the need_reply flag - see :ref:`REPLY_ACK ` for > details. > > which says in VHOST_USER_PROTOCOL_F_REPLY_ACK that this bit should only > be honoured when the feature has been negotiated. Which brings us to a > series of questions: > > - Should QEMU have preserved VhostUserVirtioFeatures::PROTOCOL_FEATURES > when doing the eventual VHOST_USER_SET_FEATURES reply? Hmm looks like a bug indeed ... Anyone wants to look into fixing that? Marc-André? > - Is vhost.rs being to strict or libvhost-user too lax in interpreting > the negotiated features before processing the ``need_reply`` [Bit 3] > field of the messages? > > - are VHOST_USER_SET_MEM_TABLE to VHOST_USER_SET_INFLIGHT_FD included > in the "list of the ones that do" require replies or do they only > reply when REPLY_ACK has been negotiated as the ambiguous "seealso::" > box out seems to imply? > > Currently I have some hacks in: > > https://github.com/stsquad/vhost/tree/my-hacks > > which gets my daemon booting up to the point we actually need to do a > transaction. However I won't submit a PR until I've worked out exactly > where the problems are. > > -- > Alex Bennée From alex.bennee at linaro.org Thu Feb 25 14:21:56 2021 From: alex.bennee at linaro.org (=?UTF-8?q?Alex=20Benn=C3=A9e?=) Date: Thu, 25 Feb 2021 14:21:56 +0000 Subject: [Rust-VMM] [RFC PATCH] vhost_user/slave_reg_handler: relax the reply_ack_flag test Message-ID: <20210225142156.28328-1-alex.bennee@linaro.org> In reality the real guest may never see PROTOCOL_FEATURES bit when it probes the device configuration. It shouldn't matter to it anyway as they details of reply acknowledgement have already been negotiated between the two ends of the vhost-user protocol having been advertised by the backend on start-up. This prevents a hang with the QEMU remote end which makes it's determination to ask for a reply based only on the negotiated protocol feature set: bool reply_supported = virtio_has_feature(dev->protocol_features, VHOST_USER_PROTOCOL_F_REPLY_ACK); Let's be liberal in what we accept. Signed-off-by: Alex Bennée Cc: Michael S. Tsirkin --- src/vhost_user/slave_req_handler.rs | 1 - 1 file changed, 1 deletion(-) diff --git a/src/vhost_user/slave_req_handler.rs b/src/vhost_user/slave_req_handler.rs index f3b0770..f7c9aeb 100644 --- a/src/vhost_user/slave_req_handler.rs +++ b/src/vhost_user/slave_req_handler.rs @@ -537,7 +537,6 @@ impl SlaveReqHandler { let vflag = VhostUserVirtioFeatures::PROTOCOL_FEATURES.bits(); let pflag = VhostUserProtocolFeatures::REPLY_ACK; if (self.virtio_features & vflag) != 0 - && (self.acked_virtio_features & vflag) != 0 && self.protocol_features.contains(pflag) && (self.acked_protocol_features & pflag.bits()) != 0 { -- 2.20.1 From dgreid at chromium.org Thu Feb 25 04:19:48 2021 From: dgreid at chromium.org (Dylan Reid) Date: Wed, 24 Feb 2021 20:19:48 -0800 Subject: [Rust-VMM] vhost reply_ack negotiation (a.k.a differences in vhost-user behaviour with libvhost-user and vhost-user-backend.rs) In-Reply-To: <20210223064312-mutt-send-email-mst@kernel.org> References: <8735xskm7j.fsf@linaro.org> <20210223064312-mutt-send-email-mst@kernel.org> Message-ID: On Tue, Feb 23, 2021 at 8:20 AM Michael S. Tsirkin wrote: > > Cc: Raphael > > On Fri, Feb 19, 2021 at 04:04:34PM +0000, Alex Bennée wrote: > > Hi, > > > > I finally got a chance to get down into the guts of vhost-user while > > attempting to port my original C RPMB daemon to Rust using the > > vhost-user-backend and related crates. I ended up with this hang during > > negotiation: > > > > startup > > > > vhost_user_write req:1 flags:0x1 > > vhost_user_read_start > > vhost_user_read req:1 flags:0x5 > > vhost_user_backend_init: we got 170000000 > > vhost_user_write req:15 flags:0x1 > > vhost_user_read_start > > vhost_user_read req:15 flags:0x5 > > vhost_user_set_protocol_features: 2008 > > vhost_user_write req:16 flags:0x1 > > vhost_user_write req:3 flags:0x1 > > vhost_user_write req:1 flags:0x1 > > vhost_user_read_start > > vhost_user_read req:1 flags:0x5 > > vhost_user_write req:13 flags:0x1 > > > > kernel initialises device > > > > virtio_rpmb virtio1: init done! > > vhost_user_write req:13 flags:0x1 > > vhost_dev_set_features: 130000000 > > vhost_user_set_features: 130000000 > > vhost_user_write req:2 flags:0x1 > > vhost_user_write req:5 flags:0x9 > > vhost_user_read_start > > > > The proximate cause is the vhost crate handling: > > > > MasterReq::SET_MEM_TABLE => { > > let res = self.set_mem_table(&hdr, size, &buf, rfds); > > self.send_ack_message(&hdr, res)?; > > } > > > > which gates on the replay_ack_enabled flag: > > > > fn send_ack_message( > > &mut self, > > req: &VhostUserMsgHeader, > > res: Result<()>, > > ) -> Result<()> { > > if dbg!(self.reply_ack_enabled) { > > let hdr = self.new_reply_header::(req, 0)?; > > let val = match res { > > Ok(_) => 0, > > Err(_) => 1, > > }; > > let msg = VhostUserU64::new(val); > > self.main_sock.send_message(&hdr, &msg, None)?; > > } > > Ok(()) > > } > > > > which is only set when we have all the appropriate acknowledged flags: > > > > fn update_reply_ack_flag(&mut self) { > > let vflag = VhostUserVirtioFeatures::PROTOCOL_FEATURES.bits(); > > let pflag = VhostUserProtocolFeatures::REPLY_ACK; > > if (self.virtio_features & vflag) != 0 > > && (self.acked_virtio_features & vflag) != 0 > > && self.protocol_features.contains(pflag) > > && (self.acked_protocol_features & pflag.bits()) != 0 > > { > > self.reply_ack_enabled = true; > > } else { > > self.reply_ack_enabled = false; > > } > > } > > > > which from above you can see QEMU helpfully dropped those bits in the > > reply. It does however work in the C/libvhost version: > > > > virtio_rpmb virtio1: init done! > > vhost_user_write req:13 flags:0x1 > > vhost_dev_set_features: 130000000 > > vhost_user_set_features: 130000000 > > vhost_user_write req:2 flags:0x1 > > vhost_user_write req:37 flags:0x9 > > vhost_user_read_start > > vhost_user_read req:37 flags:0x5 > > vhost_user_write req:8 flags:0x1 > > vhost_user_write req:10 flags:0x1 > > vhost_user_write req:9 flags:0x1 > > vhost_user_write req:12 flags:0x1 > > vhost_user_write req:13 flags:0x1 > > > > albeit with a slightly different message sequence > > (VHOST_USER_ADD_MEM_REG instead of VHOST_USER_SET_MEM_TABLE). Reading > > the C code you can see why: > > > > need_reply = vmsg.flags & VHOST_USER_NEED_REPLY_MASK; > > > > reply_requested = vu_process_message(dev, &vmsg); > > if (!reply_requested && need_reply) { > > vmsg_set_reply_u64(&vmsg, 0); > > reply_requested = 1; > > } > > > > So regardless of what may have been negotiated it will always reply with > > something if the master requested it do so. This points us at the > > specification which reads: > > > > - Bit 3 is the need_reply flag - see :ref:`REPLY_ACK ` for > > details. > > > > which says in VHOST_USER_PROTOCOL_F_REPLY_ACK that this bit should only > > be honoured when the feature has been negotiated. Which brings us to a > > series of questions: > > > > - Should QEMU have preserved VhostUserVirtioFeatures::PROTOCOL_FEATURES > > when doing the eventual VHOST_USER_SET_FEATURES reply? > > Hmm looks like a bug indeed ... Anyone wants to look > into fixing that? Marc-André? chirantan and keiichi will be implementing vhost-user-vitio-fs on Chrome OS, maybe one of you two can take a look? > > > > > - Is vhost.rs being to strict or libvhost-user too lax in interpreting > > the negotiated features before processing the ``need_reply`` [Bit 3] > > field of the messages? > > > > - are VHOST_USER_SET_MEM_TABLE to VHOST_USER_SET_INFLIGHT_FD included > > in the "list of the ones that do" require replies or do they only > > reply when REPLY_ACK has been negotiated as the ambiguous "seealso::" > > box out seems to imply? > > > > Currently I have some hacks in: > > > > https://github.com/stsquad/vhost/tree/my-hacks > > > > which gets my daemon booting up to the point we actually need to do a > > transaction. However I won't submit a PR until I've worked out exactly > > where the problems are. > > > > -- > > Alex Bennée > > > _______________________________________________ > Rust-vmm mailing list > Rust-vmm at lists.opendev.org > http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm From keiichiw at chromium.org Thu Feb 25 06:36:07 2021 From: keiichiw at chromium.org (Keiichi Watanabe) Date: Thu, 25 Feb 2021 15:36:07 +0900 Subject: [Rust-VMM] vhost reply_ack negotiation (a.k.a differences in vhost-user behaviour with libvhost-user and vhost-user-backend.rs) In-Reply-To: References: <8735xskm7j.fsf@linaro.org> <20210223064312-mutt-send-email-mst@kernel.org> Message-ID: On Thu, Feb 25, 2021 at 1:20 PM Dylan Reid wrote: > On Tue, Feb 23, 2021 at 8:20 AM Michael S. Tsirkin wrote: > > > > Cc: Raphael > > > > On Fri, Feb 19, 2021 at 04:04:34PM +0000, Alex Bennée wrote: > > > Hi, > > > > > > I finally got a chance to get down into the guts of vhost-user while > > > attempting to port my original C RPMB daemon to Rust using the > > > vhost-user-backend and related crates. I ended up with this hang during > > > negotiation: > > > > > > startup > > > > > > vhost_user_write req:1 flags:0x1 > > > vhost_user_read_start > > > vhost_user_read req:1 flags:0x5 > > > vhost_user_backend_init: we got 170000000 > > > vhost_user_write req:15 flags:0x1 > > > vhost_user_read_start > > > vhost_user_read req:15 flags:0x5 > > > vhost_user_set_protocol_features: 2008 > > > vhost_user_write req:16 flags:0x1 > > > vhost_user_write req:3 flags:0x1 > > > vhost_user_write req:1 flags:0x1 > > > vhost_user_read_start > > > vhost_user_read req:1 flags:0x5 > > > vhost_user_write req:13 flags:0x1 > > > > > > kernel initialises device > > > > > > virtio_rpmb virtio1: init done! > > > vhost_user_write req:13 flags:0x1 > > > vhost_dev_set_features: 130000000 > > > vhost_user_set_features: 130000000 > > > vhost_user_write req:2 flags:0x1 > > > vhost_user_write req:5 flags:0x9 > > > vhost_user_read_start > > > > > > The proximate cause is the vhost crate handling: > > > > > > MasterReq::SET_MEM_TABLE => { > > > let res = self.set_mem_table(&hdr, size, &buf, rfds); > > > self.send_ack_message(&hdr, res)?; > > > } > > > > > > which gates on the replay_ack_enabled flag: > > > > > > fn send_ack_message( > > > &mut self, > > > req: &VhostUserMsgHeader, > > > res: Result<()>, > > > ) -> Result<()> { > > > if dbg!(self.reply_ack_enabled) { > > > let hdr = self.new_reply_header::(req, 0)?; > > > let val = match res { > > > Ok(_) => 0, > > > Err(_) => 1, > > > }; > > > let msg = VhostUserU64::new(val); > > > self.main_sock.send_message(&hdr, &msg, None)?; > > > } > > > Ok(()) > > > } > > > > > > which is only set when we have all the appropriate acknowledged flags: > > > > > > fn update_reply_ack_flag(&mut self) { > > > let vflag = VhostUserVirtioFeatures::PROTOCOL_FEATURES.bits(); > > > let pflag = VhostUserProtocolFeatures::REPLY_ACK; > > > if (self.virtio_features & vflag) != 0 > > > && (self.acked_virtio_features & vflag) != 0 > > > && self.protocol_features.contains(pflag) > > > && (self.acked_protocol_features & pflag.bits()) != 0 > > > { > > > self.reply_ack_enabled = true; > > > } else { > > > self.reply_ack_enabled = false; > > > } > > > } > > > > > > which from above you can see QEMU helpfully dropped those bits in the > > > reply. It does however work in the C/libvhost version: > > > > > > virtio_rpmb virtio1: init done! > > > vhost_user_write req:13 flags:0x1 > > > vhost_dev_set_features: 130000000 > > > vhost_user_set_features: 130000000 > > > vhost_user_write req:2 flags:0x1 > > > vhost_user_write req:37 flags:0x9 > > > vhost_user_read_start > > > vhost_user_read req:37 flags:0x5 > > > vhost_user_write req:8 flags:0x1 > > > vhost_user_write req:10 flags:0x1 > > > vhost_user_write req:9 flags:0x1 > > > vhost_user_write req:12 flags:0x1 > > > vhost_user_write req:13 flags:0x1 > > > > > > albeit with a slightly different message sequence > > > (VHOST_USER_ADD_MEM_REG instead of VHOST_USER_SET_MEM_TABLE). Reading > > > the C code you can see why: > > > > > > need_reply = vmsg.flags & VHOST_USER_NEED_REPLY_MASK; > > > > > > reply_requested = vu_process_message(dev, &vmsg); > > > if (!reply_requested && need_reply) { > > > vmsg_set_reply_u64(&vmsg, 0); > > > reply_requested = 1; > > > } > > > > > > So regardless of what may have been negotiated it will always reply > with > > > something if the master requested it do so. This points us at the > > > specification which reads: > > > > > > - Bit 3 is the need_reply flag - see :ref:`REPLY_ACK ` for > > > details. > > > > > > which says in VHOST_USER_PROTOCOL_F_REPLY_ACK that this bit should only > > > be honoured when the feature has been negotiated. Which brings us to a > > > series of questions: > > > > > > - Should QEMU have preserved > VhostUserVirtioFeatures::PROTOCOL_FEATURES > > > when doing the eventual VHOST_USER_SET_FEATURES reply? > > > > Hmm looks like a bug indeed ... Anyone wants to look > > into fixing that? Marc-André? > > chirantan and keiichi will be implementing vhost-user-vitio-fs on > Chrome OS, maybe one of you two can take a look? > > Yeah, our team is working on vhost-user virtiofs. I think +Woody Chow will probably be able to look into this issue. > > > > > > > > > > - Is vhost.rs being to strict or libvhost-user too lax in > interpreting > > > the negotiated features before processing the ``need_reply`` [Bit 3] > > > field of the messages? > > > > > > - are VHOST_USER_SET_MEM_TABLE to VHOST_USER_SET_INFLIGHT_FD included > > > in the "list of the ones that do" require replies or do they only > > > reply when REPLY_ACK has been negotiated as the ambiguous > "seealso::" > > > box out seems to imply? > > > > > > Currently I have some hacks in: > > > > > > https://github.com/stsquad/vhost/tree/my-hacks > > > > > > which gets my daemon booting up to the point we actually need to do a > > > transaction. However I won't submit a PR until I've worked out exactly > > > where the problems are. > > > > > > -- > > > Alex Bennée > > > > > > _______________________________________________ > > Rust-vmm mailing list > > Rust-vmm at lists.opendev.org > > http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.bennee at linaro.org Fri Feb 26 11:16:19 2021 From: alex.bennee at linaro.org (=?UTF-8?q?Alex=20Benn=C3=A9e?=) Date: Fri, 26 Feb 2021 11:16:19 +0000 Subject: [Rust-VMM] [VHOST USER SPEC PATCH] vhost-user.rst: add clarifying language about protocol negotiation Message-ID: <20210226111619.21178-1-alex.bennee@linaro.org> In practice the protocol negotiation between vhost master and slave occurs before the final feature negotiation between backend and frontend. This has lead to an inconsistency between the rust-vmm vhost implementation and the libvhost-user library in their approaches to checking if all the requirements for REPLY_ACK processing were met. As this is purely a function of the protocol negotiation and not of interest to the frontend lets make the language clearer about the requirements for a successfully negotiated protocol feature. Signed-off-by: Alex Bennée Cc: Jiang Liu --- docs/interop/vhost-user.rst | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst index d6085f7045..3ac221a8c7 100644 --- a/docs/interop/vhost-user.rst +++ b/docs/interop/vhost-user.rst @@ -301,12 +301,22 @@ If *slave* detects some error such as incompatible features, it may also close the connection. This should only happen in exceptional circumstances. Any protocol extensions are gated by protocol feature bits, which -allows full backwards compatibility on both master and slave. As -older slaves don't support negotiating protocol features, a feature +allows full backwards compatibility on both master and slave. As older +slaves don't support negotiating protocol features, a device feature bit was dedicated for this purpose:: #define VHOST_USER_F_PROTOCOL_FEATURES 30 +However as the protocol negotiation something that only occurs between +parts of the backend implementation it is permissible to for the master +to mask the feature bit from the guest. As noted for the +``VHOST_USER_GET_PROTOCOL_FEATURES`` and +``VHOST_USER_SET_PROTOCOL_FEATURES`` messages this occurs before a +final ``VHOST_USER_SET_FEATURES`` comes from the guest. So the +enabling of protocol features need only require the advertising of the +feature by the slave and the successful get/set protocol features +sequence. + Starting and stopping rings --------------------------- -- 2.20.1 From no-reply at patchew.org Fri Feb 26 11:21:48 2021 From: no-reply at patchew.org (no-reply at patchew.org) Date: Fri, 26 Feb 2021 03:21:48 -0800 (PST) Subject: [Rust-VMM] [VHOST USER SPEC PATCH] vhost-user.rst: add clarifying language about protocol negotiation In-Reply-To: <20210226111619.21178-1-alex.bennee@linaro.org> Message-ID: <161433850648.15109.10656343115567123539@c667a6b167f6> Patchew URL: https://patchew.org/QEMU/20210226111619.21178-1-alex.bennee at linaro.org/ Hi, This series seems to have some coding style problems. See output below for more information: Type: series Message-id: 20210226111619.21178-1-alex.bennee at linaro.org Subject: [VHOST USER SPEC PATCH] vhost-user.rst: add clarifying language about protocol negotiation === TEST SCRIPT BEGIN === #!/bin/bash git rev-parse base > /dev/null || exit 0 git config --local diff.renamelimit 0 git config --local diff.renames True git config --local diff.algorithm histogram ./scripts/checkpatch.pl --mailback base.. === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 From https://github.com/patchew-project/qemu * [new tag] patchew/20210226111619.21178-1-alex.bennee at linaro.org -> patchew/20210226111619.21178-1-alex.bennee at linaro.org Switched to a new branch 'test' 93ba1ce vhost-user.rst: add clarifying language about protocol negotiation === OUTPUT BEGIN === ERROR: trailing whitespace #48: FILE: docs/interop/vhost-user.rst:319: + $ total: 1 errors, 0 warnings, 24 lines checked Commit 93ba1ce0fb2a (vhost-user.rst: add clarifying language about protocol negotiation) has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. === OUTPUT END === Test command exited with code: 1 The full log is available at http://patchew.org/logs/20210226111619.21178-1-alex.bennee at linaro.org/testing.checkpatch/?type=message. --- Email generated automatically by Patchew [https://patchew.org/]. Please send your feedback to patchew-devel at redhat.com From raphael.s.norwitz at gmail.com Fri Feb 26 19:58:48 2021 From: raphael.s.norwitz at gmail.com (Raphael Norwitz) Date: Fri, 26 Feb 2021 09:58:48 -1000 Subject: [Rust-VMM] vhost reply_ack negotiation (a.k.a differences in vhost-user behaviour with libvhost-user and vhost-user-backend.rs) In-Reply-To: <871rd86xrf.fsf@linaro.org> References: <8735xskm7j.fsf@linaro.org> <871rd86xrf.fsf@linaro.org> Message-ID: There are two sets of features being negotiated - virtio and vhost-user. Based on what you've posted here, I suspect the VHOST_USER_F_PROTOCOL_FEATURES virtio feature may not be negotiated by the backend, preventing the vhost-user protocol feature negotiation from happening at all. I'm not 100% sure why this would cause QEMU to assume that REPLY_ACK was negotiated though. some questions: On Mon, Feb 22, 2021 at 3:26 AM Alex Bennée wrote: > > > Dr. David Alan Gilbert writes: > > > * Alex Bennée (alex.bennee at linaro.org) wrote: > >> Hi, > >> > >> I finally got a chance to get down into the guts of vhost-user while > >> attempting to port my original C RPMB daemon to Rust using the > >> vhost-user-backend and related crates. I ended up with this hang during > >> negotiation: > >> > >> startup > >> > >> vhost_user_write req:1 flags:0x1 > >> vhost_user_read_start > >> vhost_user_read req:1 flags:0x5 > >> vhost_user_backend_init: we got 170000000 > > GET_FEATURES Do we also see a GET_PROTOCOL_FEATURES and a SET_PROTOCOL_FEATURES message here? If so can you confirm what flags they contained? vhost-user feature negotiation works as follows (see vhost_user_backend_init()): err = vhost_user_get_features(dev, &features); if (err < 0) { return err; } if (virtio_has_feature(features, VHOST_USER_F_PROTOCOL_FEATURES)) { dev->backend_features |= 1ULL << VHOST_USER_F_PROTOCOL_FEATURES; err = vhost_user_get_u64(dev, VHOST_USER_GET_PROTOCOL_FEATURES, &protocol_features); if (err < 0) { return err; } dev->protocol_features = protocol_features & VHOST_USER_PROTOCOL_FEATURE_MASK; ... err = vhost_user_set_protocol_features(dev, dev->protocol_features); if (err < 0) { return err; } } So we first get the virtio features and check if the backend advertises VHOST_USER_F_PROTOCOL_FEATURES. If it does, we proceed to negotiate vhost-user features, in which case we should see GET_PROTOCOL_FEATURES and a SET_PROTOCOL_FEATURES. Otherwise it looks like the function just returns, and we leave the vhost-user features uninitialized (presumably zeroed out?), and the backend will never even receive a GET/SET_PROTOCOL_FEATURES. dev->protocol_features is not touched anywhere else, and, if VHOST_USER_F_PROTOCOL_FEATURES is negotiated, comes directly to the backend from the protocol_features the backend &ed with VHOST_USER_PROTOCOL_FEATURE_MASK. Therefore if VHOST_USER_F_PROTOCOL_FEATURES is indeed negotiated here I'm not sure what could cause QEMU to think REPLY_ACK was negotiated while the backend does not, spare something obvious like the backend mishandling the GET/SET_PROTOCOL_FEATURES messages. I briefly checked the rustvmm code for that and didn't see anything obvious. mst - are backend devices meant to function if VHOST_USER_F_PROTOCOL_FEATURES is not advertised? Do we know of any functioning backend which does not advertise this virtio feature? If not, maybe we consider failing out here? alex - Are you sure QEMU gets stuck waiting on a reply_ack message, and not somewhere else in the setup path? I trust a SET_MEM_TABLE message was actually received by the backend. Did you confirm that QEMU was indeed stuck waiting for a reply and not somewhere else later on? > > >> vhost_user_write req:15 flags:0x1 > >> vhost_user_read_start > >> vhost_user_read req:15 flags:0x5 > >> vhost_user_set_protocol_features: 2008 > >> vhost_user_write req:16 flags:0x1 > >> vhost_user_write req:3 flags:0x1 > >> vhost_user_write req:1 flags:0x1 > >> vhost_user_read_start > >> vhost_user_read req:1 flags:0x5 > >> vhost_user_write req:13 flags:0x1 > >> > >> kernel initialises device > >> > >> virtio_rpmb virtio1: init done! > >> vhost_user_write req:13 flags:0x1 > >> vhost_dev_set_features: 130000000 > >> vhost_user_set_features: 130000000 > > SET_FEATURES This is setting virtio features - should have nothing to do with REPLY_ACK. > > >> vhost_user_write req:2 flags:0x1 > >> vhost_user_write req:5 flags:0x9 > >> vhost_user_read_start > >> > > >> > >> - Should QEMU have preserved VhostUserVirtioFeatures::PROTOCOL_FEATURES > >> when doing the eventual VHOST_USER_SET_FEATURES reply? > >> > >> - Is vhost.rs being to strict or libvhost-user too lax in interpreting > >> the negotiated features before processing the ``need_reply`` [Bit 3] > >> field of the messages? > > > > I think vhost.rs is being correctly strict - but there would be no harm > > in it flagging that you'd hit an inconsistency if it finds a need_reply > > without the feature. > > But the feature should have been negotiated. So unless the slave can > assume it is enabled because it asked I think QEMU is in the wrong by > not preserving the feature bits in it's SET_FEATURES reply. We just gets > away with it with libvhostuser being willing to reply anyway. > > > > >> - are VHOST_USER_SET_MEM_TABLE to VHOST_USER_SET_INFLIGHT_FD included > >> in the "list of the ones that do" require replies or do they only > >> reply when REPLY_ACK has been negotiated as the ambiguous "seealso::" > >> box out seems to imply? > > > > set_mem_table gives a reply when postcopy is enabled (and then qemu > > replies to the reply!) but otherwise doesn't. > > (Note there's an issue opened for .rs to support ADD_MEM_REGION > > since it's a lot better than SET_MEM_TABLE which has a fixed size table > > that's small). > > Thanks for the heads up. > > > > > Dave > > > >> Currently I have some hacks in: > >> > >> https://github.com/stsquad/vhost/tree/my-hacks > >> > >> which gets my daemon booting up to the point we actually need to do a > >> transaction. However I won't submit a PR until I've worked out exactly > >> where the problems are. > >> > >> -- > >> Alex Bennée > >> > > > -- > Alex Bennée > From raphael.s.norwitz at gmail.com Fri Feb 26 21:25:39 2021 From: raphael.s.norwitz at gmail.com (Raphael Norwitz) Date: Fri, 26 Feb 2021 11:25:39 -1000 Subject: [Rust-VMM] vhost reply_ack negotiation (a.k.a differences in vhost-user behaviour with libvhost-user and vhost-user-backend.rs) In-Reply-To: References: <8735xskm7j.fsf@linaro.org> <871rd86xrf.fsf@linaro.org> Message-ID: As an afterthought - if VHOST_USER_F_PROTOCOL_FEATURES is indeed unset, the issue may well be caused by QEMU reading an uninitialized value for dev->protocol_features. Some device types like cryptodev explicitly zero it out. As I said, it isn't set anywhere else in the source and If dev->protocol_features had REPLY_ACK set when the vhost_dev device is initialized, it would exactly explain the behavior you are seeing. On Fri, Feb 26, 2021 at 9:58 AM Raphael Norwitz wrote: > > There are two sets of features being negotiated - virtio and > vhost-user. Based on what you've posted here, I suspect the > VHOST_USER_F_PROTOCOL_FEATURES virtio feature may not be negotiated by > the backend, preventing the vhost-user protocol feature negotiation > from happening at all. I'm not 100% sure why this would cause QEMU to > assume that REPLY_ACK was negotiated though. > > some questions: > > On Mon, Feb 22, 2021 at 3:26 AM Alex Bennée wrote: > > > > > > Dr. David Alan Gilbert writes: > > > > > * Alex Bennée (alex.bennee at linaro.org) wrote: > > >> Hi, > > >> > > >> I finally got a chance to get down into the guts of vhost-user while > > >> attempting to port my original C RPMB daemon to Rust using the > > >> vhost-user-backend and related crates. I ended up with this hang during > > >> negotiation: > > >> > > >> startup > > >> > > >> vhost_user_write req:1 flags:0x1 > > >> vhost_user_read_start > > >> vhost_user_read req:1 flags:0x5 > > >> vhost_user_backend_init: we got 170000000 > > > > GET_FEATURES > > Do we also see a GET_PROTOCOL_FEATURES and a SET_PROTOCOL_FEATURES > message here? If so can you confirm what flags they contained? > > vhost-user feature negotiation works as follows (see vhost_user_backend_init()): > > err = vhost_user_get_features(dev, &features); > if (err < 0) { > return err; > } > > if (virtio_has_feature(features, VHOST_USER_F_PROTOCOL_FEATURES)) { > dev->backend_features |= 1ULL << VHOST_USER_F_PROTOCOL_FEATURES; > > err = vhost_user_get_u64(dev, VHOST_USER_GET_PROTOCOL_FEATURES, > &protocol_features); > if (err < 0) { > return err; > } > > dev->protocol_features = > protocol_features & VHOST_USER_PROTOCOL_FEATURE_MASK; > ... > > err = vhost_user_set_protocol_features(dev, dev->protocol_features); > if (err < 0) { > return err; > } > } > > So we first get the virtio features and check if the backend > advertises VHOST_USER_F_PROTOCOL_FEATURES. If it does, we proceed to > negotiate vhost-user features, in which case we should see > GET_PROTOCOL_FEATURES and a SET_PROTOCOL_FEATURES. Otherwise it looks > like the function just returns, and we leave the vhost-user features > uninitialized (presumably zeroed out?), and the backend will never > even receive a GET/SET_PROTOCOL_FEATURES. > > dev->protocol_features is not touched anywhere else, and, if > VHOST_USER_F_PROTOCOL_FEATURES is negotiated, comes directly to the > backend from the protocol_features the backend &ed with > VHOST_USER_PROTOCOL_FEATURE_MASK. Therefore if > VHOST_USER_F_PROTOCOL_FEATURES is indeed negotiated here I'm not sure > what could cause QEMU to think REPLY_ACK was negotiated while the > backend does not, spare something obvious like the backend mishandling > the GET/SET_PROTOCOL_FEATURES messages. I briefly checked the rustvmm > code for that and didn't see anything obvious. > > mst - are backend devices meant to function if > VHOST_USER_F_PROTOCOL_FEATURES is not advertised? Do we know of any > functioning backend which does not advertise this virtio feature? If > not, maybe we consider failing out here? > > alex - Are you sure QEMU gets stuck waiting on a reply_ack message, > and not somewhere else in the setup path? I trust a SET_MEM_TABLE > message was actually received by the backend. Did you confirm that > QEMU was indeed stuck waiting for a reply and not somewhere else later > on? > > > > > >> vhost_user_write req:15 flags:0x1 > > >> vhost_user_read_start > > >> vhost_user_read req:15 flags:0x5 > > >> vhost_user_set_protocol_features: 2008 > > >> vhost_user_write req:16 flags:0x1 > > >> vhost_user_write req:3 flags:0x1 > > >> vhost_user_write req:1 flags:0x1 > > >> vhost_user_read_start > > >> vhost_user_read req:1 flags:0x5 > > >> vhost_user_write req:13 flags:0x1 > > >> > > >> kernel initialises device > > >> > > >> virtio_rpmb virtio1: init done! > > >> vhost_user_write req:13 flags:0x1 > > >> vhost_dev_set_features: 130000000 > > >> vhost_user_set_features: 130000000 > > > > SET_FEATURES > > This is setting virtio features - should have nothing to do with REPLY_ACK. > > > > > >> vhost_user_write req:2 flags:0x1 > > >> vhost_user_write req:5 flags:0x9 > > >> vhost_user_read_start > > >> > > > > >> > > >> - Should QEMU have preserved VhostUserVirtioFeatures::PROTOCOL_FEATURES > > >> when doing the eventual VHOST_USER_SET_FEATURES reply? > > >> > > >> - Is vhost.rs being to strict or libvhost-user too lax in interpreting > > >> the negotiated features before processing the ``need_reply`` [Bit 3] > > >> field of the messages? > > > > > > I think vhost.rs is being correctly strict - but there would be no harm > > > in it flagging that you'd hit an inconsistency if it finds a need_reply > > > without the feature. > > > > But the feature should have been negotiated. So unless the slave can > > assume it is enabled because it asked I think QEMU is in the wrong by > > not preserving the feature bits in it's SET_FEATURES reply. We just gets > > away with it with libvhostuser being willing to reply anyway. > > > > > > > >> - are VHOST_USER_SET_MEM_TABLE to VHOST_USER_SET_INFLIGHT_FD included > > >> in the "list of the ones that do" require replies or do they only > > >> reply when REPLY_ACK has been negotiated as the ambiguous "seealso::" > > >> box out seems to imply? > > > > > > set_mem_table gives a reply when postcopy is enabled (and then qemu > > > replies to the reply!) but otherwise doesn't. > > > (Note there's an issue opened for .rs to support ADD_MEM_REGION > > > since it's a lot better than SET_MEM_TABLE which has a fixed size table > > > that's small). > > > > Thanks for the heads up. > > > > > > > > Dave > > > > > >> Currently I have some hacks in: > > >> > > >> https://github.com/stsquad/vhost/tree/my-hacks > > >> > > >> which gets my daemon booting up to the point we actually need to do a > > >> transaction. However I won't submit a PR until I've worked out exactly > > >> where the problems are. > > >> > > >> -- > > >> Alex Bennée > > >> > > > > > > -- > > Alex Bennée > > From alex.bennee at linaro.org Sat Feb 27 12:23:08 2021 From: alex.bennee at linaro.org (Alex =?utf-8?Q?Benn=C3=A9e?=) Date: Sat, 27 Feb 2021 12:23:08 +0000 Subject: [Rust-VMM] vhost reply_ack negotiation (a.k.a differences in vhost-user behaviour with libvhost-user and vhost-user-backend.rs) In-Reply-To: References: <8735xskm7j.fsf@linaro.org> <871rd86xrf.fsf@linaro.org> Message-ID: <871rd1u29v.fsf@linaro.org> Raphael Norwitz writes: > As an afterthought - if VHOST_USER_F_PROTOCOL_FEATURES is indeed > unset, the issue may well be caused by QEMU reading an uninitialized > value for dev->protocol_features. Some device types like cryptodev > explicitly zero it out. As I said, it isn't set anywhere else in the > source and If dev->protocol_features had REPLY_ACK set when the > vhost_dev device is initialized, it would exactly explain the behavior > you are seeing. Can I point you at the proposed clarification of the vhost-user specification: Subject: [VHOST USER SPEC PATCH] vhost-user.rst: add clarifying language about protocol negotiation Date: Fri, 26 Feb 2021 11:16:19 +0000 Message-Id: <20210226111619.21178-1-alex.bennee at linaro.org> > > On Fri, Feb 26, 2021 at 9:58 AM Raphael Norwitz > wrote: >> >> There are two sets of features being negotiated - virtio and >> vhost-user. Based on what you've posted here, I suspect the >> VHOST_USER_F_PROTOCOL_FEATURES virtio feature may not be negotiated by >> the backend, preventing the vhost-user protocol feature negotiation >> from happening at all. I'm not 100% sure why this would cause QEMU to >> assume that REPLY_ACK was negotiated though. >> >> some questions: >> >> On Mon, Feb 22, 2021 at 3:26 AM Alex Bennée wrote: >> > >> > >> > Dr. David Alan Gilbert writes: >> > >> > > * Alex Bennée (alex.bennee at linaro.org) wrote: >> > >> Hi, >> > >> >> > >> I finally got a chance to get down into the guts of vhost-user while >> > >> attempting to port my original C RPMB daemon to Rust using the >> > >> vhost-user-backend and related crates. I ended up with this hang during >> > >> negotiation: >> > >> >> > >> startup >> > >> >> > >> vhost_user_write req:1 flags:0x1 >> > >> vhost_user_read_start >> > >> vhost_user_read req:1 flags:0x5 >> > >> vhost_user_backend_init: we got 170000000 >> > >> > GET_FEATURES >> >> Do we also see a GET_PROTOCOL_FEATURES and a SET_PROTOCOL_FEATURES >> message here? If so can you confirm what flags they contained? >> >> vhost-user feature negotiation works as follows (see vhost_user_backend_init()): >> >> err = vhost_user_get_features(dev, &features); >> if (err < 0) { >> return err; >> } >> >> if (virtio_has_feature(features, VHOST_USER_F_PROTOCOL_FEATURES)) { >> dev->backend_features |= 1ULL << VHOST_USER_F_PROTOCOL_FEATURES; >> >> err = vhost_user_get_u64(dev, VHOST_USER_GET_PROTOCOL_FEATURES, >> &protocol_features); >> if (err < 0) { >> return err; >> } >> >> dev->protocol_features = >> protocol_features & VHOST_USER_PROTOCOL_FEATURE_MASK; >> ... >> >> err = vhost_user_set_protocol_features(dev, dev->protocol_features); >> if (err < 0) { >> return err; >> } >> } >> >> So we first get the virtio features and check if the backend >> advertises VHOST_USER_F_PROTOCOL_FEATURES. If it does, we proceed to >> negotiate vhost-user features, in which case we should see >> GET_PROTOCOL_FEATURES and a SET_PROTOCOL_FEATURES. Otherwise it looks >> like the function just returns, and we leave the vhost-user features >> uninitialized (presumably zeroed out?), and the backend will never >> even receive a GET/SET_PROTOCOL_FEATURES. >> >> dev->protocol_features is not touched anywhere else, and, if >> VHOST_USER_F_PROTOCOL_FEATURES is negotiated, comes directly to the >> backend from the protocol_features the backend &ed with >> VHOST_USER_PROTOCOL_FEATURE_MASK. Therefore if >> VHOST_USER_F_PROTOCOL_FEATURES is indeed negotiated here I'm not sure >> what could cause QEMU to think REPLY_ACK was negotiated while the >> backend does not, spare something obvious like the backend mishandling >> the GET/SET_PROTOCOL_FEATURES messages. I briefly checked the rustvmm >> code for that and didn't see anything obvious. >> >> mst - are backend devices meant to function if >> VHOST_USER_F_PROTOCOL_FEATURES is not advertised? Do we know of any >> functioning backend which does not advertise this virtio feature? If >> not, maybe we consider failing out here? >> >> alex - Are you sure QEMU gets stuck waiting on a reply_ack message, >> and not somewhere else in the setup path? I trust a SET_MEM_TABLE >> message was actually received by the backend. Did you confirm that >> QEMU was indeed stuck waiting for a reply and not somewhere else later >> on? >> >> > >> > >> vhost_user_write req:15 flags:0x1 >> > >> vhost_user_read_start >> > >> vhost_user_read req:15 flags:0x5 >> > >> vhost_user_set_protocol_features: 2008 >> > >> vhost_user_write req:16 flags:0x1 >> > >> vhost_user_write req:3 flags:0x1 >> > >> vhost_user_write req:1 flags:0x1 >> > >> vhost_user_read_start >> > >> vhost_user_read req:1 flags:0x5 >> > >> vhost_user_write req:13 flags:0x1 >> > >> >> > >> kernel initialises device >> > >> >> > >> virtio_rpmb virtio1: init done! >> > >> vhost_user_write req:13 flags:0x1 >> > >> vhost_dev_set_features: 130000000 >> > >> vhost_user_set_features: 130000000 >> > >> > SET_FEATURES >> >> This is setting virtio features - should have nothing to do with REPLY_ACK. >> >> > >> > >> vhost_user_write req:2 flags:0x1 >> > >> vhost_user_write req:5 flags:0x9 >> > >> vhost_user_read_start >> > >> >> > >> > >> >> > >> - Should QEMU have preserved VhostUserVirtioFeatures::PROTOCOL_FEATURES >> > >> when doing the eventual VHOST_USER_SET_FEATURES reply? >> > >> >> > >> - Is vhost.rs being to strict or libvhost-user too lax in interpreting >> > >> the negotiated features before processing the ``need_reply`` [Bit 3] >> > >> field of the messages? >> > > >> > > I think vhost.rs is being correctly strict - but there would be no harm >> > > in it flagging that you'd hit an inconsistency if it finds a need_reply >> > > without the feature. >> > >> > But the feature should have been negotiated. So unless the slave can >> > assume it is enabled because it asked I think QEMU is in the wrong by >> > not preserving the feature bits in it's SET_FEATURES reply. We just gets >> > away with it with libvhostuser being willing to reply anyway. >> > >> > > >> > >> - are VHOST_USER_SET_MEM_TABLE to VHOST_USER_SET_INFLIGHT_FD included >> > >> in the "list of the ones that do" require replies or do they only >> > >> reply when REPLY_ACK has been negotiated as the ambiguous "seealso::" >> > >> box out seems to imply? >> > > >> > > set_mem_table gives a reply when postcopy is enabled (and then qemu >> > > replies to the reply!) but otherwise doesn't. >> > > (Note there's an issue opened for .rs to support ADD_MEM_REGION >> > > since it's a lot better than SET_MEM_TABLE which has a fixed size table >> > > that's small). >> > >> > Thanks for the heads up. >> > >> > > >> > > Dave >> > > >> > >> Currently I have some hacks in: >> > >> >> > >> https://github.com/stsquad/vhost/tree/my-hacks >> > >> >> > >> which gets my daemon booting up to the point we actually need to do a >> > >> transaction. However I won't submit a PR until I've worked out exactly >> > >> where the problems are. >> > >> >> > >> -- >> > >> Alex Bennée >> > >> >> > >> > >> > -- >> > Alex Bennée >> > -- Alex Bennée