From aliguori at amazon.com  Mon Dec 17 23:30:58 2018
From: aliguori at amazon.com (Liguori, Anthony)
Date: Mon, 17 Dec 2018 23:30:58 +0000
Subject: [Rust-vmm] Goals for this list
Message-ID: <1545089457255.82861@amazon.com>

Hi!


Very excited about this list and eager to begin discussions.  I wanted to start with what I see as the low hanging fruit and I what I see as the goal of this effort.


I'd like to build a set of crates, preferrably with a common name, that make it really easy to write VMMs in Rust.  I think Rust is an ideal language for building VMMs and that if the right primitives are there, a lot of interesting VMMs will emerge.


Some ideas I have for what would make good core creates:


1) KVM bindgen bindings.  Right now, there is an issue with bindgen misgenerating code when encountering empty trailing arrays which are used in KVM.  Both crosvm and Firecracker get around this by modifying the headers by hand and then committing the results of bindgen.  Dylan also mentioned that they were unhappy about bindgen performance but this seems like an eventual optimization to me.


As a side note, I think having OS X hypervisor framework bindings and whatever the new Windows thing is would be pretty cool.  Maybe there's room for a wrapper around the three but it's not something that interesting to me in the near term.


2) The crosvm data_model crate.  This one is super critical but easy to misunderstand as it allows for safe access to volatile memory.  Somewhat related is the mmap() bits from sys_util.  Not sure how the crosvm folks feel but I think there is some refactoring here that could be useful to build a memory crate.


3) Some traits for device model implementations.  It's easy to really bike shed here so I reckon it's best to start with a concrete device model like a UART, work through what is required for interfaces, and then iterate from there.


4) Common device models with only a single implementation (i.e. 16650A).  Not sure about virtio, maybe.


Curious what other people think here too.  I've got a couple weeks off coming up so I'd love to have some projects to have on here.


Regards,


Anthony Liguori
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendev.org/pipermail/rust-vmm/attachments/20181217/db916bf6/attachment.html>

From fandree at amazon.com  Tue Dec 18 15:31:56 2018
From: fandree at amazon.com (Florescu, Andreea)
Date: Tue, 18 Dec 2018 15:31:56 +0000
Subject: [Rust-VMM] Goals for this list
In-Reply-To: <1545089457255.82861@amazon.com>
References: <1545089457255.82861@amazon.com>
Message-ID: <1545147115058.67079@amazon.com>

Hi,


I also started a "research" so to say about what crates we can share. I am working on a proposal and should have something ready in a few days.

For the device models, I think a better approach would be to also include virtio devices, but export each of them individually using rust features (like we are currently doing in Firecracker with the vsock device).


One question I have is where would we host the crates? An organization on GitHub with all these crates would be super cool. Something like "rust-vmm" maybe?

Andreea


________________________________
From: Liguori, Anthony <aliguori at amazon.com>
Sent: Tuesday, December 18, 2018 1:30 AM
To: rust-vmm at lists.opendev.org
Subject: [UNVERIFIED SENDER] [Rust-vmm] Goals for this list


Hi!


Very excited about this list and eager to begin discussions.  I wanted to start with what I see as the low hanging fruit and I what I see as the goal of this effort.


I'd like to build a set of crates, preferrably with a common name, that make it really easy to write VMMs in Rust.  I think Rust is an ideal language for building VMMs and that if the right primitives are there, a lot of interesting VMMs will emerge.


Some ideas I have for what would make good core creates:


1) KVM bindgen bindings.  Right now, there is an issue with bindgen misgenerating code when encountering empty trailing arrays which are used in KVM.  Both crosvm and Firecracker get around this by modifying the headers by hand and then committing the results of bindgen.  Dylan also mentioned that they were unhappy about bindgen performance but this seems like an eventual optimization to me.


As a side note, I think having OS X hypervisor framework bindings and whatever the new Windows thing is would be pretty cool.  Maybe there's room for a wrapper around the three but it's not something that interesting to me in the near term.


2) The crosvm data_model crate.  This one is super critical but easy to misunderstand as it allows for safe access to volatile memory.  Somewhat related is the mmap() bits from sys_util.  Not sure how the crosvm folks feel but I think there is some refactoring here that could be useful to build a memory crate.


3) Some traits for device model implementations.  It's easy to really bike shed here so I reckon it's best to start with a concrete device model like a UART, work through what is required for interfaces, and then iterate from there.


4) Common device models with only a single implementation (i.e. 16650A).  Not sure about virtio, maybe.


Curious what other people think here too.  I've got a couple weeks off coming up so I'd love to have some projects to have on here.


Regards,


Anthony Liguori



Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendev.org/pipermail/rust-vmm/attachments/20181218/9a0e8159/attachment.html>

From liuj97 at gmail.com  Tue Dec 18 15:57:19 2018
From: liuj97 at gmail.com (Jiang Liu)
Date: Tue, 18 Dec 2018 23:57:19 +0800
Subject: [Rust-VMM] Goals for this list
In-Reply-To: <1545147115058.67079@amazon.com>
References: <1545089457255.82861@amazon.com> <1545147115058.67079@amazon.com>
Message-ID: <9220C653-7D07-471F-9CD4-81F2A40DD54B@gmail.com>

Hi,
	Glad to meet you all here! I’m learning firecracker design and code these days, it’s really a great work.
 I have some questions about the memory model and virtio subsystems.
	1) Currently the memory_model subsystem is designed for specific scenarios, is it possible to extend the memory_model to support
more scenarios, such as memfd based mapping and memory subregions with different permission properties?
	2) Do you have any plan to support vhost-user VirtIO drivers?
	3) Is it possible to customize KVM kernel module for serverless usage cases?
Thanks,
Gerry

> On Dec 18, 2018, at 11:31 PM, Florescu, Andreea <fandree at amazon.com> wrote:
> 
> Hi,
> 
> I also started a "research" so to say about what crates we can share. I am working on a proposal and should have something ready in a few days.
> For the device models, I think a better approach would be to also include virtio devices, but export each of them individually using rust features (like we are currently doing in Firecracker with the vsock device).
> 
> One question I have is where would we host the crates? An organization on GitHub with all these crates would be super cool. Something like "rust-vmm" maybe?
> 
> Andreea
> 
> From: Liguori, Anthony <aliguori at amazon.com <mailto:aliguori at amazon.com>>
> Sent: Tuesday, December 18, 2018 1:30 AM
> To: rust-vmm at lists.opendev.org <mailto:rust-vmm at lists.opendev.org>
> Subject: [UNVERIFIED SENDER] [Rust-vmm] Goals for this list
>  
> Hi!
> 
> Very excited about this list and eager to begin discussions.  I wanted to start with what I see as the low hanging fruit and I what I see as the goal of this effort.
> 
> I'd like to build a set of crates, preferrably with a common name, that make it really easy to write VMMs in Rust.  I think Rust is an ideal language for building VMMs and that if the right primitives are there, a lot of interesting VMMs will emerge.
> 
> Some ideas I have for what would make good core creates:
> 
> 1) KVM bindgen bindings.  Right now, there is an issue with bindgen misgenerating code when encountering empty trailing arrays which are used in KVM.  Both crosvm and Firecracker get around this by modifying the headers by hand and then committing the results of bindgen.  Dylan also mentioned that they were unhappy about bindgen performance but this seems like an eventual optimization to me.
> 
> As a side note, I think having OS X hypervisor framework bindings and whatever the new Windows thing is would be pretty cool.  Maybe there's room for a wrapper around the three but it's not something that interesting to me in the near term.
> 
> 2) The crosvm data_model crate.  This one is super critical but easy to misunderstand as it allows for safe access to volatile memory.  Somewhat related is the mmap() bits from sys_util.  Not sure how the crosvm folks feel but I think there is some refactoring here that could be useful to build a memory crate.
> 
> 3) Some traits for device model implementations.  It's easy to really bike shed here so I reckon it's best to start with a concrete device model like a UART, work through what is required for interfaces, and then iterate from there.
> 
> 4) Common device models with only a single implementation (i.e. 16650A).  Not sure about virtio, maybe.
> 
> Curious what other people think here too.  I've got a couple weeks off coming up so I'd love to have some projects to have on here.
> 
> Regards,
> 
> Anthony Liguori
> 
> Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
> _______________________________________________
> Rust-vmm mailing list
> Rust-vmm at lists.opendev.org <mailto:Rust-vmm at lists.opendev.org>
> http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm <http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendev.org/pipermail/rust-vmm/attachments/20181218/6565fb0e/attachment-0001.html>

From dgreid at chromium.org  Tue Dec 18 19:02:11 2018
From: dgreid at chromium.org (Dylan Reid)
Date: Tue, 18 Dec 2018 11:02:11 -0800
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <1545089457255.82861@amazon.com>
References: <1545089457255.82861@amazon.com>
Message-ID: <CAEUnVG43MONqvtdL4HDtqoBVCVV7+o0bi=5BfQc26L0gi6HCww@mail.gmail.com>

On Mon, Dec 17, 2018 at 3:31 PM Liguori, Anthony <aliguori at amazon.com> wrote:
>
> Hi!
>
>
> Very excited about this list and eager to begin discussions.  I wanted to start with what I see as the low hanging fruit and I what I see as the goal of this effort.
>
>
> I'd like to build a set of crates, preferrably with a common name, that make it really easy to write VMMs in Rust.  I think Rust is an ideal language for building VMMs and that if the right primitives are there, a lot of interesting VMMs will emerge.
>
>
> Some ideas I have for what would make good core creates:
>
>
> 1) KVM bindgen bindings.  Right now, there is an issue with bindgen misgenerating code when encountering empty trailing arrays which are used in KVM.  Both crosvm and Firecracker get around this by modifying the headers by hand and then committing the results of bindgen.  Dylan also mentioned that they were unhappy about bindgen performance but this seems like an eventual optimization to me.

My concerns are only around build time. That's certainly something
that can be optimized later and isn't specific to this use of bindgen.

It would be great to improve the current, manual bindgen process. It
will make using new KVM features easier.

>
>
> As a side note, I think having OS X hypervisor framework bindings and whatever the new Windows thing is would be pretty cool.  Maybe there's room for a wrapper around the three but it's not something that interesting to me in the near term.
>
>
> 2) The crosvm data_model crate.  This one is super critical but easy to misunderstand as it allows for safe access to volatile memory.  Somewhat related is the mmap() bits from sys_util.  Not sure how the crosvm folks feel but I think there is some refactoring here that could be useful to build a memory crate.

sys_util can be broken in to smaller, easier to consume parts.

The data_model crate has proven very useful, there is also a similar
craten in the Fuchsia tree:
https://fuchsia.googlesource.com/garnet/+/master/public/rust/zerocopy/src/lib.rs
It might be worth considering too.

>
>
> 3) Some traits for device model implementations.  It's easy to really bike shed here so I reckon it's best to start with a concrete device model like a UART, work through what is required for interfaces, and then iterate from there.
>
>
> 4) Common device models with only a single implementation (i.e. 16650A).  Not sure about virtio, maybe.
>
>
> Curious what other people think here too.  I've got a couple weeks off coming up so I'd love to have some projects to have on here.
>
>
> Regards,
>
>
> Anthony Liguori
>
> _______________________________________________
> Rust-vmm mailing list
> Rust-vmm at lists.opendev.org
> http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm

On Mon, Dec 17, 2018 at 3:31 PM Liguori, Anthony <aliguori at amazon.com> wrote:
>
> Hi!
>
>
> Very excited about this list and eager to begin discussions.  I wanted to start with what I see as the low hanging fruit and I what I see as the goal of this effort.
>
>
> I'd like to build a set of crates, preferrably with a common name, that make it really easy to write VMMs in Rust.  I think Rust is an ideal language for building VMMs and that if the right primitives are there, a lot of interesting VMMs will emerge.
>
>
> Some ideas I have for what would make good core creates:
>
>
> 1) KVM bindgen bindings.  Right now, there is an issue with bindgen misgenerating code when encountering empty trailing arrays which are used in KVM.  Both crosvm and Firecracker get around this by modifying the headers by hand and then committing the results of bindgen.  Dylan also mentioned that they were unhappy about bindgen performance but this seems like an eventual optimization to me.
>
>
> As a side note, I think having OS X hypervisor framework bindings and whatever the new Windows thing is would be pretty cool.  Maybe there's room for a wrapper around the three but it's not something that interesting to me in the near term.
>
>
> 2) The crosvm data_model crate.  This one is super critical but easy to misunderstand as it allows for safe access to volatile memory.  Somewhat related is the mmap() bits from sys_util.  Not sure how the crosvm folks feel but I think there is some refactoring here that could be useful to build a memory crate.
>
>
> 3) Some traits for device model implementations.  It's easy to really bike shed here so I reckon it's best to start with a concrete device model like a UART, work through what is required for interfaces, and then iterate from there.
>
>
> 4) Common device models with only a single implementation (i.e. 16650A).  Not sure about virtio, maybe.
>
>
> Curious what other people think here too.  I've got a couple weeks off coming up so I'd love to have some projects to have on here.
>
>
> Regards,
>
>
> Anthony Liguori
>
> _______________________________________________
> Rust-vmm mailing list
> Rust-vmm at lists.opendev.org
> http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm


From allison at lohutok.net  Tue Dec 18 20:10:35 2018
From: allison at lohutok.net (Allison Randal)
Date: Tue, 18 Dec 2018 15:10:35 -0500
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <CAEUnVG43MONqvtdL4HDtqoBVCVV7+o0bi=5BfQc26L0gi6HCww@mail.gmail.com>
References: <1545089457255.82861@amazon.com>
 <CAEUnVG43MONqvtdL4HDtqoBVCVV7+o0bi=5BfQc26L0gi6HCww@mail.gmail.com>
Message-ID: <aaf1c200-7585-c35e-9c4a-85b5b896f0bd@lohutok.net>

On 12/18/18 2:02 PM, Dylan Reid wrote:
> On Mon, Dec 17, 2018 at 3:31 PM Liguori, Anthony <aliguori at amazon.com> wrote:
>>
>> 1) KVM bindgen bindings.  Right now, there is an issue with bindgen misgenerating code when encountering empty trailing arrays which are used in KVM.  Both crosvm and Firecracker get around this by modifying the headers by hand and then committing the results of bindgen.  Dylan also mentioned that they were unhappy about bindgen performance but this seems like an eventual optimization to me.
> 
> My concerns are only around build time. That's certainly something
> that can be optimized later and isn't specific to this use of bindgen.
> 
> It would be great to improve the current, manual bindgen process. It
> will make using new KVM features easier.

My experience with bindgen specifically for KVM, was that the result was
a horrible mess, because linux/kvm.h and ioctl.h made such extensive use
of #defines embedded within #defines, which bindgen translates into very
poor Rust macros. (Rust macros are great, far more powerful and
expressive than C # defines, but the Rust macros that bindgen generates
automatically from C #defines are poor.)

I experimented with simply defining the consts[1] and structs[2] in
Rust, instead of using bindgen for translation, and found that a
reasonable, modern interface to KVM actually only needs a small subset
of ioctl.h and linux/kvm.h, and that Rust definitions of all the consts
and structs are cleaner, more maintainable, and easier to review for
security than the pile of garbage generated by bindgen. Also, the KVM
ioctl interface is in more of a stable maintenance phase of development,
rather than rapid evolution, so it really isn't much effort to maintain
the Rust consts and structs in parallel to the C header files.

In general, I think bindgen is great, but it's not a good fit for KVM
specifically. There is a chance that bindgen could be modified to
produce more sane Rust consts from C #defines, but at the moment, a
human translator can do far better work than bindgen, and it's faster
and more reliable to just do the hand-translation than it is to attempt
to modify bindgen to do extensive compile-time optimizations on macros.

I thought about finishing up the Rust consts and structs, and offering
to contribute them to the KVM project. I could still do that if it would
be useful to anyone. My own work has moved beyond a focus on Rust
hypervisors, but I'm actively interested in seeing them move forward, so
I'm happy to help where I can.

Allison

[1] https://gitlab.com/preon/libkvm/blob/master/src/linux/kvm_consts.rs
(not complete, just a proof-of-concept)

[2] https://gitlab.com/preon/libkvm/blob/master/src/linux/kvm_structs.rs
(not complete, just a proof-of-concept)


From pbonzini at redhat.com  Wed Dec 19 21:13:23 2018
From: pbonzini at redhat.com (Paolo Bonzini)
Date: Wed, 19 Dec 2018 22:13:23 +0100
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <1545089457255.82861@amazon.com>
References: <1545089457255.82861@amazon.com>
Message-ID: <c8a972ec-cf55-6f2f-34fa-d6032a86320c@redhat.com>

On 18/12/18 00:30, Liguori, Anthony wrote:
> As a side note, I think having OS X hypervisor framework bindings
> and whatever the new Windows thing is would be pretty cool.

Yes, indeed.  Hypervisor.framework however is much more complex than KVM
or WHP because you deal manually with VMCSes and have to do instruction
emulation in userspace.  QEMU takes a stab at it, but it's not as stable
as KVM.

*However* Google does have patches for KVM to do instruction emulation
in userspace, and I'd like to apply them upstream too now that KVM has
an API test framework (and thus we can know they won't bitrot).
(Steve, you are in Cc because hint, hint :)).  Once that is in place, I
guess a minimal x86 emulator written in Rust, porting the emulator code
that QEMU has for Hypervisor.framework, would be a fun GSoC project for
a very good student.

> 2) The crosvm data_model crate.  This one is super critical but easy to
> misunderstand as it allows for safe access to volatile memory.  Somewhat
> related is the mmap() bits from sys_util.  Not sure how the crosvm folks
> feel but I think there is some refactoring here that could be useful to
> build a memory crate.
> 
> 3) Some traits for device model implementations.  It's easy to really
> bike shed here so I reckon it's best to start with a concrete device
> model like a UART, work through what is required for interfaces, and
> then iterate from there.
> 
> 4) Common device models with only a single implementation (i.e.
> 16650A).  Not sure about virtio, maybe.

virtio would be interesting.  One initial target could be a demo
vhost-user client, it has to set up a memory map, parse vrings, handle
endianness, etc.  It would be an interesting benchmark for a DMA API.

The control plane (your item 3) by comparison is a bit less interesting.

Paolo


From aliguori at amazon.com  Wed Dec 19 22:15:30 2018
From: aliguori at amazon.com (Anthony Liguori)
Date: Wed, 19 Dec 2018 14:15:30 -0800
Subject: [Rust-VMM] Goals for this list
In-Reply-To: <1545147115058.67079@amazon.com>
References: <1545089457255.82861@amazon.com> <1545147115058.67079@amazon.com>
Message-ID: <cig332k1k5qi2l.fsf@u54e1add816995a33037d.ant.amazon.com>

"Florescu, Andreea" <fandree at amazon.com> writes:

> Hi,
>
> I also started a "research" so to say about what crates we can share.
> I am working on a proposal and should have something ready in a few
> days.
>
> For the device models, I think a better approach would be to also
> include virtio devices, but export each of them individually using
> rust features (like we are currently doing in Firecracker with the
> vsock device).
>
> One question I have is where would we host the crates? An organization
> on GitHub with all these crates would be super cool. Something like
> "rust-vmm" maybe?

I think we should get some code for the easy stuff (like KVM crate),
demonstrate it working with Firecracker and Crosvm, and then we can
figure this out.

Probably a new GitHub org though.

Regards,

Anthony Liguori

>
> Andreea
>
> ----------------------------------------------------------------------
> From: Liguori, Anthony <aliguori at amazon.com>
> Sent: Tuesday, December 18, 2018 1:30 AM
> To: rust-vmm at lists.opendev.org
> Subject: [UNVERIFIED SENDER] [Rust-vmm] Goals for this list 
>
> Hi!
>
> Very excited about this list and eager to begin discussions. I wanted
> to start with what I see as the low hanging fruit and I what I see as
> the goal of this effort.
>
> I'd like to build a set of crates, preferrably with a common name,
> that make it really easy to write VMMs in Rust. I think Rust is an
> ideal language for building VMMs and that if the right primitives are
> there, a lot of interesting VMMs will emerge.
>
> Some ideas I have for what would make good core creates:
>
> 1) KVM bindgen bindings. Right now, there is an issue with bindgen
> misgenerating code when encountering empty trailing arrays which are
> used in KVM. Both crosvm and Firecracker get around this by modifying
> the headers by hand and then committing the results of bindgen. Dylan
> also mentioned that they were unhappy about bindgen performance but
> this seems like an eventual optimization to me.
>
> As a side note, I think having OS X hypervisor framework bindings and
> whatever the new Windows thing is would be pretty cool. Maybe there's
> room for a wrapper around the three but it's not something that
> interesting to me in the near term.
>
> 2) The crosvm data_model crate. This one is super critical but easy to
> misunderstand as it allows for safe access to volatile memory.
> Somewhat related is the mmap() bits from sys_util. Not sure how the
> crosvm folks feel but I think there is some refactoring here that
> could be useful to build a memory crate.
>
> 3) Some traits for device model implementations. It's easy to really
> bike shed here so I reckon it's best to start with a concrete device
> model like a UART, work through what is required for interfaces, and
> then iterate from there.
>
> 4) Common device models with only a single implementation (i.e.
> 16650A). Not sure about virtio, maybe.
>
> Curious what other people think here too. I've got a couple weeks off
> coming up so I'd love to have some projects to have on here.
>
> Regards,
>
> Anthony Liguori


From aliguori at amazon.com  Wed Dec 19 22:16:24 2018
From: aliguori at amazon.com (Anthony Liguori)
Date: Wed, 19 Dec 2018 14:16:24 -0800
Subject: [Rust-VMM] Goals for this list
In-Reply-To: <9220C653-7D07-471F-9CD4-81F2A40DD54B@gmail.com>
References: <1545089457255.82861@amazon.com> <1545147115058.67079@amazon.com>
 <9220C653-7D07-471F-9CD4-81F2A40DD54B@gmail.com>
Message-ID: <cig332h8f9qi13.fsf@u54e1add816995a33037d.ant.amazon.com>

Jiang Liu <liuj97 at gmail.com> writes:

> Hi,
> 	Glad to meet you all here! I’m learning firecracker design and code these days, it’s really a great work.
>  I have some questions about the memory model and virtio subsystems.
> 	1) Currently the memory_model subsystem is designed for specific scenarios, is it possible to extend the memory_model to support
> more scenarios, such as memfd based mapping and memory subregions with different permission properties?
> 	2) Do you have any plan to support vhost-user VirtIO drivers?
> 	3) Is it possible to customize KVM kernel module for serverless usage cases?
> Thanks,
> Gerry

You should ask this at:

https://lists.infradead.org/mailman/listinfo/firecracker-devel

Regards,

Anthony Liguori

>
>> On Dec 18, 2018, at 11:31 PM, Florescu, Andreea <fandree at amazon.com> wrote:
>> 
>> Hi,
>> 
>> I also started a "research" so to say about what crates we can share. I am working on a proposal and should have something ready in a few days.
>> For the device models, I think a better approach would be to also include virtio devices, but export each of them individually using rust features (like we are currently doing in Firecracker with the vsock device).
>> 
>> One question I have is where would we host the crates? An organization on GitHub with all these crates would be super cool. Something like "rust-vmm" maybe?
>> 
>> Andreea
>> 
>> From: Liguori, Anthony <aliguori at amazon.com <mailto:aliguori at amazon.com>>
>> Sent: Tuesday, December 18, 2018 1:30 AM
>> To: rust-vmm at lists.opendev.org <mailto:rust-vmm at lists.opendev.org>
>> Subject: [UNVERIFIED SENDER] [Rust-vmm] Goals for this list
>>  
>> Hi!
>> 
>> Very excited about this list and eager to begin discussions.  I wanted to start with what I see as the low hanging fruit and I what I see as the goal of this effort.
>> 
>> I'd like to build a set of crates, preferrably with a common name, that make it really easy to write VMMs in Rust.  I think Rust is an ideal language for building VMMs and that if the right primitives are there, a lot of interesting VMMs will emerge.
>> 
>> Some ideas I have for what would make good core creates:
>> 
>> 1) KVM bindgen bindings.  Right now, there is an issue with bindgen misgenerating code when encountering empty trailing arrays which are used in KVM.  Both crosvm and Firecracker get around this by modifying the headers by hand and then committing the results of bindgen.  Dylan also mentioned that they were unhappy about bindgen performance but this seems like an eventual optimization to me.
>> 
>> As a side note, I think having OS X hypervisor framework bindings and whatever the new Windows thing is would be pretty cool.  Maybe there's room for a wrapper around the three but it's not something that interesting to me in the near term.
>> 
>> 2) The crosvm data_model crate.  This one is super critical but easy to misunderstand as it allows for safe access to volatile memory.  Somewhat related is the mmap() bits from sys_util.  Not sure how the crosvm folks feel but I think there is some refactoring here that could be useful to build a memory crate.
>> 
>> 3) Some traits for device model implementations.  It's easy to really bike shed here so I reckon it's best to start with a concrete device model like a UART, work through what is required for interfaces, and then iterate from there.
>> 
>> 4) Common device models with only a single implementation (i.e. 16650A).  Not sure about virtio, maybe.
>> 
>> Curious what other people think here too.  I've got a couple weeks off coming up so I'd love to have some projects to have on here.
>> 
>> Regards,
>> 
>> Anthony Liguori
>> 
>> Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
>> _______________________________________________
>> Rust-vmm mailing list
>> Rust-vmm at lists.opendev.org <mailto:Rust-vmm at lists.opendev.org>
>> http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm <http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm>


From aliguori at amazon.com  Wed Dec 19 22:18:15 2018
From: aliguori at amazon.com (Anthony Liguori)
Date: Wed, 19 Dec 2018 14:18:15 -0800
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <CAEUnVG43MONqvtdL4HDtqoBVCVV7+o0bi=5BfQc26L0gi6HCww@mail.gmail.com>
References: <1545089457255.82861@amazon.com>
 <CAEUnVG43MONqvtdL4HDtqoBVCVV7+o0bi=5BfQc26L0gi6HCww@mail.gmail.com>
Message-ID: <cig332efadqhy0.fsf@u54e1add816995a33037d.ant.amazon.com>

Dylan Reid <dgreid at chromium.org> writes:

> On Mon, Dec 17, 2018 at 3:31 PM Liguori, Anthony <aliguori at amazon.com> wrote:
>>
>> Hi!
>>
>>
>> Very excited about this list and eager to begin discussions.  I
>> wanted to start with what I see as the low hanging fruit and I what I
>> see as the goal of this effort.
>>
>>
>> I'd like to build a set of crates, preferrably with a common name,
>> that make it really easy to write VMMs in Rust.  I think Rust is an
>> ideal language for building VMMs and that if the right primitives are
>> there, a lot of interesting VMMs will emerge.
>>
>>
>> Some ideas I have for what would make good core creates:
>>
>>
>> 1) KVM bindgen bindings.  Right now, there is an issue with bindgen
>> misgenerating code when encountering empty trailing arrays which are
>> used in KVM.  Both crosvm and Firecracker get around this by
>> modifying the headers by hand and then committing the results of
>> bindgen.  Dylan also mentioned that they were unhappy about bindgen
>> performance but this seems like an eventual optimization to me.

>
> My concerns are only around build time. That's certainly something
> that can be optimized later and isn't specific to this use of bindgen.
>
> It would be great to improve the current, manual bindgen process. It
> will make using new KVM features easier.

Now that I've thought a bit about it, I think I prefer to commit bindgen
results because then the version of the crate becomes meaningful.  New
KVM features are tied to specific crate versions and you can set
dependencies accordingly.

The build.rs can definitely be improved though to make it easy to update
the committed results.

>> As a side note, I think having OS X hypervisor framework bindings and
>> whatever the new Windows thing is would be pretty cool.  Maybe
>> there's room for a wrapper around the three but it's not something
>> that interesting to me in the near term.
>>
>>
>> 2) The crosvm data_model crate.  This one is super critical but easy
>> to misunderstand as it allows for safe access to volatile memory.
>> Somewhat related is the mmap() bits from sys_util.  Not sure how the
>> crosvm folks feel but I think there is some refactoring here that
>> could be useful to build a memory crate.
>
> sys_util can be broken in to smaller, easier to consume parts.
>
> The data_model crate has proven very useful, there is also a similar
> craten in the Fuchsia tree:
> https://fuchsia.googlesource.com/garnet/+/master/public/rust/zerocopy/src/lib.rs
> It might be worth considering too.

Oh, neat.  I will take a look.

Regards,

Anthony Liguori


From aliguori at amazon.com  Wed Dec 19 22:19:26 2018
From: aliguori at amazon.com (Anthony Liguori)
Date: Wed, 19 Dec 2018 14:19:26 -0800
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <aaf1c200-7585-c35e-9c4a-85b5b896f0bd@lohutok.net>
References: <1545089457255.82861@amazon.com>
 <CAEUnVG43MONqvtdL4HDtqoBVCVV7+o0bi=5BfQc26L0gi6HCww@mail.gmail.com>
 <aaf1c200-7585-c35e-9c4a-85b5b896f0bd@lohutok.net>
Message-ID: <cig332bm5hqhw1.fsf@u54e1add816995a33037d.ant.amazon.com>

Allison Randal <allison at lohutok.net> writes:

> On 12/18/18 2:02 PM, Dylan Reid wrote:
>> On Mon, Dec 17, 2018 at 3:31 PM Liguori, Anthony <aliguori at amazon.com> wrote:
>>>
>>> 1) KVM bindgen bindings.  Right now, there is an issue with bindgen misgenerating code when encountering empty trailing arrays which are used in KVM.  Both crosvm and Firecracker get around this by modifying the headers by hand and then committing the results of bindgen.  Dylan also mentioned that they were unhappy about bindgen performance but this seems like an eventual optimization to me.
>> 
>> My concerns are only around build time. That's certainly something
>> that can be optimized later and isn't specific to this use of bindgen.
>> 
>> It would be great to improve the current, manual bindgen process. It
>> will make using new KVM features easier.
>
> My experience with bindgen specifically for KVM, was that the result was
> a horrible mess, because linux/kvm.h and ioctl.h made such extensive use
> of #defines embedded within #defines, which bindgen translates into very
> poor Rust macros. (Rust macros are great, far more powerful and
> expressive than C # defines, but the Rust macros that bindgen generates
> automatically from C #defines are poor.)
>
> I experimented with simply defining the consts[1] and structs[2] in
> Rust, instead of using bindgen for translation, and found that a
> reasonable, modern interface to KVM actually only needs a small subset
> of ioctl.h and linux/kvm.h, and that Rust definitions of all the consts
> and structs are cleaner, more maintainable, and easier to review for
> security than the pile of garbage generated by bindgen. Also, the KVM
> ioctl interface is in more of a stable maintenance phase of development,
> rather than rapid evolution, so it really isn't much effort to maintain
> the Rust consts and structs in parallel to the C header files.
>
> In general, I think bindgen is great, but it's not a good fit for KVM
> specifically. There is a chance that bindgen could be modified to
> produce more sane Rust consts from C #defines, but at the moment, a
> human translator can do far better work than bindgen, and it's faster
> and more reliable to just do the hand-translation than it is to attempt
> to modify bindgen to do extensive compile-time optimizations on macros.
>
> I thought about finishing up the Rust consts and structs, and offering
> to contribute them to the KVM project. I could still do that if it would
> be useful to anyone. My own work has moved beyond a focus on Rust
> hypervisors, but I'm actively interested in seeing them move forward, so
> I'm happy to help where I can.

I did look at he work you've done and it's great!  I do think it's
important to cover everything though and especially to get architectures
like aarch64.

I think while the bindgen results are not beautiful, having a bindgen
layer gives us something we can build nicer interfaces on top of.

Regards,

Anthony Liguori

>
> Allison
>
> [1] https://gitlab.com/preon/libkvm/blob/master/src/linux/kvm_consts.rs
> (not complete, just a proof-of-concept)
>
> [2] https://gitlab.com/preon/libkvm/blob/master/src/linux/kvm_structs.rs
> (not complete, just a proof-of-concept)


From aliguori at amazon.com  Wed Dec 19 22:23:30 2018
From: aliguori at amazon.com (Anthony Liguori)
Date: Wed, 19 Dec 2018 14:23:30 -0800
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <c8a972ec-cf55-6f2f-34fa-d6032a86320c@redhat.com>
References: <1545089457255.82861@amazon.com>
 <c8a972ec-cf55-6f2f-34fa-d6032a86320c@redhat.com>
Message-ID: <cig3328t0lqhp9.fsf@u54e1add816995a33037d.ant.amazon.com>

Paolo Bonzini <pbonzini at redhat.com> writes:

> On 18/12/18 00:30, Liguori, Anthony wrote:
>> As a side note, I think having OS X hypervisor framework bindings
>> and whatever the new Windows thing is would be pretty cool.
>
> Yes, indeed.  Hypervisor.framework however is much more complex than KVM
> or WHP because you deal manually with VMCSes and have to do instruction
> emulation in userspace.  QEMU takes a stab at it, but it's not as stable
> as KVM.
>
> *However* Google does have patches for KVM to do instruction emulation
> in userspace, and I'd like to apply them upstream too now that KVM has
> an API test framework (and thus we can know they won't bitrot).
> (Steve, you are in Cc because hint, hint :)).  Once that is in place, I
> guess a minimal x86 emulator written in Rust, porting the emulator code
> that QEMU has for Hypervisor.framework, would be a fun GSoC project for
> a very good student.

That does sound like an awesome project.

>> 2) The crosvm data_model crate.  This one is super critical but easy to
>> misunderstand as it allows for safe access to volatile memory.  Somewhat
>> related is the mmap() bits from sys_util.  Not sure how the crosvm folks
>> feel but I think there is some refactoring here that could be useful to
>> build a memory crate.
>> 
>> 3) Some traits for device model implementations.  It's easy to really
>> bike shed here so I reckon it's best to start with a concrete device
>> model like a UART, work through what is required for interfaces, and
>> then iterate from there.
>> 
>> 4) Common device models with only a single implementation (i.e.
>> 16650A).  Not sure about virtio, maybe.
>
> virtio would be interesting.  One initial target could be a demo
> vhost-user client, it has to set up a memory map, parse vrings, handle
> endianness, etc.  It would be an interesting benchmark for a DMA API.

Maybe.  I'll poke around.

One challenge with DMA is that you have to decide whether you are only
supporting synchronous DMA or you are also trying to support
asynchronous DMA.

My general preference is to avoid complex infrastructure and keep to
simple traits.  A DMA trait can be pretty darn simple with all of the
gooey logic being up to the implementor.  Not sure you would get
meaningful benchmarks there.

Regards,

Anthony Liguori

> The control plane (your item 3) by comparison is a bit less interesting.
>
> Paolo


From pbonzini at redhat.com  Wed Dec 19 22:42:53 2018
From: pbonzini at redhat.com (Paolo Bonzini)
Date: Wed, 19 Dec 2018 23:42:53 +0100
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <cig3328t0lqhp9.fsf@u54e1add816995a33037d.ant.amazon.com>
References: <1545089457255.82861@amazon.com>
 <c8a972ec-cf55-6f2f-34fa-d6032a86320c@redhat.com>
 <cig3328t0lqhp9.fsf@u54e1add816995a33037d.ant.amazon.com>
Message-ID: <bcc3ea49-742e-2910-6320-40068f8f39ec@redhat.com>

On 19/12/18 23:23, Anthony Liguori wrote:
>> virtio would be interesting.  One initial target could be a demo
>> vhost-user client, it has to set up a memory map, parse vrings, handle
>> endianness, etc.  It would be an interesting benchmark for a DMA API.
> Maybe.  I'll poke around.
> 
> One challenge with DMA is that you have to decide whether you are only
> supporting synchronous DMA or you are also trying to support
> asynchronous DMA.
> 
> My general preference is to avoid complex infrastructure and keep to
> simple traits.  A DMA trait can be pretty darn simple with all of the
> gooey logic being up to the implementor.  Not sure you would get
> meaningful benchmarks there.

Sorry, I meant benchmark in the (non-computer-science) dictionary
meaning, i.e. a basis to evaluate the API in a simplified but still
realistic scenario.  It would be more or less what you wrote in your
original message: a few full-fledged basic crates (mostly data_model, or
alternatively the Fuchsia one), a toyish implementation of standardized
memory map and DMA traits, and a sample application (vhost-user-blk).

Paolo


From srutherford at google.com  Wed Dec 19 21:21:32 2018
From: srutherford at google.com (Steve Rutherford)
Date: Wed, 19 Dec 2018 13:21:32 -0800
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <c8a972ec-cf55-6f2f-34fa-d6032a86320c@redhat.com>
References: <1545089457255.82861@amazon.com>
 <c8a972ec-cf55-6f2f-34fa-d6032a86320c@redhat.com>
Message-ID: <CABayD+cjmy1UpR22=53cKqcLEyeVQg3eQ_gukv92N9r0MSDT+g@mail.gmail.com>

Hint received :)
Getting userspace instruction emulation to work with nested has been a
bit of a fight, and we've been waiting to push stuff upstream until we
have it everywhere.

On Wed, Dec 19, 2018 at 1:13 PM Paolo Bonzini <pbonzini at redhat.com> wrote:
>
> On 18/12/18 00:30, Liguori, Anthony wrote:
> > As a side note, I think having OS X hypervisor framework bindings
> > and whatever the new Windows thing is would be pretty cool.
>
> Yes, indeed.  Hypervisor.framework however is much more complex than KVM
> or WHP because you deal manually with VMCSes and have to do instruction
> emulation in userspace.  QEMU takes a stab at it, but it's not as stable
> as KVM.
>
> *However* Google does have patches for KVM to do instruction emulation
> in userspace, and I'd like to apply them upstream too now that KVM has
> an API test framework (and thus we can know they won't bitrot).
> (Steve, you are in Cc because hint, hint :)).  Once that is in place, I
> guess a minimal x86 emulator written in Rust, porting the emulator code
> that QEMU has for Hypervisor.framework, would be a fun GSoC project for
> a very good student.
>
> > 2) The crosvm data_model crate.  This one is super critical but easy to
> > misunderstand as it allows for safe access to volatile memory.  Somewhat
> > related is the mmap() bits from sys_util.  Not sure how the crosvm folks
> > feel but I think there is some refactoring here that could be useful to
> > build a memory crate.
> >
> > 3) Some traits for device model implementations.  It's easy to really
> > bike shed here so I reckon it's best to start with a concrete device
> > model like a UART, work through what is required for interfaces, and
> > then iterate from there.
> >
> > 4) Common device models with only a single implementation (i.e.
> > 16650A).  Not sure about virtio, maybe.
>
> virtio would be interesting.  One initial target could be a demo
> vhost-user client, it has to set up a memory map, parse vrings, handle
> endianness, etc.  It would be an interesting benchmark for a DMA API.
>
> The control plane (your item 3) by comparison is a bit less interesting.
>
> Paolo


From dgreid at chromium.org  Wed Dec 19 23:02:15 2018
From: dgreid at chromium.org (Dylan Reid)
Date: Wed, 19 Dec 2018 15:02:15 -0800
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <CABayD+cjmy1UpR22=53cKqcLEyeVQg3eQ_gukv92N9r0MSDT+g@mail.gmail.com>
References: <1545089457255.82861@amazon.com>
 <c8a972ec-cf55-6f2f-34fa-d6032a86320c@redhat.com>
 <CABayD+cjmy1UpR22=53cKqcLEyeVQg3eQ_gukv92N9r0MSDT+g@mail.gmail.com>
Message-ID: <CAEUnVG7WfPj0XsTesQ=1E3p6pN=Ykzf+WrLAedTwP2GGxxdXEw@mail.gmail.com>

On Wed, Dec 19, 2018 at 2:59 PM Steve Rutherford <srutherford at google.com> wrote:
>
> Hint received :)
> Getting userspace instruction emulation to work with nested has been a
> bit of a fight, and we've been waiting to push stuff upstream until we
> have it everywhere.

The crosvm team is currently investigating moving the PIT and APIC
emulation to user space. If that works, instruction emulation will be
next on the list, including helping to get the kernel side landed
upstream.

>
> On Wed, Dec 19, 2018 at 1:13 PM Paolo Bonzini <pbonzini at redhat.com> wrote:
> >
> > On 18/12/18 00:30, Liguori, Anthony wrote:
> > > As a side note, I think having OS X hypervisor framework bindings
> > > and whatever the new Windows thing is would be pretty cool.
> >
> > Yes, indeed.  Hypervisor.framework however is much more complex than KVM
> > or WHP because you deal manually with VMCSes and have to do instruction
> > emulation in userspace.  QEMU takes a stab at it, but it's not as stable
> > as KVM.
> >
> > *However* Google does have patches for KVM to do instruction emulation
> > in userspace, and I'd like to apply them upstream too now that KVM has
> > an API test framework (and thus we can know they won't bitrot).
> > (Steve, you are in Cc because hint, hint :)).  Once that is in place, I
> > guess a minimal x86 emulator written in Rust, porting the emulator code
> > that QEMU has for Hypervisor.framework, would be a fun GSoC project for
> > a very good student.
> >
> > > 2) The crosvm data_model crate.  This one is super critical but easy to
> > > misunderstand as it allows for safe access to volatile memory.  Somewhat
> > > related is the mmap() bits from sys_util.  Not sure how the crosvm folks
> > > feel but I think there is some refactoring here that could be useful to
> > > build a memory crate.
> > >
> > > 3) Some traits for device model implementations.  It's easy to really
> > > bike shed here so I reckon it's best to start with a concrete device
> > > model like a UART, work through what is required for interfaces, and
> > > then iterate from there.
> > >
> > > 4) Common device models with only a single implementation (i.e.
> > > 16650A).  Not sure about virtio, maybe.
> >
> > virtio would be interesting.  One initial target could be a demo
> > vhost-user client, it has to set up a memory map, parse vrings, handle
> > endianness, etc.  It would be an interesting benchmark for a DMA API.
> >
> > The control plane (your item 3) by comparison is a bit less interesting.
> >
> > Paolo
>
> _______________________________________________
> Rust-vmm mailing list
> Rust-vmm at lists.opendev.org
> http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm


From josh at joshtriplett.org  Wed Dec 19 23:07:39 2018
From: josh at joshtriplett.org (Josh Triplett)
Date: Wed, 19 Dec 2018 15:07:39 -0800
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <CAEUnVG7WfPj0XsTesQ=1E3p6pN=Ykzf+WrLAedTwP2GGxxdXEw@mail.gmail.com>
References: <1545089457255.82861@amazon.com>
 <c8a972ec-cf55-6f2f-34fa-d6032a86320c@redhat.com>
 <CABayD+cjmy1UpR22=53cKqcLEyeVQg3eQ_gukv92N9r0MSDT+g@mail.gmail.com>
 <CAEUnVG7WfPj0XsTesQ=1E3p6pN=Ykzf+WrLAedTwP2GGxxdXEw@mail.gmail.com>
Message-ID: <20181219230739.GA11421@localhost>

On Wed, Dec 19, 2018 at 03:02:15PM -0800, Dylan Reid wrote:
> On Wed, Dec 19, 2018 at 2:59 PM Steve Rutherford <srutherford at google.com> wrote:
> >
> > Hint received :)
> > Getting userspace instruction emulation to work with nested has been a
> > bit of a fight, and we've been waiting to push stuff upstream until we
> > have it everywhere.
> 
> The crosvm team is currently investigating moving the PIT and APIC
> emulation to user space. If that works, instruction emulation will be
> next on the list, including helping to get the kernel side landed
> upstream.

That sounds great; the less complexity and attack surface in the kernel,
the better.

(That also makes it easier to turn off that emulation entirely, for
specialty VMs.)


From sebastien.boeuf at intel.com  Wed Dec 19 23:17:05 2018
From: sebastien.boeuf at intel.com (Boeuf, Sebastien)
Date: Wed, 19 Dec 2018 23:17:05 +0000
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <CAEUnVG7WfPj0XsTesQ=1E3p6pN=Ykzf+WrLAedTwP2GGxxdXEw@mail.gmail.com>
References: <1545089457255.82861@amazon.com>
 <c8a972ec-cf55-6f2f-34fa-d6032a86320c@redhat.com>
 <CABayD+cjmy1UpR22=53cKqcLEyeVQg3eQ_gukv92N9r0MSDT+g@mail.gmail.com>
 <CAEUnVG7WfPj0XsTesQ=1E3p6pN=Ykzf+WrLAedTwP2GGxxdXEw@mail.gmail.com>
Message-ID: <1545261425.24277.8.camel@intel.com>

On Wed, 2018-12-19 at 15:02 -0800, Dylan Reid wrote:
> On Wed, Dec 19, 2018 at 2:59 PM Steve Rutherford <srutherford at google.
> com> wrote:
> > 
> > 
> > Hint received :)
> > Getting userspace instruction emulation to work with nested has
> > been a
> > bit of a fight, and we've been waiting to push stuff upstream until
> > we
> > have it everywhere.
> The crosvm team is currently investigating moving the PIT and APIC
> emulation to user space. If that works, instruction emulation will be
> next on the list, including helping to get the kernel side landed
> upstream.
> 

That's nice! Quick question, why do you want to emulate the PIT? If you
emulate APIC feature X86_FEATURE_TSC_DEADLINE_TIMER, the legacy timer
emulation should not be required, right?
Oh I guess that's because some other architectures might need the PIT.

Are you planning to make this modular so that we could choose to pick
only the APIC emulation?

> > 
> > 
> > On Wed, Dec 19, 2018 at 1:13 PM Paolo Bonzini <pbonzini at redhat.com>
> > wrote:
> > > 
> > > 
> > > On 18/12/18 00:30, Liguori, Anthony wrote:
> > > > 
> > > > As a side note, I think having OS X hypervisor framework
> > > > bindings
> > > > and whatever the new Windows thing is would be pretty cool.
> > > Yes, indeed.  Hypervisor.framework however is much more complex
> > > than KVM
> > > or WHP because you deal manually with VMCSes and have to do
> > > instruction
> > > emulation in userspace.  QEMU takes a stab at it, but it's not as
> > > stable
> > > as KVM.
> > > 
> > > *However* Google does have patches for KVM to do instruction
> > > emulation
> > > in userspace, and I'd like to apply them upstream too now that
> > > KVM has
> > > an API test framework (and thus we can know they won't bitrot).
> > > (Steve, you are in Cc because hint, hint :)).  Once that is in
> > > place, I
> > > guess a minimal x86 emulator written in Rust, porting the
> > > emulator code
> > > that QEMU has for Hypervisor.framework, would be a fun GSoC
> > > project for
> > > a very good student.
> > > 
> > > > 
> > > > 2) The crosvm data_model crate.  This one is super critical but
> > > > easy to
> > > > misunderstand as it allows for safe access to volatile
> > > > memory.  Somewhat
> > > > related is the mmap() bits from sys_util.  Not sure how the
> > > > crosvm folks
> > > > feel but I think there is some refactoring here that could be
> > > > useful to
> > > > build a memory crate.
> > > > 
> > > > 3) Some traits for device model implementations.  It's easy to
> > > > really
> > > > bike shed here so I reckon it's best to start with a concrete
> > > > device
> > > > model like a UART, work through what is required for
> > > > interfaces, and
> > > > then iterate from there.
> > > > 
> > > > 4) Common device models with only a single implementation (i.e.
> > > > 16650A).  Not sure about virtio, maybe.
> > > virtio would be interesting.  One initial target could be a demo
> > > vhost-user client, it has to set up a memory map, parse vrings,
> > > handle
> > > endianness, etc.  It would be an interesting benchmark for a DMA
> > > API.
> > > 
> > > The control plane (your item 3) by comparison is a bit less
> > > interesting.
> > > 
> > > Paolo
> > _______________________________________________
> > Rust-vmm mailing list
> > Rust-vmm at lists.opendev.org
> > http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm
> _______________________________________________
> Rust-vmm mailing list
> Rust-vmm at lists.opendev.org
> http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm

From liuj97 at gmail.com  Thu Dec 20 01:40:37 2018
From: liuj97 at gmail.com (Jiang Liu)
Date: Thu, 20 Dec 2018 09:40:37 +0800
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <1545261425.24277.8.camel@intel.com>
References: <1545089457255.82861@amazon.com>
 <c8a972ec-cf55-6f2f-34fa-d6032a86320c@redhat.com>
 <CABayD+cjmy1UpR22=53cKqcLEyeVQg3eQ_gukv92N9r0MSDT+g@mail.gmail.com>
 <CAEUnVG7WfPj0XsTesQ=1E3p6pN=Ykzf+WrLAedTwP2GGxxdXEw@mail.gmail.com>
 <1545261425.24277.8.camel@intel.com>
Message-ID: <0A1DF8C3-D11D-4051-A1D7-37446FB32756@gmail.com>



> On Dec 20, 2018, at 7:17 AM, Boeuf, Sebastien <sebastien.boeuf at intel.com> wrote:
> 
> On Wed, 2018-12-19 at 15:02 -0800, Dylan Reid wrote:
>> On Wed, Dec 19, 2018 at 2:59 PM Steve Rutherford <srutherford at google.
>> com> wrote:
>>> 
>>> 
>>> Hint received :)
>>> Getting userspace instruction emulation to work with nested has
>>> been a
>>> bit of a fight, and we've been waiting to push stuff upstream until
>>> we
>>> have it everywhere.
>> The crosvm team is currently investigating moving the PIT and APIC
>> emulation to user space. If that works, instruction emulation will be
>> next on the list, including helping to get the kernel side landed
>> upstream.
>> 
> 
> That's nice! Quick question, why do you want to emulate the PIT? If you
> emulate APIC feature X86_FEATURE_TSC_DEADLINE_TIMER, the legacy timer
> emulation should not be required, right?
> Oh I guess that's because some other architectures might need the PIT.
> 
> Are you planning to make this modular so that we could choose to pick
> only the APIC emulation?
> 
Yeah, APIC deadline timer could be used here so we could remove PIT, even
PIC and Local APIC. We have done a quick PoC by using NetBSD and uKVM.
I think it should work with Linux too. 


>>> 
>>> 
>>> On Wed, Dec 19, 2018 at 1:13 PM Paolo Bonzini <pbonzini at redhat.com>
>>> wrote:
>>>> 
>>>> 
>>>> On 18/12/18 00:30, Liguori, Anthony wrote:
>>>>> 
>>>>> As a side note, I think having OS X hypervisor framework
>>>>> bindings
>>>>> and whatever the new Windows thing is would be pretty cool.
>>>> Yes, indeed.  Hypervisor.framework however is much more complex
>>>> than KVM
>>>> or WHP because you deal manually with VMCSes and have to do
>>>> instruction
>>>> emulation in userspace.  QEMU takes a stab at it, but it's not as
>>>> stable
>>>> as KVM.
>>>> 
>>>> *However* Google does have patches for KVM to do instruction
>>>> emulation
>>>> in userspace, and I'd like to apply them upstream too now that
>>>> KVM has
>>>> an API test framework (and thus we can know they won't bitrot).
>>>> (Steve, you are in Cc because hint, hint :)).  Once that is in
>>>> place, I
>>>> guess a minimal x86 emulator written in Rust, porting the
>>>> emulator code
>>>> that QEMU has for Hypervisor.framework, would be a fun GSoC
>>>> project for
>>>> a very good student.
>>>> 
>>>>> 
>>>>> 2) The crosvm data_model crate.  This one is super critical but
>>>>> easy to
>>>>> misunderstand as it allows for safe access to volatile
>>>>> memory.  Somewhat
>>>>> related is the mmap() bits from sys_util.  Not sure how the
>>>>> crosvm folks
>>>>> feel but I think there is some refactoring here that could be
>>>>> useful to
>>>>> build a memory crate.
>>>>> 
>>>>> 3) Some traits for device model implementations.  It's easy to
>>>>> really
>>>>> bike shed here so I reckon it's best to start with a concrete
>>>>> device
>>>>> model like a UART, work through what is required for
>>>>> interfaces, and
>>>>> then iterate from there.
>>>>> 
>>>>> 4) Common device models with only a single implementation (i.e.
>>>>> 16650A).  Not sure about virtio, maybe.
>>>> virtio would be interesting.  One initial target could be a demo
>>>> vhost-user client, it has to set up a memory map, parse vrings,
>>>> handle
>>>> endianness, etc.  It would be an interesting benchmark for a DMA
>>>> API.
>>>> 
>>>> The control plane (your item 3) by comparison is a bit less
>>>> interesting.
>>>> 
>>>> Paolo
>>> _______________________________________________
>>> Rust-vmm mailing list
>>> Rust-vmm at lists.opendev.org
>>> http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm
>> _______________________________________________
>> Rust-vmm mailing list
>> Rust-vmm at lists.opendev.org
>> http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm
> _______________________________________________
> Rust-vmm mailing list
> Rust-vmm at lists.opendev.org <mailto:Rust-vmm at lists.opendev.org>
> http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm <http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendev.org/pipermail/rust-vmm/attachments/20181220/a501b5d5/attachment-0001.html>

From liuj97 at gmail.com  Thu Dec 20 01:55:04 2018
From: liuj97 at gmail.com (Jiang Liu)
Date: Thu, 20 Dec 2018 09:55:04 +0800
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <0A1DF8C3-D11D-4051-A1D7-37446FB32756@gmail.com>
References: <1545089457255.82861@amazon.com>
 <c8a972ec-cf55-6f2f-34fa-d6032a86320c@redhat.com>
 <CABayD+cjmy1UpR22=53cKqcLEyeVQg3eQ_gukv92N9r0MSDT+g@mail.gmail.com>
 <CAEUnVG7WfPj0XsTesQ=1E3p6pN=Ykzf+WrLAedTwP2GGxxdXEw@mail.gmail.com>
 <1545261425.24277.8.camel@intel.com>
 <0A1DF8C3-D11D-4051-A1D7-37446FB32756@gmail.com>
Message-ID: <0D950779-9F83-4285-B431-4CC97DBABCC8@gmail.com>



> On Dec 20, 2018, at 9:40 AM, Jiang Liu <liuj97 at gmail.com> wrote:
> 
> 
> 
>> On Dec 20, 2018, at 7:17 AM, Boeuf, Sebastien <sebastien.boeuf at intel.com <mailto:sebastien.boeuf at intel.com>> wrote:
>> 
>> On Wed, 2018-12-19 at 15:02 -0800, Dylan Reid wrote:
>>> On Wed, Dec 19, 2018 at 2:59 PM Steve Rutherford <srutherford at google.
>>> com> wrote:
>>>> 
>>>> 
>>>> Hint received :)
>>>> Getting userspace instruction emulation to work with nested has
>>>> been a
>>>> bit of a fight, and we've been waiting to push stuff upstream until
>>>> we
>>>> have it everywhere.
>>> The crosvm team is currently investigating moving the PIT and APIC
>>> emulation to user space. If that works, instruction emulation will be
>>> next on the list, including helping to get the kernel side landed
>>> upstream.
>>> 
>> 
>> That's nice! Quick question, why do you want to emulate the PIT? If you
>> emulate APIC feature X86_FEATURE_TSC_DEADLINE_TIMER, the legacy timer
>> emulation should not be required, right?
>> Oh I guess that's because some other architectures might need the PIT.
>> 
>> Are you planning to make this modular so that we could choose to pick
>> only the APIC emulation?
>> 
> Yeah, APIC deadline timer could be used here so we could remove PIT, even
> PIC and Local APIC. We have done a quick PoC by using NetBSD and uKVM.
> I think it should work with Linux too. 
Sorry, “local APIC” should be “IO APIC”, we still use the in-kernel Local APIC
for MSI and IPI.

> 
> 
>>>> 
>>>> 
>>>> On Wed, Dec 19, 2018 at 1:13 PM Paolo Bonzini <pbonzini at redhat.com <mailto:pbonzini at redhat.com>>
>>>> wrote:
>>>>> 
>>>>> 
>>>>> On 18/12/18 00:30, Liguori, Anthony wrote:
>>>>>> 
>>>>>> As a side note, I think having OS X hypervisor framework
>>>>>> bindings
>>>>>> and whatever the new Windows thing is would be pretty cool.
>>>>> Yes, indeed.  Hypervisor.framework however is much more complex
>>>>> than KVM
>>>>> or WHP because you deal manually with VMCSes and have to do
>>>>> instruction
>>>>> emulation in userspace.  QEMU takes a stab at it, but it's not as
>>>>> stable
>>>>> as KVM.
>>>>> 
>>>>> *However* Google does have patches for KVM to do instruction
>>>>> emulation
>>>>> in userspace, and I'd like to apply them upstream too now that
>>>>> KVM has
>>>>> an API test framework (and thus we can know they won't bitrot).
>>>>> (Steve, you are in Cc because hint, hint :)).  Once that is in
>>>>> place, I
>>>>> guess a minimal x86 emulator written in Rust, porting the
>>>>> emulator code
>>>>> that QEMU has for Hypervisor.framework, would be a fun GSoC
>>>>> project for
>>>>> a very good student.
>>>>> 
>>>>>> 
>>>>>> 2) The crosvm data_model crate.  This one is super critical but
>>>>>> easy to
>>>>>> misunderstand as it allows for safe access to volatile
>>>>>> memory.  Somewhat
>>>>>> related is the mmap() bits from sys_util.  Not sure how the
>>>>>> crosvm folks
>>>>>> feel but I think there is some refactoring here that could be
>>>>>> useful to
>>>>>> build a memory crate.
>>>>>> 
>>>>>> 3) Some traits for device model implementations.  It's easy to
>>>>>> really
>>>>>> bike shed here so I reckon it's best to start with a concrete
>>>>>> device
>>>>>> model like a UART, work through what is required for
>>>>>> interfaces, and
>>>>>> then iterate from there.
>>>>>> 
>>>>>> 4) Common device models with only a single implementation (i.e.
>>>>>> 16650A).  Not sure about virtio, maybe.
>>>>> virtio would be interesting.  One initial target could be a demo
>>>>> vhost-user client, it has to set up a memory map, parse vrings,
>>>>> handle
>>>>> endianness, etc.  It would be an interesting benchmark for a DMA
>>>>> API.
>>>>> 
>>>>> The control plane (your item 3) by comparison is a bit less
>>>>> interesting.
>>>>> 
>>>>> Paolo
>>>> _______________________________________________
>>>> Rust-vmm mailing list
>>>> Rust-vmm at lists.opendev.org <mailto:Rust-vmm at lists.opendev.org>
>>>> http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm
>>> _______________________________________________
>>> Rust-vmm mailing list
>>> Rust-vmm at lists.opendev.org <mailto:Rust-vmm at lists.opendev.org>
>>> http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm
>> _______________________________________________
>> Rust-vmm mailing list
>> Rust-vmm at lists.opendev.org <mailto:Rust-vmm at lists.opendev.org>
>> http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm <http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendev.org/pipermail/rust-vmm/attachments/20181220/e4c081ad/attachment.html>

From sebastien.boeuf at intel.com  Thu Dec 20 07:50:32 2018
From: sebastien.boeuf at intel.com (Boeuf, Sebastien)
Date: Thu, 20 Dec 2018 07:50:32 +0000
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <0D950779-9F83-4285-B431-4CC97DBABCC8@gmail.com>
References: <1545089457255.82861@amazon.com>
 <c8a972ec-cf55-6f2f-34fa-d6032a86320c@redhat.com>
 <CABayD+cjmy1UpR22=53cKqcLEyeVQg3eQ_gukv92N9r0MSDT+g@mail.gmail.com>
 <CAEUnVG7WfPj0XsTesQ=1E3p6pN=Ykzf+WrLAedTwP2GGxxdXEw@mail.gmail.com>
 <1545261425.24277.8.camel@intel.com>
 <0A1DF8C3-D11D-4051-A1D7-37446FB32756@gmail.com>
 <0D950779-9F83-4285-B431-4CC97DBABCC8@gmail.com>
Message-ID: <1545292232.30274.6.camel@intel.com>

On Thu, 2018-12-20 at 09:55 +0800, Jiang Liu wrote:


On Dec 20, 2018, at 9:40 AM, Jiang Liu <liuj97 at gmail.com<mailto:liuj97 at gmail.com>> wrote:



On Dec 20, 2018, at 7:17 AM, Boeuf, Sebastien <sebastien.boeuf at intel.com<mailto:sebastien.boeuf at intel.com>> wrote:

On Wed, 2018-12-19 at 15:02 -0800, Dylan Reid wrote:
On Wed, Dec 19, 2018 at 2:59 PM Steve Rutherford <srutherford at google.
com> wrote:


Hint received :)
Getting userspace instruction emulation to work with nested has
been a
bit of a fight, and we've been waiting to push stuff upstream until
we
have it everywhere.
The crosvm team is currently investigating moving the PIT and APIC
emulation to user space. If that works, instruction emulation will be
next on the list, including helping to get the kernel side landed
upstream.


That's nice! Quick question, why do you want to emulate the PIT? If you
emulate APIC feature X86_FEATURE_TSC_DEADLINE_TIMER, the legacy timer
emulation should not be required, right?
Oh I guess that's because some other architectures might need the PIT.

Are you planning to make this modular so that we could choose to pick
only the APIC emulation?

Yeah, APIC deadline timer could be used here so we could remove PIT, even
PIC and Local APIC. We have done a quick PoC by using NetBSD and uKVM.
I think it should work with Linux too.
Sorry, “local APIC” should be “IO APIC”, we still use the in-kernel Local APIC
for MSI and IPI.

Yes, if you don't expect any pin based interrupt, then removing the IOAPIC and the PIC should also work just fine and the APIC (local APIC) itself should be enough.
Talking about linux, if you take a look here: https://github.com/torvalds/linux/blob/master/arch/x86/kernel/apic/apic.c#L818, you see that having the deadline timer would skip the whole calibration of the APIC timer, hence this will prevent from falling back onto the legacy timers (https://github.com/torvalds/linux/blob/master/arch/x86/kernel/apic/apic.c#L840-L842) which are specifically expected to be a PIT or a HPET.






On Wed, Dec 19, 2018 at 1:13 PM Paolo Bonzini <pbonzini at redhat.com<mailto:pbonzini at redhat.com>>
wrote:


On 18/12/18 00:30, Liguori, Anthony wrote:

As a side note, I think having OS X hypervisor framework
bindings
and whatever the new Windows thing is would be pretty cool.
Yes, indeed.  Hypervisor.framework however is much more complex
than KVM
or WHP because you deal manually with VMCSes and have to do
instruction
emulation in userspace.  QEMU takes a stab at it, but it's not as
stable
as KVM.

*However* Google does have patches for KVM to do instruction
emulation
in userspace, and I'd like to apply them upstream too now that
KVM has
an API test framework (and thus we can know they won't bitrot).
(Steve, you are in Cc because hint, hint :)).  Once that is in
place, I
guess a minimal x86 emulator written in Rust, porting the
emulator code
that QEMU has for Hypervisor.framework, would be a fun GSoC
project for
a very good student.


2) The crosvm data_model crate.  This one is super critical but
easy to
misunderstand as it allows for safe access to volatile
memory.  Somewhat
related is the mmap() bits from sys_util.  Not sure how the
crosvm folks
feel but I think there is some refactoring here that could be
useful to
build a memory crate.

3) Some traits for device model implementations.  It's easy to
really
bike shed here so I reckon it's best to start with a concrete
device
model like a UART, work through what is required for
interfaces, and
then iterate from there.

4) Common device models with only a single implementation (i.e.
16650A).  Not sure about virtio, maybe.
virtio would be interesting.  One initial target could be a demo
vhost-user client, it has to set up a memory map, parse vrings,
handle
endianness, etc.  It would be an interesting benchmark for a DMA
API.

The control plane (your item 3) by comparison is a bit less
interesting.

Paolo
_______________________________________________
Rust-vmm mailing list
Rust-vmm at lists.opendev.org<mailto:Rust-vmm at lists.opendev.org>
http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm
_______________________________________________
Rust-vmm mailing list
Rust-vmm at lists.opendev.org<mailto:Rust-vmm at lists.opendev.org>
http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm
_______________________________________________
Rust-vmm mailing list
Rust-vmm at lists.opendev.org<mailto:Rust-vmm at lists.opendev.org>
http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendev.org/pipermail/rust-vmm/attachments/20181220/e81dc29d/attachment-0001.html>

From allison at lohutok.net  Thu Dec 20 14:12:45 2018
From: allison at lohutok.net (Allison Randal)
Date: Thu, 20 Dec 2018 09:12:45 -0500
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <cig332bm5hqhw1.fsf@u54e1add816995a33037d.ant.amazon.com>
References: <1545089457255.82861@amazon.com>
 <CAEUnVG43MONqvtdL4HDtqoBVCVV7+o0bi=5BfQc26L0gi6HCww@mail.gmail.com>
 <aaf1c200-7585-c35e-9c4a-85b5b896f0bd@lohutok.net>
 <cig332bm5hqhw1.fsf@u54e1add816995a33037d.ant.amazon.com>
Message-ID: <6acbefa5-dac8-e527-3be7-00df9586b645@lohutok.net>

On 12/19/18 5:19 PM, Anthony Liguori wrote:
> 
> I did look at he work you've done and it's great!  I do think it's
> important to cover everything though and especially to get architectures
> like aarch64.

Thanks, it really was just a proof-of-concept. Rust is much better than
C at selective compilation for different architectures, so that's
another area where well-written Rust would be a huge improvement over
poorly-translated C-to-Rust.

> I think while the bindgen results are not beautiful, having a bindgen
> layer gives us something we can build nicer interfaces on top of.

My primary concern was tight control over security at the lower layers,
hence the emphasis on keeping that interface to KVM extremely clean and
small. Totally understandable that other Rust hypervisor implementations
may have other priorities.


I'm curious whether rewriting (a subset of) the KVM kernel module itself
in Rust would end up being a significant advantage for memory safety and
security at an even lower layer. (Rust's ability to drop in as a
compatible replacement for C components is a great advantage for
gradually improving the memory safety of critical pieces of legacy C
code.) I haven't decided whether to do a proof-of-concept on that yet,
though, as my current focus is security vulnerabilities at the hardware
level that punch through VM isolation (L1TF, etc.).

Allison


From aliguori at amazon.com  Thu Dec 20 15:05:28 2018
From: aliguori at amazon.com (Anthony Liguori)
Date: Thu, 20 Dec 2018 07:05:28 -0800
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <6acbefa5-dac8-e527-3be7-00df9586b645@lohutok.net>
References: <1545089457255.82861@amazon.com>
 <CAEUnVG43MONqvtdL4HDtqoBVCVV7+o0bi=5BfQc26L0gi6HCww@mail.gmail.com>
 <aaf1c200-7585-c35e-9c4a-85b5b896f0bd@lohutok.net>
 <cig332bm5hqhw1.fsf@u54e1add816995a33037d.ant.amazon.com>
 <6acbefa5-dac8-e527-3be7-00df9586b645@lohutok.net>
Message-ID: <cig332ftus8chz.fsf@u54e1add816995a33037d.ant.amazon.com>

Allison Randal <allison at lohutok.net> writes:

> On 12/19/18 5:19 PM, Anthony Liguori wrote:
>> 
>> I did look at he work you've done and it's great!  I do think it's
>> important to cover everything though and especially to get architectures
>> like aarch64.
>
> Thanks, it really was just a proof-of-concept. Rust is much better than
> C at selective compilation for different architectures, so that's
> another area where well-written Rust would be a huge improvement over
> poorly-translated C-to-Rust.
>
>> I think while the bindgen results are not beautiful, having a bindgen
>> layer gives us something we can build nicer interfaces on top of.
>
> My primary concern was tight control over security at the lower layers,
> hence the emphasis on keeping that interface to KVM extremely clean and
> small. Totally understandable that other Rust hypervisor implementations
> may have other priorities.
>
>
> I'm curious whether rewriting (a subset of) the KVM kernel module itself
> in Rust would end up being a significant advantage for memory safety and
> security at an even lower layer. (Rust's ability to drop in as a
> compatible replacement for C components is a great advantage for
> gradually improving the memory safety of critical pieces of legacy C
> code.) I haven't decided whether to do a proof-of-concept on that yet,
> though, as my current focus is security vulnerabilities at the hardware
> level that punch through VM isolation (L1TF, etc.).

Yes!  But in addition to rewriting that subset, moving it to userspace
gives you yet another layer of protection.

The two biggest sources of CVEs in KVM have been instruction emulation
and device emulation.  Moving the x86_emulate code to userspace and
rewritting it in Rust would eliminate one of the larger attack surfaces
in KVM and likewise, moving IO APIC and PIT emulation to userspace would
help a lot there too.

On modern processors, LAPIC is handled almost entirely in hardware so
the remaining complexity in KVM is really around EPT handling and
hardware interaction.  I don't think either can reasonably be moved.

Regards,

Anthony Liguori

>
> Allison


From pbonzini at redhat.com  Thu Dec 20 15:34:08 2018
From: pbonzini at redhat.com (Paolo Bonzini)
Date: Thu, 20 Dec 2018 16:34:08 +0100
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <cig332ftus8chz.fsf@u54e1add816995a33037d.ant.amazon.com>
References: <1545089457255.82861@amazon.com>
 <CAEUnVG43MONqvtdL4HDtqoBVCVV7+o0bi=5BfQc26L0gi6HCww@mail.gmail.com>
 <aaf1c200-7585-c35e-9c4a-85b5b896f0bd@lohutok.net>
 <cig332bm5hqhw1.fsf@u54e1add816995a33037d.ant.amazon.com>
 <6acbefa5-dac8-e527-3be7-00df9586b645@lohutok.net>
 <cig332ftus8chz.fsf@u54e1add816995a33037d.ant.amazon.com>
Message-ID: <86f95429-4381-9501-798d-ad56f264a34c@redhat.com>

On 20/12/18 16:05, Anthony Liguori wrote:
> The two biggest sources of CVEs in KVM have been instruction emulation
> and device emulation.  Moving the x86_emulate code to userspace and
> rewritting it in Rust would eliminate one of the larger attack surfaces
> in KVM and likewise, moving IO APIC and PIT emulation to userspace would
> help a lot there too.
> 
> On modern processors, LAPIC is handled almost entirely in hardware so
> the remaining complexity in KVM is really around EPT handling and
> hardware interaction.  I don't think either can reasonably be moved.

Note that userspace PIT/PIC/IOAPIC emulation is already supported by KVM
(Linux 4.4 or newer I think; QEMU will make it the default for the q35
machine type in the next release, for now you need -machine
kernel_irqchip=split).

Paolo


From mutexlox at google.com  Thu Dec 20 19:53:58 2018
From: mutexlox at google.com (Miriam Zimmerman)
Date: Thu, 20 Dec 2018 11:53:58 -0800
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
Message-ID: <CAHSjozAqQwjUehcktyhRCSj385QL4JS0a+6d=5yih8NYHSTasw@mail.gmail.com>

Dylan just let me know about this list. I've started working on a userspace
PIT (in Rust) for crosvm, based heavily on Google's existing implementation
for GCE. I'm tracking overall work on getting split-irqchip support for
crosvm in https://crbug.com/908689.

I'd really appreciate any thoughts people have on the project or any
devices we can skip without limiting which guest OSs we can run.

Miriam
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendev.org/pipermail/rust-vmm/attachments/20181220/3d74a517/attachment.html>

From dgreid at google.com  Thu Dec 20 19:55:49 2018
From: dgreid at google.com (Dylan Reid)
Date: Thu, 20 Dec 2018 11:55:49 -0800
Subject: [Rust-VMM] [Rust-vmm] Goals for this list
In-Reply-To: <86f95429-4381-9501-798d-ad56f264a34c@redhat.com>
References: <1545089457255.82861@amazon.com>
 <CAEUnVG43MONqvtdL4HDtqoBVCVV7+o0bi=5BfQc26L0gi6HCww@mail.gmail.com>
 <aaf1c200-7585-c35e-9c4a-85b5b896f0bd@lohutok.net>
 <cig332bm5hqhw1.fsf@u54e1add816995a33037d.ant.amazon.com>
 <6acbefa5-dac8-e527-3be7-00df9586b645@lohutok.net>
 <cig332ftus8chz.fsf@u54e1add816995a33037d.ant.amazon.com>
 <86f95429-4381-9501-798d-ad56f264a34c@redhat.com>
Message-ID: <CAEUnVG4jnRiVrbJ1s28a_+LqxdD9iUAiTHQHN9gDKHqKLUaXyg@mail.gmail.com>

On Thu, Dec 20, 2018, 7:34 AM Paolo Bonzini <pbonzini at redhat.com wrote:

> On 20/12/18 16:05, Anthony Liguori wrote:
> > The two biggest sources of CVEs in KVM have been instruction emulation
> > and device emulation.  Moving the x86_emulate code to userspace and
> > rewritting it in Rust would eliminate one of the larger attack surfaces
> > in KVM and likewise, moving IO APIC and PIT emulation to userspace would
> > help a lot there too.
> >
> > On modern processors, LAPIC is handled almost entirely in hardware so
> > the remaining complexity in KVM is really around EPT handling and
> > hardware interaction.  I don't think either can reasonably be moved.
>
> Note that userspace PIT/PIC/IOAPIC emulation is already supported by KVM
> (Linux 4.4 or newer I think; QEMU will make it the default for the q35
> machine type in the next release, for now you need -machine
> kernel_irqchip=split).
>

+ Miriam who is working on pit and apic on crosvm


> Paolo
>
> _______________________________________________
> Rust-vmm mailing list
> Rust-vmm at lists.opendev.org
> http://lists.opendev.org/cgi-bin/mailman/listinfo/rust-vmm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendev.org/pipermail/rust-vmm/attachments/20181220/e125a2ac/attachment.html>

From fandree at amazon.com  Mon Dec 24 13:41:15 2018
From: fandree at amazon.com (Florescu, Andreea)
Date: Mon, 24 Dec 2018 13:41:15 +0000
Subject: [Rust-VMM] Firecracker & Crosvm common crates
Message-ID: <1545658874428.39674@amazon.com>

Hey everyone,


I've created a gist [1] so we can start a discussion about what would be worth contributing back to Crosvm.

The document is far from being completed, but I think it is a good idea to get feedback as early as possible. That gist can also be a starting point for talking about crates that can be shared between the teams (and potentially other rust VMMs).


Let me know what you think.


Thanks and happy holidays,

Andreea


[1] https://gist.github.com/andreeaflorescu/ce4190b442dea2c48a0d783019579ad3




Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendev.org/pipermail/rust-vmm/attachments/20181224/db74974f/attachment-0001.html>