[Rust-VMM] Buildkite CI - multiple pipelines?

[Florescu, Andreea]

Hey everyone,


During PTG we discussed about having a centralized place for the buildkite pipeline. The buildkite pipeline is the definition of steps (or commands) that run as part of the CI.


Since all crates will have to pass common tests like `cargo test`, `cargo build` and `cargo fmt` the idea would be to have these steps in a centralized repository so we do not have to duplicate the pipeline in all repositories. I created an issue for this purpose [1]. The pipeline would live in a `buildkite-ci` repository or similar. Maybe rust-vmm-ci is a better option so we can have the coverage test in the same repository as well, but that is a different discussion.


Each GitHub repository would then use `buildkite-ci` as a git submodule. To better explain this, let's take an example.


Let's say we have `buildkite-ci.git` and in it we have a `.buildkite` directory. In the buildkite directory we have `pipeline.yml` which defines steps/commands like cargo build and cargo test.

Now, say we want to enable the buildkite CI for the vm-memory repository. To do that, we would need to do the following:

1. Add `buildlite-ci` as a git submodule of vm-memory.

2. Add a new vm-memory-ci pipeline using the Buildkite web interface.

3. Specify which pipeline to use to run the tests. This is done via a command called "pipeline upload" which takes as a parameter a path that is relative to the root of the repository. You can read more about it here [2]. The command in this case will look something like:

buildkite-agent pipeline upload buildkite-ci/.buidkite/pipeline.yml


With this design you get 2 things:

1. You don't have to duplicate the pipeline for all repositories

2. You can still have custom integration tests for each repository


Now, after some thought I started to believe that the better choice is to actually provide multiple pipelines instead of one. My proposal would be to have one pipeline for each platform and operating system we want to support. And let me tell you why:

1. Some crates (like vm-memory) will also have support for Windows so you want to the integration tests to pass on windows as well. But you can't have it in the default pipeline because some crates cannot have it (like kvm-ioctls).

2. Ideally we would have *most* of the crates work on arm and x86. That means that in our pipelines we would have same tests on x86 and arm as well. But some tools are not available on arm, so the steps are not exactly the same. Examples of things that aren't available (yet) on arm are clippy and kcov. Now, when we start development we might not want to add support from both arm and x86 from the first PR because it might turn out to be very complex and hard to review. So we can start with x86 and only use the x86 pipeline till support for arm is added.


With our current configuration of crates, platforms and operating systems we would end up having 3 pipelines:

- x86_linux_pipeline.yml (used by kvm-ioctls, vm-memory, vmm-sys-utils)

- x86_windows_pipeline.yml (used by vm-memory)

- arm_linux_pipeline.yml (used by kvm-ioctls, vmm-sys-utils)


What do you all think?


Regards,

Andreea


[1] https://github.com/rust-vmm/community/issues/56

[2] https://buildkite.com/docs/pipelines/defining-steps



Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendev.org/pipermail/rust-vmm/attachments/20190509/09fc6a81/attachment.html>