February 2018 - Zuul-discuss - lists.opendev.org

[nodepool] Webapp label_list
by David Shrewsbury 01 Mar '18

01 Mar '18

Greetings! We have recently merged a change to add the label_list command to nodepool's webapp [1]. For that change, we briefly discussed in IRC [2] the meaning of the "count" parameter that was originally within the output of that command. The tl;dr from me in that discussion was that count without considering node state has no meaning. The final PS of that review left count out of the results. There is now another change proposing to add that count back in [3]. I think this functionality needs some clarification on what it is actually supposed to mean, because I am a bit confused. If it is supposed to be a list of all labels that are configured within nodepool, then it will never be a complete set unless there are nodes booted for each label. If it is supposed to be a list of labels that are immediately available for jobs, then we will have to take node state into account (as well as whether the node is locked or already allocated to another request). So I guess my question is (and I suppose this should have been clarified earlier) .... what is label_list supposed to mean for a user? [1] https://review.openstack.org/535563 [2] http://eavesdrop.openstack.org/irclogs/%23zuul/%23zuul.2018-02-07.log.html#… [3] https://review.openstack.org/548134 -Dave -- David Shrewsbury (Shrews)

4 8

[nodepool] Restricting images to specific nodepool builders
by Ian Wienand 20 Feb '18

20 Feb '18

Hi, How should we go about restricting certain image builds to specific nodepool builder instances? My immediate issue is with ARM64 image builds, which I only want to happen on a builder hosted in an ARM64 cloud. Currently, the builders go through the image list and check "is the existing image missing or too old, if so, build" [1]. Additionally, all builders share a configuration file [2]; so builders don't know "who they are". I'd propose we add an arbitrary tag/match system so that builders can pickup only those builds they mark themselves capable of building? e.g. diskimages would specify required builder tags similar to: --- diskimages: - name: arm64-ubuntu-xenial elements: - block-device-efi - vm - ubuntu-minimal ... env-vars: TMPDIR: /opt/dib_tmp DIB_CHECKSUM: '1' ... builder-requires: architecture: arm64 --- The nodepool.yaml would grow another section similar: --- builder-provides: architecture: arm64 something_else_unique_about_this_buidler: true --- For OpenStack, we would template this section in the config file via puppet in [2], ensuring above that only our theoretical ARM64 build machine had that section in it's config. The nodepool-buidler build loop can then check that its builder-provides section has all the tags specified in an image's "builder-requires" section before deciding to start building. Thoughts welcome :) -i [1] https://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/build… [2] https://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool…

6 10

How to determine what has changed in a test
by corvus＠inaugust.com 08 Feb '18

08 Feb '18

Hi, The problem of how to determine what has changed in the change under test has come to my attention twice today, and it's still early, so I thought it'd be a good topic for a ML discussion. The two issues that have come to my attention represent two similar kinds of tests: a) Inspect the specific commit(s) under test (e.g., inspect the commit message). b) Inspect the delta of applying the patch (e.g., run coverage twice and calculate the difference). Simon Westphahl has proposed a patch focused on the first case, which adds a zuul.commits variable with the sha(s) of the patch(es) under test: https://review.openstack.org/539975 Meanwhile, here is an example of a test which relies on the existence of an origin remote: http://logs.openstack.org/58/473658/19/check/openstack-tox-cover/0f6849c/jo… We also have the minor example of pip emitting warnings if there is no origin remote; if we can fix that here, that's fine, but I don't think it's significant enough to drive our design. The patch to add zuul.commits makes case (a) fairly simple -- you can have ansible iterate over the values in the variable. I think that case (b) might be a bit harder with zuul.commits -- specialized tooling would be needed to create the alternative repo state (ie, a new branch) in order to perform the git checkout or diff operations. Because in (b) we are focused more on the previous state than the commit, we can't just checkout zuul.commit, we need to checkout the parent of zuul.commit. However, if we were to create an origin remote, and populate it with the "previous" git repo state, (that is, either the current state of the tree, or that with the changes ahead of this one applied), it seems we can accommodate both (a) and (b) in an intuitive manner. You could get the values that would be put in zuul.commits by performing 'git diff origin/master..master', which addresses (a), and you could address (b) with 'git checkout origin/master'. Among the reasons we don't have any remotes on the git checkouts are: * What would the URL be? * What would the content be? * Concern about jobs altering the git repo under test by performing naïve and incorrect git operations. We've addressed the content question, that leaves the other two. Ideally, if we did this, I'd like to have a remote with no URL. However, while that may be possible, some investigation suggests that we may want to avoid that as it looks like it may be a bit of an edge case that not all tools or libraries would support correctly. However, if we set it to a bogus URL, that may be sufficient to satisfy the tooling, as well as prevent git repo manipulation errors. I'm inclined to favor this approach, as I think it addresses both cases and is fairly intuitive. Do folks think it would work? Are there things we would be missing as compared to the zuul.commits approach? Thanks, Jim

3 3

Zuul website
by corvus＠inaugust.com 02 Feb '18

02 Feb '18

Hi, We now have a website at https://zuul-ci.org/ The content is in git in the openstack-infra/zuul-website repo: https://git.openstack.org/cgit/openstack-infra/zuul-website/tree/ I put together the site using a CC-BY 3.0 template, and I expect that the content of that repo will be licensed CC-BY as well (as most of our docs-only repos are). I enthusiastically invite other folks to contribute to the website by proposing changes to that repo. Currently the zuul-core team has approval rights, but we can expand it as warranted -- I expect there's an overlapping group of folks who would like to contribute in this area. I am not emotionally attached to anything about the current website. Please feel free to propose changes to absolutely everything -- including replacing the site entirely, using a different template, title image, etc. What's there is a starting point, and an attempt to organize information and present it to folks in a way that helps them find the right resources quickly. I only ask that any changes try to keep this in mind: the site has many audiences, and we should think about how they all interact with it. The site itself is currently simply static HTML. You can view it on your workstation with no special tools. I looked into static site generators, and there are promising tools in this space, but ultimately I felt that until we get to the point where we want more than a handful of pages, the extra work isn't worth it. Some parts of the template are not in use in the current site. Most notable is the navigation bar. We can add those back in easily as warranted. You may want to visit the live demo of the template to see all of its offerings: https://html5up.net/arcana So as far as the infrastructure goes, I suggest that we continue with simple static pages for now, and when we get to the point where we know what kind of content we want to add that warrants more tooling, we adopt a static site generator then. I do think we may want to start publishing the zuul, nodepool, and zuul-jobs docs under this domain name as well. But we can do that in the same way we do with OpenStack -- publish the docs in a subdirectory, so they won't be part of the 'zuul-website' repo. We can start looking at adapting those jobs soon. Thanks, Jim

2 1

Feature branch merge update
by corvus＠inaugust.com 02 Feb '18

02 Feb '18

Hi, After merging the feature/zuulv3 branches of Nodepool and Zuul back into master, a number of open changes on the master branch remained from before the merge (and, in many cases, much older than that). Some of us have gone through to try to triage those changes. Some of them were implemented on the feature/zuulv3 branch or otherwise just don't apply anymore (a component was removed, or functions differently). These changes were abandoned with a short explanation of why, and an invitation to restore them if the, admittedly cursory, analysis was wrong. Some of them may still apply, but in those cases, almost certainly need to be rebased. These changes we marked with Workflow -1: Work in Progress, with the hope that the author will update them when time permits. At this point, all of the open changes on Zuul and Nodepool which are not marked WIP should be relevant to the current state of the tree, and should be reviewed. So if you've only been looking at a subset of open changes to avoid the older changes on master which may not be relevant, you should be able to remove that filter and consider all open, non-WIP changes. Thanks, Jim

1 0