- Zuul-discuss - lists.opendev.org

Software Factory 3.0 released
by Tristan Cacqueray 30 Mar '18

30 Mar '18

Greeting folks, We are pleased to announce the release of Software Factory version 3.0: http://www.softwarefactory-project.io/releases/3.0 It is a software development forge that integrates the last version of Zuul and Nodepool along with a few patches that couldn't make it into the upstream release: * Ansible-2.4 support: https://review.openstack.org/535839 * Dynamic tenant configuration: https://review.openstack.org/535878 * MQTT reporter: https://review.openstack.org/535543 * Job and Projects dashboard: https://review.openstack.org/550979 * Broken config protection: https://review.openstack.org/535511 * Zookeeper auto-retry: https://review.openstack.org/535537 * OpenContainer Nodepool provider: https://review.openstack.org/535556 * Kubernetes Nodepool provider: https://review.openstack.org/535557 Best regards, The Software Factory Team

3 3

Next steps after 3.0.0 release
by corvus＠inaugust.com 29 Mar '18

29 Mar '18

Hi, Now that 3.0.0 is released[1] (yay!), it's a good time to go over what we need to do next. The first order of business is to land the changes to add reno: https://review.openstack.org/536861 https://review.openstack.org/262597 You should read the docs that those changes introduce (they'll be in the developer docs for future reference), but the key bit is: When adding a noteworthy feature, fixing a noteworthy bug or introducing a behavior change that a user or operator should know about, it is a good idea to add a release note to the same patch. Every release from this point out will have release notes included in the documentation. We relaxed our review standards significantly during the feature branch phase, but we've been returning to normal for the past few months. At this point, the master branches of Nodepool and Zuul should always be kept working and ready for release. Gating usually handles the technical side of that, but don't lose track of the intent when it comes to other aspects. Please make sure that changes have documentation if they affect a user-visible aspect of the system, have a release note if the change is noteworthy, and have tests if they alter behavior. If a change is missing these things, please ask about it. These rules don't need to be too rigid, but if we flex them, we should be in the habit of discussing it and establishing that there's a good reason. We need to be careful about making changes to the zuul-jobs and zuul-base-jobs repos now. We'll write up a policy and include it in the docs for those repos, but the important thing is that we can't use new Zuul features in those repos until we've given folks a chance to upgrade (so probably some number of weeks after the feature appears in a release). And we can't remove features from Zuul until we've removed their use from zuul-jobs, and given folks who might be running a fork of zuul-jobs time to update. I expect us to release Zuul and Nodepool much more often now. Any time we make a significant change and establish that the software is working as intended, we should release. We've assigned some very specific meaning to version numbers recently, but don't think that will continue. The versions we release will be what they are depending on what has landed when we release. Thanks everyone for your work on the release, and I look forward to reviewing all the patches that are waiting in the wings! -Jim [1] http://lists.zuul-ci.org/pipermail/zuul-announce/2018-March/000006.html

1 0

Docs and git repos on zuul-ci.org
by corvus＠inaugust.com 28 Mar '18

28 Mar '18

Hi, We now have all of the Zuul documentation hosted on the Zuul website[1], so folks may want to start referencing those URLs, rather than docs.openstack.org. We also have the Zuul git repos mirrored under the Zuul domain at [2]. That page shows only the Zuul-related git repos. Additionally, if you're referencing the zuul-jobs or zuul-base-jobs repos, you may want to switch your connection to use git.zuul-ci.org instead. Note that if you do that, the project name will be "git.zuul-ci.org/zuul-jobs" -- with no "openstack-infra" prefixed to the name, as there are no organization prefixes on repositories mirrored at git.zuul-ci.org. [1] https://zuul-ci.org/docs/ [2] http://git.zuul-ci.org/ -Jim

2 2

Re: [OpenStack-Infra] Zuul: Question about Github webhook
by Melvin Hillsman 22 Mar '18

22 Mar '18

I also have went through the "ZFS" at least once and ran into quite a few issues I think and started documenting[0] them. Bernd not sure if you have run into additional issues but feel free to add to the etherpad; it is not organized at all right now but the plan was/is to report back a list of things and patch(es). [0] https://etherpad.openstack.org/p/zuul-quickstart-and-zfs(zuul-from-scratch) On Wed, Mar 21, 2018 at 3:26 AM, <Tobias.Henkel(a)bmw.de> wrote: > Hi Bernd, > > On 21.03.18, 08:54, "Bernd Bausch" <berndbausch(a)gmail.com> wrote: > > Sorry for this newbie intrusion on the distribution list. I am trying > to > follow the Zuul from Scratch instructions[1] to set up Zuul at home. > The > purpose is twofold: Learn about Zuul, and check whether Zuul from > Scratch > instructions are accurate and make sense. > > Currently I am at a roadblock. > > The problem is the Github webhook, through which Github sends data to > Zuul. > It should be > http://<zuul-hostname>/connection/github/payload [2], but on the Zuul > server > that > I have deployed so far, nothing listens at port 80. Thus, Github gets > a "connection refused" when trying to contact my server. I used > tcpdump to > confirm that my server does indeed receive an http POST from Github. > > Either something is wrong with my setup, which prevents Zuul from > listening > at > 80. Or the webhook isn't documented correctly. > > How can I confirm what could be the problem? > > Bernd > > [1] https://docs.openstack.org/infra/zuul/admin/zuul-from-scratch.html > [2] https://docs.openstack.org/infra/zuul/admin/drivers/github.html, > https://docs.openstack.org/infra/zuul/admin/zuul-from- > scratch.html#configure > -github > > > It looks like you discovered a bug in the zuul-from-scratch docs. Zuul-web > by default > listens on port 9000. While the documented webhook config in github > assumes port 80. > So you need to either change the webhook to use 9000 or change the zuul > config to use > port 80 [1]. > > Further I pushed up a change to fix the doc [2]. > > Sidenote: we now have zuul specific mailing lists [3] so you might want to > also subscribe > to them. > > [1] https://docs.openstack.org/infra/zuul/admin/components. > html#attr-web.port > [2] https://review.openstack.org/554829 > [3] http://lists.zuul-ci.org > > Kind regards > Tobias > > _______________________________________________ > OpenStack-Infra mailing list > OpenStack-Infra(a)lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra -- Kind regards, Melvin Hillsman mrhillsman(a)gmail.com mobile: (832) 264-2646

3 4

Services missing tenant scope (nodepool-*, zuul-executor)
by Tristan Cacqueray 19 Mar '18

19 Mar '18

Hello folks, Is there a plan to add a tenant scope to all the services? For example, we could add the tenant name to NodeRequest and a 'tenant' setting in nodepool.yaml so that a provider and its labels could be restricted to a particular tenant. Similarly for the executors, we could be able to isolate tenant job execution from each other. Has this been discussed at the PTG? Regards, -Tristan

2 2

Zookeeper ConnectionLoss/SessionExpired issues
by Tristan Cacqueray 16 Mar '18

16 Mar '18

Hello folks, Here is a recap of the issues happening when the connection to Zookeeper is unstable: * Nodepool marks dib build as failed, * ConnectionLoss/SessionExpired exceptions happen when the connection drop after the "self._zk.suspended or self._zk.lost" checks, * Zuul lost NodeRequest which may lock change in queued state, and * Zookeeper needs to be running before starting Zuul or Nodepool, otherwise the daemon stay stuck in limbo. This used to happen frequently in our deployment (running in an OpenStack tenant with zookeeper on a dedicated instance). It seems like the current connection listener doesn't work properly and the errors seems to mostly happen when the connection is actually used. There are a couple of patches[0] to leverage Kazoo retry facilities and to mitigate above issues by automatically retrying operations until the service is available. The nodepool patch adds a uuid to Sequence object as suggested by the Zookeeper ErrorHandling documentation[1]. The last remaining issue is the Zuul ephemeral NodeRequest code that doesn't re-create the request if they got deleted when the connection is reset. I think those patches improve the overall service reliability and we have been using them successfully without issues in software-factory packages. Regards, -Tristan [0]: https://review.openstack.org/#/q/topic:zookeeper-retry [1]: https://cwiki.apache.org/confluence/display/ZOOKEEPER/ErrorHandling

5 16

Disabling nullwrap
by corvus＠inaugust.com 13 Mar '18

13 Mar '18

Hi, The recent security fixes have underscored that bubblewrap is currently key to Zuul executor security. While Zuul does have an option to run without bubblewrap (called "nullwrap"), it is not recommended, and it doesn't support secrets, which is a fairly important feature. Nullwrap was written with the idea of aiding those use-cases where it is not possible to run with bubblewrap. However, I think that if such a case still exists, we should work to address that -- nullwrap really shouldn't be the answer. I'd like to disable the ability to use nullwrap to avoid any potential confusion on the subject in the future. If anyone is currently using nullwrap, please speak up so we can start working on a solution. -Jim

4 3

Landing changes to zuul-web javascript
by Monty Taylor 09 Mar '18

09 Mar '18

Hey everybody! Something just occurred to me while reviewing the next set of patches from Tristan to zuul-web that I think is important to share more broadly. In OpenStack's deployment of Zuul, HTML/JS changes to zuul-web go live basically immediately (give or take 15 minutes) No services need to be restarted or anything. However, while the python code will also be _installed_ at the same time, it will not go live immediately. It's also completely possible for someone to deploy web assets to a separate web server or even an object store like swift. For that reason, I'd like to suggest we make a practice of splitting web changes into patches that add methods to the REST interface on zuul-web, and patches that add Javascript consumption. Then make sure when approving patches that the REST changes have gone live - which will let us verify the draft changes from the build-javascript-content job. We could obviously add feature-flags to the javascript so that the two patches could land together. However, I don't think it's time for that yet for two reasons: a) this dashboard is still pretty new and we're still figuring some things out but more importantly b) we can't verify the html/js changes live in the draft change until the REST side has gone live anyway Anyway - I'm sure there will be exceptions where it's fine - so people should use good judgment as always - but at least be aware that landing javascript changes will essentially result in a near-immediate application of them to production for OpenStack and judge accordingly. Monty

3 4

working on zuul trigger plugin for jenkins
by Darragh Bailey 07 Mar '18

07 Mar '18

Hi, Spend some time last week hacking away at a zuul trigger plugin for jenkins, really just about getting some better understanding of how this should hang together and what questions need to be asked. Lots of time spent reading details on gRPC as it was suggested at https://etherpad.openstack.org/p/zuulv3-jenkins-integration So far I've got a (very) rudimentary gRPC server that responds to a single status request with OK working as a Jenkins plugin that starts up automatically. I'll try to get it released somewhere soon so it's possible to discuss code, though if it is of interest to the OpenStack community maybe it should be hosted on review.openstack.org? Some general things I've noted: - authenticating requests doesn't appear to be straight forward with gRPC - unclear whether this can share the default jenkins http port even though gRPC and REST (currently using dedicated port) can technically do this - given the number of messages exchanged for a single job, should this use bidirectional streaming - started with a proto layout that is mostly unary client calls and server streaming responses - but should the trigger really be a single connection with multiple messages around the hand-off coordination? I've also had a bunch of questions to bounce off Monty around how to deal with Zuulv3 and Jenkins node coordination, the upshot of those is to leave that to nodepool and probably help out with the Jenkins nodepool plugin and make that a dependency of the zuul trigger plugin. For more detail, originally I was worrying about how to ensure zuul trigger plugin would work without using the nodepool plugin, and it would end up needing to query Jenkins as to whether it had the resources to execute the on before triggering, and this would require a deeper integration into Zuul. It was suggested that this would complicate it deeply and would be better to have nodepool manage the resources, so that zuul would only start the ansible job that would call the zuul trigger API endpoints if nodepool had sufficient resources for Jenkins to use to execute the job. It appears clear any migration path to Zuulv3 involving Jenkins would want to use Zuulv2 -> Gearman -> Jenkins -> Nodepoolv3 first, then migrate Zuulv3. So I'm looking to help on the Jenkins Nodepool plugin first, and that will probably make it much more straight forward to understand what needs to be done in the trigger plugin. A couple of weeks ago someone was in the Zuul room chatting about working on the Nodepool plugin for Jenkins, wondering if there has been any progress there, and what I can help out with in the future. -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool"

1 0

[nodepool] Webapp label_list
by David Shrewsbury 28 Feb '18

28 Feb '18

Greetings! We have recently merged a change to add the label_list command to nodepool's webapp [1]. For that change, we briefly discussed in IRC [2] the meaning of the "count" parameter that was originally within the output of that command. The tl;dr from me in that discussion was that count without considering node state has no meaning. The final PS of that review left count out of the results. There is now another change proposing to add that count back in [3]. I think this functionality needs some clarification on what it is actually supposed to mean, because I am a bit confused. If it is supposed to be a list of all labels that are configured within nodepool, then it will never be a complete set unless there are nodes booted for each label. If it is supposed to be a list of labels that are immediately available for jobs, then we will have to take node state into account (as well as whether the node is locked or already allocated to another request). So I guess my question is (and I suppose this should have been clarified earlier) .... what is label_list supposed to mean for a user? [1] https://review.openstack.org/535563 [2] http://eavesdrop.openstack.org/irclogs/%23zuul/%23zuul.2018-02-07.log.html#… [3] https://review.openstack.org/548134 -Dave -- David Shrewsbury (Shrews)

4 8

[nodepool] Restricting images to specific nodepool builders
by Ian Wienand 19 Feb '18

19 Feb '18

Hi, How should we go about restricting certain image builds to specific nodepool builder instances? My immediate issue is with ARM64 image builds, which I only want to happen on a builder hosted in an ARM64 cloud. Currently, the builders go through the image list and check "is the existing image missing or too old, if so, build" [1]. Additionally, all builders share a configuration file [2]; so builders don't know "who they are". I'd propose we add an arbitrary tag/match system so that builders can pickup only those builds they mark themselves capable of building? e.g. diskimages would specify required builder tags similar to: --- diskimages: - name: arm64-ubuntu-xenial elements: - block-device-efi - vm - ubuntu-minimal ... env-vars: TMPDIR: /opt/dib_tmp DIB_CHECKSUM: '1' ... builder-requires: architecture: arm64 --- The nodepool.yaml would grow another section similar: --- builder-provides: architecture: arm64 something_else_unique_about_this_buidler: true --- For OpenStack, we would template this section in the config file via puppet in [2], ensuring above that only our theoretical ARM64 build machine had that section in it's config. The nodepool-buidler build loop can then check that its builder-provides section has all the tags specified in an image's "builder-requires" section before deciding to start building. Thoughts welcome :) -i [1] https://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/build… [2] https://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool…

6 10

How to determine what has changed in a test
by corvus＠inaugust.com 07 Feb '18

07 Feb '18

Hi, The problem of how to determine what has changed in the change under test has come to my attention twice today, and it's still early, so I thought it'd be a good topic for a ML discussion. The two issues that have come to my attention represent two similar kinds of tests: a) Inspect the specific commit(s) under test (e.g., inspect the commit message). b) Inspect the delta of applying the patch (e.g., run coverage twice and calculate the difference). Simon Westphahl has proposed a patch focused on the first case, which adds a zuul.commits variable with the sha(s) of the patch(es) under test: https://review.openstack.org/539975 Meanwhile, here is an example of a test which relies on the existence of an origin remote: http://logs.openstack.org/58/473658/19/check/openstack-tox-cover/0f6849c/jo… We also have the minor example of pip emitting warnings if there is no origin remote; if we can fix that here, that's fine, but I don't think it's significant enough to drive our design. The patch to add zuul.commits makes case (a) fairly simple -- you can have ansible iterate over the values in the variable. I think that case (b) might be a bit harder with zuul.commits -- specialized tooling would be needed to create the alternative repo state (ie, a new branch) in order to perform the git checkout or diff operations. Because in (b) we are focused more on the previous state than the commit, we can't just checkout zuul.commit, we need to checkout the parent of zuul.commit. However, if we were to create an origin remote, and populate it with the "previous" git repo state, (that is, either the current state of the tree, or that with the changes ahead of this one applied), it seems we can accommodate both (a) and (b) in an intuitive manner. You could get the values that would be put in zuul.commits by performing 'git diff origin/master..master', which addresses (a), and you could address (b) with 'git checkout origin/master'. Among the reasons we don't have any remotes on the git checkouts are: * What would the URL be? * What would the content be? * Concern about jobs altering the git repo under test by performing naïve and incorrect git operations. We've addressed the content question, that leaves the other two. Ideally, if we did this, I'd like to have a remote with no URL. However, while that may be possible, some investigation suggests that we may want to avoid that as it looks like it may be a bit of an edge case that not all tools or libraries would support correctly. However, if we set it to a bogus URL, that may be sufficient to satisfy the tooling, as well as prevent git repo manipulation errors. I'm inclined to favor this approach, as I think it addresses both cases and is fairly intuitive. Do folks think it would work? Are there things we would be missing as compared to the zuul.commits approach? Thanks, Jim

3 3

Zuul website
by corvus＠inaugust.com 01 Feb '18

01 Feb '18

Hi, We now have a website at https://zuul-ci.org/ The content is in git in the openstack-infra/zuul-website repo: https://git.openstack.org/cgit/openstack-infra/zuul-website/tree/ I put together the site using a CC-BY 3.0 template, and I expect that the content of that repo will be licensed CC-BY as well (as most of our docs-only repos are). I enthusiastically invite other folks to contribute to the website by proposing changes to that repo. Currently the zuul-core team has approval rights, but we can expand it as warranted -- I expect there's an overlapping group of folks who would like to contribute in this area. I am not emotionally attached to anything about the current website. Please feel free to propose changes to absolutely everything -- including replacing the site entirely, using a different template, title image, etc. What's there is a starting point, and an attempt to organize information and present it to folks in a way that helps them find the right resources quickly. I only ask that any changes try to keep this in mind: the site has many audiences, and we should think about how they all interact with it. The site itself is currently simply static HTML. You can view it on your workstation with no special tools. I looked into static site generators, and there are promising tools in this space, but ultimately I felt that until we get to the point where we want more than a handful of pages, the extra work isn't worth it. Some parts of the template are not in use in the current site. Most notable is the navigation bar. We can add those back in easily as warranted. You may want to visit the live demo of the template to see all of its offerings: https://html5up.net/arcana So as far as the infrastructure goes, I suggest that we continue with simple static pages for now, and when we get to the point where we know what kind of content we want to add that warrants more tooling, we adopt a static site generator then. I do think we may want to start publishing the zuul, nodepool, and zuul-jobs docs under this domain name as well. But we can do that in the same way we do with OpenStack -- publish the docs in a subdirectory, so they won't be part of the 'zuul-website' repo. We can start looking at adapting those jobs soon. Thanks, Jim

2 1

Feature branch merge update
by corvus＠inaugust.com 01 Feb '18

01 Feb '18

Hi, After merging the feature/zuulv3 branches of Nodepool and Zuul back into master, a number of open changes on the master branch remained from before the merge (and, in many cases, much older than that). Some of us have gone through to try to triage those changes. Some of them were implemented on the feature/zuulv3 branch or otherwise just don't apply anymore (a component was removed, or functions differently). These changes were abandoned with a short explanation of why, and an invitation to restore them if the, admittedly cursory, analysis was wrong. Some of them may still apply, but in those cases, almost certainly need to be rebased. These changes we marked with Workflow -1: Work in Progress, with the hope that the author will update them when time permits. At this point, all of the open changes on Zuul and Nodepool which are not marked WIP should be relevant to the current state of the tree, and should be reviewed. So if you've only been looking at a subset of open changes to avoid the older changes on master which may not be relevant, you should be able to remove that filter and consider all open, non-WIP changes. Thanks, Jim

1 0

Job parent branches
by corvus＠inaugust.com 24 Jan '18

24 Jan '18

Hi, We've run into an issue a few times where a complex job defined in a project repo doesn't run because it can't find a parent job that matches the branch. In all the cases I've seen so far, it's children of OpenStack's devstack job which are at issue. That's because the devstack job is defined in the devstack repo (an untrusted project repo) with branches. That means that it has implied branch matchers attached to it. We could, of course, turn those off, however, the intention is for the devstack job to be branched. Running the Queens release is not the same as running Ocata, so the different branch variants should be used under different circumstances. The problem occurs when someone wants to inherit from the devstack job in a different repo, and there is not a devstack branch variant which matches the branch of the child job. The way Zuul pieces together the final job that is run is to take the name of the job specified in the project-pipeline config, and then collect all the variants of that job which match the change. Each time it finds a match, it looks at the parent of that variant, and recursively collects all the variants of the parent which match the change (eliminating duplicates). If it can't find at least one variant of the parent which matches the change, the job is not run. The first time this came to my attention was for the tempest job. During development, the devstack job has only been defined on the master branch of devstack. The tempest job is in a repo with no branches, so it doesn't have an implied branch matcher. When any project invokes it, it will match the branch, and then Zuul will look for its parent job, 'devstack', and as long as the change is on the master branch, it will find it and run. However, if the change is on stable/ocata, and there is no devstack job defined in the stable/ocata branch of devstack, then it won't run. I think the solution to this case is to backport the devstack job to devstack's stable branches. That's the situation we will ultimately be in after our natural branching progression anyway, and we should treat this as a one-time cost of the transition to Zuul v3. The other two instances where this has come up are for projects which should run a child of the devstack job on a branch which does not, and never will, exist in devstack. When nodepool had a feature/zuulv3 branch, we were unable to run the devstack integration test job on that branch because of this. Ansible's default branch is 'devel', which means in order for us to run the shade integration test job on changes to Ansible, Zuul looks for a devstack variant which matches the 'devel' branch. In all cases, these issues could technically be solved by creating devstack job variants which matched these other branches. However, it should not be incumbent on the devstack maintainers to anticipate all of the branch names of other projects which might want to use the devstack job. We could create a devstack job variant with an explicit branch matcher which matches all branches, and treat the stable branch variants as deltas from that, however, that is more difficult to manage and doesn't fit our branch creation model. I can think of two solutions to this problem: 1) Have Zuul use a default branch when searching for parents with no matching branch variants. Every project in Zuul has a default branch (which can be specified in the 'project' config stanza, but the default default is 'master'). If, when searching for the parents of a job, Zuul finds no matching variants, it could perform the search a second time with the default branch of the parent job's project (all variants of a job are defined in the same repo, so we know which project the job is in, so there's only one default branch to search for each parent). 2) Allow someone creating a child job to specify a fallback branch for Zuul to use when searching for parent jobs. Essentially, we would modify the algorithm in the same way as the previous suggestion, except instead of determining the fallback branch based on the default branch of the parent job, we would let the child job supply the fallback branch. At each level of recursion as we walk up the inheritance path, a new fallback branch could be provided. I'm inclined to implement option (1) at least. That is very similar to what already happens today with git repo checkouts. I.e., if you did manage to run one of the jobs in the examples I described, you would get the master branch of devstack checked out in the job because of the existing fallback behavior. It makes sense that the job matching algorithm would be similar. And I think in many of the cases where this issue comes up, it would do the right thing automatically (certainly it would have in all the examples thus far). I'm on the fence about option (2). I see no reason not to add it other than perhaps it's unnecessary complexity. Maybe we should avoid implementing it until we find a compelling case for it? Or do folks anticipate a need for that now? Thanks, Jim

3 5

feature/zuulv3 branches merged into master
by corvus＠inaugust.com 18 Jan '18

18 Jan '18

Hi, Today we merged the feature/zuulv3 branches of Nodepool and Zuul into master. In order to delete the feature branch, now that it's no longer necessary, we need to close all of the changes on that branch. In order to do that, I have marked all of them as abandoned. I'd like to stress that this is a technicality and of course the changes themselves are still very much welcome. If you have a pending change to that branch, please re-propose the change to the master branch. If, when doing so, you could include a link to the previous version of the change, especially if previous reviews had been completed, that would be very helpful to reviewers. I'm including a summary of all the changes which were open on feature/zuulv3 shortly before I abandoned them. All 113 changes in status:open branch:feature/zuulv3 Number Subject Project Branch Topic Owner Updated C V W 383437 Increase build worker main loop delay nodepool feature/zuulv3 nodepool-zk James E. Blair 2017-12-18 1 -1 408951 Add command list into server.py zuul feature/zuulv3 launcher Joshua Hesketh 2017-12-18 -1 -1 426861 WIP: Add reporter for Federated Message Bus (fedmsg) zuul feature/zuulv3 426861 Paul Belanger 2017-12-18 -1 -1 429850 WIP: Re-enable test_time_database test zuul feature/zuulv3 enable-tests Paul Belanger 2017-12-18 -1 -1 446172 Add reconfigure_handler for executor logging zuul feature/zuulv3 Paul Belanger 2017-12-18 1 -1 450897 Re-enable test_timer_sshkey zuul feature/zuulv3 test-conflict Clint 'SpamapS' B 2017-12-18 -1 454826 [WIP] Integration: Get static nodes from nodepool zuul feature/zuulv3 Cullen Taylor 2017-12-18 -1 456090 Set the git sshkey for the connection zuul feature/zuulv3 zuulv3 Joshua Hesketh 2017-12-18 -1 -1 465852 Add Dockerfile nodepool feature/zuulv3 Tobias Henkel 2017-05-23 -1 1 -1 465912 Add Dockerfile zuul feature/zuulv3 Tobias Henkel 2017-12-28 2 1 -1 466071 Switch from testrepository to stestr zuul feature/zuulv3 stestr Clint 'SpamapS' B 2017-09-21 1 -1 468670 Re-enable test_merge_conflict_reports zuul feature/zuulv3 re-enable-tests Clint 'SpamapS' B 2017-09-21 2 -1 471175 WIP: Status branch protection checking for github zuul feature/zuulv3 branch-protection Jamie Lennox 2017-12-22 1 473811 Add build.started state flag zuul feature/zuulv3 streamer_testing Monty Taylor 2017-12-18 1 -1 -1 473985 Only prepend hostname on multi-node plays zuul feature/zuulv3 streamer_testing Monty Taylor 2017-12-18 1 -1 478675 Add ability to auto-generate simple one-line shell playbooks zuul feature/zuulv3 Monty Taylor 2017-12-18 -1 -1 480843 Ensure build.start_time is defined zuul feature/zuulv3 Tristan Cacqueray 2017-09-21 -1 1 481134 Add v3 update slides zuul feature/zuulv3 480759 Monty Taylor 2017-12-18 1 -1 487538 Use yarn and webpack to manage zuul-web javascript zuul feature/zuulv3 zuul-javascript-t Monty Taylor 2017-12-21 2 1 -1 528295 ├─Add babel transpiling enabling use of ES6 features zuul feature/zuulv3 zuul-javascript-t Monty Taylor 2017-12-21 2 1 528296 │ └─Add StandardJS linting and analysis zuul feature/zuulv3 zuul-javascript-t Monty Taylor 2017-12-21 2 1 528373 │ └─Fix source_url handling for jobs view zuul feature/zuulv3 zuul-javascript-t Monty Taylor 2017-12-21 2 1 528297 │ └─Fix StandardJS warnings and turn them to errors zuul feature/zuulv3 zuul-javascript-t Monty Taylor 2017-12-21 2 1 528298 │ └─Add bundle analysis to the lint target zuul feature/zuulv3 zuul-javascript-t Monty Taylor 2017-12-22 2 1 529193 │ └─Inject url endpoint information zuul feature/zuulv3 zuul-javascript-t Monty Taylor 2017-12-21 2 1 528374 │ └─Make bundle of build web content zuul feature/zuulv3 zuul-javascript-t Monty Taylor 2017-12-21 2 -1 528437 │ └─Remove use strict zuul feature/zuulv3 zuul-javascript-t Monty Taylor 2017-12-21 2 -1 529686 ├─Add nodeenv and plumb it in to tox zuul feature/zuulv3 zuul-javascript-t Monty Taylor 2017-12-21 -1 529690 └─Add web server URL tests zuul feature/zuulv3 James E. Blair 2017-12-21 -1 494655 WIP Replace paste/webob with aiohttp zuul feature/zuulv3 aiohttp Monty Taylor 2017-09-21 -1 497962 Add zookeeper-server to bindep for Red Hat nodepool feature/zuulv3 Monty Taylor 2017-08-25 -1 -1 499224 Keep a cache of branches for projects and update on events zuul feature/zuulv3 github-ratelimit Monty Taylor 2017-09-21 -1 -1 499969 Add /node-list to the webapp nodepool feature/zuulv3 Tristan Cacqueray 2018-01-09 2 1 500254 └─Add /label-list to the webapp nodepool feature/zuulv3 Tristan Cacqueray 2018-01-09 2 1 531390 └─Refactor status functions, add web endpoints, allow params nodepool feature/zuulv3 500254 Matthieu Huin 2018-01-09 1 531703 └─Add a separate module for node management commands nodepool feature/zuulv3 500254 Matthieu Huin 2018-01-09 1 531718 └─webapp: add optional admin endpoint nodepool feature/zuulv3 500254 Matthieu Huin 2018-01-10 1 500159 Always generate the ARA report, even on failure zuul feature/zuulv3 zuulv3 David Moreau Sima 2017-09-21 1 -1 502468 Emit a message to the job log if ansible crashes zuul feature/zuulv3 Monty Taylor 2017-09-21 -1 503166 Don't gather host keys for non ssh connections nodepool feature/zuulv3 windows-support Tobias Henkel 05:42 AM -1 504112 └─Add connection-port to provider diskimage nodepool feature/zuulv3 windows-support Tobias Henkel 05:45 AM -1 504238 Delete IncludeRole object from result object for include_role tas zuul feature/zuulv3 David Moreau Sima 2018-01-09 1 -1 504267 Move github webhook from webapp to zuul-web zuul feature/zuulv3 zuul-web Jesse Keating 2017-12-11 1 1 504464 Consume server.id from shade create exception nodepool feature/zuulv3 Monty Taylor 2017-12-18 2 -1 504526 Add SELinux type enforcement zuul feature/zuulv3 zuulv3 Tristan Cacqueray 2017-09-21 1 504809 WIP: add abstract job attribute zuul feature/zuulv3 James E. Blair 2017-09-21 -1 505419 Disable action and lookup plugins from 2.4 zuul feature/zuulv3 zuulv3 Monty Taylor 2017-09-21 2 -1 505430 └─Port in changes from ansible 2.4 command module zuul feature/zuulv3 zuulv3 Monty Taylor 2017-09-21 -1 506871 Add restricted-node-labels global limits zuul feature/zuulv3 zuulv3 Tristan Cacqueray 2017-11-07 1 508774 Add noop job description zuul feature/zuulv3 zuul-web Tristan Cacqueray 05:11 PM 2 -2 1 508793 WIP add repl zuul feature/zuulv3 James E. Blair 2017-10-02 508960 Add memory awareness to system load governor zuul feature/zuulv3 Clint 'SpamapS' B 05:03 AM 2 1 509294 Add jinja filters for manipulating zuul.projects zuul feature/zuulv3 Monty Taylor 2017-10-03 -1 509484 Log exceptions from MODULE FAILURE more consistently zuul feature/zuulv3 Monty Taylor 2017-10-05 2 -1 509531 Provide resource_name in logging as structured data nodepool feature/zuulv3 zuulv3 Monty Taylor 2017-12-18 -1 -1 509903 Remove references to pipelines, queues, and layouts on dequeue zuul feature/zuulv3 zuulv3-fixes James E. Blair 2017-10-06 -1 -1 510301 Add a CRD multi-branch test zuul feature/zuulv3 James E. Blair 2017-10-07 -1 512130 Enabled ssh retries for ansible zuul feature/zuulv3 zuulv3-fixes Paul Belanger 2017-10-17 2 1 -1 513003 WIP: Add script for deterministic key generation zuul feature/zuulv3 Tobias Henkel 2017-10-18 -1 513368 Add regex support to project stanzas zuul feature/zuulv3 Tobias Henkel 2018-01-05 1 513387 Add node list to webapp nodepool feature/zuulv3 Tobias Henkel 2017-10-19 1 513885 Refactor Zuul job to use lightweight Vue components zuul feature/zuulv3 cleanup-status Mohammed Naser 2017-12-12 1 1 513886 └─Switch change list to use Vue.js zuul feature/zuulv3 cleanup-status Mohammed Naser 2017-10-21 1 515169 Support autoholding nodes for specific changes/refs zuul feature/zuulv3 autohold-changese Krzysztof Klimond 2017-12-14 1 516920 builder: do not cleanup image for driver not managing image nodepool feature/zuulv3 nodepool-drivers Tristan Cacqueray 2018-01-17 2 1 468624 └─Implement a static driver for Nodepool nodepool feature/zuulv3 nodepool-drivers Tristan Cacqueray 2018-01-17 -1 526325 └─Refactor run_handler to be generic nodepool feature/zuulv3 nodepool-drivers Tristan Cacqueray 2018-01-17 1 532450 └─Refactor NodeLauncher to be generic nodepool feature/zuulv3 nodepool-drivers Tristan Cacqueray 2018-01-17 1 468753 └─Implement an OpenContainer driver nodepool feature/zuulv3 nodepool-drivers Tristan Cacqueray 2018-01-17 -2 -1 521356 └─Implement a Kubernetes driver nodepool feature/zuulv3 nodepool-drivers Tristan Cacqueray 2018-01-17 1 528982 └─Implement an Amazon EC2 driver nodepool feature/zuulv3 nodepool-drivers Tristan Cacqueray 2018-01-17 1 517067 Add BaseSource.getProjectReadonly and refactor zuul feature/zuulv3 517078 Clint 'SpamapS' B 2017-11-03 -1 -1 517078 └─Do not add invalid projets via the /keys API zuul feature/zuulv3 517078 Clint 'SpamapS' B 2017-11-16 2 -1 518279 mqtt: add basic reporter zuul feature/zuulv3 zuulv3 Tristan Cacqueray 2017-11-29 2 1 519222 Add client unit testing zuul feature/zuulv3 enqueue-ref Ian Wienand 2017-12-06 2 1 519596 Allow run to be list of playbooks zuul feature/zuulv3 Paul Belanger 2017-11-20 -1 1 519654 Override HOME environment variable in bubblewrap zuul feature/zuulv3 bwrap-set-home Clint 'SpamapS' B 2017-11-16 -1 -1 520657 Run image object autocleanup after uploading images nodepool feature/zuulv3 Monty Taylor 2017-11-29 -1 520664 WIP: Convert from legacy to native devstack job nodepool feature/zuulv3 devstack_native David Shrewsbury 2018-01-02 1 520855 Ensure the merger prunes on update zuul feature/zuulv3 fetch-prune Ian Wienand 2017-11-20 -1 1 521625 Add angular to fetch-dependencies.sh zuul feature/zuulv3 Monty Taylor 2017-11-21 -1 1 523640 zk: use kazoo retry facilities nodepool feature/zuulv3 zuulv3 Tristan Cacqueray 05:28 AM -2 1 523697 executor: add log_stream_port and log_stream_file settings zuul feature/zuulv3 zuulv3 Tristan Cacqueray 2017-11-30 -1 1 524463 Remove zuul._projects zuul feature/zuulv3 project-transitio Ian Wienand 2017-12-01 -1 -1 524773 handler: fix support for handler without launch_manager nodepool feature/zuulv3 nodepool-drivers Tristan Cacqueray 2018-01-17 2 1 524620 └─Add a plugin interface for drivers nodepool feature/zuulv3 nodepool-drivers Tristan Cacqueray 02:53 AM 1 525851 zk: automatically retry command when connection is lost zuul feature/zuulv3 zuulv3 Tristan Cacqueray 2017-12-27 -2 1 527579 web: add /{tenant}/jobs/{job_name} route zuul feature/zuulv3 zuul-web Tristan Cacqueray 2017-12-13 -1 1 528588 Use bandit security scanner zuul feature/zuulv3 bandit Clint 'SpamapS' B 2017-12-17 1 528729 requirements: remove paramiko <2.0 cap zuul feature/zuulv3 zuulv3 Tristan Cacqueray 2018-01-05 2 1 528969 config: add statsd-server config parameter nodepool feature/zuulv3 zuulv3 Tristan Cacqueray 2018-01-15 2 -1 529013 web: add OpenAPI documentation zuul feature/zuulv3 zuulv3 Tristan Cacqueray 2017-12-27 2 529060 Make Zuul able to start with a broken config zuul feature/zuulv3 529060 Fabien Boucher 2018-01-08 1 529293 Set remote url on every getRepo in merger zuul feature/zuulv3 remote-url Tobias Henkel 2018-01-08 1 529821 Add winrm certificate handling zuul feature/zuulv3 windows-support Tobias Henkel 2018-01-17 1 530521 Slack driver zuul feature/zuulv3 slack-reporter Clint 'SpamapS' B 2018-01-10 -2 1 531009 Allow Ansible 2.4 zuul feature/zuulv3 ansible24 David Shrewsbury 2018-01-11 -2 1 531057 license: remove dos line break nodepool feature/zuulv3 zuulv3 Tristan Cacqueray 2018-01-04 -1 531171 WIP Rework log streaming to use logging zuul feature/zuulv3 zuul-stream-rewor Monty Taylor 2018-01-04 -1 531510 Delete stale jobdirs at startup zuul feature/zuulv3 James E. Blair 2018-01-12 -1 -1 531742 Also prefix the indexes if needed zuul feature/zuulv3 prefix-indexes Tobias Henkel 2018-01-08 1 531934 Add job for testing against devel of ansible zuul feature/zuulv3 turn-on-ansible Monty Taylor 2018-01-16 1 532023 Fix sql reporting with postgres zuul feature/zuulv3 fix-postgres Tobias Henkel 2018-01-09 1 532310 Switch to python3-dev / python3-devel in bindep.txt zuul feature/zuulv3 Paul Belanger 2018-01-10 2 1 532718 scheduler: better handle format status error zuul feature/zuulv3 zuulv3 Tristan Cacqueray 2018-01-11 1 533256 Add DIB faq to docs nodepool feature/zuulv3 dib-faq Clark Boylan 2018-01-12 1 533509 ansible: honor command no_log module attribute zuul feature/zuulv3 zuulv3 Tristan Cacqueray 2018-01-15 1 534190 Use re2 for change_matcher zuul feature/zuulv3 fix-zuul-re Clint 'SpamapS' B 2018-01-16 -1 534588 pep8: ignore E124 rule zuul feature/zuulv3 zuulv3 Tristan Cacqueray 2018-01-17 1 534956 Handle secrets in branches zuul feature/zuulv3 James E. Blair 2018-01-17 2 1 534974 └─Handle nodesets in branches zuul feature/zuulv3 James E. Blair 05:23 PM 2 1 534988 zk: check for client in properties nodepool feature/zuulv3 zuulv3 Tristan Cacqueray 02:20 AM 1 535423 WIP: Add provider info command nodepool feature/zuulv3 info_cmd David Shrewsbury 05:44 PM -1 Thanks for your understanding and your help, Jim

1 0

Secret handling in branches
by corvus＠inaugust.com 18 Jan '18

18 Jan '18

Hi, Currently our handling of secrets in branches is less than ideal. There are two problems: 1) A secret may not be used by a job in another branch of the same project. 2) A secret may not be defined in more than one branch of the same project. The first issue is inconvenient for users which want to, for example, upload an image to dockerhub for all of the branches of their project. The second prohibits the obvious fix for the first issue, but it also means that as soon as a project with a secret definition branches, Zuul will have a configuration error. I believe the original intent was not to restrict secrets to a single branch. If we accept that a secret should be available for use by any branch of a project, the fix for that is simple. The fix for the second issue is also not too difficult -- we can simply permit duplicate definitions, but only on different branches of the same project and with the same content. That allows us to create a new branch from an existing one without a configuration error. I've implemented these fixes in this change: https://review.openstack.org/534956 In an IRC conversation, we briefly considered supporting distinct secrets on multiple branches -- that is, a secret with the same name on multiple branches of the same project but with potentially different values on each branch, but that's a bit more difficult to reason about, and ultimately doesn't make this more secure. An important thing to note about the change is that by permitting the use of secrets across branches, if a project has different group maintaining the different branches, it could allow a secret to leak from one group to the other. In other words, if a project uses secrets, you need to trust the maintainers of all the branches of that project equally. It's also worth noting that even in the current situation, where we (perhaps accidentally) only allowed the secret to be used within a single branch, it is equally trivial for a maintainer of another branch of the same project to simply copy the secret to their branch under a different name and expose it, since all branches of a project share a single private key. Preventing this would require different keys for different branches, which would make branching impossible. I think the proposed solution is the best way of handling this, given the constraints. I know some folks would like to use secrets in a production-promotion system, where distinct groups have differing levels of access control. I think in those cases, those secrets and jobs will have to be located in distinct projects, not branches. Please let me know your thoughts and if this affects your use of secrets. -Jim

5 7

Zuul Setup - VPN or Environment with No Public Access
by Melvin Hillsman 18 Jan '18

18 Jan '18

Hey everyone, I wanted to start a discussion here of something I would like to setup in at least two environments and also consolidate into a tutorial/blog for others to build on/integrate. Please see image of what I have setup as one of the environments. Unfortunately one of the things documentation (could be I just have not come across it) does not explain is the flow/dependencies upon each other of the different daemons. I know some folks tried to explain what could be done but I was still a little loss so hoping this thread can flush out the details. -- Kind regards, Melvin Hillsman mrhillsman(a)gmail.com mobile: (832) 264-2646

3 4