May 2018 - Zuul-discuss - lists.opendev.org

Toggle gzip in upload-logs
by Paul Belanger 12 Oct '18

12 Oct '18

Greetings, I've proposed a patch to upload-logs[1] that will toggle the compressing of our console logs files. The main change is, by default we no longer compress them. Jobs would need to set zuul_log_compress: true to work as we do today. The main reason for this, is to reduce the amount of extra work a user needs to view our console logs in apache / nginx. Today, if compressed, the browser will try to download the file not render it. I think we could add documentation explaining how to enable compression and which settings would be needed for apache / nginx but in this case, seems like an optimization over necessity. Thoughts? [1] https://review.openstack.org/567696/

3 4

Spec for a tenant-scoped admin web API
by Matthieu Huin 26 Sep '18

26 Sep '18

Hello, Following Jim's example with the containers spec, I've opened a spec review in the zuul repository: https://review.openstack.org/562321 I've been toying around with adding a few "admin" endpoints to zuul and nodepool for some time now [1][2]. Using Software Factory's authorization engine to limit access to these endpoints to specific users, and consuming germqtt's event feed to react to comments on a review, I demonstrated PoCs for features that I think would be of interest to Zuul users in general: * Ability to re-enqueue builds from zuul's web UI * Ability to trigger a autohold by commenting "autohold <job> on <tenant>" on gerrit review The aforementioned patches were meant as experiments; now I would like to go further and work to have this integrated into Zuul in a way that would be tenant-scoped. Before I rework these extensively I seek your guidance and opinions on how to proceed with this; hence this spec review. There are surely things I have overlooked, and my proposal to use groups to implement policies might need discussing. Please let me know what you think, also if it is the right place for the spec (the review on infra-specs is still open) MHU [1] https://review.openstack.org/536319 [2] https://review.openstack.org/539004

1 1

Nodepool driver interface
by David Shrewsbury 13 Sep '18

13 Sep '18

Greetings all, We just merged the last piece needed [1] for all of Tristan's various experimental nodepool drivers. This has reminded me of an idea that I've had in the back of my mind for a while now. Currently, there are reviews up for Kubernetes, Amazon EC2, Ansible, OpenContainer, VMWare, and Azure. I'd like for us to at least discuss the possibility of a pluggable driver system so that the drivers can be developed independently from nodepool, rather than in-tree to nodepool. Why? A. Having them in-tree leads to the assumption that they are tested. We do not have access to all of these services, so actively testing them within nodepool is not (currently) possible. B. Nodepool reviewers will be responsible for maintaining the drivers. I, myself, do not have experience with these services, nor access to them, nor the desire or time to learn some of them. I would venture a guess that most of us share some of these deficiencies. This puts us in a spot of just guessing at fixes, or just ignoring them hoping someone else will pick up the ball. C. With plugins, users could get driver updates/fixes without waiting for a nodepool release. I think keeping the OpenStack driver in-tree makes sense because we actively test and use it. I can even get on board with keeping the static driver too. But the others, to me, would be best kept out of tree *unless* we can add access to those services to our Zuul environment and actively test them as well (that at least takes care of the A reason above, and most of B if we have excellent tests setup). Thoughts? [1] https://review.openstack.org/535555 -- David Shrewsbury (Shrews)

7 9

Upgrade to Ansible 2.5
by corvus＠inaugust.com 05 Jun '18

05 Jun '18

Hi, We have some patches ready to upgrade Zuul to use Ansible 2.5, but there is some concern about whether there may be incompatibilities with playbooks and roles in zuul-jobs. I know some folks are running with local patches to use later Ansible versions -- if you are one of them, and you are using zuul-jobs, please let me know so we have an idea of what to expect. How do folks feel about merging the upgrade-to-2.5 changes and releasing those as Zuul 3.1.0 -- say, next Thursday, May 10? If we do that, and we find that there are incompatibilities, we will need to begin merging fixes to zuul-jobs immediately. Hopefully, however, those fixes can use the "ansible_version" variable in order to be backwards compatible, so zuul-jobs continues to work for folks who haven't upgraded yet. Finally, we hope to soon have a spec detailing multi-version support for Zuul, so that future upgrades can be handled more gracefully. -Jim

5 8

Templates in job vars
by Ian Wienand 29 May '18

29 May '18

Hi, In templating out some diskimage-builder jobs, there is a situation where we pass a list of tests to run, but some platforms can't run various tests. So something like --- - job: name: base abstract: yes ... vars: dib_functests: - gentoo - debian - job: name: test-on-bionic parent: base - job: name: test-on-xenial parent: base vars: dib_functests: - gentoo - debian - suse # also run suse, redefine whole list --- That the base list of tests is repeated to override the dib_functests is a little ugly. I would prefer if the base tests that all platforms can run was defined only once (so if it's updated, you can't miss one). To my mind, it would be better if I could jinja on the variable, so I could add (or remove) a test; above it would become like: # also run opensuse tests on xenial vars: dib_functests: "{{ dib_functests + ['opensuse'] }}" I ended up just appending a second variable [1], but it feels a bit hacky (if you had more levels, you'd need to keep appending more and more variables). Seeing as I'm pretty sure jinja templates are Turing complete, I can also see issues with expanding each vars: entry with it :) Terrible idea? Already better ways I don't know about? Cheers, -i [1] https://review.openstack.org/#/c/566779/60/roles/dib-functests/defaults/mai…

2 1

Zuul project update 2018-05-14
by corvus＠inaugust.com 14 May '18

14 May '18

Hi, Shrews is beginning implementation of multi-label support in nodepool as part of his continuing work on better integrating the static driver. I spent most of last week and will continue to spend most of this week preparing for the upcoming OpenStack Summit and OpenDev events. However, at the end of the week, I needed a break from that, so I started work on replacing our use of aiohttp with cherrypy. Aiohttp (and related libraries) have broken us a few times recently by rapidly dropping support for what they consider "older" python versions -- yet versions that are still under support on the operating systems we use. Also, several of us have found the python async implementation unnecessarily complex and verbose. Cherrypy supports a thread-based model which fits in well with the rest of Zuul. I don't plan on sending this email next week due to the Summit, so expect the next installment on May 28. Please help write that email by updating this etherpad: https://etherpad.openstack.org/p/zuul-update-email Thanks, Jim

1 0

Mission statement and tagline
by corvus＠inaugust.com 09 May '18

09 May '18

Hi, The Foundation marketing team has indicated that it would be good to have a mission statement and tagline. The mission statement can be used to help convey what we're trying to accomplish as a community, and the tagline will likely feature prominently on the website and other material. I'd like us all to be involved in creating these, but we need to have the process finished by May 10 in order to be able to update marketing material before the Summit/OpenDev. I propose the following process: * We make (and refine) suggestions this week. Add suggestions to https://etherpad.openstack.org/p/zuul-mission and discuss in the etherpad or here on the mailing list. Feedback from the zuul-core team and the Foundation marketing team (some of whom are subscribed to this list) will be important to formulate good candidates. * Next Monday, the zuul-core team and Foundation marketing team veto any suggestions which are inappropriate. I don't think it's fair to have a mission statement the core team doesn't agree with, and we should trust the marketing folks as domain experts to help us avoid pitfalls. Email me with veto requests; I'll produce a final list. * If there isn't a clear favorite, well have a condorcet election with the remaining candidates, to be complete by May 10. I'll run the election. * If we've vetoed everything, we'll have a special IRC meeting to come up with *something*. To start things off, I've populated the etherpad with a mission statement suggestion from Clark, and some of the favored tagline suggestions from earlier brainstorming. -Jim

2 2

Zuul Stickers
by James Cole 08 May '18

08 May '18

Hey everyone! I just wanted to share a pic of the stickers we’re giving away at the Summit in Vancouver. A link is below in case you can’t see the attachment. https://www.dropbox.com/s/dwb8df3wuqey5i6/Zuul_Stickers.jpg?dl=0 Come grab one in Vancouver! James Cole Graphic Designer OpenStack Foundation

1 0

Polling users of Nodepool's static node driver
by David Shrewsbury 08 May '18

08 May '18

Hi, Is anyone currently using the static driver? And if so, are you making us of the multi-label feature it supports for the static nodes? I ask because I've been trying to change the behavior of the driver to pre-populate ZooKeeper with the static nodes (more in line with how the OpenStack driver works), rather than only having the ZK nodes appear when the node is actually in-use. Doing so fundamentally changes how the static driver works, and this deep dive of the code has revealed this multi-label usage which I think many of us (certainly me) have missed. Changing the driver to operate more like how we expect will mean changing nodepool to support multiple labels across *all* drivers. That's not a simple change, and affects other things, such as statsd reporting (which reports per-label stats). (And if you were wondering how the static driver was getting away with supporting multiple labels before, it was the fact that it *didn't* pre-register its nodes with ZK that allowed that.) One one hand, I can see how supporting multiple labels might be useful (e.g., a node labeled with both ubuntu-xenial and ubuntu-latest). But I don't see a great benefit to doing so. If no one is depending on this multi-label behavior, I'd like to remove it from the static driver. -Dave -- David Shrewsbury (Shrews)

4 6

Zuul project update 2018-05-07
by corvus＠inaugust.com 07 May '18

07 May '18

Hi, This past week has seen the following: Shrews began attempting to fix nodepool static driver to pre-register static nodes in ZooKeeper. This led to the discussion in [1] I pushed patches to add our new logo to the website and docs, as well as some other changes to the website suggested by the marketing team. The OpenDev conference and OpenStack summit are fast approaching. I have a lot to prepare before then -- both in terms of preparing for talks as well as polishing the website and documentation as much as possible, so I'm probably going to focus on those for the next two weeks rather than take up any significant new development work. Please help write next week's email by updating this etherpad: https://etherpad.openstack.org/p/zuul-update-email Thanks, Jim [1] http://lists.zuul-ci.org/pipermail/zuul-discuss/2018-May/000197.html

1 0