- Zuul-discuss - lists.opendev.org

An URLTrigger driver for Zuul
by Fabien Boucher 12 Feb '19

12 Feb '19

Hello, I would like to propose a solution to trigger jobs based on the state of an external resource accessible at an URL. The use case is quite common and actually Jenkins propose this kind of feature via the URLTrigger plugin [0]. I proposed two implementations of an URLTrigger driver for Zuul: - A trigger driver [1]: It is quite similar to the timer driver. The CronTrigger is used to provide the resource check rate. At each loop it checks the resource state and keep the state in a cache. When the cache state has changed the event is sent into the scheduler. - A require driver [2]: Idea was to rely on the timer driver to provide the resource check rate. Then attach a requirement in the pipeline via the require driver. The requirement is a RefFilter that match when the resource state change. Let me know what do you think about this URLTrigger idea. Regards, Fabien [0]: https://wiki.jenkins.io/display/JENKINS/URLTrigger+Plugin [1]: https://review.openstack.org/#/c/635567/ [2]: https://review.openstack.org/#/c/635554/

3 3

Zuul project update 2019-02-11
by corvus＠inaugust.com 11 Feb '19

11 Feb '19

Hi, Tristan has: Worked on config introspection and build a dashboard to list configuration relationship and identify unused items: https://review.openstack.org/#/q/topic:config-introspection Worked on displaying the job's hierarchy: https://review.openstack.org/#/q/topic:jobs-hierarchy Worked on improving the build/buildset zuul-web interface: https://review.openstack.org/#/q/topic:web-build Paul has been working on allowing job.run to be a list of playbooks: https://review.openstack.org/592160/ Tobias has been working on: Updating multi ansible spec: https://review.openstack.org/623927 Force merge broken config hardening: https://review.openstack.org/633514 Configurable clone timeout for supporting larger repos: https://review.openstack.org/633936 Worked on analysis of executor memleak. Initial try: https://review.openstack.org/634598 And I am continuing work on speculative docker images described in a previous mailing list post -- https://review.openstack.org/#/q/topic:docker-registry ********************************************************************** I believe the multi-ansible spec is ready for final review. I invite everyone with an interest to review https://review.openstack.org/623927 this week, and assuming there are no major changes required, I plan on merging that at the end of the week. It has also been suggested that this change may be significant enough to warrant a major version bump, to Zuul v4.0. When you review the spec, feel free to provide your thoughts on that as well. ********************************************************************** Help write the next update email by upadting https://etherpad.openstack.org/p/zuul-update-email Thanks, Jim

1 0

[all] Denver Open Infrastructure Summit Community Contributor Awards!
by Kendall Nelson 06 Feb '19

06 Feb '19

Hello Everyone! As we approach the Summit (still a ways away thankfully), its time to kick off the Community Contributor Award nominations[1]! For those of you that have never heard of the CCA, I'll briefly explain what they are :) We all know people in our communities that do the dirty jobs, we all know people that will bend over backwards trying to help someone new, we all know someone that is a savant in some area of the code we could never hope to understand. These people rarely get the thanks they deserve and the Community Contributor Awards are a chance to make sure they know that they are appreciated for the amazing work they do and skills they have. As always, participation is voluntary :) Nominations will close on April 14th at 7:00 UTC and recipients will be announced at the Open Infrastructure Summit in Denver[2]. Recipients will be selected by a panel of top-level OSF project representatives who wish to participate. Finally, congrats again to recipients in Berlin[3]! -Kendall Nelson (diablo_rojo) [1] https://openstackfoundation.formstack.com/forms/train_cca_nominations [2]https://www.openstack.org/summit/denver-2019/ <https://www.openstack.org/summit/berlin-2018/> [3] http://superuser.openstack.org/articles/openstack-community-contributor-awa…

1 0

Zuul 2019 planning
by Lauren Sell 05 Feb '19

05 Feb '19

Hi everyone, We’d like to work with the Zuul community on 2019 planning so we can best support and promote the project this year. If the mailing list is the best way to start the conversation, we can use the below etherpad, or have a discussion on this thread. We’re also happy to host a quick call if folks want to brainstorm in a high bandwidth setting. The main things we’re looking to document are the goals for the community this year (for example, attract more contributors or document more users in production or make Zuul easier to try out and use…) along with any key milestones, features, topics, etc. that we’ll want to communicate. Please take a look at the etherpad and add your thoughts over the next week: https://etherpad.openstack.org/p/zuul-2019-planning <https://etherpad.openstack.org/p/zuul-2019-planning> Thanks! Lauren

2 2

Superuser Award nominations open until March 22
by Claire Massey 05 Feb '19

05 Feb '19

Hey everyone The Superuser Award nominations are now open for the Denver Open Infrastructure Summit. March 22 is the deadline for submissions. The Superuser Awards recognize teams using open infrastructure to meaningfully improve business and differentiate in a competitive industry, while also contributing back to the open-source community. The awards will be given out during the Denver Summit. *This is a great opportunity to nominate companies that are using Zuul!* You can learn more about the Superuser Awards here: https://superuser.openstack.org/articles/superuser-awards-nominations-open-… <https://superuser.openstack.org/articles/superuser-awards-nominations-open-…>. Complete this form to nominate a team: https://openstackfoundation.formstack.com/forms/superuserawards_denver2019 <https://openstackfoundation.formstack.com/forms/superuserawards_denver2019> Thanks, Claire

1 0

Open Infrastructure Summit Denver - Community Voting Open
by Claire Massey 31 Jan '19

31 Jan '19

Hi everyone, Community voting for the Open Infrastructure Summit Denver CFP sessions is open! You can VOTE HERE <https://www.openstack.org/summit/denver-2019/vote-for-speakers>, but what does that mean? Now that the Call for Presentations (CFP) has closed, all submissions are available for community vote and input. After community voting closes, the volunteer Programming Committee members will receive the presentations to review and determine the final selections for Summit schedule. While community votes are meant to help inform the decision, Programming Committee members are expected to exercise judgment in their area of expertise and help ensure diversity of sessions and speakers. View full details of the session selection process here <https://www.openstack.org/summit/denver-2019/call-for-presentations/selecti…>. In order to vote, you need an OSF community membership. If you do not have an account, please create one by going to openstack.org/join <http://openstack.org/join>. If you need to reset your password, you can do that here <https://www.openstack.org/Security/lostpassword>. Hurry, voting closes Monday, February 4 at 11:59pm Pacific Time (Tuesday, February 5 at 7:59 UTC). Continue to visit https://www.openstack.org/summit/denver-2019 <https://www.openstack.org/summit/denver-2019> for all Summit-related information. REGISTER Register for the Summit <https://openinfrasummitandptgdenver.eventbrite.com/> before prices increase in late February! VISA APPLICATION PROCESS Make sure to secure your Visa soon. More information <https://www.openstack.org/summit/denver-2019/travel/#travel-support> about the Visa application process. TRAVEL SUPPORT PROGRAM February 27 is the last day to submit applications. Please submit your applications <https://www.openstack.org/summit/denver-2019/travel/#travel-support> by 11:59pm Pacific Time (February 28 at 7:59am UTC). If you have any questions, please email summit(a)openstack.org <mailto:summit@openstack.org>. Cheers, Claire

1 0

Using nodepool v3 with Jenkins
by Darragh Bailey 31 Jan '19

31 Jan '19

Hi, Hoping that we can make use of nodepool v3 with Jenkins to solve some of our existing problems, and at the same time move a bit closer to upgrading from zuul v2 -> v3. Going to outline our current problem, my initial thinking that resulted in some of my questions last night in IRC and some further thinking on it that may provide a way forward for us. Hoping that someone can chime in and identify if this is realistic or if we're heading down the wrong path before getting to far. Keep in mind that I'm only starting with nodepool, so I'm unsure if I even understand some of the concepts. e.g. does nodepool get a request for a particular type of node or is it infered by there being none of that type available in the queues in zookeeper? Current problem: We have some resources that are limited and are not directly accessible as workers for Jenkins, and so need a way to manage the allocation of resources to certain jobs. Some of these resources are provided by teams and some are from a set of resources managed by the CI team. So we probably have ~4 sets of these types of resources. At the moment this is done using some (too much) hackier with groovy, the Global Event Listener Plugin (GELP), and Lockable Resources plugin. The GELP allows us to hook into various points in Jenkins, such as when the job is scheduled but not yet running Jenkins (QUEUED). Unfortunately there doesn't appear to be any way with the gearman protocol to inspect what job is going to be assigned before it is picked up, so we can only do manipulation after the job has been scheduled and before it starts running. The data required to provide the job with the details required to use the limited resource is retrieved from vault and injected right before the job gets fully launched on Jenkins. This is done by creating lockable resource entries dynamically based on the number of the resources available, and then catching the event on when Jenkins allows the job needing this to start and injecting the required data into the environment. Unfortunately this means the queuing for the jobs requiring these limited resources occurs within Jenkins instead of Zuul because the job has still be scheduled even if it is not yet running. We've run into a number of problems resulting in workers getting allocated for jobs that don't get to start, and taking them away from jobs that could be run, and effectively lightly DOSing the system. Initial thoughts: So enter nodepool and the hope that if we move the management of both these resources and the workers out of Jenkins and into something a little more flexible, it can prevent the job from getting scheduled in the first place if not enough of the limited resources are available, and allow jobs that do not need these to be scheduled because a worker isn't getting assigned to the job that is queued waiting. Initially I was thinking of having two resources, making the one representing the limited resource a fake resource of some kind that Jenkins doesn't connect to, but as it appears that it's a requirement of the Jenkins Nodepool agent that nodes returned must be accessible to Jenkins, that doesn't seem to fly. Next idea was whether it would be possible to have a static pool and a another driver that could grab a node from the static pool if a request for the limited resource node type came in, which prompted some of my questions in IRC last night. Further thinking done: After spending a bit more time with the static nodepool driver, and depending on how it works (there are lots of gaps in my understanding), I'm wondering if what we need is to create a custom driver and place all of our static nodes in a internal pool inside of this driver. Additionally query our vault instance for a list of these limited resource groups and build some pools of those internally as well. And assign labels as part of the requirements for the jobs using these limited resources. Then instead of immediate registering all of the nodes with zookeeper, instead depending on the node label request that comes in, populate zookeeper with an entry containing suitable labels. If it is one requiring a limited resource identified by a name such as teamA-resources, the driver code can test if there are any not in use and if so create a node entry in zookeeper that contains labels "teamA-resources" and "teamA-resource-5". The second label can be used in Jenkins by our GELP code to determine which set of data to inject into the environment of the job scheduled on that node. Obviously there is still some hackier in there, but does a custom driver building on the idea of combining some of the ideas of the static driver along with the dynamic drivers where instead of launching a new node, it simply creates a node entry on demand using both a static node and details on what resource is required? Possibly I'm misunderstanding how nodepool works in this case, and maybe it just populates a queue in zookeeper and tries to keep some in there, which may mean that what needs to happen is that one entry allocating a node to each of the sets of limited resources needs to be added as entries to zookeeper queues by the driver with further entries added if there are enough resources available as those queues are consumed? Which is a bit different, but seems like it would still work to limit the amount of damage where multiple jobs were launched and required more of the limited resource than was available. Currently can end up seeing 3 queue instances for a single limited resource queued up in Jenkins, so as we migrate another case were this will be needed we could end up seeing a queue of 9-10. Seems nodepool might limit it to only losing one per team, until we can migrate to zuulv3 and use some of the mutex capabilities to handle this a bit better. Thoughts? Is this the best way to dig up as not digging isn't an option here... -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool"

2 4

Zuul jobs for container publishing
by corvus＠inaugust.com 31 Jan '19

31 Jan '19

Hi, We recently added a new pipeline to OpenStack's Zuul, and three new jobs and roles to the zuul-jobs repo to support container publishing. Zuul users may be interested in adopting both the pipeline and the jobs into their own workflows. The new pipeline is the "promote" pipeline. It runs jobs on changes after they merge, but unlike the "post" pipeline, it does not run with the resulting branch state after the merge -- it runs in the context of the change. This means that it's not suitable for building release artifacts, but that's not its purpose. Its purpose is to promote artifacts previously built in the gate pipeline to production. That is safe to do because anything built in the gate pipeline is built with the full future state of in-flight changes to all related projects and branches (even if it's built before they land). We build a lot of artifacts in the gate pipeline now, and then throw them away only to build them again in post. The promote pipeline is an attempt to avoid that waste and instead preserve artifacts built in gate and directly publish them -- but only if and when the changes for which they were built are merged. The first functional use of this system is in building container images and publishing them on Docker Hub. We have begun using this in openstack-infra to build container images of third-party applications, but this will work just as well and as easily for building images of applications in our infrastructure (for example, we are also starting to use this to publish Zuul images). To implement this, we wrote three jobs which are available now from zuul-jobs (and therefore can be used in any project or job): * build-docker-image [1] * upload-docker-image [2] * promote-docker-image [3] The first job should run in the check pipeline and simply runs "docker build" on the arguments passed to it. It is used to verify that the build succeeds. The second job should run in the gate pipeline. It uploads the image to Docker Hub into the final repository location, but it only does so with a single tag with the form "change_123456" where '123456' is the Gerrit change number for the change under consideration. This will mean that docker pulls won't accidentally fetch the image yet. But the image is staged in Docker Hub and is ready to be promoted if the change passes all its tests and merges. The third job runs in the promote pipeline. This job, unlike the others, only performs Docker Hub API calls and requires no build resources. Therefore it is a zero-node job in Zuul -- a job which only runs on the Zuul executor. This job finds a previously uploaded image with a "change_" tag, and re-tags it with any specified tags ("latest" by default, but you can also tag version numbers or anything else). It also cleans up unused "change_" tags so they don't clutter up the repository. The nice thing about zero-node jobs is that they start and finish very quickly (since they don't have to deal with resource allocation and contention). With this system, an image will be fully published within about a minute after the change merges. A complete example .zuul.yaml might look like this: - secret: name: myproject-dockerhub data: username: dockerhubuser password: !encrypted/pkcs1-oaep - DFlbrDM5eUMptMGIVMXV1g455xOJLi92UYF08Z2/JlIGu3t6v052o9FKlVyj1ZmpXs5+2 JTa5jHkLTvTsYs9fCaNcQc2nmViCyWNlbOMzjB17uiZOaYFNs1sMqZcUZbGEz7Y8ds6Qq NBXI10jWFPTah4QxUuBvUbT3vmjnUToCzexl5ZGhKgijcnROWfUsnlCdugpgoNIcPsUki zty5FotDihnrC8n8vIomVK6EClY38ty97pLrADzFDd+Cos/OUlvi2xooUhzx8Bn020rJA lqEU5v8LGXp5QkHx0MSDx6JY6KppJ/4p/yM+4By6l+A20zdcimxmgiNc9rMWPwDj7xsao m7NAZWmWqOO0Xkhgt6WOfugwgt9X46sgs2+yDEfbnI5ok8uRbAB/4FWj/KdpyXwhcf+O2 wEfhxLwDbAoGONQPjb4YcZmCXtmR7Qe5t+n2jyczWXvrbaBDUQP5a+YtVNN/xhmQ7D740 POlxv7bLxJAixzqaQ3d8Rz9ZEv6zzRuhWph32UQtZ1JxSNww+EvmXm2eEi2Q2z6pT1Cx/ j2OrFyA2GL/UJOVb15VHKF6bgHPHWJtpjPFhqdcvBhVute4BWB+KPcWH+y+apHN1enK3H tNJO9iqm34nKwSuj5ExmFw50LtwR5/9FyRuRPq/vBL+8y82v8FDmeYsBeobn5M= - job: name: myproject-build-image parent: build-docker-image description: Build Docker images. vars: &image_vars docker_images: - context: . repository: myproject/imagename - job: name: myproject-upload-image parent: upload-docker-image description: Build Docker images and upload to Docker Hub. secrets: - name: docker_credentials secret: myproject-dockerhub pass-to-parent: true vars: *image_vars - job: name: myproject-promote-image parent: promote-docker-image description: Promote previously uploaded Docker images. secrets: - name: docker_credentials secret: myproject-dockerhub pass-to-parent: true vars: *image_vars - project: check: jobs: - myproject-build-image gate: jobs: - myproject-upload-image promote: jobs: - myproject-promote-image And the pipeline definition looks like: - pipeline: name: promote description: | This pipeline runs jobs that operate after each change is merged in order to promote artifacts generated in the gate pipeline. manager: supercedent precedence: low post-review: True trigger: gerrit: - event: change-merged success: gerrit: {} mysql: failure: gerrit: {} mysql: If you find that you need jobs which behave slightly differently, you may be able to inherit from these jobs and add pre or post playbooks. Or if you need something particularly complicated, you can re-implement the jobs using the underlying roles in zuul-jobs, which have the same names an the jobs which invoke them. [1] https://zuul-ci.org/docs/zuul-jobs/jobs.html#job-build-docker-image [2] https://zuul-ci.org/docs/zuul-jobs/jobs.html#job-upload-docker-image [3] https://zuul-ci.org/docs/zuul-jobs/jobs.html#job-promote-docker-image [4] https://zuul-ci.org/docs/zuul/user/encryption.html -Jim

2 1

[summit] Open Infrastructure Summit Shanghai - Dates and Location
by Claire Massey 31 Jan '19

31 Jan '19

Hello Zuul team - We are excited to announce that following Denver, the Open Infrastructure Summit will be traveling to Shanghai, China the week of November 4, 2019. The Summit will be held at the Shanghai Expo Centre. Registration will be available soon. Sign up for updates[1] and reach out to summit(a)openstack.org with any questions. [1] https://www.openstack.org/summit/shanghai-2019 Cheers, Claire

1 0

Zuul project update 2019-01-28
by corvus＠inaugust.com 28 Jan '19

28 Jan '19

Hi, Over the past two weeks, Tristan has been working on the topic:freeze_job and got the "zuul-runner execute" command to run a zuul-job locally, as well as a buildset endpoint and a buildset page to show all the build result for a change. I've been working on image building jobs, and will continue to work on improvements to those this week (see two recent emails to list). Please help write next week's email by updating this etherpad: https://etherpad.openstack.org/p/zuul-update-email -Jim

1 0

Configurable autohold list of allowed build results.
by Gabriele Cerami 24 Jan '19

24 Jan '19

Hi, autohold currently allows users to examine a live environment to proceed with post-mortem examination. In some occasions, trying to find the root cause of a bug by just examining the logs and the timeline of the failed environment may prove too difficult, and the preferable way may be to catch the failure at its root exactly when the root cause happens. In a mechanism similar to breakpoints, we may want to stop the build at a certain point, the proceed manually to prod for weak spots. The only way to get this behaviour from zuul currently is to request an autohold, and add a step that injects a fake failure to the build. With [1] I'd like to suggest an alternative method, the list of build results that are allowed to put a node in hold state can be passed as argument, and not just hardcoded. In this way, the build may be configured to be brought forward up to a certain known to work point, without having to expressly fail, then begin the step by step manually debugging on the live environment. It may be best suitable for non voting, experimental jobs, or during the development of new jobs, but a useful option nonetheless. Any feedback welcome. Thanks. [1] https://review.openstack.org/632498

2 1

CFP Open Until January 23, Open Infrastructure Summit in Denver
by Claire Massey 22 Jan '19

22 Jan '19

Hi everyone, FYI - the CFP is now open for the first Open Infrastructure Summit <https://www.openstack.org/summit/denver-2019/> (formerly the OpenStack Summit) which will be held in Denver, Colorado April 29 - May 1, 2019. Wednesday, *January 23* is the deadline to Submit presentations <https://www.openstack.org/summit/denver-2019/call-for-presentations>. The Open Infrastructure Summit is organized by OSF and designed to be a place where open source infrastructure communities can come together and collaborate in the open. Zuul will have a large and prominent presence at the event so please submit talks to the CFP and plan to attend! SUBMIT YOUR PRESENTATION <https://www.openstack.org/summit/denver-2019/call-for-presentations> Important info: Based on previous Program Committee and attendee feedback, we have added / updated three Tracks: Security, Getting Started, and Open Development (previously Open Source Community). You can find the Track descriptions here <https://www.openstack.org/summit/denver-2019/summit-categories>. All of the OSF pilot projects —including Airship, Kata Containers, StarlingX and Zuul — will be front and center alongside other open source communities like Ansible, Cloud Foundry, Docker, Kubernetes, and many more. The Open Infrastructure Summit (formerly the OpenStack Summit), has evolved to recognize our diverse audience, and to signal to the market that the event is relevant for all IT infrastructure decision makers. If you’re interested in influencing the Summit content, apply to be a Programming Committee member <https://openstackfoundation.formstack.com/forms/openstackdenver2019_program…>*, where you can also find a full list of time requirements and expectations. Nominations will close on January 4, 2019. The content submission process for the Forum and Project Teams Gathering will be managed separately in the upcoming months. *OSF Staff will serve as the Programming Committee for the Getting Started Track. Denver Summit registration and sponsor sales are currently open. Learn more <https://www.openstack.org/summit/denver-2019> and email summit(a)openstack.org <mailto:summit@openstack.org> with any questions. Please email speakersupport(a)openstack.org <mailto:speakersupport@openstack.org> with any questions or feedback. Thanks, Claire

1 1

Two new Zuul ideas pass-to-parent: true and named project groups
by Monty Taylor 21 Jan '19

21 Jan '19

Hey all! I had an idea for a zuul feature to make the containers jobs (and jobs like them) easier. And then one that also came to mind to make some of our central jobs easier to deal with in openstack. First idea: Right now, secrets, for good reason, are only usable by jobs in their own repo. However, for use-cases like the containers job, or publishing to pypi, or readthedocs even - it's hard to have a job in zuul-jobs that is usable for doing those tasks because they all need secrets. The idea is to add a flag to the secret request, like this: - job: name: system-config-build-image parent: build-container-image abstract: true allowed-projects: openstack-infra/system-config - job: name: system-config-upload-image parent: system-config-build-image secrets: name: credentials secret: system-config-dockerhub allowed-projects: openstack-infra/system-config secrets: name: credentials secret: system-config-dockerhub pass-to-parent: true By passing the flag, we tell zuul to pass the secret variable file to every playbook in the inheritance chain above the job (but still not child jobs). Obviously this requires that the user of the job trust the parent job. In fact, maybe we even add a "warning" - so that if someone submits a job config to zuul with a pass-to-parent flag, it leaves an inline comment on the change (without a negative vote) warning about the implications of choosing to do that - that way reviewers can be sure they understand that they're trusting someone upstream from them. Second idea: Somewhere - maybe in config repo - but maybe in main.yaml - or maybe it's even just a normal global object - we allow for the creation of arbitrary named groups of projects. So - for instance, if we go for config-projects, something like: - project-group: name: openstack projects: - openstack/nova - openstack/swift - project-group: name: opendev projects: - openstack-infra/system-config - openstack-infra/bindep Then, we allow project-group entries to be used in allowed-projects, like this: - job: name: publish-openstack-sphinx-docs parent: publish-openstack-sphinx-docs-base final: true pre-run: post-run: - playbooks/sphinx/post.yaml allowed-projects: - project-group: openstack This would allow people (say AJaeger, or whoever) to maintain a list of "official" projects that can publish docs to docs.openstack.org ... but let any of those projects add the job, so that we didn't have to put the jobs into project pipelines in project-config. We could do a similar thing for container jobs. Say put this in system-config: - job: name: opendev-upload-image parent: opendev-build-image secrets: name: credentials secret: opendev-dockerhub allowed-projects: - project-group: opendev Then in bindep we have: - job: name: bindep-upload-image parent: opendev-upload-image final: true vars: &jinja-init_vars images: - target: bindepo repository: opendevorg/bindep The more I type things, the more I think there's no need to restrict the creation of named project groups. I could see openstack projects, like say kolla, wanting to create a project-group in one of their repos, so that they could put a secret into a job in a central repo for their dockerhub org, but use the job they make in each of their individual repos. Some of this could be done with tenants - but those are a bit more heavy-weight - and we already see that inside of OpenStack there are teams with many repos. There's also some similarity in problem solved between the two - both make for better reuse of jobs that needs privileges. But I think it's two different use-cases. One is about facilitating writing jobs in zuul-jobs (or other generic job locations) that need secrets to be passed to them but are otherwise generic and parameterizable. The second is about having discreet jobs that have specific secrets that should only be used in a subset of projects - but where it's possible that a given subset of projects might get access to a group of discreet credentialed jobs and managing membership in the group makes more logical sense than managing project-pipeline assignment. It also allows for having child-jobs of discreet credentialed jobs (like the container example) Anywho - what do people think? Monty

4 4

Infra team upgrading review.openstack.org Gerrit from 2.13.9 to 2.13.12 January 18 at about 1700UTC
by Clark Boylan 18 Jan '19

18 Jan '19

We will be performing a minor Gerrit upgrade to version 2.13.12 tomorrow (January 18, 2019) at 1700UTC. We've tested this upgrade on our dev server, https://review-dev.openstack.org, and expect it to be a quick upgrade. Any outage shouldn't last more than 10 minutes. We will let our configuration management tooling manage the upgrade so we won't have an exact time, but will try to get it as close to 1700UTC as possible. Feel free to test out the new version on the dev server. We are happy to answer any questions you might have as well. Sorry for the short notice and the cross post, Clark

1 0

Discrepancies in zuul-jobs
by Matthieu Huin 17 Jan '19

17 Jan '19

Hello, There seems to be a few discrepancies in the way some roles are implemented in zuul-jobs. The main problems I have encountered are: [1] discrepancies in dependencies management: for example the upload-to-pypi job assumes twine is installed on the build host, and will fail if twine is missing; whereas the npm-related jobs install nodejs and npm in the playbooks. [2] discrepancies in OS support: some of the npm-related jobs assume they're running on Debian or Ubuntu (explicit call to apt to install dependencies), other jobs go with bindep. Concerning [1], maybe guidelines for writing jobs and the kind of assumptions that can be made about execution environments should be set; I don't remember seeing any in the documentation but let me know if I am wrong. Anyhow, should we assume that jobs will handle pre-requisites and dependencies or should this be left up to the node images maintainers? Concerning [2], the Software Factory is working on expanding third party CI on zuul-jobs by adding coverage on CentOS and Fedora. This is actually how I encountered the problem with the npm jobs. This will help detect these kinds of problems. As with [1], guidelines should be issued about OS support of zuul-jobs. If you think this is the way to go I can open a review to start a section on guidelines for writing jobs in the documentation. WDYT ? Cheers, MHU

5 6

Zuul featured in Open Infrastructure Community Newsletter
by Claire Massey 17 Jan '19

17 Jan '19

Hi everyone, Today, OSF distributed the next Open Infrastructure Community Newsletter. The bi-weekly newsletters provide a digest of the latest developments and activities across open infrastructure projects, events and users. This week, we highlighted the latest with *Zuul* as well as a brief update about the Open Infrastructure Summit and additional OSF project news. You can checkout the full newsletter on Superuser [1], and if you are interested in receiving the upcoming newsletters, you can subscribe here [2]. If you would like to contribute or have feedback, please reach out to community(a)openstack.org <mailto:community@openstack.org>. Thanks, Claire [1] http://superuser.openstack.org/articles/inside-open-infrastructure-1-16/ <http://superuser.openstack.org/articles/inside-open-infrastructure-1-16/> [2] https://www.openstack.org/community/email-signup <https://www.openstack.org/community/email-signup>

1 0

Is it possible to support merge-mode: rebase in zuul merger?
by Yang, Feng 3. (NSB - CN/Hangzhou) 15 Jan '19

15 Jan '19

Hello Zuul experts, Happy New Year! I didn’t find clues in the historical mail list. So please allow me to ask here. We are using zuul v2 for ~2 years, recently we encountered a special case: merged codes in zuul merger is NOT the same with gerrit rebase codes. On zuul side, we are using default mode ‘git merge -s resolve’ On Gerrit repo side, we are using “rebase if necessary” strategy. Then we are thinking if it is possible to add a new merge-mode: rebase into zuul merger. Did you plan to add rebase strategy into zuul before? Or do you have any suggestions? I really appreciate it. Thanks. Looking forward your comments. Thank you in advance. 😊 Thanks Br, Finn

4 4

Zuul project update 2019-01-14
by corvus＠inaugust.com 15 Jan '19

15 Jan '19

Hi, Today I approved our formal governance document[1], after receiving votes from all of the current maintainers. Thanks, everyone, for helping to create that! David is working on nodepool image build timeouts[2]. We made a Zuul release last week, and are about to make a Nodepool release. Please help write next week's email by updating this etherpad: https://etherpad.openstack.org/p/zuul-update-email [1] https://zuul-ci.org/docs/zuul/governance.html [2] https://review.openstack.org/629923 -Jim

1 0

Zubbi 1.0.0 released
by Felix.FS.Schmidt＠bmw.de 14 Jan '19

14 Jan '19

Hello, With the start of the new year we are happy to announce that Zubbi 1.0.0 is now available on PyPI: https://pypi.org/project/zubbi/. The new Zubbi version provides support for Gerrit repositories. As opposed to GitHub, Zubbi yet doesn't support event handling for Gerrit. Thus, Gerrit repositories can only be scraped periodically. During the OpenStack summit, the question for a test instance came up. So we deployed a Zubbi demo showing the jobs and roles from the openstack-infra/zuul-jobs repo at http://ad43b5505180211e99b0002e3f6a3815-573606705.eu-west-1.elb.amazonaws.c…. Feel free to explore Zubbi :-) We hope that you like the demo and are interested in setting up a Zubbi on your own. For this case, we've added a quickstart guide in our README, explaining how to set up Zubbi with a minimal set of configuration (the same as the demo). In case you have any further questions or need help for setting up Zubbi, don't hesitate to contact us or create an issue in the GitHub repository: https://github.com/bmwcarit/zubbi. We would be happy to see a Zubbi for your Zuul setups :-) Best regards Felix

1 0

Zuul project update 2019-01-07
by corvus＠inaugust.com 08 Jan '19

08 Jan '19

Hi, Welcome back from the holidays! Fabian has been working on a driver for Pagure (https://pagure.io/ ) and now has a usable implementation here: https://review.openstack.org/#/c/604404/ I have continued work on artifact URL storage in the db in service of building a 'promote' pipeline; I should be able to have jobs and roles demonstrating it soon. I'd like to call everyone's attention to three items in review with wide-ranging impact; it would be good for as many folks as possible to review these: Governance document: https://review.openstack.org/622439 Ansible multi-version support spec: https://review.openstack.org/623927 Scheduler scale-out spec: https://review.openstack.org/621479 Please help write next week's email by updating this etherpad: https://etherpad.openstack.org/p/zuul-update-email -Jim

2 1

AW: AW: Different queues across branches?
by Mike Pagel 29 Dec '18

29 Dec '18

Happy holidays to everyone! I wrote: > In that case however, having multiple queue names is an > erroneous condition. Just wanted to wrap this up. I realize that raising an exception (or similar behavior) to complain about this condition will break existing installations, which is not acceptable. In particular not for this kind of thing. Nevertheless the discussion helped me to understand the behavior in my setup. Thanks a lot, Mike -----Ursprüngliche Nachricht----- Von: Mike Pagel [mailto:mike@mpagel.de] Gesendet: Donnerstag, 20. Dezember 2018 18:42 An: 'zuul-discuss(a)lists.zuul-ci.org' <zuul-discuss(a)lists.zuul-ci.org> Betreff: AW: AW: Different queues across branches? > Yes -- when creating shared change queues from projects, the first > appearance of a queue name is used, and this is true for any branch. > So, unlike jobs definitions and invocations, a project's membership in a shared queue is global and immutable. > It doesn't change based on branch, or the change under test. It is simply a characteristic of the project itself. I understand and I accept that this is the current state. In that case however, having multiple queue names is an erroneous condition. Depending on what "first" actually means I could imagine scenarios, where setting a different queue name in a branch can remove my project from the original queue used before on master and essentially break master's gating, because other (required) projects are still using the original queue name... Would you agree? Would it make sense to guard against this? Where would that kind of checking code go? What kind of exception should be thrown? Where would I find an example of such kind of configuration check code and associated tests? Mike

1 0

Different queues across branches?
by Mike Pagel 20 Dec '18

20 Dec '18

Hey zuulers, we recently had to create a maintenance branch across all repositories required to build the project. Since the two branches no longer affect each other in terms of a green build I intended to decouple the gating process of master and branch by changing the gate's queue name in the branch. I was expecting the branch gates to be executed in the new queue and the master gates being executed in the original one. However, it looks like all gates are still happening in the same original queue. Did I expect something wrong? In case it matters, we are using implied branch matching for our jobs. Thanks a lot, Mike

3 8

How to define zuul.executor.inventory_file on a localhost
by Akhil Jayakumar (akjayaku) 19 Dec '18

19 Dec '18

Hi All, Our openstack cinder ci jobs are failing due to below error. Do you know where we need to define zuul.executor variable “inventory_file”? Appreciate your help. 2018-12-18 06:48:58.993454 | TASK [log-inventory : Copy ansible inventory to logs dir] 2018-12-18 06:48:59.045029 | primary | ERROR 2018-12-18 06:48:59.045544 | primary | { 2018-12-18 06:48:59.045637 | primary | "msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'inventory_file'\n\nThe error appears to have been in '/tmp/03e62a72bdbc4480937f15e45000abb3/trusted/project_1/git.zuul-ci.org/zuul-jobs/roles/log-inventory/tasks/main.yaml': line 11, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n# https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.4.ht… name: Copy ansible inventory to logs dir\n ^ here\n" 2018-12-18 06:48:59.045713 | primary | } 2018-12-18 06:48:59.048055 | 2018-12-18 06:48:59.048294 | PLAY RECAP 2018-12-18 06:48:59.048470 | primary | ok: 18 changed: 10 unreachable: 0 failed: 1 Thanks, Akhil.

3 3

Zuul project update 2018-12-17
by corvus＠inaugust.com 18 Dec '18

18 Dec '18

Hi, The past two weeks in the Zuul project have seen: David discovered an issue in node deletion that was leaving empty ZooKeeper nodes and added code to nodepool to clean those up automatically. He also fixed a race in the test_handler_poll_session_expired nodepool test. Tobias worked on spec for scale out scheduler (what we've been informally calling Zuul v4 -- where we likely connect all components via Zookeeper and remove our use of Gearman). He also fixed an issue with github status checks. Paul added nodepool.node_id support for job inventory files (so that performance problems in jobs can be traced back to the hypervisor on which they ran). And I've continued the relative-priority work (including a change to group projects in check as they are grouped in gate), as well as a revision to our proposed governance document. A number of us are likely to be less active over the next few weeks. I don't plan to send another status update email until January 7. Please help write the next update email by updating this etherpad: https://etherpad.openstack.org/p/zuul-update-email -Jim

3 2

Creation of OSF Project Confirmation Guidelines
by Claire Massey 14 Dec '18

14 Dec '18

Hi everyone, As you know, Zuul is a pilot project supported by the OpenStack Foundation (OSF). As such, we wanted to make sure you’re aware of an effort recently kicked off a sub group of the OSF Board of Directors to draft guidelines[1] for confirming pilot projects as full, top-level, open infrastructure project of the Foundation. The current pilot projects that will be eligible for confirmation review in 2019-2020 are Airship, Kata Containers, StarlingX and Zuul. Currently the OpenStack project is the only confirmed project. The discussion about drafting these guidelines is just getting started, and *we invite you to participate in this effort* by reviewing the working draft in this etherpad and adding in your feedback and thoughts starting on line 139: https://etherpad.openstack.org/p/BrainstormingOSFProjectConfirmationGuideli… <https://etherpad.openstack.org/p/BrainstormingOSFProjectConfirmationGuideli…> There will most likely be an open call scheduled in early 2019 where additional feedback will also be welcomed. I will share that invitation with you once it’s schedule. The goal is to come up with a set of points that the Board can use when reviewing pilot projects for confirmation. This topic has previously been discussed at a very high-level at previous Board meetings, including the September 18th meeting[2] (starting around slide 38). 1. https://etherpad.openstack.org/p/ProjectConfirmationGuidelines <https://etherpad.openstack.org/p/ProjectConfirmationGuidelines> 2. https://docs.google.com/presentation/d/10UyCpxkjPqC3kT-dYRpBxzNT39i2OhlggJv… <https://docs.google.com/presentation/d/10UyCpxkjPqC3kT-dYRpBxzNT39i2OhlggJv…> Thanks, Claire

1 0