- Zuul-discuss - lists.opendev.org

nodepoolcmd + secure.conf
by Paul Belanger 25 Mar '19

25 Mar '19

Greetings, I wanted to discuss maybe reverting the following change[1]. On friday I updated our local nodepool.yaml configuration to move zookeeper-servers section into secure.conf. This was mostly to prepare for the upcoming patch adding SSL support for zookeeper. However, because 480310 removed the default value of the -s flag, I now find myself having to type the following everytime I want to interact with nodepool: $ nodepool -s /etc/nodepool/secure.conf list $ nodepool -s /etc/nodepool/secure.conf dib-image-list ... I actually don't think a full revert is the best case here, as that would mean ever nodepool install would need to add /etc/nodepool/secure.conf, even if empty to have the CLI command not raise an exception for missing file. I also don't think silently failing, if /etc/nodepool/secure.conf is missing is the best way forward either as this could mask deployment issues. Does anybody have an idea how we could fix this to be a little more user friendly? Thanks! - Paul [1] https://review.openstack.org/480310/

2 2

Speculative docker images
by corvus＠inaugust.com 22 Mar '19

22 Mar '19

Hi, As you know, with respect to git repositories, Zuul frequently operates well into the future, allowing jobs to build software with changes that haven't landed yet under the assurance that they will. As more of us have incorporated container image building into our processes, we find that we want the same thing there -- that not only would our git repositories represent the future that Zuul sees, but also our container image registries. As a concrete and simple example, we have an integration job called 'zuul-quick-start' which uses both Nodepool and Zuul container images. Those images themselves depend on build images created from the openstack-infra/system-config repository, which they use via a "FROM" line in Dockerfiles, and extending the zuul-quick-start job to be able to rebuild and use those builder images in the case of a Depends-On change is too complex to consider. In particular, the "FROM" line is problematic, as the only way to realistically use a speculative image as a base image is to alter the Dockerfile or make the image available in a registry. Therefore, this is a proposal for a system for building speculative images which can then be used by further speculative images. Every item in a Zuul pipeline has its own future state which is related to, but separate from all the other items. It needs to be able to build an image for a project and store it in a registry under the final published name and tag, without affecting any of the changes ahead of it. Therefore, each item needs its own image registry. In the case of a single, simple job such as zuul-quick-start, we can just run a docker registry on the node on which the job is running. In the case of a more complicated system where we want to build an image once and use it in multiple dependent jobs in a job graph, we will start an image registry at the root of the job graph, pause the job, and the dependent jobs will use it. Now that we have the registry for the change, we need to have that registry reflect the state of any changes ahead. We need to know when the change ahead has finished building its images so that we can then add those images into our change's registry. To accomplish this, we can extend Zuul so that jobs have a provides/requires interface. If our job requires docker images produced by a job in a dependent change, we can say: - job: name: zuul-quick-start requires: docker-images - job: name: system-config-build-image-python-builder provides: docker-images Zuul will see that zuul-quick-start should not start until all of the jobs for all of the changes ahead which provide "docker-images" have completed. Then zuul-quick-start will be able to retrieve images from the preceding changes' jobs and add them to its own registry. In the gate, the behavior of this is obvious: if a change has a job with a requirement, iterate over all changes ahead of it in the shared queue and search for jobs on those changes which provide that requirement. If there are none, or all of them have completed or paused, then that job is ready to start. Otherwise, wait. In an independent pipeline, we do include dependent changes ahead in the shared queue, but they are not live -- they don't run any actual jobs. So in that case, we're going to have to peek outside our shared queue and see if our dependent changes show up anywhere else in the same pipeline, and if they do, wait for their jobs. If they don't show up, then we know they completed earlier, and it's safe to start. There's one more thing about this change that will help us later: a job that "provides" something should be able to pass data to a job which "requires" it. So our image build jobs can use zuul_return to provide information about which images they built so that it's easier for later jobs to import those images into their registries. Once again, the independent pipelines make this tricky, as since those jobs may have run some time in the past, the scheduler may have forgotten the data they returned. However, once the SQL database is required, we can rely on that to fill in any missing data. That's the only change we need to Zuul itself to support this system. The rest is all playbooks and roles. That brings us to the challenge of how to actually transfer these artifacts between items. I can think of two approaches. They are not mutually exclusive -- we can implement both if we want. They both have pros and cons. But I would like input from Zuul operators (including openstack-infra/opendev) as to which they would prefer, and I'll implement that approach in OpenDev for Zuul's own upstream use. If we have a split decision, someone else can implement the second approach. I think we can support both in zuul-jobs, but I'd like us to run the most widely applicable approach upstream. Alternative 1: Use the Logserver ================================ In the case of a simple job, at the end of the job, or in the case of a complex job graph, a child job dependent on the completion of the image build process (so we don't have to wait for all the jobs to complete), we will export the images to the filesystem and upload their layers (minus any layers which are already on Docker Hub, in order to save space) to the logserver. This next bit is a little complicated, but remember that this will all be encapsulated in roles so we only have to implement it once; the user interface will resemble "roles: [fetch-dependent-images]". At the end of the image build job, for every image that we built: Export it to a directory on the filesystem. For each layer in the image: Perform a HEAD request against the Docker Hub registry to see if the layer exists in the target repository. If that exists, layer exists in docker, remove from filesystem. rsync image directories to logserver. zuul_return image location on logserver. At the start of a job which requires images: For each previous image (as provided by zuul_return): Rsync image directory from logserver to filesystem. For each layer in image manifest: If layer is missing from filesystem: Get the layer from Docker Hub. Import image into local registry. The advantages of this approach: * No external operational dependencies (i.e., you don't have to run a docker registry). The disadvantages of this approach: * Images can't easily be fetched and used for debugging. * Implementation is slightly more complex. Alternative 2: Use a Central Registry ===================================== In this approach, we would run a central registry to store speculative images. It would not store final images at all, and every image uploaded to it would be tagged only with a tag like "pipeline_check_change_1234_original_tag" to uniquely identify its build context. At the end of the image build job, for every image that we built: Copy it from the per-change registry to the central registry, and tag it with pipeline and change id. zuul_return image location and tag. At the start of a job which requires images: For each previous image (as provided by zuul_return): Copy it from the central registry to the local registry, and tag it with the original tag. The advantages of this approach: * Developers can fetch images from the central registry by pulling them with the crazy tag. * Implementation is a little easier, and uses more standardized tools. The disadvantages of this approach: * An external service is required. * We will need to prune old images from the registry. Conclusion ========== I don't think this system will take long to implement with either of the alternatives. Please let me know what you think of the system, and of the storage/transfer alternatives. -Jim

4 4

Parameterized Build
by Tristan Cacqueray 20 Mar '19

20 Mar '19

Hello, It seems like it would help the migration of Jenkins users if they could have a similar feature[0] in Zuul to manually trigger a job with custom parameters. One solution is to create a fake change or branch to trigger a job, but perhaps Zuul can offer a better user experience. In particular, the Jenkins implementation offers a nice form to set the parameters. I started to implement this feature with a jobtrigger driver[1] along with a web page[2] to expose the job's variables but James asked to describe the use case first. Our current use case is for QA/QE engineers who would like to run a single (usualy long running) job with custom parameters such as the product version, the system version, ... With Jenkins, we can have lots of jobs grouped as multijob running in different phases, and the parameterized build is a very convenient way to run a single job, sometime using a parameter to add a breakpoint and get access to the environment. Another solution would be to build on top of the zuul-runner to provide a similar workflow, but it seems like more efficient if we could re-use the zuul-scheduler and let it request/cleanup the nodes. In summary, could we adjust the architecture to support executing an individual job instead of an entire job graph and could we support a zuul-web trigger? Regards, -Tristan [0]: https://wiki.jenkins.io/display/JENKINS/Parameterized+Build [1]: https://review.openstack.org/555153 [2]: https://review.openstack.org/554839

3 9

Re: Zuul 3.7.0 released
by Jimmy McArthur 20 Mar '19

20 Mar '19

Congrats to the whole Zuul community :) > James E. Blair <mailto:corvus@inaugust.com> > March 20, 2019 at 3:33 PM > The Zuul team is pleased to announce the release of Zuul 3.7.0. > > Download it from PyPI: https://pypi.org/project/zuul > > The release notes, included below, are also available in the > documentation with hyperlink references: > > https://zuul-ci.org/docs/zuul/releasenotes.html#relnotes-3-7-0 > > Upgrade Notes > ============= > > * The default ansible version is now 2.7. In case this breaks any jobs > this can be overridden in job.ansible-version, > tenant.default-ansible-version or scheduler.default_ansible_version > > * In order to support several ansible versions installed simultaneously > Zuul now handles them itself in virtual environments. By default Zuul > installs the needed ansible versions on startup so there is no further > user action required. However it is recommended to pre-install the > ansible environments during installation by invoking > zuul-manage-ansible. > > * The user value in the [fingergw] configuration section previously > defaulted to zuul, but now is unset by default, which will cause > fingergw not to drop privileges. It is recommended that this value be > explicitly set to an unprivileged user. > > * The hosts value in the [zookeeper] configuration section previously > defaulted to localhost:2181, but now is unset by default. This value > is required and must be explicitly set (and the documentation has > always indicated this). > > New Features > ============ > > * Jobs may now specify which ansible version is used to run them. The > ansible version to use can now be specified by job.ansible-version. > > * Zuul now supports ansible 2.6 for running jobs. > > * Zuul now supports ansible 2.7 for running jobs. > > * The ansible_python_interpreter variable is now whitelisted for the > add_host task. > > _______________________________________________ > Zuul-announce mailing list > Zuul-announce(a)lists.zuul-ci.org > http://lists.zuul-ci.org/cgi-bin/mailman/listinfo/zuul-announce

1 0

Re: Migrating git repos to OpenDev
by corvus＠inaugust.com 20 Mar '19

20 Mar '19

corvus(a)inaugust.com (James E. Blair) writes: > Hi, > > As discussed in November[1], the OpenStack project infrastructure is > being rebranded as "OpenDev" to better support a wider community of > projects. > > We are nearly ready to perform the part of this transition with the > largest impact: moving the authoritative git repositories for existing > projects. Working with the folks representing each of the affected OSF projects, we've scheduled the migration for Friday, April 19, 2019. This is shortly after the next OpenStack release. Please see [2] for further details about what this will involve. [1] http://lists.openstack.org/pipermail/openstack-dev/2018-November/136403.html [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.ht… -Jim

1 0

BoF/WG Space at Open Infrastructure Summit Denver
by Jimmy McArthur 19 Mar '19

19 Mar '19

Does the Zuul Community have any interest in BoF or Working Group space in Denver? For those unfamiliar, Working Groups are informal groups of people sharing a common interest for a task or code subsystem. Birds of a Feather is an effort to interest new community members to your project through a presentation and Q&A format. Each session type is 40 minutes. Cheers, Jimmy

4 11

Zuul Board Approval Presentation
by Jimmy McArthur 18 Mar '19

18 Mar '19

Hi All - I'm working with Zuul on their pending project confirmation on behalf of the OpenStack Foundation. As you may or may not know, Zuul will need to do a presentation in front of the OpenStack Foundation Board. I've started an etherpad [1] with a rough outline, but ultimately this should be a community driven effort. If you are able to fill in some blanks, we welcome your contribution! Cheers, Jimmy [1] https://etherpad.openstack.org/p/Zuul_2019_OSF_Project_Confirmation_Process

1 0

Zuul generic jobs
by Tomasz Szandała 13 Mar '19

13 Mar '19

Greetings, I have a project, that requires compilation for different targets. Same commands, but each one has to be done separately. I have seen how OpenStack does it: https://github.com/openstack-infra/openstack-zuul-jobs/blob/master/zuul.d/j… so we define dozens of jobs separately even if they differ only by single element. My idea was to define job compilation, in which it is Ansible playbook that determines which target it should choose, based on job name: (compilation.yaml - playbook) - name: Set hardware and technology set_fact: hardware: "{{ zuul.job.split('.')[1] }}" technology: "{{ zuul.job.split('.')[2] }}" Now in project definition I can have something like this: - project: verification: jobs: - compilation.x86_64.old: parent: compilation - compilation.i686.old: parent: compilation - compilation.i686.future: parent: compilation But Zuul says, that compiation.x86_64.old is not defined. In other words: I want to define job inside project definition. This will mean that adding new variant I can only modify only 2 lines. Is there any reason why it is impossible? Kind regards, Tomasz

7 9

Zuul project update 2019-03-11
by corvus＠inaugust.com 12 Mar '19

12 Mar '19

Hi, Since the last email update, we merged the Multi-Ansible spec: https://zuul-ci.org/docs/zuul/developer/specs/multiple-ansible-versions.html Tristan worked on adding a job filter to the trigger event, as discussed previously on this list. The webtrigger patch is now based on that. Tristan also continued working on zuul-runner and a set of patches to support nodes with python3 only. I finally finished the changes related to buildset docker registries (also mentioned previously on this list). In the end, the system looks to be easy to use, but there are a few moving parts. My next task is to document how to set such a system up. ****************************** There are a lot of outstanding changes, and I'd like to attempt to prioritize and sequence them to help us work through them together: 1) The multi-ansible series While this is newer than the others on the list, it does address significant technical debt. It would be really good to get this finished and released before Ansible 2.5 goes out of maintenance (it's the oldest supported Ansible and only receives security fixes). 2) The zuul-runner series This has been in development for a long time, and has a significant impact on the executor. It will almost certainly conflict with the multi-ansible series, so let's work on getting this merged after that. 3) The web authentication spec This has been in development for a while, and folks working on it recently signaled it's ready for review. This shouldn't conflict with anything ahead of it, so we can review it as time permits. 4) The web authentication series Once the spec lands, we can review the implementation which has been developing in parallel. Of course, there's lots of other stuff out there we can review as well, and anyone is free to review anything they want. :) However, I think if we agree to prioritize these major efforts as I suggest, we will be able to focus on them together and make the most efficient progress. Help write the next update email by updating https://etherpad.openstack.org/p/zuul-update-email Thanks, Jim

5 5

404 - Tenant does not exist
by Tomasz Szandała 10 Mar '19

10 Mar '19

Greetings, I have set up zuul v3 using docker-compose, so on my host I can see following containers running: CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES b47be7107613 zuul/zuul-web "/usr/bin/dumb-init …" 12 hours ago Up 18 minutes 0.0.0.0:9000->9000/tcp docker_web_1 5b4fcd2a2238 zuul/zuul-executor "/usr/bin/dumb-init …" 12 hours ago Up 18 minutes docker_executor_1 4fe739ae6ace zuul/zuul-scheduler "/usr/bin/dumb-init …" 12 hours ago Up 7 minutes docker_scheduler_1 1c88df15bd55 zuul/nodepool-launcher "/usr/bin/dumb-init …" 12 hours ago Up 18 minutes 0.0.0.0:8022->8022/tcp docker_launcher_1 682c8f4ece17 zookeeper "/docker-entrypoint.…" 12 hours ago Up 18 minutes 2181/tcp, 2888/tcp, 3888/tcp docker_zk_1 3bcfaf604728 docker_logs "httpd-foreground" 12 hours ago Up 18 minutes 0.0.0.0:8000->80/tcp docker_logs_1 151eee2a9f4a mariadb "docker-entrypoint.s…" 12 hours ago Up 12 hours 3306/tcp Everything in real time works perfectly, but the problem is that I cannot see the builds history. My web service cannot get http://zuul-master.internal.com:9000/api/tenant/igor/buildsets nor any other /api call. It receives 404, tenant not found. I doesn't look like firewall issue, nor I see any errors in docker-compose logs. Any idea what did I set wrongly here? Kind regards, Tomasz

1 1

Re: [all] [forum] Forum Submissions are open!
by Kendall Nelson 27 Feb '19

27 Feb '19

Hello :) On Wed, Feb 27, 2019 at 1:26 PM Chris Morgan <mihalis68(a)gmail.com> wrote: > I think the issue is that forum submissions building on what gets > discussed in Berlin can't be expected to be finalised whilst attendees to > the berlin meetup are still traveling. It's not that Erik can't pull these > things together, in fact he's an old hand at this, it's more that this > process isn't reasonable if there's so little time to collate what we learn > in Berlin and feed it forward to Denver. Frankly it sounds like because the > planning committee needs 5 weeks, Erik can have two days. Seem unfair. > Honestly, the decision process doesn't take much time, aside from organizing a time that all 10 people can meet across x timezones (a thing unto itself). Its the community feedback period, giving people enough time to secure travel approval from their management, loading the sessions into the actual schedule app, and other print deadlines that force us to have everything set this far out. I will definitely help the ops community in whatever way I can! Do you have remote attendance set up for the meetup? > Chris > > On Wed, Feb 27, 2019 at 2:29 PM Kendall Nelson <kennelson11(a)gmail.com> > wrote: > >> Another- nother thought: You could take a look at what is submitted by >> project teams closer to the deadline and see if your ideas might fit well >> with theirs since they are looking for feedback from operators anyway. In >> the past I have always hoped for more engagement in the forum sessions I've >> submitted but only ever had one or two operators able to join us. >> >> -Kendall (diablo_rojo) >> >> On Wed, Feb 27, 2019 at 11:14 AM Kendall Nelson <kennelson11(a)gmail.com> >> wrote: >> >>> Hello :) >>> >>> On Wed, Feb 27, 2019 at 11:08 AM Jimmy McArthur <jimmy(a)openstack.org> >>> wrote: >>> >>>> Erik, >>>> >>>> I definitely understand the timeline is tight. One of the reasons that >>>> we publish the schedule so early is to enable community members to plan >>>> their schedule early, especially as there is more overlap with the main >>>> Summit Schedule in Denver. Additionally, travel approval is often >>>> predicated upon someone showing they're leading/moderating a session. >>>> >>>> Before publishing the schedule, we print a draft Forum schedule for >>>> community feedback and start promotion of the schedule, which we have to >>>> put up on the OpenStack website and apps at 5 weeks out. Extending the date >>>> beyond the 10th won't give the Forum Selection Committee enough time to >>>> complete those tasks. >>>> >>>> I think if the Ops team can come up with some high level discussion >>>> topics, we'll be happy to put some holds in the Forum schedule for >>>> Ops-specific content. diablo_rojo has also offered to attend some of the >>>> Ops sessions remotely as well, if that would help you all shape some things >>>> into actual sessions. >>>> >>> >>> I'm definitely happy to help as much as I can. If you'll have something >>> set up that I can call into (zoom, webex, bluejeans, hangout, whatever), I >>> definitely will. I could also read through etherpads you take notes in and >>> help summarize things into forum proposals. >>> >>> Another thing to note is that whatever you/we submit, it doesn't have to >>> be award winning :) Its totally possible to change session descriptions and >>> edit who the speaker is later. >>> >>> Other random thought, I know Sean McGinnis has attended a lot of the >>> Operators stuff in the past so maybe he could help narrow things down too? >>> Not to sign him up for more work, but I know he's written a forum propsal >>> or two in the past ;) >>> >>> >>>> >>>> I wish I could offer a further extension, but extending it another week >>>> would push too far into the process. >>>> >>>> Cheers, >>>> Jimmy >>>> >>>> Erik McCormick <emccormick(a)cirrusseven.com> >>>> February 27, 2019 at 12:43 PM >>>> >>>> Jimmy, >>>> >>>> I won't even get home until the 10th much less have time to follow up >>>> with anyone. The formation of those sessions often come from >>>> discussions spawned at the meetup and expanded upon later with folks >>>> who could not attend. Could we at least get until 3/17? I understand >>>> your desire to finalize the schedule, but 6 weeks out should be more >>>> than enough time, no? >>>> >>>> Thanks, >>>> Erik >>>> >>>> Jimmy McArthur <jimmy(a)openstack.org> >>>> February 27, 2019 at 12:04 PM >>>> >>>> Hi Erik, >>>> >>>> We are able to extend the deadline to 11:59PM Pacific, March 10th. >>>> That should give the weekend to get any additional stragglers in and still >>>> allow the Forum Programming Committee enough time to manage the rest of the >>>> approval and publishing process in time for people's travel needs, etc... >>>> >>>> For the Ops Meetup specifically, I'd suggest going a bit broader with >>>> the proposals and offering to fill in the blanks later. For example, if >>>> something comes up and everyone agrees it should go to the Forum, just >>>> submit before the end of the Ops session. Kendall or myself would be happy >>>> to help you add details a bit later in the process, should clarification be >>>> necessary. We typically have enough spots for the majority of proposed >>>> Forum sessions. That's not a guarantee, but food for thought. >>>> >>>> Cheers, >>>> Jimmy >>>> >>>> >>>> _______________________________________________ >>>> Airship-discuss mailing list >>>> Airship-discuss(a)lists.airshipit.org >>>> http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss >>>> >>>> Erik McCormick <emccormick(a)cirrusseven.com> >>>> February 27, 2019 at 11:31 AM >>>> Would it be possible to push the deadline back a couple weeks? I expect >>>> there to be a few session proposals that will come out of the Ops Meetup >>>> which ends the day before the deadline. It would be helpful to have a >>>> little time to organize and submit things afterwards. >>>> >>>> Thanks, >>>> Erik >>>> >>>> Jimmy McArthur <jimmy(a)openstack.org> >>>> February 27, 2019 at 10:40 AM >>>> Hi Everyone - >>>> >>>> A quick reminder that we are accepting Forum [1] submissions for the >>>> 2019 Open Infrastructure Summit in Denver [2]. Please submit your ideas >>>> through the Summit CFP tool [3] through March 8th. Don't forget to put >>>> your brainstorming etherpad up on the Denver Forum page [4]. >>>> >>>> This is not a classic conference track with speakers and presentations. >>>> OSF community members (participants in development teams, operators, >>>> working groups, SIGs, and other interested individuals) discuss the topics >>>> they want to cover and get alignment on and we welcome your participation. >>>> The Forum is your opportunity to help shape the development of future >>>> project releases. More information about the Forum [1]. >>>> >>>> If you have questions or concerns, please reach out to >>>> speakersupport(a)openstack.org. >>>> >>>> Cheers, >>>> Jimmy >>>> >>>> [1] https://wiki.openstack.org/wiki/Forum >>>> [2] https://www.openstack.org/summit/denver-2019/ >>>> [3] https://www.openstack.org/summit/denver-2019/call-for-presentations >>>> [4] https://wiki.openstack.org/wiki/Forum/Denver2019 >>>> ___________________________________________ >>>> >>>> >>> Hopefully that helps! >>> >>> -Kendall (diablo_rojo) >>> >> > > -- > Chris Morgan <mihalis68(a)gmail.com> > - Kendall Nelson (diablo_rojo)

1 0

Re: [all] [forum] Forum Submissions are open!
by Jimmy McArthur 27 Feb '19

27 Feb '19

Erik, I definitely understand the timeline is tight. One of the reasons that we publish the schedule so early is to enable community members to plan their schedule early, especially as there is more overlap with the main Summit Schedule in Denver. Additionally, travel approval is often predicated upon someone showing they're leading/moderating a session. Before publishing the schedule, we print a draft Forum schedule for community feedback and start promotion of the schedule, which we have to put up on the OpenStack website and apps at 5 weeks out. Extending the date beyond the 10th won't give the Forum Selection Committee enough time to complete those tasks. I think if the Ops team can come up with some high level discussion topics, we'll be happy to put some holds in the Forum schedule for Ops-specific content. diablo_rojo has also offered to attend some of the Ops sessions remotely as well, if that would help you all shape some things into actual sessions. I wish I could offer a further extension, but extending it another week would push too far into the process. Cheers, Jimmy > Erik McCormick <mailto:emccormick@cirrusseven.com> > February 27, 2019 at 12:43 PM > Jimmy, > > I won't even get home until the 10th much less have time to follow up > with anyone. The formation of those sessions often come from > discussions spawned at the meetup and expanded upon later with folks > who could not attend. Could we at least get until 3/17? I understand > your desire to finalize the schedule, but 6 weeks out should be more > than enough time, no? > > Thanks, > Erik > Jimmy McArthur <mailto:jimmy@openstack.org> > February 27, 2019 at 12:04 PM > Hi Erik, > > We are able to extend the deadline to 11:59PM Pacific, March 10th. > That should give the weekend to get any additional stragglers in and > still allow the Forum Programming Committee enough time to manage the > rest of the approval and publishing process in time for people's > travel needs, etc... > > For the Ops Meetup specifically, I'd suggest going a bit broader with > the proposals and offering to fill in the blanks later. For example, > if something comes up and everyone agrees it should go to the Forum, > just submit before the end of the Ops session. Kendall or myself > would be happy to help you add details a bit later in the process, > should clarification be necessary. We typically have enough spots for > the majority of proposed Forum sessions. That's not a guarantee, but > food for thought. > > Cheers, > Jimmy > > > _______________________________________________ > Airship-discuss mailing list > Airship-discuss(a)lists.airshipit.org > http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss > Erik McCormick <mailto:emccormick@cirrusseven.com> > February 27, 2019 at 11:31 AM > Would it be possible to push the deadline back a couple weeks? I > expect there to be a few session proposals that will come out of the > Ops Meetup which ends the day before the deadline. It would be helpful > to have a little time to organize and submit things afterwards. > > Thanks, > Erik > > Jimmy McArthur <mailto:jimmy@openstack.org> > February 27, 2019 at 10:40 AM > Hi Everyone - > > A quick reminder that we are accepting Forum [1] submissions for the > 2019 Open Infrastructure Summit in Denver [2]. Please submit your > ideas through the Summit CFP tool [3] through March 8th. Don't forget > to put your brainstorming etherpad up on the Denver Forum page [4]. > > This is not a classic conference track with speakers and > presentations. OSF community members (participants in development > teams, operators, working groups, SIGs, and other interested > individuals) discuss the topics they want to cover and get alignment > on and we welcome your participation. The Forum is your opportunity > to help shape the development of future project releases. More > information about the Forum [1]. > > If you have questions or concerns, please reach out to > speakersupport(a)openstack.org <mailto:speakersupport@openstack.org>. > > Cheers, > Jimmy > > [1] https://wiki.openstack.org/wiki/Forum > [2] https://www.openstack.org/summit/denver-2019/ > [3] https://www.openstack.org/summit/denver-2019/call-for-presentations > [4] https://wiki.openstack.org/wiki/Forum/Denver2019 > ___________________________________________ >

2 2

Re: [all] [forum] Forum Submissions are open!
by Jimmy McArthur 27 Feb '19

27 Feb '19

Hi Erik, We are able to extend the deadline to 11:59PM Pacific, March 10th. That should give the weekend to get any additional stragglers in and still allow the Forum Programming Committee enough time to manage the rest of the approval and publishing process in time for people's travel needs, etc... For the Ops Meetup specifically, I'd suggest going a bit broader with the proposals and offering to fill in the blanks later. For example, if something comes up and everyone agrees it should go to the Forum, just submit before the end of the Ops session. Kendall or myself would be happy to help you add details a bit later in the process, should clarification be necessary. We typically have enough spots for the majority of proposed Forum sessions. That's not a guarantee, but food for thought. Cheers, Jimmy > Erik McCormick <mailto:emccormick@cirrusseven.com> > February 27, 2019 at 11:31 AM > Would it be possible to push the deadline back a couple weeks? I > expect there to be a few session proposals that will come out of the > Ops Meetup which ends the day before the deadline. It would be helpful > to have a little time to organize and submit things afterwards. > > Thanks, > Erik > > Jimmy McArthur <mailto:jimmy@openstack.org> > February 27, 2019 at 10:40 AM > Hi Everyone - > > A quick reminder that we are accepting Forum [1] submissions for the > 2019 Open Infrastructure Summit in Denver [2]. Please submit your > ideas through the Summit CFP tool [3] through March 8th. Don't forget > to put your brainstorming etherpad up on the Denver Forum page [4]. > > This is not a classic conference track with speakers and > presentations. OSF community members (participants in development > teams, operators, working groups, SIGs, and other interested > individuals) discuss the topics they want to cover and get alignment > on and we welcome your participation. The Forum is your opportunity > to help shape the development of future project releases. More > information about the Forum [1]. > > If you have questions or concerns, please reach out to > speakersupport(a)openstack.org <mailto:speakersupport@openstack.org>. > > Cheers, > Jimmy > > [1] https://wiki.openstack.org/wiki/Forum > [2] https://www.openstack.org/summit/denver-2019/ > [3] https://www.openstack.org/summit/denver-2019/call-for-presentations > [4] https://wiki.openstack.org/wiki/Forum/Denver2019 > ___________________________________________ >

1 0

[all] [forum] Forum Submissions are open!
by Jimmy McArthur 27 Feb '19

27 Feb '19

Hi Everyone - A quick reminder that we are accepting Forum [1] submissions for the 2019 Open Infrastructure Summit in Denver [2]. Please submit your ideas through the Summit CFP tool [3] through March 8th. Don't forget to put your brainstorming etherpad up on the Denver Forum page [4]. This is not a classic conference track with speakers and presentations. OSF community members (participants in development teams, operators, working groups, SIGs, and other interested individuals) discuss the topics they want to cover and get alignment on and we welcome your participation. The Forum is your opportunity to help shape the development of future project releases. More information about the Forum [1]. If you have questions or concerns, please reach out to speakersupport(a)openstack.org <mailto:speakersupport@openstack.org>. Cheers, Jimmy [1] https://wiki.openstack.org/wiki/Forum [2] https://www.openstack.org/summit/denver-2019/ [3] https://www.openstack.org/summit/denver-2019/call-for-presentations [4] https://wiki.openstack.org/wiki/Forum/Denver2019 ___________________________________________

1 0

OpenInfra Day Vietnam 2019 - pre-CFP
by Trinh Nguyen 26 Feb '19

26 Feb '19

Hello, This is the pre-calling for presentations at the OpenInfra Day in Vietnam this year. If you love to visit Hanoi <https://en.wikipedia.org/wiki/Hanoi>, the capital of Vietnam, and share your passion for the Open Infrastructure of any topic (container, CI, deployment, edge computing, etc.), please let me know by replying to this email. Below is the tentative information of the event: - Date: 31 August 2019 - Location: Hanoi, Vietnam We are working with the OpenStack Foundation to organize the Upstream Institute at the day so this will be a great opportunity for potential contributors to come and learn. There is also a couple of PTLs and projects core members have shown their interest in visiting Hanoi for this event. We will send out the official call-for-presentations after we've done with the logistic vendors and It would be around the beginning of May or sooner. If you have any questions, please do not hesitate to contact me. See you in Hanoi :) Bests, On behalf of VietOpenInfra Group. P/S: Have a look at our last OpenInfra Day: https://2018.vietopenstack.org/ -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

1 0

Zuul project update 2019-02-25
by corvus＠inaugust.com 25 Feb '19

25 Feb '19

Matthieu is working on authenticated web API, and thus enabling tenant- or project-scoped privileged actions. The spec needs feedback, especially the authorization definition in Zuul: https://review.openstack.org/562321 There is also a review topic with WIP implementation of the current state of the spec: https://review.openstack.org/#/q/topic:zuul_admin_web Tristan worked on web and AMQP triggers: https://review.openstack.org/#/q/topic:webtrigger https://review.openstack.org/#/q/topic:amqp-trigger As well as adding python3 only node support: https://review.openstack.org/#/q/topic:python3-interpreter Fabien worked on an URLTrigger driver: https://review.openstack.org/635567/ I have gotten speculative docker images are working, but we need buildset node affinity between jobs and their paused parents to complete implementation, so I'll be working on that this week. -Jim

1 0

Escaping text in Zuul inventory file
by corvus＠inaugust.com 24 Feb '19

24 Feb '19

Hi, Recently Zuul was modified to include the change message (i.e., commit message in Gerrit, PR message in GitHub) as a Zuul variable: https://review.openstack.org/631207 Ansible applies jinja templating to the inventory file (which we rely on for variables in many jobs). If the change message incidentally included jinja escapes, then Ansible may fail to run due to syntax errors or undefined variables. To correct this, we agreed that allowing templating of the change message was not useful and should be disabled. An initial approach using "{% raw %}" / "{% endraw %}" was deemed too fragile (a change message with "{% endraw %}" in it would defeat the fix). So we have merged this change which uses an Ansible-specific YAML tag, "!unsafe", to mark the string as untrusted input which should not be templated: https://review.openstack.org/633930 This appears to be an elegant and complete solution. Yet before we release the next version of Zuul with this, I think we should consider possible ramifications a bit more. I can think of two things that including YAML tags in the inventory *might* cause problems with, and I'd like input on whether folks think these could be actual problems, or are, instead, easily overcome. And of course, if you think of other potential problems. First, the YAML tag may make it more difficult for users who perform automatic processing of inventory files collected after the job runs. I know of know examples of that myself, and it seems like it should be safe for us to say that if you're going to parse an Ansible YAML inventory file, you should be using a YAML parser that understands Ansible YAML tags. But perhaps someone knows of why this would be an undue hardship. The second is really a special case of the first. As our build page in the Zuul web UI continues to become more sophisticated, we might want to have it parse the inventory file much as it is now currently parsing the job-output.json file. I don't recall any discussions of this, but if we think it's likely, we should probably make sure that any Javascript YAML parser we might use could handle the tag. If we think that these issues, or any others that folks may think of, make it worth avoiding the !unsafe tag, we could solve the problem by base64 encoding the change message. Ansible has convenient jinja filters for base64 decoding, and in fact, Ansible modules such as slurp provide data in base64-encoded form. It would be easy for a job which needed to inspect the change message to do so by running it through the b64decode filter first. However, if we think these issues are not blockers, the !unsafe tag presents the simplest and most human-readable approach. -Jim

6 7

[all] Forum Submissions are Open!
by Kendall Nelson 22 Feb '19

22 Feb '19

Hello Everyone! We are now accepting Forum [1] submissions for the 2019 Open Infrastructure Summit in Denver [2]. Please submit your ideas through the Summit CFP tool [3] through March 8th. Don't forget to put your brainstorming etherpad up on the Denver Forum page [4]. This is not a classic conference track with speakers and presentations. OSF community members (participants in development teams, operators, working groups, SIGs, and other interested individuals) discuss the topics they want to cover and get alignment on and we welcome your participation. The Forum is your opportunity to help shape the development of future project releases. More information about the Forum [1]. The timeline for submissions is as follows: Feb 22nd | Formal topic submission tool opens: https://www.openstack.org/summit/denver-2019/call-for-presentations. March 8th | Deadline for proposing Forum topics. Scheduling committee meeting to make draft agenda. March 22nd | Draft Forum schedule published. Crowd sourced session conflict detection. Forum promotion begins. March 29th | Scheduling committee final meeting April 5th | Forum schedule final April 29th-May 1st | Forum Time! If you have questions or concerns, please reach out to speakersupport(a)openstack.org. Cheers, Kendall Nelson (diablo_rojo) [1] https://wiki.openstack.org/wiki/Forum [2] https://www.openstack.org/summit/denver-2019/ [3] https://www.openstack.org/summit/denver-2019/call-for-presentations [4] https://wiki.openstack.org/wiki/Forum/Denver2019

1 0

Using add-build-sshkey with static node
by Tristan Cacqueray 20 Feb '19

20 Feb '19

Hello, The add-build-sshkey isn't great with static node because the key added to the authorized_keys isn't removed if the remove-build-sshkey doesn't run. This results in key being leaked in authorized_keys without bound. Thus I proposed to add an option to cleanup keys before adding a new one: https://review.openstack.org/632620 . However this isn't great for static node that can be used by more than one job concurrently (multi-slot). Though, multi-slot already needs special trick to isolate workspaces and it seems like many other roles assume the node is dedicated to the job. Multi-slot node usage could be improved by using a dedicated user for each job, e.g. the operator would create zuul-%d user, and nodepool would set unique username for each zknode. In the meantime, can we please consider 632620 as this works well for single user static node. Regards, -Tristan

1 0

Zuul talks at Open Infrastructure Summit Denver in April
by Claire Massey 20 Feb '19

20 Feb '19

Hey Zuul community, The agenda is now live for the Open Infrastructure Summit and there are quite a few Zuul talks - congratulations to all of the speakers! View the agenda for the Open Infrastructure Summit Denver <https://www.openstack.org/summit/denver-2019/summit-schedule#day=2019-04-29>. Zuul sessions include: Moving your testing infrastructure from Jenkins to Zuul Unleashing the power of Fabric: Orchestrating new performance features for SR-IOV VNFs Getting a Neural Network Up and Running with OpenLab Best Practice to Run Zuul on Kubernetes Profiling and Optimizing Container Image Builds Digital Transformation in the Finance Industry If you haven’t already registered for the Summit, you’ll want to do so now before prices increase on February 27! Register here <https://openinfrasummitandptgdenver.eventbrite.com/>. I’ll continue to share more information about the Summit as the date approaches. Let me know if you have any questions. Thanks, Claire

1 0

Zuul/Nodepool tracing
by Simon.Westphahl＠bmw.de 20 Feb '19

20 Feb '19

Hi, recently I was getting sick of grepping through the logs of scheduler, executor and nodepool to debug an issue. My main problem was that there was no single identifier that I could use to find all logs that are related to a specific change. You have to look for the change/PR to find the matching 'Change' object. Then use the found object id to continue searching for the node request id, and so on. I decided to try to add tracing[0] to Zuul, as I was hoping that would simplify my use-case. For this I used the OpenTracing Python library[1] with Jaeger[2][3] as the trace backend (both hosted by the CNCF). Proof of concept patches for Zuul and Nodepool are available at [4] and [5]. You can also find some screenshots at [6]. Without initializing a concrete tracing backend (e.g. Jaeger) the tracing API will be a noop. So this should not impact the performance of Zuul/Nodepool in a significant way. The main downside of tracing is probably that you need to add instrumentation to the code. Tracing is of course not only useful for debugging issues but also e.g. for performance profiling. Curious to hear what you guys think! - Simon [0] https://opentracing.io/docs/overview/ [1] https://github.com/opentracing/opentracing-python [2] https://www.jaegertracing.io/ [3] https://github.com/jaegertracing/jaeger-client-python [4] https://review.openstack.org/#/c/637505/ [5] https://review.openstack.org/#/c/637506/ [6] https://imgur.com/gallery/00FTNej

4 4

[all] Denver Forum Brainstorming
by Kendall Nelson 19 Feb '19

19 Feb '19

Hello Everyone! Welcome to the topic selection process for our Forum in Denver! This is not a classic conference track with speakers and presentations. OSF community members (participants in development teams, operators, working groups, SIGs, and other interested individuals) discuss the topics they want to cover and get alignment on and we welcome your participation. The Forum is your opportunity to help shape the development of future project releases. For OpenStack Denver marks the beginning of Train’s release cycle, where ideas and requirements will be gathered. We should come armed with feedback from the upcoming Stein release if at all possible. We aim to ensure the broadest coverage of topics that will allow for multiple parts of the community getting together to discuss key areas within our community/projects. For OSF Projects (StarlingX, Zuul, Airship, Kata Containers) As a refresher, the idea is to gather ideas and requirements for your project’s upcoming release. Look to https://wiki.openstack.org/wiki/Forum for an idea of how to structure fishbowls and discussions for your project. The idea is to ensure the broadest coverage of topics, while allowing for the project community to discuss critical areas of concern. To make sure we are presenting the best topics for discussion, we have asked representatives of each of your projects to help us out in the Forum selection process. There are two stages to the brainstorming: 1.If you haven’t already, its encouraged that you set up an etherpad with your team and start discussing ideas you'd like to talk about at the Forum and work out which ones to submit. 2. On the 22nd of February, we will open up a more formal web-based tool for you to submit abstracts for the most popular sessions that came out of your brainstorming. Make an etherpad and add it to the list at: https://wiki.openstack.org/wiki/Forum/Denver2018 <https://wiki.openstack.org/wiki/Forum/Berlin2018> This is your opportunity to think outside the box and talk with other projects, groups, and individuals that you might not see during Summit sessions. Look for interested parties to collaborate with and share your ideas. Examples of typical sessions that make for a great Forum: - Strategic, whole-of-community discussions, to think about the big picture, including beyond just one release cycle and new technologies - eg Making OpenStack One Platform for containers/VMs/Bare Metal (Strategic session) the entire community congregates to share opinions on how to make OpenStack achieve its integration engine goal - Cross-project sessions, in a similar vein to what has happened at past forums, but with increased emphasis on issues that are of relevant to all areas of the community - eg Rolling Upgrades at Scale (Cross-Project session) – the Large Deployments Team collaborates with Nova, Cinder and Keystone to tackle issues that come up with rolling upgrades when there’s a large number of machines. - Project-specific sessions, where community members most interested in a specific project can discuss their experience with the project over the last release and provide feedback, collaborate on priorities, and present or generate 'blue sky' ideas for the next release - eg Neutron Pain Points (Project-Specific session) – Co-organized by neutron developers and users. Neutron developers bring some specific questions about implementation and usage. Neutron users bring feedback from the latest release. All community members interested in Neutron discuss ideas about the future. Think about what kind of session ideas might end up as: Project-specific, cross-project or strategic/whole-of-community discussions. There'll be more slots for the latter two, so do try and think outside the box! This part of the process is where we gather broad community consensus - in theory the second part is just about fitting in as many of the good ideas into the schedule as we can. Further details about the forum can be found at: https://wiki.openstack.org/wiki/Forum Thanks all! Kendall Nelson, on behalf of the OpenStack Foundation, User Committee & Technical Committee

1 0

Job at periodic pipeline with "files" section
by Felix Enrique Llorente Pastora 19 Feb '19

19 Feb '19

Hi All, Here at tripleo CI we have try to run a tripleo job with "files" section from parent and it does not run at periodic, after doing some testing with zuul quickstart looks like it does not work, kind of make sense. Have open a bug here https://storyboard.openstack.org/#!/story/2005040, but don't know if it make sense to fix this. BR -- Quique Llorente Openstack TripleO CI

2 4

zuul-preview - a helper service for JAMstack preview sites
by Monty Taylor 14 Feb '19

14 Feb '19

Heya, One of the cool things we do in Zuul jobs are our static html preview sites. We use this for zuul-dashboard preview, as well as storyboard, docs builds and others. Recently some of us in OpenDev land have been working with the OpenStack Foundation folks to support using netlify-cms with gerrit, which lets people who would prefer to edit web content via vim/emacs via git collaborate with those who would prefer to do such a thing in a visual CMS-like manner. It's really cool - except, one of the things that came out of it is the realization that a non-zero number of static-site-generators and/or JAMstack applications make some assumptions that they exist at the root of a URL. While we could argue that it would be better if they didn't make that assumption, supporting them would be potentially friendlier. To that end, over the weekend Jim and I hacked up a service using Apache's mod_rewrite and its ability to call out to a helper program to find the target URL. We wrote a program that takes the input from apache, hits zuul's REST API and returns the appropriate url. With that, given a url like: http://site.926bb0aaddad4bc3853269451e115dcb.openstack.preview.opendev.org/ It can serve the content published on the fileserver, or in swift - basically allowing us to serve preview content as-if it was at the root of a url without sacrifcing the zuul pattern of jobs being the ones to tell zuul where they published the content. It is a small program bundled with pre-configured Apache in a Docker container, so anyone who wants to run this at their site can simply run the published docker container as-is. We think it would be appropriate for this service to be part of Zuul in general so that it's easily available to sites which want to run it, and so we wanted to see what people thought about Zuul adopting it. https://github.com/emonty/upgraded-octo-giggle We wrote it in C++, for two reasons: 1) It's a performance sensitive service, so C++ is helpful. 2) We've talked about potentially writing some of zuul's components in not-python in the future. We're not broadly ready for that yet, but we thought having a simple program in C++ (that's not likely to get any more complex) but that goes things like REST and JSON interactions, would give us a basis for a more informed discussion later. So ... what do people think? Should the Zuul project adopt a small c++ service? Monty

8 13

Zuul-config reload
by Tomasz Szandała 13 Feb '19

13 Feb '19

Greetings, how can I reload main configuration repository? It seems like my commits to Gerrit's zuul-config still requires to restart zuul's docker (I am using docker-compose setup). Kind regards, Tomasz

2 3

An URLTrigger driver for Zuul
by Fabien Boucher 11 Feb '19

11 Feb '19

Hello, I would like to propose a solution to trigger jobs based on the state of an external resource accessible at an URL. The use case is quite common and actually Jenkins propose this kind of feature via the URLTrigger plugin [0]. I proposed two implementations of an URLTrigger driver for Zuul: - A trigger driver [1]: It is quite similar to the timer driver. The CronTrigger is used to provide the resource check rate. At each loop it checks the resource state and keep the state in a cache. When the cache state has changed the event is sent into the scheduler. - A require driver [2]: Idea was to rely on the timer driver to provide the resource check rate. Then attach a requirement in the pipeline via the require driver. The requirement is a RefFilter that match when the resource state change. Let me know what do you think about this URLTrigger idea. Regards, Fabien [0]: https://wiki.jenkins.io/display/JENKINS/URLTrigger+Plugin [1]: https://review.openstack.org/#/c/635567/ [2]: https://review.openstack.org/#/c/635554/

3 3

Zuul project update 2019-02-11
by corvus＠inaugust.com 11 Feb '19

11 Feb '19

Hi, Tristan has: Worked on config introspection and build a dashboard to list configuration relationship and identify unused items: https://review.openstack.org/#/q/topic:config-introspection Worked on displaying the job's hierarchy: https://review.openstack.org/#/q/topic:jobs-hierarchy Worked on improving the build/buildset zuul-web interface: https://review.openstack.org/#/q/topic:web-build Paul has been working on allowing job.run to be a list of playbooks: https://review.openstack.org/592160/ Tobias has been working on: Updating multi ansible spec: https://review.openstack.org/623927 Force merge broken config hardening: https://review.openstack.org/633514 Configurable clone timeout for supporting larger repos: https://review.openstack.org/633936 Worked on analysis of executor memleak. Initial try: https://review.openstack.org/634598 And I am continuing work on speculative docker images described in a previous mailing list post -- https://review.openstack.org/#/q/topic:docker-registry ********************************************************************** I believe the multi-ansible spec is ready for final review. I invite everyone with an interest to review https://review.openstack.org/623927 this week, and assuming there are no major changes required, I plan on merging that at the end of the week. It has also been suggested that this change may be significant enough to warrant a major version bump, to Zuul v4.0. When you review the spec, feel free to provide your thoughts on that as well. ********************************************************************** Help write the next update email by upadting https://etherpad.openstack.org/p/zuul-update-email Thanks, Jim

1 0

[all] Denver Open Infrastructure Summit Community Contributor Awards!
by Kendall Nelson 06 Feb '19

06 Feb '19

Hello Everyone! As we approach the Summit (still a ways away thankfully), its time to kick off the Community Contributor Award nominations[1]! For those of you that have never heard of the CCA, I'll briefly explain what they are :) We all know people in our communities that do the dirty jobs, we all know people that will bend over backwards trying to help someone new, we all know someone that is a savant in some area of the code we could never hope to understand. These people rarely get the thanks they deserve and the Community Contributor Awards are a chance to make sure they know that they are appreciated for the amazing work they do and skills they have. As always, participation is voluntary :) Nominations will close on April 14th at 7:00 UTC and recipients will be announced at the Open Infrastructure Summit in Denver[2]. Recipients will be selected by a panel of top-level OSF project representatives who wish to participate. Finally, congrats again to recipients in Berlin[3]! -Kendall Nelson (diablo_rojo) [1] https://openstackfoundation.formstack.com/forms/train_cca_nominations [2]https://www.openstack.org/summit/denver-2019/ <https://www.openstack.org/summit/berlin-2018/> [3] http://superuser.openstack.org/articles/openstack-community-contributor-awa…

1 0

Zuul 2019 planning
by Lauren Sell 05 Feb '19

05 Feb '19

Hi everyone, We’d like to work with the Zuul community on 2019 planning so we can best support and promote the project this year. If the mailing list is the best way to start the conversation, we can use the below etherpad, or have a discussion on this thread. We’re also happy to host a quick call if folks want to brainstorm in a high bandwidth setting. The main things we’re looking to document are the goals for the community this year (for example, attract more contributors or document more users in production or make Zuul easier to try out and use…) along with any key milestones, features, topics, etc. that we’ll want to communicate. Please take a look at the etherpad and add your thoughts over the next week: https://etherpad.openstack.org/p/zuul-2019-planning <https://etherpad.openstack.org/p/zuul-2019-planning> Thanks! Lauren

2 2

Superuser Award nominations open until March 22
by Claire Massey 04 Feb '19

04 Feb '19

Hey everyone The Superuser Award nominations are now open for the Denver Open Infrastructure Summit. March 22 is the deadline for submissions. The Superuser Awards recognize teams using open infrastructure to meaningfully improve business and differentiate in a competitive industry, while also contributing back to the open-source community. The awards will be given out during the Denver Summit. *This is a great opportunity to nominate companies that are using Zuul!* You can learn more about the Superuser Awards here: https://superuser.openstack.org/articles/superuser-awards-nominations-open-… <https://superuser.openstack.org/articles/superuser-awards-nominations-open-…>. Complete this form to nominate a team: https://openstackfoundation.formstack.com/forms/superuserawards_denver2019 <https://openstackfoundation.formstack.com/forms/superuserawards_denver2019> Thanks, Claire

1 0

Open Infrastructure Summit Denver - Community Voting Open
by Claire Massey 31 Jan '19

31 Jan '19

Hi everyone, Community voting for the Open Infrastructure Summit Denver CFP sessions is open! You can VOTE HERE <https://www.openstack.org/summit/denver-2019/vote-for-speakers>, but what does that mean? Now that the Call for Presentations (CFP) has closed, all submissions are available for community vote and input. After community voting closes, the volunteer Programming Committee members will receive the presentations to review and determine the final selections for Summit schedule. While community votes are meant to help inform the decision, Programming Committee members are expected to exercise judgment in their area of expertise and help ensure diversity of sessions and speakers. View full details of the session selection process here <https://www.openstack.org/summit/denver-2019/call-for-presentations/selecti…>. In order to vote, you need an OSF community membership. If you do not have an account, please create one by going to openstack.org/join <http://openstack.org/join>. If you need to reset your password, you can do that here <https://www.openstack.org/Security/lostpassword>. Hurry, voting closes Monday, February 4 at 11:59pm Pacific Time (Tuesday, February 5 at 7:59 UTC). Continue to visit https://www.openstack.org/summit/denver-2019 <https://www.openstack.org/summit/denver-2019> for all Summit-related information. REGISTER Register for the Summit <https://openinfrasummitandptgdenver.eventbrite.com/> before prices increase in late February! VISA APPLICATION PROCESS Make sure to secure your Visa soon. More information <https://www.openstack.org/summit/denver-2019/travel/#travel-support> about the Visa application process. TRAVEL SUPPORT PROGRAM February 27 is the last day to submit applications. Please submit your applications <https://www.openstack.org/summit/denver-2019/travel/#travel-support> by 11:59pm Pacific Time (February 28 at 7:59am UTC). If you have any questions, please email summit(a)openstack.org <mailto:summit@openstack.org>. Cheers, Claire

1 0

Using nodepool v3 with Jenkins
by Darragh Bailey 31 Jan '19

31 Jan '19

Hi, Hoping that we can make use of nodepool v3 with Jenkins to solve some of our existing problems, and at the same time move a bit closer to upgrading from zuul v2 -> v3. Going to outline our current problem, my initial thinking that resulted in some of my questions last night in IRC and some further thinking on it that may provide a way forward for us. Hoping that someone can chime in and identify if this is realistic or if we're heading down the wrong path before getting to far. Keep in mind that I'm only starting with nodepool, so I'm unsure if I even understand some of the concepts. e.g. does nodepool get a request for a particular type of node or is it infered by there being none of that type available in the queues in zookeeper? Current problem: We have some resources that are limited and are not directly accessible as workers for Jenkins, and so need a way to manage the allocation of resources to certain jobs. Some of these resources are provided by teams and some are from a set of resources managed by the CI team. So we probably have ~4 sets of these types of resources. At the moment this is done using some (too much) hackier with groovy, the Global Event Listener Plugin (GELP), and Lockable Resources plugin. The GELP allows us to hook into various points in Jenkins, such as when the job is scheduled but not yet running Jenkins (QUEUED). Unfortunately there doesn't appear to be any way with the gearman protocol to inspect what job is going to be assigned before it is picked up, so we can only do manipulation after the job has been scheduled and before it starts running. The data required to provide the job with the details required to use the limited resource is retrieved from vault and injected right before the job gets fully launched on Jenkins. This is done by creating lockable resource entries dynamically based on the number of the resources available, and then catching the event on when Jenkins allows the job needing this to start and injecting the required data into the environment. Unfortunately this means the queuing for the jobs requiring these limited resources occurs within Jenkins instead of Zuul because the job has still be scheduled even if it is not yet running. We've run into a number of problems resulting in workers getting allocated for jobs that don't get to start, and taking them away from jobs that could be run, and effectively lightly DOSing the system. Initial thoughts: So enter nodepool and the hope that if we move the management of both these resources and the workers out of Jenkins and into something a little more flexible, it can prevent the job from getting scheduled in the first place if not enough of the limited resources are available, and allow jobs that do not need these to be scheduled because a worker isn't getting assigned to the job that is queued waiting. Initially I was thinking of having two resources, making the one representing the limited resource a fake resource of some kind that Jenkins doesn't connect to, but as it appears that it's a requirement of the Jenkins Nodepool agent that nodes returned must be accessible to Jenkins, that doesn't seem to fly. Next idea was whether it would be possible to have a static pool and a another driver that could grab a node from the static pool if a request for the limited resource node type came in, which prompted some of my questions in IRC last night. Further thinking done: After spending a bit more time with the static nodepool driver, and depending on how it works (there are lots of gaps in my understanding), I'm wondering if what we need is to create a custom driver and place all of our static nodes in a internal pool inside of this driver. Additionally query our vault instance for a list of these limited resource groups and build some pools of those internally as well. And assign labels as part of the requirements for the jobs using these limited resources. Then instead of immediate registering all of the nodes with zookeeper, instead depending on the node label request that comes in, populate zookeeper with an entry containing suitable labels. If it is one requiring a limited resource identified by a name such as teamA-resources, the driver code can test if there are any not in use and if so create a node entry in zookeeper that contains labels "teamA-resources" and "teamA-resource-5". The second label can be used in Jenkins by our GELP code to determine which set of data to inject into the environment of the job scheduled on that node. Obviously there is still some hackier in there, but does a custom driver building on the idea of combining some of the ideas of the static driver along with the dynamic drivers where instead of launching a new node, it simply creates a node entry on demand using both a static node and details on what resource is required? Possibly I'm misunderstanding how nodepool works in this case, and maybe it just populates a queue in zookeeper and tries to keep some in there, which may mean that what needs to happen is that one entry allocating a node to each of the sets of limited resources needs to be added as entries to zookeeper queues by the driver with further entries added if there are enough resources available as those queues are consumed? Which is a bit different, but seems like it would still work to limit the amount of damage where multiple jobs were launched and required more of the limited resource than was available. Currently can end up seeing 3 queue instances for a single limited resource queued up in Jenkins, so as we migrate another case were this will be needed we could end up seeing a queue of 9-10. Seems nodepool might limit it to only losing one per team, until we can migrate to zuulv3 and use some of the mutex capabilities to handle this a bit better. Thoughts? Is this the best way to dig up as not digging isn't an option here... -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool"

2 4

Zuul jobs for container publishing
by corvus＠inaugust.com 31 Jan '19

31 Jan '19

Hi, We recently added a new pipeline to OpenStack's Zuul, and three new jobs and roles to the zuul-jobs repo to support container publishing. Zuul users may be interested in adopting both the pipeline and the jobs into their own workflows. The new pipeline is the "promote" pipeline. It runs jobs on changes after they merge, but unlike the "post" pipeline, it does not run with the resulting branch state after the merge -- it runs in the context of the change. This means that it's not suitable for building release artifacts, but that's not its purpose. Its purpose is to promote artifacts previously built in the gate pipeline to production. That is safe to do because anything built in the gate pipeline is built with the full future state of in-flight changes to all related projects and branches (even if it's built before they land). We build a lot of artifacts in the gate pipeline now, and then throw them away only to build them again in post. The promote pipeline is an attempt to avoid that waste and instead preserve artifacts built in gate and directly publish them -- but only if and when the changes for which they were built are merged. The first functional use of this system is in building container images and publishing them on Docker Hub. We have begun using this in openstack-infra to build container images of third-party applications, but this will work just as well and as easily for building images of applications in our infrastructure (for example, we are also starting to use this to publish Zuul images). To implement this, we wrote three jobs which are available now from zuul-jobs (and therefore can be used in any project or job): * build-docker-image [1] * upload-docker-image [2] * promote-docker-image [3] The first job should run in the check pipeline and simply runs "docker build" on the arguments passed to it. It is used to verify that the build succeeds. The second job should run in the gate pipeline. It uploads the image to Docker Hub into the final repository location, but it only does so with a single tag with the form "change_123456" where '123456' is the Gerrit change number for the change under consideration. This will mean that docker pulls won't accidentally fetch the image yet. But the image is staged in Docker Hub and is ready to be promoted if the change passes all its tests and merges. The third job runs in the promote pipeline. This job, unlike the others, only performs Docker Hub API calls and requires no build resources. Therefore it is a zero-node job in Zuul -- a job which only runs on the Zuul executor. This job finds a previously uploaded image with a "change_" tag, and re-tags it with any specified tags ("latest" by default, but you can also tag version numbers or anything else). It also cleans up unused "change_" tags so they don't clutter up the repository. The nice thing about zero-node jobs is that they start and finish very quickly (since they don't have to deal with resource allocation and contention). With this system, an image will be fully published within about a minute after the change merges. A complete example .zuul.yaml might look like this: - secret: name: myproject-dockerhub data: username: dockerhubuser password: !encrypted/pkcs1-oaep - DFlbrDM5eUMptMGIVMXV1g455xOJLi92UYF08Z2/JlIGu3t6v052o9FKlVyj1ZmpXs5+2 JTa5jHkLTvTsYs9fCaNcQc2nmViCyWNlbOMzjB17uiZOaYFNs1sMqZcUZbGEz7Y8ds6Qq NBXI10jWFPTah4QxUuBvUbT3vmjnUToCzexl5ZGhKgijcnROWfUsnlCdugpgoNIcPsUki zty5FotDihnrC8n8vIomVK6EClY38ty97pLrADzFDd+Cos/OUlvi2xooUhzx8Bn020rJA lqEU5v8LGXp5QkHx0MSDx6JY6KppJ/4p/yM+4By6l+A20zdcimxmgiNc9rMWPwDj7xsao m7NAZWmWqOO0Xkhgt6WOfugwgt9X46sgs2+yDEfbnI5ok8uRbAB/4FWj/KdpyXwhcf+O2 wEfhxLwDbAoGONQPjb4YcZmCXtmR7Qe5t+n2jyczWXvrbaBDUQP5a+YtVNN/xhmQ7D740 POlxv7bLxJAixzqaQ3d8Rz9ZEv6zzRuhWph32UQtZ1JxSNww+EvmXm2eEi2Q2z6pT1Cx/ j2OrFyA2GL/UJOVb15VHKF6bgHPHWJtpjPFhqdcvBhVute4BWB+KPcWH+y+apHN1enK3H tNJO9iqm34nKwSuj5ExmFw50LtwR5/9FyRuRPq/vBL+8y82v8FDmeYsBeobn5M= - job: name: myproject-build-image parent: build-docker-image description: Build Docker images. vars: &image_vars docker_images: - context: . repository: myproject/imagename - job: name: myproject-upload-image parent: upload-docker-image description: Build Docker images and upload to Docker Hub. secrets: - name: docker_credentials secret: myproject-dockerhub pass-to-parent: true vars: *image_vars - job: name: myproject-promote-image parent: promote-docker-image description: Promote previously uploaded Docker images. secrets: - name: docker_credentials secret: myproject-dockerhub pass-to-parent: true vars: *image_vars - project: check: jobs: - myproject-build-image gate: jobs: - myproject-upload-image promote: jobs: - myproject-promote-image And the pipeline definition looks like: - pipeline: name: promote description: | This pipeline runs jobs that operate after each change is merged in order to promote artifacts generated in the gate pipeline. manager: supercedent precedence: low post-review: True trigger: gerrit: - event: change-merged success: gerrit: {} mysql: failure: gerrit: {} mysql: If you find that you need jobs which behave slightly differently, you may be able to inherit from these jobs and add pre or post playbooks. Or if you need something particularly complicated, you can re-implement the jobs using the underlying roles in zuul-jobs, which have the same names an the jobs which invoke them. [1] https://zuul-ci.org/docs/zuul-jobs/jobs.html#job-build-docker-image [2] https://zuul-ci.org/docs/zuul-jobs/jobs.html#job-upload-docker-image [3] https://zuul-ci.org/docs/zuul-jobs/jobs.html#job-promote-docker-image [4] https://zuul-ci.org/docs/zuul/user/encryption.html -Jim

2 1

[summit] Open Infrastructure Summit Shanghai - Dates and Location
by Claire Massey 30 Jan '19

30 Jan '19

Hello Zuul team - We are excited to announce that following Denver, the Open Infrastructure Summit will be traveling to Shanghai, China the week of November 4, 2019. The Summit will be held at the Shanghai Expo Centre. Registration will be available soon. Sign up for updates[1] and reach out to summit(a)openstack.org with any questions. [1] https://www.openstack.org/summit/shanghai-2019 Cheers, Claire

1 0

Zuul project update 2019-01-28
by corvus＠inaugust.com 28 Jan '19

28 Jan '19

Hi, Over the past two weeks, Tristan has been working on the topic:freeze_job and got the "zuul-runner execute" command to run a zuul-job locally, as well as a buildset endpoint and a buildset page to show all the build result for a change. I've been working on image building jobs, and will continue to work on improvements to those this week (see two recent emails to list). Please help write next week's email by updating this etherpad: https://etherpad.openstack.org/p/zuul-update-email -Jim

1 0

Configurable autohold list of allowed build results.
by Gabriele Cerami 24 Jan '19

24 Jan '19

Hi, autohold currently allows users to examine a live environment to proceed with post-mortem examination. In some occasions, trying to find the root cause of a bug by just examining the logs and the timeline of the failed environment may prove too difficult, and the preferable way may be to catch the failure at its root exactly when the root cause happens. In a mechanism similar to breakpoints, we may want to stop the build at a certain point, the proceed manually to prod for weak spots. The only way to get this behaviour from zuul currently is to request an autohold, and add a step that injects a fake failure to the build. With [1] I'd like to suggest an alternative method, the list of build results that are allowed to put a node in hold state can be passed as argument, and not just hardcoded. In this way, the build may be configured to be brought forward up to a certain known to work point, without having to expressly fail, then begin the step by step manually debugging on the live environment. It may be best suitable for non voting, experimental jobs, or during the development of new jobs, but a useful option nonetheless. Any feedback welcome. Thanks. [1] https://review.openstack.org/632498

2 1

CFP Open Until January 23, Open Infrastructure Summit in Denver
by Claire Massey 22 Jan '19

22 Jan '19

Hi everyone, FYI - the CFP is now open for the first Open Infrastructure Summit <https://www.openstack.org/summit/denver-2019/> (formerly the OpenStack Summit) which will be held in Denver, Colorado April 29 - May 1, 2019. Wednesday, *January 23* is the deadline to Submit presentations <https://www.openstack.org/summit/denver-2019/call-for-presentations>. The Open Infrastructure Summit is organized by OSF and designed to be a place where open source infrastructure communities can come together and collaborate in the open. Zuul will have a large and prominent presence at the event so please submit talks to the CFP and plan to attend! SUBMIT YOUR PRESENTATION <https://www.openstack.org/summit/denver-2019/call-for-presentations> Important info: Based on previous Program Committee and attendee feedback, we have added / updated three Tracks: Security, Getting Started, and Open Development (previously Open Source Community). You can find the Track descriptions here <https://www.openstack.org/summit/denver-2019/summit-categories>. All of the OSF pilot projects —including Airship, Kata Containers, StarlingX and Zuul — will be front and center alongside other open source communities like Ansible, Cloud Foundry, Docker, Kubernetes, and many more. The Open Infrastructure Summit (formerly the OpenStack Summit), has evolved to recognize our diverse audience, and to signal to the market that the event is relevant for all IT infrastructure decision makers. If you’re interested in influencing the Summit content, apply to be a Programming Committee member <https://openstackfoundation.formstack.com/forms/openstackdenver2019_program…>*, where you can also find a full list of time requirements and expectations. Nominations will close on January 4, 2019. The content submission process for the Forum and Project Teams Gathering will be managed separately in the upcoming months. *OSF Staff will serve as the Programming Committee for the Getting Started Track. Denver Summit registration and sponsor sales are currently open. Learn more <https://www.openstack.org/summit/denver-2019> and email summit(a)openstack.org <mailto:summit@openstack.org> with any questions. Please email speakersupport(a)openstack.org <mailto:speakersupport@openstack.org> with any questions or feedback. Thanks, Claire

1 1

Two new Zuul ideas pass-to-parent: true and named project groups
by Monty Taylor 20 Jan '19

20 Jan '19

Hey all! I had an idea for a zuul feature to make the containers jobs (and jobs like them) easier. And then one that also came to mind to make some of our central jobs easier to deal with in openstack. First idea: Right now, secrets, for good reason, are only usable by jobs in their own repo. However, for use-cases like the containers job, or publishing to pypi, or readthedocs even - it's hard to have a job in zuul-jobs that is usable for doing those tasks because they all need secrets. The idea is to add a flag to the secret request, like this: - job: name: system-config-build-image parent: build-container-image abstract: true allowed-projects: openstack-infra/system-config - job: name: system-config-upload-image parent: system-config-build-image secrets: name: credentials secret: system-config-dockerhub allowed-projects: openstack-infra/system-config secrets: name: credentials secret: system-config-dockerhub pass-to-parent: true By passing the flag, we tell zuul to pass the secret variable file to every playbook in the inheritance chain above the job (but still not child jobs). Obviously this requires that the user of the job trust the parent job. In fact, maybe we even add a "warning" - so that if someone submits a job config to zuul with a pass-to-parent flag, it leaves an inline comment on the change (without a negative vote) warning about the implications of choosing to do that - that way reviewers can be sure they understand that they're trusting someone upstream from them. Second idea: Somewhere - maybe in config repo - but maybe in main.yaml - or maybe it's even just a normal global object - we allow for the creation of arbitrary named groups of projects. So - for instance, if we go for config-projects, something like: - project-group: name: openstack projects: - openstack/nova - openstack/swift - project-group: name: opendev projects: - openstack-infra/system-config - openstack-infra/bindep Then, we allow project-group entries to be used in allowed-projects, like this: - job: name: publish-openstack-sphinx-docs parent: publish-openstack-sphinx-docs-base final: true pre-run: post-run: - playbooks/sphinx/post.yaml allowed-projects: - project-group: openstack This would allow people (say AJaeger, or whoever) to maintain a list of "official" projects that can publish docs to docs.openstack.org ... but let any of those projects add the job, so that we didn't have to put the jobs into project pipelines in project-config. We could do a similar thing for container jobs. Say put this in system-config: - job: name: opendev-upload-image parent: opendev-build-image secrets: name: credentials secret: opendev-dockerhub allowed-projects: - project-group: opendev Then in bindep we have: - job: name: bindep-upload-image parent: opendev-upload-image final: true vars: &jinja-init_vars images: - target: bindepo repository: opendevorg/bindep The more I type things, the more I think there's no need to restrict the creation of named project groups. I could see openstack projects, like say kolla, wanting to create a project-group in one of their repos, so that they could put a secret into a job in a central repo for their dockerhub org, but use the job they make in each of their individual repos. Some of this could be done with tenants - but those are a bit more heavy-weight - and we already see that inside of OpenStack there are teams with many repos. There's also some similarity in problem solved between the two - both make for better reuse of jobs that needs privileges. But I think it's two different use-cases. One is about facilitating writing jobs in zuul-jobs (or other generic job locations) that need secrets to be passed to them but are otherwise generic and parameterizable. The second is about having discreet jobs that have specific secrets that should only be used in a subset of projects - but where it's possible that a given subset of projects might get access to a group of discreet credentialed jobs and managing membership in the group makes more logical sense than managing project-pipeline assignment. It also allows for having child-jobs of discreet credentialed jobs (like the container example) Anywho - what do people think? Monty

4 4

Infra team upgrading review.openstack.org Gerrit from 2.13.9 to 2.13.12 January 18 at about 1700UTC
by Clark Boylan 17 Jan '19

17 Jan '19

We will be performing a minor Gerrit upgrade to version 2.13.12 tomorrow (January 18, 2019) at 1700UTC. We've tested this upgrade on our dev server, https://review-dev.openstack.org, and expect it to be a quick upgrade. Any outage shouldn't last more than 10 minutes. We will let our configuration management tooling manage the upgrade so we won't have an exact time, but will try to get it as close to 1700UTC as possible. Feel free to test out the new version on the dev server. We are happy to answer any questions you might have as well. Sorry for the short notice and the cross post, Clark

1 0

Discrepancies in zuul-jobs
by Matthieu Huin 17 Jan '19

17 Jan '19

Hello, There seems to be a few discrepancies in the way some roles are implemented in zuul-jobs. The main problems I have encountered are: [1] discrepancies in dependencies management: for example the upload-to-pypi job assumes twine is installed on the build host, and will fail if twine is missing; whereas the npm-related jobs install nodejs and npm in the playbooks. [2] discrepancies in OS support: some of the npm-related jobs assume they're running on Debian or Ubuntu (explicit call to apt to install dependencies), other jobs go with bindep. Concerning [1], maybe guidelines for writing jobs and the kind of assumptions that can be made about execution environments should be set; I don't remember seeing any in the documentation but let me know if I am wrong. Anyhow, should we assume that jobs will handle pre-requisites and dependencies or should this be left up to the node images maintainers? Concerning [2], the Software Factory is working on expanding third party CI on zuul-jobs by adding coverage on CentOS and Fedora. This is actually how I encountered the problem with the npm jobs. This will help detect these kinds of problems. As with [1], guidelines should be issued about OS support of zuul-jobs. If you think this is the way to go I can open a review to start a section on guidelines for writing jobs in the documentation. WDYT ? Cheers, MHU

5 6

Zuul featured in Open Infrastructure Community Newsletter
by Claire Massey 16 Jan '19

16 Jan '19

Hi everyone, Today, OSF distributed the next Open Infrastructure Community Newsletter. The bi-weekly newsletters provide a digest of the latest developments and activities across open infrastructure projects, events and users. This week, we highlighted the latest with *Zuul* as well as a brief update about the Open Infrastructure Summit and additional OSF project news. You can checkout the full newsletter on Superuser [1], and if you are interested in receiving the upcoming newsletters, you can subscribe here [2]. If you would like to contribute or have feedback, please reach out to community(a)openstack.org <mailto:community@openstack.org>. Thanks, Claire [1] http://superuser.openstack.org/articles/inside-open-infrastructure-1-16/ <http://superuser.openstack.org/articles/inside-open-infrastructure-1-16/> [2] https://www.openstack.org/community/email-signup <https://www.openstack.org/community/email-signup>

1 0

Is it possible to support merge-mode: rebase in zuul merger?
by Yang, Feng 3. (NSB - CN/Hangzhou) 14 Jan '19

14 Jan '19

Hello Zuul experts, Happy New Year! I didn’t find clues in the historical mail list. So please allow me to ask here. We are using zuul v2 for ~2 years, recently we encountered a special case: merged codes in zuul merger is NOT the same with gerrit rebase codes. On zuul side, we are using default mode ‘git merge -s resolve’ On Gerrit repo side, we are using “rebase if necessary” strategy. Then we are thinking if it is possible to add a new merge-mode: rebase into zuul merger. Did you plan to add rebase strategy into zuul before? Or do you have any suggestions? I really appreciate it. Thanks. Looking forward your comments. Thank you in advance. 😊 Thanks Br, Finn

4 4

Zuul project update 2019-01-14
by corvus＠inaugust.com 14 Jan '19

14 Jan '19

Hi, Today I approved our formal governance document[1], after receiving votes from all of the current maintainers. Thanks, everyone, for helping to create that! David is working on nodepool image build timeouts[2]. We made a Zuul release last week, and are about to make a Nodepool release. Please help write next week's email by updating this etherpad: https://etherpad.openstack.org/p/zuul-update-email [1] https://zuul-ci.org/docs/zuul/governance.html [2] https://review.openstack.org/629923 -Jim

1 0

Zubbi 1.0.0 released
by Felix.FS.Schmidt＠bmw.de 14 Jan '19

14 Jan '19

Hello, With the start of the new year we are happy to announce that Zubbi 1.0.0 is now available on PyPI: https://pypi.org/project/zubbi/. The new Zubbi version provides support for Gerrit repositories. As opposed to GitHub, Zubbi yet doesn't support event handling for Gerrit. Thus, Gerrit repositories can only be scraped periodically. During the OpenStack summit, the question for a test instance came up. So we deployed a Zubbi demo showing the jobs and roles from the openstack-infra/zuul-jobs repo at http://ad43b5505180211e99b0002e3f6a3815-573606705.eu-west-1.elb.amazonaws.c…. Feel free to explore Zubbi :-) We hope that you like the demo and are interested in setting up a Zubbi on your own. For this case, we've added a quickstart guide in our README, explaining how to set up Zubbi with a minimal set of configuration (the same as the demo). In case you have any further questions or need help for setting up Zubbi, don't hesitate to contact us or create an issue in the GitHub repository: https://github.com/bmwcarit/zubbi. We would be happy to see a Zubbi for your Zuul setups :-) Best regards Felix

1 0

Zuul project update 2019-01-07
by corvus＠inaugust.com 08 Jan '19

08 Jan '19

Hi, Welcome back from the holidays! Fabian has been working on a driver for Pagure (https://pagure.io/ ) and now has a usable implementation here: https://review.openstack.org/#/c/604404/ I have continued work on artifact URL storage in the db in service of building a 'promote' pipeline; I should be able to have jobs and roles demonstrating it soon. I'd like to call everyone's attention to three items in review with wide-ranging impact; it would be good for as many folks as possible to review these: Governance document: https://review.openstack.org/622439 Ansible multi-version support spec: https://review.openstack.org/623927 Scheduler scale-out spec: https://review.openstack.org/621479 Please help write next week's email by updating this etherpad: https://etherpad.openstack.org/p/zuul-update-email -Jim

2 1

AW: AW: Different queues across branches?
by Mike Pagel 29 Dec '18

29 Dec '18

Happy holidays to everyone! I wrote: > In that case however, having multiple queue names is an > erroneous condition. Just wanted to wrap this up. I realize that raising an exception (or similar behavior) to complain about this condition will break existing installations, which is not acceptable. In particular not for this kind of thing. Nevertheless the discussion helped me to understand the behavior in my setup. Thanks a lot, Mike -----Ursprüngliche Nachricht----- Von: Mike Pagel [mailto:mike@mpagel.de] Gesendet: Donnerstag, 20. Dezember 2018 18:42 An: 'zuul-discuss(a)lists.zuul-ci.org' <zuul-discuss(a)lists.zuul-ci.org> Betreff: AW: AW: Different queues across branches? > Yes -- when creating shared change queues from projects, the first > appearance of a queue name is used, and this is true for any branch. > So, unlike jobs definitions and invocations, a project's membership in a shared queue is global and immutable. > It doesn't change based on branch, or the change under test. It is simply a characteristic of the project itself. I understand and I accept that this is the current state. In that case however, having multiple queue names is an erroneous condition. Depending on what "first" actually means I could imagine scenarios, where setting a different queue name in a branch can remove my project from the original queue used before on master and essentially break master's gating, because other (required) projects are still using the original queue name... Would you agree? Would it make sense to guard against this? Where would that kind of checking code go? What kind of exception should be thrown? Where would I find an example of such kind of configuration check code and associated tests? Mike

1 0

Different queues across branches?
by Mike Pagel 20 Dec '18

20 Dec '18

Hey zuulers, we recently had to create a maintenance branch across all repositories required to build the project. Since the two branches no longer affect each other in terms of a green build I intended to decouple the gating process of master and branch by changing the gate's queue name in the branch. I was expecting the branch gates to be executed in the new queue and the master gates being executed in the original one. However, it looks like all gates are still happening in the same original queue. Did I expect something wrong? In case it matters, we are using implied branch matching for our jobs. Thanks a lot, Mike

3 8

How to define zuul.executor.inventory_file on a localhost
by Akhil Jayakumar (akjayaku) 19 Dec '18

19 Dec '18

Hi All, Our openstack cinder ci jobs are failing due to below error. Do you know where we need to define zuul.executor variable “inventory_file”? Appreciate your help. 2018-12-18 06:48:58.993454 | TASK [log-inventory : Copy ansible inventory to logs dir] 2018-12-18 06:48:59.045029 | primary | ERROR 2018-12-18 06:48:59.045544 | primary | { 2018-12-18 06:48:59.045637 | primary | "msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'inventory_file'\n\nThe error appears to have been in '/tmp/03e62a72bdbc4480937f15e45000abb3/trusted/project_1/git.zuul-ci.org/zuul-jobs/roles/log-inventory/tasks/main.yaml': line 11, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n# https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.4.ht… name: Copy ansible inventory to logs dir\n ^ here\n" 2018-12-18 06:48:59.045713 | primary | } 2018-12-18 06:48:59.048055 | 2018-12-18 06:48:59.048294 | PLAY RECAP 2018-12-18 06:48:59.048470 | primary | ok: 18 changed: 10 unreachable: 0 failed: 1 Thanks, Akhil.

3 3

Zuul project update 2018-12-17
by corvus＠inaugust.com 18 Dec '18

18 Dec '18

Hi, The past two weeks in the Zuul project have seen: David discovered an issue in node deletion that was leaving empty ZooKeeper nodes and added code to nodepool to clean those up automatically. He also fixed a race in the test_handler_poll_session_expired nodepool test. Tobias worked on spec for scale out scheduler (what we've been informally calling Zuul v4 -- where we likely connect all components via Zookeeper and remove our use of Gearman). He also fixed an issue with github status checks. Paul added nodepool.node_id support for job inventory files (so that performance problems in jobs can be traced back to the hypervisor on which they ran). And I've continued the relative-priority work (including a change to group projects in check as they are grouped in gate), as well as a revision to our proposed governance document. A number of us are likely to be less active over the next few weeks. I don't plan to send another status update email until January 7. Please help write the next update email by updating this etherpad: https://etherpad.openstack.org/p/zuul-update-email -Jim

3 2

Creation of OSF Project Confirmation Guidelines
by Claire Massey 13 Dec '18

13 Dec '18

Hi everyone, As you know, Zuul is a pilot project supported by the OpenStack Foundation (OSF). As such, we wanted to make sure you’re aware of an effort recently kicked off a sub group of the OSF Board of Directors to draft guidelines[1] for confirming pilot projects as full, top-level, open infrastructure project of the Foundation. The current pilot projects that will be eligible for confirmation review in 2019-2020 are Airship, Kata Containers, StarlingX and Zuul. Currently the OpenStack project is the only confirmed project. The discussion about drafting these guidelines is just getting started, and *we invite you to participate in this effort* by reviewing the working draft in this etherpad and adding in your feedback and thoughts starting on line 139: https://etherpad.openstack.org/p/BrainstormingOSFProjectConfirmationGuideli… <https://etherpad.openstack.org/p/BrainstormingOSFProjectConfirmationGuideli…> There will most likely be an open call scheduled in early 2019 where additional feedback will also be welcomed. I will share that invitation with you once it’s schedule. The goal is to come up with a set of points that the Board can use when reviewing pilot projects for confirmation. This topic has previously been discussed at a very high-level at previous Board meetings, including the September 18th meeting[2] (starting around slide 38). 1. https://etherpad.openstack.org/p/ProjectConfirmationGuidelines <https://etherpad.openstack.org/p/ProjectConfirmationGuidelines> 2. https://docs.google.com/presentation/d/10UyCpxkjPqC3kT-dYRpBxzNT39i2OhlggJv… <https://docs.google.com/presentation/d/10UyCpxkjPqC3kT-dYRpBxzNT39i2OhlggJv…> Thanks, Claire

1 0