August 2018 - Zuul-discuss - lists.opendev.org

Toggle gzip in upload-logs
by Paul Belanger 13 Oct '18

13 Oct '18

Greetings, I've proposed a patch to upload-logs[1] that will toggle the compressing of our console logs files. The main change is, by default we no longer compress them. Jobs would need to set zuul_log_compress: true to work as we do today. The main reason for this, is to reduce the amount of extra work a user needs to view our console logs in apache / nginx. Today, if compressed, the browser will try to download the file not render it. I think we could add documentation explaining how to enable compression and which settings would be needed for apache / nginx but in this case, seems like an optimization over necessity. Thoughts? [1] https://review.openstack.org/567696/

3 4

Spec for a tenant-scoped admin web API
by Matthieu Huin 26 Sep '18

26 Sep '18

Hello, Following Jim's example with the containers spec, I've opened a spec review in the zuul repository: https://review.openstack.org/562321 I've been toying around with adding a few "admin" endpoints to zuul and nodepool for some time now [1][2]. Using Software Factory's authorization engine to limit access to these endpoints to specific users, and consuming germqtt's event feed to react to comments on a review, I demonstrated PoCs for features that I think would be of interest to Zuul users in general: * Ability to re-enqueue builds from zuul's web UI * Ability to trigger a autohold by commenting "autohold <job> on <tenant>" on gerrit review The aforementioned patches were meant as experiments; now I would like to go further and work to have this integrated into Zuul in a way that would be tenant-scoped. Before I rework these extensively I seek your guidance and opinions on how to proceed with this; hence this spec review. There are surely things I have overlooked, and my proposal to use groups to implement policies might need discussing. Please let me know what you think, also if it is the right place for the spec (the review on infra-specs is still open) MHU [1] https://review.openstack.org/536319 [2] https://review.openstack.org/539004

1 1

Feedback on Zuul Overview One Pager
by Claire Massey 26 Sep '18

26 Sep '18

Hi Zuul team, We'll have a Zuul branded booth at AnsibleFest in Austin, October 2-3. More information about our booth presence and how you can participate will be shared in the coming weeks. For now we’re working on creating all of the branded materials needed for the booth, including a one page piece of collateral that gives a nice overview of Zuul. A working draft of the one pager is available here: https://etherpad.openstack.org/p/Zuul_one_pager <https://etherpad.openstack.org/p/Zuul_one_pager>. If you have any feedback or edits on the copy, please provide it in the etherpad by next Monday, August 20. The completed document will end up looking polished like the Kata Containers overview one pager shown at https://katacontainers.io/media/uploads/katacontainers/uploads/katacontaine… <https://katacontainers.io/media/uploads/katacontainers/uploads/katacontaine…>. Thanks! Claire

7 12

Nodepool driver interface
by David Shrewsbury 13 Sep '18

13 Sep '18

Greetings all, We just merged the last piece needed [1] for all of Tristan's various experimental nodepool drivers. This has reminded me of an idea that I've had in the back of my mind for a while now. Currently, there are reviews up for Kubernetes, Amazon EC2, Ansible, OpenContainer, VMWare, and Azure. I'd like for us to at least discuss the possibility of a pluggable driver system so that the drivers can be developed independently from nodepool, rather than in-tree to nodepool. Why? A. Having them in-tree leads to the assumption that they are tested. We do not have access to all of these services, so actively testing them within nodepool is not (currently) possible. B. Nodepool reviewers will be responsible for maintaining the drivers. I, myself, do not have experience with these services, nor access to them, nor the desire or time to learn some of them. I would venture a guess that most of us share some of these deficiencies. This puts us in a spot of just guessing at fixes, or just ignoring them hoping someone else will pick up the ball. C. With plugins, users could get driver updates/fixes without waiting for a nodepool release. I think keeping the OpenStack driver in-tree makes sense because we actively test and use it. I can even get on board with keeping the static driver too. But the others, to me, would be best kept out of tree *unless* we can add access to those services to our Zuul environment and actively test them as well (that at least takes care of the A reason above, and most of B if we have excellent tests setup). Thoughts? [1] https://review.openstack.org/535555 -- David Shrewsbury (Shrews)

7 9

Zuul Preparations for Berlin Forum
by Claire Massey 01 Sep '18

01 Sep '18

Hi everyone, In preparation to the OpenStack Summit in Berlin it’s time to start thinking about the Forum (https://wiki.openstack.org/wiki/Forum <https://wiki.openstack.org/wiki/Forum>) that consists of a set of 40-minute long collaborative sessions in parallel to the conference.The Forum provides a feedback loop between developers and users and gives an opportunity to do strategic planning for upcoming releases of OSF projects. The participants use etherpad to plan the agenda and capture the notes from the sessions which have a format of open discussion. We're currently in the brainstorming phase for Forum topics. The purpose of this period is to identify areas for feedback or new requirements and collect ideas for cross-project discussions and planning that are also relevant for users. To collect ideas we've created an etherpad: https://etherpad.openstack.org/p/BER-zuul-brainstorming <https://etherpad.openstack.org/p/BER-zuul-brainstorming>. Please start adding topics to the etherpad now. Note that the Forum session submission period starts on September 12 and closes on September 26. CALL TO ACTION: We’re also looking for *one* active member of the Zuul team to volunteer to serve as a representative on the Selection Committee for the Forum. If you're interested in volunteering for this role, please reply to this thread by August 30. The Forum Selection Committee is comprised of members of the OSF staff, OpenStack TC and UC and one representative each from new top level projects StarlingX, Airship, Zuul and Kata Containers. The Selection Committee will collaboratively select the topics and program the Forum agenda between September 26 - October 24. Once the Zuul representative is confirmed, our team, led by Jimmy McArthur, will work closely with them on next steps. Please let me know if you have any questions. Thanks, Claire

2 1

ARA Records Ansible 0.16.0 has been released
by David Moreau Simard 30 Aug '18

30 Aug '18

Hi, Please take note that ARA 0.16 has been released today. It includes a bugfix which prevented customized configuration on the sqlite WSGI middleware which may be deployed for browsing Zuul ARA reports. A new feature can be used to prevent specific host facts from being saved -- by default this includes 'ansible_env' due to the potential sensitivity of the vars that can be in the Ansible shell environment. You can find the full changelog here [1]. Please let me know if you see any issues or have any questions ! [1]: https://github.com/openstack/ara/releases/tag/0.16.0 David Moreau Simard Service Reliability Engineering | OpenShift dmsimard = [irc, github, twitter]

1 0

Re: Invitation: Aug 30 Community Webinar: OSF Project Updates
by Jeremy Stanley 30 Aug '18

30 Aug '18

On 2018-08-27 12:30:54 -0500 (-0500), Claire Massey wrote: > This Thursday, August 30, we’ll host an OSF community webinar at > 8:00am PT (1500 UTC). This meeting is open to all community > members so please join us! > > We’ll cover: [...] > --High level updates from new OSF projects: Airship, Kata > Containers, StarlingX, and Zuul [...] For the sake of transparency, the OSF marketing team was looking on fairly short notice for someone to riff on Zuul for 5 minutes and I volunteered. Anne has a pair of slides one of which is just "Zuul: Stop Merging Broken Code; Get Involved: zuul-ci.org" and the other has "Zuul is Continuous Everything: Merged CI/CD workflow w/ Ansible; Fully dynamic configuration; Gerrit and GitHub support" on it. I'm just going to be dialling in on a phone because I don't trust Zoom's proprietary browser wizardry and am not set up with a webcam anyway. The target audience is mostly marketers and press analysts who have at best heard the name before, so it'll be really low-level. I'll spend a few seconds acknowledging that Zuul originated in Spring of 2012, growing out of the need to scale more traditional CI models to test the ever increasing volume of developer activity in the OpenStack community with a focus on parallel predictive testing of multiple future code states, with the third major release marking six years of continued effort in that regard, but that it also now brings a broader scope with in-repository configuration tested securely on proposal, adoption of a real automation system (Ansible) as its job configuration language and executor implementation, and adds support for integrating dependencies across multiple code review platforms (ability to mix multiple Gerrit and GitHub connections). That's right... rambling. Nevertheless if you decide to dial in and have concerns about what I'm saying, I'll be in #zuul on Freenode the entire time should you wish to course-correct my comments. ;) -- Jeremy Stanley

1 1

Early Bird Pricing Ends Tomorrow - OpenStack Summit Berlin
by Claire Massey 27 Aug '18

27 Aug '18

Hi everyone, Friendly reminder that the early bird ticket price deadline for the OpenStack Summit Berlin <http://www.openstack.org/summit> is tomorrow, August 28 at 11:59pm PT (August 29, 6:59 UTC). Register NOW <https://openstacksummit2018berlin.eventbrite.com/> before the price increases to $999 USD! In Berlin, there will be sessions and workshops around open infrastructure use cases, including CI/CD, container infrastructure, edge computing, HPC / AI / GPUs, private & hybrid cloud, public cloud and NFV. In case you haven’t seen it, the initial agenda with CFP selected content is now live <https://www.openstack.org/summit/berlin-2018/summit-schedule/> and includes sessions and workshops from Ocado Technology, Metronom, Oerlikon, and more! In addition, make sure to check out the Edge Hackathon hosted by Open Telekom Cloud <https://www.openstack.org/summit/berlin-2018/summit-schedule/events/22723/b…> the weekend prior to the Summit. Interested in sponsoring the Summit? Find out more here <https://www.openstack.org/summit/berlin-2018/sponsors/> or email summit(a)openstack.org <mailto:summit@openstack.org>. Thanks, Claire

1 0

Invitation: Aug 30 Community Webinar: OSF Project Updates
by Claire Massey 27 Aug '18

27 Aug '18

Hi Zuul team, This Thursday, August 30, we’ll host an OSF community webinar at 8:00am PT (1500 UTC). This meeting is open to all community members so please join us! We’ll cover: —What’s new in OpenStack’s latest release - Rocky. (This info will still be at a fairly high level, so might not be new information if you’re someone who stays up to date in the OpenStack dev ML or is actively involved in upstream work) —High level updates from new OSF projects: Airship, Kata Containers, StarlingX, and Zuul —What you can expect at the Berlin Summit in November This meeting will be run over Zoom and will be recorded, so if you can’t make the time, don’t panic! Details below. Thanks, Claire When: Aug 30, 2018 8:00 AM Pacific Time (US and Canada) Topic: OSF Community Meeting Please click the link below to join the webinar: https://zoom.us/j/551803657 <https://zoom.us/j/551803657> Or iPhone one-tap : US: +16699006833,,551803657# or +16468769923,,551803657# Or Telephone: Dial(for higher quality, dial a number based on your current location): US: +1 669 900 6833 or +1 646 876 9923 Webinar ID: 551 803 657 International numbers available: https://zoom.us/u/bh2jVweqf <https://zoom.us/u/bh2jVweqf>

1 0

Berlin Community Contributor Awards
by Kendall Nelson 24 Aug '18

24 Aug '18

Hello Everyone! As we approach the Summit (still a ways away thankfully), its time to kick off the Community Contributor Award nominations! For those of you that might already know what they are, here is the form[1]. For those of you that have never heard of the CCA, I'll briefly explain what they are :) We all know people in the community that do the dirty jobs, we all know people that will bend over backwards trying to help someone new, we all know someone that is a savant in some area of the code we could never hope to understand. These people rarely get the thanks they deserve and the Community Contributor Awards are a chance to make sure they know that they are appreciated for the amazing work they do and skills they have. So go forth and nominate these amazing community members[1]! Nominations will close on October 21st at 7:00 UTC and winners will be announced at the OpenStack Summit in Berlin[2]. -Kendall Nelson (diablo_rojo) [1] https://openstackfoundation.formstack.com/forms/berlin_stein_ccas [2]https://www.openstack.org/summit/berlin-2018/

1 1

Zuul project update 2018-08-20
by corvus＠inaugust.com 21 Aug '18

21 Aug '18

Hi, Several folks are on vacation around this time of year, so things are quieter than normal (and may be for the next couple of weeks). Shrews is working on getting images published via pbrx[1]. I plan to continue work on per-project/tenant ssh keys. Please help write next week's email by updating this etherpad: https://etherpad.openstack.org/p/zuul-update-email -Jim [1] https://review.openstack.org/#/q/topic:push-job

1 0

Proposal to use React for the web dashboard
by Tristan Cacqueray 20 Aug '18

20 Aug '18

Hello, As we are about to add more functionalities to the web dashboard, I would like to suggest we replace Angular by React. In the context of the log-classify project, I experimented with different frameworks to write user web interfaces. I tried Angular, VueJs and React, and the latter works best. React seems less obscure than Angular and it is more widely used than VueJs. The main advantage for me is that React's components are in charge of the rendering and you can follow the code from main() to actual page code. Here is a prototype for the Zuul web interface re-written in React: https://review.openstack.org/#/c/591604/ The highlights are: * The status page jquery code has been rewritten, * It uses patternfly for the new "Masthead" bar, * Tenant name is displayed on the top right, and * Routing uses HTML5 history API (links doesn't have .html extension) The downsides are: * The new routing needs extra rewrite rules, and * Sub-directory deployment needs to rebuild the files with the desired static assets serving location. We could simplify the sub-directory use-case using a custom dynamic discovery mecanism, for example with a /.well-known/zuul-manifest.json file, but this has not been implemented yet. There may be other options, but by default React app are served from the / top location. Please find further details about this specific issue. Are there many zuul web dashboards served from a sub-directory? Regards, -Tristan Details about sub-directory and static files location: I first tried to set the "homepage" setting to "./" so that static files would be loaded relative to the requested path, but that doesn't work for url like "/job/{name}" where static files will be loaded from "/job/static/js/..." The "homepage" default settings is "/" and the index.html loads static assets from "/static/". The current solution is to manually change the "homepage" to the desired location, e.g. "/zuul/" and rebuild the UI so that the index.html loads static assets from "/zuul/static", even when it is accessed from "/zuul/job/{name}" Or without rebuilding, the zuul static assets needs to be served from /, even when client are using a sub-directory. This is not very clean and it may cause conflicts. Another solution is to use the HashRouter instead of the BrowserRouter, but then url will look like this: "zuul.openstack.org/#/job/{name}"

1 0

Berlin Summit Schedule Live!
by Claire Massey 14 Aug '18

14 Aug '18

Hi Zuul team, The schedule for the Berlin Summit <http://www.openstack.org/summit/berlin-2018/summit-schedule> is live with all of the sessions that were accepted via the CFP. It includes a several talks <https://www.openstack.org/summit/berlin-2018/summit-schedule/global-search?…> that feature Zuul. You can see the full CI/CD track here <https://www.openstack.org/summit/berlin-2018/summit-schedule/#track=249>. Check out 100+ sessions, demos, and workshops covering 35+ open source projects in the following Tracks: • CI/CD • Container Infrastructure • Edge Computing • HPC / GPU / AI • Private & Hybrid Cloud • Public Cloud • Telecom & NFV Log in with your OpenStackID <https://openstackid.org/> and start building your schedule now! Register for the Summit <https://openstacksummit2018berlin.eventbrite.com/> - Get your Summit ticket for USD $699 before the price increases on August 21 at 11:59pm PT (August 22 at 6:59 UTC) For speakers with accepted sessions, look for an email from speakersupport(a)openstack.org <mailto:speakersupport@openstack.org> for next steps on registration. Thank you to our Programming Committee! They have once again taken time out of their busy schedules to help create another round of outstanding content for the OpenStack Summit. The OpenStack Foundation relies on the community-nominated Programming Committee, along with your Community Votes to select the content of the summit. If you're curious about this process, you can read more about it here <https://www.openstack.org/summit/berlin-2018/call-for-presentations/selecti…> where we have also listed the Programming Committee members. Interested in sponsoring the Berlin Summit? Learn more here <https://www.openstack.org/summit/berlin-2018/sponsors/#how-to-sponsor>. Thanks, Claire

1 0

Zuul project update 2018-08-13
by corvus＠inaugust.com 14 Aug '18

14 Aug '18

Hi, Tristan is continuing work on the container spec implementation. I'm finishing up support for line comments as well as logs in swift. Later this week, I plan to start working on implementation of an item initially described in the Zuul v3 spec but which was not implemented, the per-project and per-tenant ssh keys described here: http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html#co… The recent discussions about add_host have reminded us that with that piece in place, add_host would become much more useful for untrusted projects. -Jim

1 0

zuul-jobs : Removal of readthedocs playbook
by Ian Wienand 09 Aug '18

09 Aug '18

Hello, The zuul-jobs:playbooks/python/readthedocs.yaml playbook is scheduled for removal [1] in two weeks on or about 27th August, 2018 (per the deprecation policy [2]). This playbook was used to trigger documentation builds on the readthedocs.org service. The playbook uses an API endpoint that is no longer available [3]. The trigger-readthedocs role is provided for anyone wishing to migrate to the authenticated API endpoint [4]. For more details, see [5]. Thanks, -i [1] https://review.openstack.org/#/c/583836/ [2] https://docs.openstack.org/infra/zuul-jobs/deprecation.html [3] https://github.com/rtfd/readthedocs.org/commit/a66bd56bada66f99c778aa424042… [4] https://zuul-ci.org/docs/zuul-jobs/roles.html#role-trigger-readthedocs [5] http://lists.openstack.org/pipermail/openstack-dev/2018-August/132836.html

1 0

Zuul project update 2018-08-06
by corvus＠inaugust.com 08 Aug '18

08 Aug '18

Hi, Sorry for the delay, I was a bit distracted earlier this week. Tobias has fixed issue with implicit branch matchers[1]. I've mostly completed the effort to store logs in swift. We still need to add the icons if we want them to show up in the generated HTML indexes, but otherwise, it should be ready. For those interested, I expect OpenStack to start using this at scale shortly. The changes to add line comments to Gerrit (and add the framework for GitHub to use later) are in. I still need to see if we can map line numbers across speculative changes in the mergers. We released Zuul 3.2.0. Folks also may be interested to know that in OpenStack, we've seen a significant performance improvement on the executors by upgrading to more recent kernels (4.15.0). Please help write next week's email by updating this etherpad: https://etherpad.openstack.org/p/zuul-update-email -Jim [1] https://review.openstack.org/588201

1 0

Software Factory 3.1 released
by Tristan Cacqueray 06 Aug '18

06 Aug '18

We are pleased to announce the release of Software Factory version 3.1: http://www.softwarefactory-project.io/releases/3.1 This new release adds support for tenant deployment and Zuul configuration management from the resources. Here is the list of features we picked in the Software Factory version of Zuul and Nodepool that are still under review upstream. We are confident these features will eventually land upstream, but they might evolve by then. They should therefore be considered a "tech preview", although we will do our best to integrate these in the least disruptive way possible: * Improved dashboard: https://review.openstack.org/#/q/topic:zuul-web-routes https://review.openstack.org/#/q/topic:zuul-ui-pages Nodepool drivers: * RunC: https://review.openstack.org/535556 * Kubernetes: https://review.openstack.org/535557 * Openshift: https://review.openstack.org/570667 Best regards, The Software Factory Team

1 0

Interview with Podcast.__init__
by Monty Taylor 03 Aug '18

03 Aug '18

Hey all, Podcast.__init__, which is a python related podcast, recently interviewed me about Zuul: https://www.podcastinit.com/zuul-with-monty-taylor-episode-172/ If you think you can handle listening to me talk for an hour, check it out. For those of you who think you've heard all of my old grandpa stories ... this podcast contains the story of when Jim and I first met. Monty

1 0