- Zuul-discuss - lists.opendev.org

Increasing project visibility
by Jeremy Stanley 06 Feb '20

06 Feb '20

With AnsibleFest just around the corner, I'm looking forward to seeing lots of you again. As many on this list are probably already aware, there will be a Zuul booth and I expect to spend a fair amount of time there talking to new folks about what Zuul is and why I think it's so amazing. I hear there will be some talks there which mention Zuul as well, so hopefully that helps spread the word to even more attendees. This got me thinking about how growing Zuul depends on growing our community around it, but to do that people need to know it exists in the first place. Speaking engagements and talking to folks at conferences are great ways, we have a slick looking Web site, some members of the community write articles or get interviewed about cool things they've done with Zuul, we're featured prominently in OSF marketing materials and newsletters... but what else can we do to get out the message, and how can we tell what's working to reach our future community and what isn't? Let's see what more we can come up with! Who's got suggestions? -- Jeremy Stanley

9 23

Zuul community activity metrics
by Thierry Carrez 04 Feb '20

04 Feb '20

Hi everyone, As some of you know, I have been working with Bitergia to provide a dashboard of community activity metrics for Zuul. This dashboard is now available at: https://zuul.biterg.io/ I customized the default dashboard so that it primarily shows Gerrit activity (merged changes that we generally count as "contributions"), rather than the default Bitergia dashboard. The rest of the dashboards are standard Bitergia dashboards, so you might find some panels more interesting than others. It currently tracks Git data, Gerrit activity, and mailing-list activity. We have the possibility to customize or add panels, so if you see some analysis or panel that would be specifically useful for Zuul, let me know ! Bitergia does its own affiliation tracking. We might develop a sync in the future to ingest OSF affiliation data automatically, but for the time being the affiliation mapping is separate, and requires manual fixing when the automated ingest ends up being incorrect. I did a pass at correcting the most obvious errors, but let me know if there is other entries to urgently fix. Also some identities may be duplicated (especially across data sources as people don't use the same email address in for example git and the ML), and some others may still be "unknown". Let me know if you have corrections. We plan to organize an open training session where Bitergia folks would cover the basic usage of the platform and answer community questions. I'll keep y'all posted. Cheers, -- Thierry Carrez (ttx)

2 2

Fosdem
by Mark Meyer 31 Jan '20

31 Jan '20

Moin, Any Zuul people say fosdem? Cheers Mark -- Mark Meyer mark(a)ofosos.org

2 2

2019 OSF Annual Report Now Available
by Jimmy McArthur 30 Jan '20

30 Jan '20

The OSF community had a productive year, merging 58k code changes to produce open source infrastructure software like Airship, Kata Containers, StarlingX, and Zuul, along with the third most active OSS project in the world, OpenStack. With 100,000 members and millions more visiting OSF websites in 2019 to get involved, the community made huge strides in addressing the 7.7B market for OpenStack and more than $12B combined OpenStack & Containers markets in the future. Each individual member, working group, SIG, and contributor was instrumental in continuing to support the OSF mission: helping people build and operate open infrastructure. The OSF 2019 Annual Report was published today highlighting the achievements across the community and the goals for the year ahead. Let’s break down some of the highlights of last year: -- The OSF confirmed three new open infrastructure projects to complement OpenStack in powering the world’s open infrastructure; -- OpenStack is one of the top three most active open source projects in number of changes, and is projected to be a $7.7 billion USD market by 2023; -- Some of the world’s largest brands—AT&T, Baidu, Blizzard Entertainment, BMW, China UnionPay, Walmart, and Volvo among others—shared their open source infrastructure use cases and learnings; -- Upstream contributors continued to prioritize cross-project integration with open source projects including Ceph, Kubernetes, Ansible, and Tungsten Fabric. -- New contributors were on-boarded through multiple internship and mentoring programs as well as OpenStack Upstream Institute, which was held in seven countries last year! The OSF would like to extend a huge thanks to the global community for all of the work that went into 2019 and is continuing in 2020 to help people build and operate open source infrastructure. Check out the full OSF 2019 Annual Report on the OpenStack website! Jimmy McArthur OSF

1 1

Help Shape the Track Content for OpenDev + PTG, June 8-11 in Vancouver
by Jimmy McArthur 30 Jan '20

30 Jan '20

Hi everyone, Hopefully by now you’ve heard about our upcoming event, OpenDev + PTG Vancouver, June 8-11,2020 <https://opendevptg2020.eventbrite.com>. We need your help shaping this event! Our vision is for the content to be programmed by you-- the community. OSF is looking to kick things off by selecting members for OpenDev Programming Committees for each Track. That Program Committee will then select Moderators who will lead interactive discussions on a particular topic within the track. Below you'll have the opportunity to nominate yourself for a position on the Programming Committee, as a Moderator, or both, as well as suggesting specific Topics within each Track. PTG programming will kick off in the coming weeks. If you’re interested in volunteering as an OpenDev Programming Committee member, discussion Moderator, or would like to suggest topics for moderated discussions within a particular Track, please read the details below, and then fill out this form <https://openstackfoundation.formstack.com/forms/opendev_vancouver2020_volun…>. We’re looking for subject matter experts on the following OpenDev Tracks: - Hardware Automation (accelerators, provisioning hardware, networking) - Large-scale Usage of Open Source Infrastructure Software (scale pain points, multi-location, CI/CD) - Containers in Production (isolation, virtualization, telecom containers) - Key Challenges for Open Source in 2020 (beyond licensing, public clouds, ethics) OpenDev Programming Committeemembers will: * Work with other Committee members, which will include OSF representatives, to curate OpenDev content based on subject expertise, community input, and relevance to open source infrastructure * Promote the individual Tracks within your networks * Review community input and suggestions for Track discussions * Solicit moderators from your network if you know someone who is a subject matter expert * Ensure diversity of speakers and companies represented in your Track * Focus topics around on real-world user stories and technical, in-the-trenches experiences Programming Committee members need to be available during the following dates/time commitments: * 8 - 10 hours from February- May for bi-weekly calls with your Track's Programming Committee (plus a couple of OSF representatives to facilitate the call) * OpenDev, June 8 - 10, 2020 (not required, but preferred) o Programming Committee members will receive a complimentary pass to the event Programming Committees will be comprised of a few people per Track who will work to select a handful of topics and moderators for each Track. The exact topic counts will be determined before Committees begin deciding. OpenDev Discussion Moderatorswill * Be appointed by the Programming Committees * Facilitate discussions within a particular Track * Have adequate knowledge and experience to lead and moderate discussion around certain topics during the event * Work with Programming Committee to decide focal point of discussion Moderators need to be available to attend OpenDev, June 8 - 10, 2020, and will receive a complimentary pass. Programming Committee nominations are open until February 11.Deadlines to volunteer to be a moderator and suggest topics will be in late February. Nominate yourself or suggest discussion topics here: <https://openstackfoundation.formstack.com/forms/opendev_vancouver2020_volun…>https://openstackfoundation.formstack.com/forms/opendev_vancouver2020_volun… Cheers, The OpenStack Foundation

1 1

Retiring as zuul-maint
by Clint Byrum 28 Jan '20

28 Jan '20

Hi everyone. I'd like to publicly state that I'm unable to continue as a Zuul maintainer. I've lost touch with the current status-quo, and I don't have any day-job time to work on Zuul. My hobby time spent on Zuul is no longer a joy, as I find working on zuul-jobs to be overbearing and unproductive, and the internals of Zuul have changed enough that I don't feel confident approaching core issues anymore. Thanks everyone for all the great stuff you've been doing. I still use Zuul every day, and believe strongly in the ethos of the project, but I just have to be realistic and lay down my +2.

3 2

Fetch roles improvement to enable running zuul-jobs on kubectl connection
by Tristan Cacqueray 25 Jan '20

25 Jan '20

Hello folks, I have been working on supporting kubectl connection for zuul-jobs. The issue is that Ansible synchronize module doesn't work[0]. One solution would be to include the fix in Zuul library[1], but that's not ideal because zuul's synchronize would be slightly different. Alternatively we can synchronize files using oc rsync[2] command but that can't be done from untrusted context. Thus we need to adapt every fetch roles to not perform the synchronize task, and instead rely on the generic fetch-output role. As a first step I added a zuul_use_fetch_output toggle to enable the new behavior, but we should consider enforcing the fetch-output job and make it the default behavior. Indeed It is much easier to copy or link file to ~/zuul-output and let the base job retrieve and publish instead of having each fetch roles bother with running synchronize to zuul.executor.work_root or log_root. A zuul_use_fetch_output toggle implementation is proposed[3] along with a third-party-ci build that demonstrates a full tox-py36[4] job running in a container. Regards, -Tristan [0]: https://github.com/ansible/ansible/pull/62107 [1]: https://review.opendev.org/681553 [2]: https://review.opendev.org/#/c/631402/6/roles/prepare-workspace-openshift/t… [3]: https://review.opendev.org/#/q/topic:zuul-jobs-with-kubectl [4]: https://review.opendev.org/682049

3 2

Next steps for zuul-operator
by corvus＠inaugust.com 25 Jan '20

25 Jan '20

Hi, The zuul-operator project seems to have stalled. I think that's partly due to time constraints, but also we have had a number of proposals for alternative implementations and no clear direction. Originally we said we would use the Ansible operator framework (and were vague about how to handle the more programmatic parts; one assumes Python of some sort). It has also been proposed that we use the golang framework. A third proposal is Ansible with most of the transformation handled by dhall. We might have enough people with enough time interested in finishing this up if we are able to agree on a path forward, so I'd like to have a meeting to discuss this. I think a voice conference might be appropriate for this, due to the nuances of the discussion. I've checked a number of folks active in the discussion so far, and it looks like Wednesday, January 15 at 15:30 UTC is a good time. Let's use the conferencing system[1] room 6001. Anyone may feel free to join, and please let me know if you think we should reschedule. [1] https://wiki.openstack.org/wiki/Infrastructure/Conferencing -Jim

3 9

zuul schema validation and zuul-lint
by Sorin Sbarnea 13 Jan '20

13 Jan '20

Over the last weekend I managed to finally produce a validation schema for zuul config files and also publish https://pypi.org/project/zuul-lint/ The goal is better described at https://storyboard.openstack.org/#!/story/2003989 but I will summarize below the goal: * ability to lint zuul config file syntax * ability to get vscode live validation of zuul config files * be able to lint zuul config syntax using pre-commit tool (not hook) All these are working now and I was able to run zuul-lint on all* zuul files from opendev and rdo. My initial desire was to make this part of zuul itself but I realized that installing zuul package comes with two big problems: - huge dependency chain (most not needed to perform a linting) - broken on macos due to some deps While the second bullet probably can be fixed I am not so sure about the first one, mainly because we cannot use a `zuul[lint]` extras for linter requirements as it would be additive to default ones. This could be sorted only if we decide to add extras for server and client and keep almost no default dependencies on zuul package itself. Another alternative would be to make zuul repository produce more than one python package, so the linter can be kept isolated. Still I am not sure how easy it would be to achieve that. It would be great to have some feedback from some very-early adopters. Please note that this is only schema validation, is not validating if the configuration is compatible with a specific zuul instance, as this is a totally different aspect, one that is unlikely to be possible to be done without a CR/PR (missing jobs) * "all" that I already had cloned locally, did not had time to do a full batch. /sorin

1 0

Changing default of stage_compress_logs in stage-output role
by Clark Boylan 08 Jan '20

08 Jan '20

The zuul-jobs stage-output role is useful in preparing logs for permanent storage. Previously the default behavior of this role was to compress logs that it collects to reduce storage space requirements (via a default true value for stage_compress_logs). Unfortunately, we've found that depending on the permanent storage system this may be undesireable. Specifically when storing logs in object storage services we can upload and store a compressed version of the file while still logically referring to it via its uncompressed name. Then the object storage service will convert it to the uncompressed version when retrieved. This is problematic if you want a compressed file to be worked with in its compressed form, like a .tar.gz tarball. In order to disambiguate between files that are intentionally compressed (like tarballs) and those that are compressed for storage efficiency reasons we are changing the default of stage_compress_logs to false and letting the upload roles determine how to compress things. In the case of upload-logs-swift already compressed files will be left alone and not treated special by the storage service. Any uncompressed files will be compressed on upload with hints to get the original back when retrieved. This should create a good balance of storage efficiency while still being able to work with intentionally compressed files without unexpected decompression from storage services. If you would like to keep the old stage-output behavior you can set stage_compress_logs to true in your site variables or base jobs. More details can be found in the change that updates stage-output, https://review.opendev.org/#/c/701282/. Clark

1 0