July 2020 - Zuul-discuss - lists.opendev.org

install and use ansible collections in zuul v3 ansible native job?
by Marios Andreou 24 Nov '21

24 Nov '21

hi folks long story short: can we install and use ansible collections in a zuul v3/ansible native job? In the tripleo-ci squad we are trying to use the tripleo_overcloud_image_build role from the tripleo-operator-ansible collection [1]. We are already using other roles from this collection - e.g. there [2]. This is possible because in that case we have zuul calling bash calling ansible... which means we get to configure collections in the bash bit setting ANSIBLE_COLLECTIONS_PATHS [3] before invoking ansible. For the case I'm having trouble with - what i refer to as ansible native - here https://review.opendev.org/#/c/733818/33/roles/oooci-build-images/tasks/mai… we have a zuul job playbook directly trying to install and use a collection. Currently it fails because instead of collections: (doesn't work " 'collections' is not a valid attribute? [4] ) I am trying to use include_role but then it can't find the collection's modules "ERROR! no action detected in task" [5]. I tried to set it in ansible.cfg but that won't work and anyway it is surely the wrong approach. The one thing left to try is to explicitly require ansible 2.9 for the job - but wanted to reach out and see if anyone else has any problems trying to install and use ansible collections (with modules importantly) as part of a zuul v3/ansible native job. thanks for reading and for any comments, marios [1] https://github.com/openstack/tripleo-operator-ansible/tree/master/roles/tri… [2] https://opendev.org/openstack/tripleo-quickstart-extras/src/commit/55e74bb8… [3] https://opendev.org/openstack/tripleo-quickstart/src/commit/299131bcc6dca86… [4] https://23fd78c0f6b01b9c4209-59c294b8f46eb75cda2bd004e2a1c97f.ssl.cf1.rackc… [5] https://2063b7e2bb67f86f8ec5-87fe67c942382c6ee875c514bffa4f21.ssl.cf2.rackc…

3 8

Addressing users needs via the CLI
by Matthieu Huin 20 Aug '20

20 Aug '20

Hello, I would like to discuss the current state of Zuul's CLI, especially from a user's point of view. We can identify two types of Zuul users, as reflected in its documentation: service administrators, and project contributors. The admins perform maintenance tasks on the services. The projects contributors, or users to make it simple, check the status of their builds, encrypt secrets to be used in jobs, and retrieve jobs artifacts. Historically the Zuul client CLI was meant to be used by admins to perform maintenance tasks. It assumed access to Zuul's gearman server; so usually it would be run directly on Zuul's instance by operators. Some other admin scripts also live in the tools directory of the zuul repository, like zuul-changes.py and zuul-clear-refs.py. As for users, there's encrypt_secret.py for secrets, and everything else happens in the GUI. However since v3 landed, many Zuul deployments implement multitenancy, and JWT authentication was added a while ago along with the tenant-scoped admin REST API, which covers most of the available gearman RPC commands. This introduces a third type of users: the tenant admins, who should be allowed to perform base maintenance tasks related to their tenant. I believe this is a request most of us Zuul operators have heard from our users. Until authentication lands in the web UI, the only way for tenant admins to perform their tasks is to use the REST API with an admin-issued token. This is why I have added support for the REST API in the admin CLI a while ago. As the REST API coverage increases, I imagined adding functionalities to the admin CLI that would fit more the needs of regular users, such as the possibility to stream a job's console log in a terminal. The problem is that it denatures the primary purpose of this CLI as a tool for operators. Thus I am proposing a split in the CLI, as in this review: https://review.opendev.org/#/c/729236/ The zuul-web-cli would address the use cases of tenant admins, and provide automation tools to regular users outside of the GUI. I would like for example to integrate the encrypt-secret.py tool in it, to streamline the user experience. Now the question is, should this "non-op" CLI live in the Zuul repository, or have its own dedicated project? Should it exist at all, or should we instead let users develop their own custom-tailored tools around the REST API? On the other hand, should admin tools like zuul-changes.py be integrated into the admin CLI? Let me know what you think! Cheers, -- Matthieu Huin Senior Software Developer Red Hat <https://www.redhat.com> <https://red.ht/sig>

3 3

Jenkins support in Zuul v3
by T, Mohamednisar (Nokia - IN/Bangalore) 28 Jul '20

28 Jul '20

Hello Zuul team, Thank you for developing sunch a wonderful gating system. We were using zuul v2 with Jenkins and gearman. We understood that, zuul wanted move away from Jenkins to ansible jobs. As Zuul v2 is deprecated and We would like to use Jenkins with zuul v3, I have two questions 1. Is there any plan to support Jenkins in v3. 2. If we add the feature to support Jenkins in v3 using gearman, is it possible to upstream the changes back to zuul Thanks & Regards, Nisar T

2 1

October/November PTG Date Selection
by Kendall Waters 27 Jul '20

27 Jul '20

Hello Everyone! We wanted to get your input on the best dates for holding the PTG now that its been annouced that the Summit itself will be virtual[1]. Please fill out the CIVS poll by August 3 at 15:00 UTC. We really appreicate your feedback! Poll: https://civs.cs.cornell.edu/cgi-bin/vote.pl?id=E_a69944444a9b7c93&akey=3ecc… <https://civs.cs.cornell.edu/cgi-bin/vote.pl?id=E_a69944444a9b7c93&akey=3ecc…> There are obviously many pros and cons to each option, below are the conflicts/reasons why we might not want to select that option. Option 0 - Two weeks before Summit (October 5-9) Conflicts with RC work External Event Conflicts: None? Option 1 - One week after Summit (October 26-30) Halloween on Oct 31 Perfect for release timing but event fatigue is a real concern External Event Conflicts: Open Source Summit EU, ODSC Open Data Science Conference East Cloud Foundry Summit EU, KVM Forum Option 2 - Two weeks after the Summit (November 2-6) USA Election day on Nov 3; could be a distraction, contributors needing to go vote (whatever that looks like with the virus), news, etc. Event Conflicts: None? Option 3 - Three weeks after the Summit (November 9-13) Pushes far into the cycle (4 weeks after previous release, a bit late, Veterns day/holiday - may affect France and Germany and US) External Events Conflicts: VMWorld EU -The Kendalls (diablo_rojo & wendallkaters) [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016068.html <http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016068.html>

1 0

Enabling the preferences dialog
by Sorin Sbarnea 24 Jul '20

24 Jul '20

Can we please enable the preferences dialog on Zuul? Over the last few months I was asked on multiple occasions about making some UI changes optional/configurable in zuul. So I added it, assured it fits well in the new pf4 UI, and struggle to convince other cores to review it. Please, take a look at https://review.opendev.org/#/c/723603/ and help me finish it. I guess that once first one goes in, it will much easier to review other ones. There are few other changes are waiting for the preference dialog: configure how many lines of code are included in abridged view (hardcoded to 42 now) ability to enable/disable ANSI rendering move of auto-reload option to preferences I know that previously it was mentioned a possible conflcit with "login" feature Matthew Huin was working on. Still the reality is that: user preferences is not linked to login and must work without login many deployments will not enable login due to performance concerns login change is outdated and not touched in more than a month, far away from being merged. Also please accept my apologies for nagging few of you over the last weeks about my (boring) UI changes. I hope that using the mailing list would be less intrusive and maybe more effective. Cheers, Sorin

2 1

Can me migrate zuul-jobs towards being an Ansible collection?
by Sorin Sbarnea 23 Jul '20

23 Jul '20

As I really like the quality and reusability of roles present in zuul-jobs, I would like to have bit more flexibility regarding installing them, basically being able to package them and install them as a collection. Can we agree on going into the direction of publishing it as collection in galaxy? The only major issue we have is that galaxy requires all roles not to use dashes, underlines being accepted. I think it may be possible to perform a slow transition by creating symlinks, maybe even combined with a rename of only those that are generic enough, like ensure ones. The idea is that is impossible to publish a collection that contains roles with underlines and this is not going to change. Still, it is possible to package only a set of directories in your collection, ignoring others. The second issue I am aware of is related to documentation. Galaxy expects documentation as markdown and also the meta file. That is not such a big deal as Sphinx can build documentation from both using myst-parser extension. If we agree that we want to go this path, probably it would make sense to start with a one role collection and slowly migrate others. Cheers Sorin Sbarnea

2 1

Moving zuul UI preferences in a single location
by Sorin Sbarnea 06 Jul '20

06 Jul '20

As the number of configurable user preferences seems to be growing (see list below), I think is time to put them in a single place, instead of scattering them in various places in the UI, as this would only clutter it. Also it does not scale well. - timezone config (now on top nav bar) - ability to configure log wrapping on/off (requested) - ability to enable/disable ANSI support (also requested due to performance concerns) - auto-reload on status page - other future options I proposed adding a config icon that popup a dialog where we can put these configurations. https://sbarnea.com/ss/Screen-Shot-2020-07-06-13-43-58.png https://review.opendev.org/#/c/739482/1 - adds the config dialog, dialog is now empty bu the goal was to demonstrate how it would work first. I also proposed removal of cookies and replacement with localStorage, which allowed us to simplify the code and also avoid the GDPR legal issues https://review.opendev.org/#/c/739454/ Regarding enabling ANSI support, there are two alternatives proposed (each using a different react ANSI library) https://review.opendev.org/#/c/739444/ -- via react-ansi (more terminal like, with line numbers) https://review.opendev.org/#/c/716251/ -- via ansi-to-react If we consolidate user configurable preferences into a single screen and make use of localStore it may be much easier to gradually introduce changes in zuul, as we can make them opt-ins. What do you think? Cheers Sorin

2 1

Making yum errors visible by default in zuul
by Sorin Sbarnea 05 Jul '20

05 Jul '20

Few weeks back I proposed a change that would expose the "failures" list of errors in zuul UI when a module returns it, similar to stdout and stderr. The change got in limbo after @corvus remarked that this is not a standard way to return errors for modules. He mentioned that they could get to the full output by pressing the failed button (which is another UX issue by itself because is presented as a label, only few people would guess that this is clickable). The reality more ugly than this, and finding the error for a non-veteran zuul user is a real adventure: * Once landed on the "summary" page the user would see only the "Failed to install some of the specified packages", yep **some**, not the list. * There is no butten user can press from summary page to open full return list. * Hopefully user will decide to pick "console" tab instead of "logs" which would likely confuse him more * Once landed on console page, at least he would see the failed task as red, but still w/o required details * Now he would have to guess that the red [FAILED] block is clickable * After he clicks it, he would finally get to the "failures" list. https://review.opendev.org/#/c/734833/ Returning failures was the behavior of yum module for a very long time and there are no plans to change it from Ansible side (see linked bug). Yum module is one of the most used ones, also a core plugin. Considering that my proposal was to make the field visible in summary and preview happens only when a task failed and contains that field, why do we prefer to cripple the user experience for all users? Even if Ansible would decide to magically make the msg from yum module better, this will not help us for many years. Such a change would not lang sooner than 2.11, which means that at least two more years until we will display friendly errors. I know that likely most, if not all members of zuul-discuss have a good knowledge of Zuul, so you are likely to already know stuff that is not obvious. I encourage you to step back and consider what a first-time contributor, that never used Zuul, would see and do when it gets his first CI failure. The reason why I am writing this is that I am aware of several other UX issues with zuul and I personally faced some less then pleasant feedback from first time users over the last year, especially after I helped enabling Zuul CI on Ansible molecule and ansible-lint projects. I would be very happy if you would join me in my quest for improving Zuul user experience, especially from the browser point of view. Thanks Sorin Sbarnea

2 1

[Diversity] Diversity & Inclusion WG Meeting 7/6
by Amy Marrich 03 Jul '20

03 Jul '20

The Diversity & Inclusion WG invites members of all OSF projects to our next meeting Monday, July 6th, at 17:00 UTC in the #openstack-diversity channel. The agenda can be found at https://etherpad.openstack.org/p/diversity -wg-agenda. We will be discussing changing our Wiki page to reflect the broader OSF projects and communities so that the page reflects our mission. Please feel free to add any other topics you wish to discuss at the meeting. Thanks, Amy (spotz)

1 0

Wiki article on Zuul
by Jimmy McArthur 02 Jul '20

02 Jul '20

Hi everyone, We're still working to get the Zuul wiki article accepted by Wikipedia. The latest additions we need are: * A table of release history including the date and version of each release. Similar to what you see on the OpenStack wiki [1] * Graphics, diagrams, photos from past presentations/slide decks (preferably published at a known/trusted URL) * More frequent usage of links to other Wikipedia articles You can take a look at the current progress here [2] as well as the current set of proposed edits here [3]. If you all can chip in on any of these, please let me know :) Thank you! Jimmy [1] https://en.wikipedia.org/wiki/OpenStack [2] https://en.wikipedia.org/wiki/Draft:Zuul_(software) [3] https://docs.google.com/document/d/1HOw0weCATKrksxOQrjyIM6bZG7b6Zs_fobeNdZ0…

1 0