August 2020 - Zuul-discuss - lists.opendev.org

install and use ansible collections in zuul v3 ansible native job?
by Marios Andreou 23 Nov '21

23 Nov '21

hi folks long story short: can we install and use ansible collections in a zuul v3/ansible native job? In the tripleo-ci squad we are trying to use the tripleo_overcloud_image_build role from the tripleo-operator-ansible collection [1]. We are already using other roles from this collection - e.g. there [2]. This is possible because in that case we have zuul calling bash calling ansible... which means we get to configure collections in the bash bit setting ANSIBLE_COLLECTIONS_PATHS [3] before invoking ansible. For the case I'm having trouble with - what i refer to as ansible native - here https://review.opendev.org/#/c/733818/33/roles/oooci-build-images/tasks/mai… we have a zuul job playbook directly trying to install and use a collection. Currently it fails because instead of collections: (doesn't work " 'collections' is not a valid attribute? [4] ) I am trying to use include_role but then it can't find the collection's modules "ERROR! no action detected in task" [5]. I tried to set it in ansible.cfg but that won't work and anyway it is surely the wrong approach. The one thing left to try is to explicitly require ansible 2.9 for the job - but wanted to reach out and see if anyone else has any problems trying to install and use ansible collections (with modules importantly) as part of a zuul v3/ansible native job. thanks for reading and for any comments, marios [1] https://github.com/openstack/tripleo-operator-ansible/tree/master/roles/tri… [2] https://opendev.org/openstack/tripleo-quickstart-extras/src/commit/55e74bb8… [3] https://opendev.org/openstack/tripleo-quickstart/src/commit/299131bcc6dca86… [4] https://23fd78c0f6b01b9c4209-59c294b8f46eb75cda2bd004e2a1c97f.ssl.cf1.rackc… [5] https://2063b7e2bb67f86f8ec5-87fe67c942382c6ee875c514bffa4f21.ssl.cf2.rackc…

3 8

[summit] CFP Deadline Soon! - Virtual Open Infrastructure Summit
by Jimmy McArthur 10 Sep '20

10 Sep '20

Hi everyone, It’s time to submit your Open Infrastructure virtual Summit presentations[1]! The CFP deadline is tomorrow. Submit sessions featuring open source projects including Airship, Ansible, Ceph, Kata Containers, Kubernetes, ONAP, OpenStack, OPNFV, StarlingX and Zuul. As a reminder, these are the 2020 Tracks: 5G, NFV & Edge AI, Machine Learning & HPC CI/CD Container Infrastructure Getting Started Hands-on Workshops Open Development Private & Hybrid Cloud Public Cloud Security Get your presentations, panels, and workshops in before August 4 at 11:59 pm PT (August 5 at 6:59 am UTC). The content submission process for the Forum and Project Teams Gathering (PTG) will be managed separately in the upcoming months. The Summit Programming Committee has shared topics by Track for community members interested in speaking at the upcoming Summit. Check out the submission tips[2]! Then don’t forget to register[3] for the virtual Open Infrastructure Summit taking place October 19-23 at no cost to you. Need more time? Reach out to speakersupport(a)openstack.org with any questions or concerns. Cheers, Jimmy [1] https://cfp.openstack.org/ [2] https://superuser.openstack.org/articles/virtual-open-infrastructure-summit… [3] https://openinfrasummit2020.eventbrite.com

1 1

[Diversity] Diversity & Inclusion WG Meeting 8/31 - Removing Divisive Language
by Amy Marrich 27 Aug '20

27 Aug '20

The Diversity & Inclusion WG has taken on the task from this week's Board meeting to assist with the development of the OSF's stance on the removal of Divisive Language within the OSF projects. The WG invites members of all OSF projects to participate in this effort and to join us at our next meeting Monday, August 31, at 17:00 UTC which will be held at https://meetpad.opendev.org/osf-diversity-and-inclusion. <https://meetpad.opendev.org/osf-diversity-and-inclusion> The agenda can be found at https://etherpad.openstack.org/p/diversity-wg-agenda. If you have any questions please let me and the team know here, on #openstack-diversity on IRC, or you can email me directly. Thanks, Amy Marrich (spotz)

1 0

Enabling ANSI output display in zuul
by Sorin Sbarnea 26 Aug '20

26 Aug '20

I initially proposed the first version for ANSI support about 6 months ago, but still we do have native ANSI display for output. Please take a look at review below, I even included screenshots. https://review.opendev.org/#/c/739444/ I fact I do have a longer list of changes ready for review that aim to improve the web UX experience: https://review.opendev.org/#/q/topic:zuul-ux+(status:open+OR+status:merged) Thanks Sorin

1 0

Addressing users needs via the CLI
by Matthieu Huin 20 Aug '20

20 Aug '20

Hello, I would like to discuss the current state of Zuul's CLI, especially from a user's point of view. We can identify two types of Zuul users, as reflected in its documentation: service administrators, and project contributors. The admins perform maintenance tasks on the services. The projects contributors, or users to make it simple, check the status of their builds, encrypt secrets to be used in jobs, and retrieve jobs artifacts. Historically the Zuul client CLI was meant to be used by admins to perform maintenance tasks. It assumed access to Zuul's gearman server; so usually it would be run directly on Zuul's instance by operators. Some other admin scripts also live in the tools directory of the zuul repository, like zuul-changes.py and zuul-clear-refs.py. As for users, there's encrypt_secret.py for secrets, and everything else happens in the GUI. However since v3 landed, many Zuul deployments implement multitenancy, and JWT authentication was added a while ago along with the tenant-scoped admin REST API, which covers most of the available gearman RPC commands. This introduces a third type of users: the tenant admins, who should be allowed to perform base maintenance tasks related to their tenant. I believe this is a request most of us Zuul operators have heard from our users. Until authentication lands in the web UI, the only way for tenant admins to perform their tasks is to use the REST API with an admin-issued token. This is why I have added support for the REST API in the admin CLI a while ago. As the REST API coverage increases, I imagined adding functionalities to the admin CLI that would fit more the needs of regular users, such as the possibility to stream a job's console log in a terminal. The problem is that it denatures the primary purpose of this CLI as a tool for operators. Thus I am proposing a split in the CLI, as in this review: https://review.opendev.org/#/c/729236/ The zuul-web-cli would address the use cases of tenant admins, and provide automation tools to regular users outside of the GUI. I would like for example to integrate the encrypt-secret.py tool in it, to streamline the user experience. Now the question is, should this "non-op" CLI live in the Zuul repository, or have its own dedicated project? Should it exist at all, or should we instead let users develop their own custom-tailored tools around the REST API? On the other hand, should admin tools like zuul-changes.py be integrated into the admin CLI? Let me know what you think! Cheers, -- Matthieu Huin Senior Software Developer Red Hat <https://www.redhat.com> <https://red.ht/sig>

3 3

Re: Zuul POC : Facing issues while configuring pipeline in Zuul;.
by Divya sopan Hole 19 Aug '20

19 Aug '20

Hi James, I have configured the same in /etc/zuul/ and /etc/nodepool/ as per documentations given http://zuul-ci.org/ in my ubuntu-server system and the same configuration of zuul.conf, main.yaml and nodepool.yaml in respective folders etc_zuul and etc_nodepool can be found in given GitHub repo link https://github.com/zuul-org/zuul-test I have configured jobs.yaml, project.yaml and pipelines.yaml (Github driver) under zuul.d And the jobs.yaml is dependent on testjob.yaml These are the following findings after running on localhost:9000 You can find the screenshot of* status, jobs, projects, builds, buildsets, nodes and tenant * in the given github repository mentioned above. 1. I am unable to see the build history of jobs and screen shows "Loading..." Please, provide a proper solution so that I can see my builds. 2. Where can I see a console log of builds section on my system? 3. I found error while running zuul-scheduler 4. I have tried to commit changes in repository and pushed the changes but I am not able to see any changes in the pipeline of Zuul dashboard Example - Coloured bars in check pipeline or gate pipeline I have followed the same procedure as provided by you .Thank you for your input . Thanks & Regards, Divya Hole. On Tue, Aug 18, 2020 at 6:44 PM Divya sopan Hole <divyasopanhole(a)gmail.com> wrote: > Hi James, > > I have configured the same in /etc/zuul/ and /etc/nodepool/ as per > documentations given http://zuul-ci.org/ > in my ubuntu-server system and the same configuration of zuul.conf, > main.yaml and nodepool.yaml in respective > folders etc_zuul and etc_nodepool can be found in given GitHub repo link > https://github.com/zuul-org/zuul-test > > I have configured jobs.yaml, project.yaml and pipelines.yaml (Github > driver) under zuul.d > And the jobs.yaml is dependent on testjob.yaml > > These are the following findings after running on localhost:9000 > You can find the screenshot of* status, jobs, projects, builds, > buildsets, nodes and tenant*. > > 1. I am unable to see the build history of jobs and screen shows > "Loading..." Please, provide a proper solution so that I can see my builds. > 2. Where can I see a console log of builds section on my system? > 3. I found error while running zuul-scheduler > > I have followed the same procedure as provided by you .Thank you for your > input . > Thanks & Regards, > Divya Hole. > > On Tue, Aug 18, 2020 at 3:35 AM James E. Blair <corvus(a)inaugust.com> > wrote: > >> Divya sopan Hole <divyasopanhole(a)gmail.com> writes: >> >> > Hello Team, >> > >> > We are doing POC on Zuul implementation on cloud infrastructure. >> > >> > During implementation we are stuck in pipeline creation & also not able >> to >> > see any zuul status on GitHub. >> >> Hi! Sorry you ran into problems, but hopefully we can fix them and make >> our documentation better. >> >> First, did you try going through the Zuul quickstart? >> >> You can find it here: >> https://zuul-ci.org/docs/zuul/tutorials/quick-start.html >> >> It uses Gerrit rather than Github, but the basic setup is the same, so >> it should help answer some of these questions. >> >> > Problem Title: Facing issues while configuring CI CD pipeline in Zuul >> > >> > OS , Tools & version: Ubuntu (VM), Zuul 3.0, github.com >> > >> > Description of the problem: >> > >> > 1.Where should the config files need to be placed? Config files such as >> > pipeline.yaml , .zuul.yaml etc. >> >> Pipeline and other configuration should go in a .zuul.yaml or zuul.yaml >> file in the root of a config-project. Or they can go in multiple files >> in the .zuul.d or zuul.d directory. Start by creating a dedicated git >> repo for holding these, and add files as you see fit. >> >> This part of the quickstart addresses this: >> >> >> https://zuul-ci.org/docs/zuul/tutorials/quick-start.html#configure-zuul-pip… >> >> > 2.Where should the config project and untrusted project reside and how >> to >> > define it ? >> > (Like full path or repo address) >> >> It can be named anything and reside anywhere. Since you're using >> Github, I'd suggest a Github project in the same Github org you usually >> use. For example, you might create a repo at >> github.com/example/zuul-config. >> >> After you create that, add it to the Zuul tenant config file. The >> quickstart does not talk about this. Perhaps it should. We usually >> place it in "/etc/zuul/main.yaml" on the zuul scheduler. Here's the >> documentation for it: >> >> https://zuul-ci.org/docs/zuul/reference/tenants.html >> >> You'll want to make it look something like this: >> >> - tenant: >> name: my-tenant >> report-build-page: true >> source: >> github: >> config-projects: >> - example/zuul-config >> >> Where the last line is the name of the repo you just created. You >> should also add whatever repos you want to test after that: >> >> untrusted-projects: >> - example/test-project >> >> > 3.We configured our pipelines and all in a separate github repo but does >> > not show "builds"(zuul dashboard showing continuously loading. >> > It is not showing any build reports on Github ) >> >> If you don't even see empty pipelines on the dashboard, then the >> scheduler may not have loaded the configuration correctly. Start by >> looking at the logs for the zuul-scheduler process and see if there are >> warnings or errors there. >> >> > 4.We are currently running all jobs in localhost that means no nodes >> apart >> > from localhost. >> > Do we need extra nodes to run the jobs ? >> >> No other nodes are needed, but I would recommend this: >> >> * Start by only running "noop" jobs. They are entirely internal and >> don't do anything. That helps you get the main system up and running. >> >> * You can run jobs on "localhost" (using zero-node nodesets), but what >> you can run there is fairly restricted unless the job is defined in a >> config-project. >> >> * Since you're doing a proof of concept, I'd suggest the next step after >> running noop would be to set up a nodepool launcher with a static >> node. That will let you run some basic jobs on an existing server (or >> vm, or container) without worrying too much about nodepool setup. >> >> * After that, move on to using "real" resources (whether those are >> containers, vms, or real static servers). >> >> > Steps tried: >> > >> > 1.Created Github App using the link : >> > https://probot.github.io/docs/development/#generating-a-new-app >> > >> > 2.Tried configuring Zuul pipeline.yaml under zuul-test-conf repo and >> > .zuul.yaml >> > >> > 3.Added an initial pipeline configuration to >> > the zuul-test-config repository. >> > Inside that project, we have created a zuul.yaml file. >> > >> > 4.In the zuul-test project, created a .zuul.yaml file with the following >> > contents: >> > >> > project: >> > check: >> > jobs: >> > - noop >> > >> > Can you please help us on the solution or documentation where we can >> look >> > for? >> >> That all looks good. The main thing I see missing is the tenant >> configuration. So I'm guessing if you check your scheduler log, it will >> tell you it is incorrect or missing. >> >> -Jim >> >

1 0

Zuul is now available in Fedora
by Fabien Boucher 18 Aug '20

18 Aug '20

Hi, I would like to share that Zuul is now packaged into Fedora. It will be available with the next Fedora 33 [1]. Nodepool is also part of Fedora. Zuul and Nodepool packages source are available here: [2] and [3]. If you are interested, help will be welcome to keep those packages into Fedora so do not hesitate to propose PRs. PRs are validated by Zuul already [4] (build, lint, functional tests [5]). Regards Fabien Boucher [1]: https://koji.fedoraproject.org/koji/packageinfo?packageID=24098 [2]: https://src.fedoraproject.org/rpms/zuul [3]: https://src.fedoraproject.org/rpms/nodepool [4]: https://src.fedoraproject.org/rpms/zuul/pull-request/3 [5]: https://src.fedoraproject.org/rpms/zuul/blob/master/f/tests/tests.yml

2 1

Zuul POC : Facing issues while configuring pipeline in Zuul;.
by Divya sopan Hole 18 Aug '20

18 Aug '20

Hello Team, We are doing POC on Zuul implementation on cloud infrastructure. During implementation we are stuck in pipeline creation & also not able to see any zuul status on GitHub. Problem Title: Facing issues while configuring CI CD pipeline in Zuul OS , Tools & version: Ubuntu (VM), Zuul 3.0, github.com Description of the problem: 1.Where should the config files need to be placed? Config files such as pipeline.yaml , .zuul.yaml etc. 2.Where should the config project and untrusted project reside and how to define it ? (Like full path or repo address) 3.We configured our pipelines and all in a separate github repo but does not show "builds"(zuul dashboard showing continuously loading. It is not showing any build reports on Github ) 4.We are currently running all jobs in localhost that means no nodes apart from localhost. Do we need extra nodes to run the jobs ? Steps tried: 1.Created Github App using the link : https://probot.github.io/docs/development/#generating-a-new-app 2.Tried configuring Zuul pipeline.yaml under zuul-test-conf repo and .zuul.yaml 3.Added an initial pipeline configuration to the zuul-test-config repository. Inside that project, we have created a zuul.yaml file. 4.In the zuul-test project, created a .zuul.yaml file with the following contents: project: check: jobs: - noop Can you please help us on the solution or documentation where we can look for? Regards, Divya Hole

2 1

Forum Brainstorming
by Kendall Nelson 18 Aug '20

18 Aug '20

Hello Everyone! The Virtual Forum is approaching. We would love 1 volunteer from the community for the Forum Selection Committee. Ideally, the volunteer would already be serving in some capacity in a governance role for your project. In addition to calling for volunteers for the Forum selection committee, this email kicks off the brainstorming period before the CFP tool opens for formal Forum submissions. The categories for brainstorming etherpads have already been setup here[1]. Please add your etherpads and ideas there! The CFP tool will open on August 31st and will close September 14th. For information on the upcoming virtual Summit[2]. For more information on the Forum[3]. Please reach out to jimmy(a)openstack.org or knelson(a)openstack.org if you're interested. Volunteers should respond on or before August 31, 2020. Thanks! Kendall (diablo_rojo) [1] Virtual Forum 2020 Wiki: https://wiki.openstack.org/wiki/Forum/Virtual2020 [2] Virtual Open Infra Summit Site: https://www.openstack.org/summit/2020 [3] General Forum Wiki: https://wiki.openstack.org/wiki/Forum

1 0

vPTG October 2020 Team Signup
by Kendall Nelson 13 Aug '20

13 Aug '20

Greetings! As you hopefully already know, our next PTG will be virtual again, and held from Monday October 26th to Friday October 30th. We will have the same schedule set up available as last time with three windows of time spread across the day to cover all timezones with breaks in between. *To signup your team, you must complete **BOTH** the survey[1] AND reserve time in the ethercalc[2] by September 11th at 7:00 UTC.* We ask that the PTL/SIG Chair/Team lead sign up for time to have their discussions in with 4 rules/guidelines. 1. Cross project discussions (like SIGs or support project teams) should be scheduled towards the start of the week so that any discussions that might shape those of other teams happen first. 2. No team should sign up for more than 4 hours per UTC day to help keep participants actively engaged. 3. No team should sign up for more than 16 hours across all time slots to avoid burning out our contributors and to enable participation in multiple teams discussions. Again, you need to fill out BOTH the ethercalc AND the survey to complete your team's sign up. If you have any issues with signing up your team, due to conflict or otherwise, please let me know! While we are trying to empower you to make your own decisions as to when you meet and for how long (after all, you know your needs and teams timezones better than we do), we are here to help! Once your team is signed up, please register! And remind your team to register! Registration is free, but since it will be how we contact you with passwords, event details, etc. it is still important! Continue to check back for updates at openstack.org/ptg. -the Kendalls (diablo_rojo & wendallkaters) [1] Team Survey: https://openstackfoundation.formstack.com/forms/june2020_virtual_ptg_survey [2] Ethercalc Signup: https://ethercalc.openstack.org/126u8ek25noy [3] PTG Registration: https://october2020ptg.eventbrite.com

1 2