April 2021 - Zuul-discuss - lists.opendev.org

install and use ansible collections in zuul v3 ansible native job?
by Marios Andreou 23 Nov '21

23 Nov '21

hi folks long story short: can we install and use ansible collections in a zuul v3/ansible native job? In the tripleo-ci squad we are trying to use the tripleo_overcloud_image_build role from the tripleo-operator-ansible collection [1]. We are already using other roles from this collection - e.g. there [2]. This is possible because in that case we have zuul calling bash calling ansible... which means we get to configure collections in the bash bit setting ANSIBLE_COLLECTIONS_PATHS [3] before invoking ansible. For the case I'm having trouble with - what i refer to as ansible native - here https://review.opendev.org/#/c/733818/33/roles/oooci-build-images/tasks/mai… we have a zuul job playbook directly trying to install and use a collection. Currently it fails because instead of collections: (doesn't work " 'collections' is not a valid attribute? [4] ) I am trying to use include_role but then it can't find the collection's modules "ERROR! no action detected in task" [5]. I tried to set it in ansible.cfg but that won't work and anyway it is surely the wrong approach. The one thing left to try is to explicitly require ansible 2.9 for the job - but wanted to reach out and see if anyone else has any problems trying to install and use ansible collections (with modules importantly) as part of a zuul v3/ansible native job. thanks for reading and for any comments, marios [1] https://github.com/openstack/tripleo-operator-ansible/tree/master/roles/tri… [2] https://opendev.org/openstack/tripleo-quickstart-extras/src/commit/55e74bb8… [3] https://opendev.org/openstack/tripleo-quickstart/src/commit/299131bcc6dca86… [4] https://23fd78c0f6b01b9c4209-59c294b8f46eb75cda2bd004e2a1c97f.ssl.cf1.rackc… [5] https://2063b7e2bb67f86f8ec5-87fe67c942382c6ee875c514bffa4f21.ssl.cf2.rackc…

3 8

GitHub issues with 4.2.0
by Artem Goncharov 25 May '21

25 May '21

Hi all, I had unfortunately no time to investigate it deeper and rolled back to 4.1 (since we deploy zuul in k8 there are currently no old logs). What I have however noted: - gating for GitHub driver doesn’t work for us. The event is empty (https://opendev.org/zuul/zuul/src/branch/master/zuul/driver/github/githubmo… <https://opendev.org/zuul/zuul/src/branch/master/zuul/driver/github/githubmo…> tells event.label is empty). The same was also happening for reviews (https://opendev.org/zuul/zuul/src/branch/master/zuul/driver/github/githubmo… <https://opendev.org/zuul/zuul/src/branch/master/zuul/driver/github/githubmo…> - "because Reviews [{'type': 'approved', 'permission': 'write'}] does not match dict_values([])”) - merger was failing: https://opendev.org/zuul/zuul/src/branch/master/zuul/merger/merger.py#L1032 <https://opendev.org/zuul/zuul/src/branch/master/zuul/merger/merger.py#L1032> - ‘branch’ key is missing. Here I might be giving wrong line reference, since this is purely from memory, but it was definitely complaining to ‘branch' Rollback to 4.1 immediately "fixed" both issues, therefore I do not think this is a problem in the configuration on its own. I would try to deploy some test instance to play with it. This is currently just a feedback to 4.2.0 and potential warning to the ones using Zuul with GitHub Regards, Artem

3 5

[Diversity] Diversity & Inclusion WG Meeting reminder
by Amy Marrich 30 Apr '21

30 Apr '21

The Diversity & Inclusion WG invites members of all OSF projects to our meeting Monday May 3rd, at 17:00 UTC in the #openinfra-diversity channel. The agenda can be found at https://etherpad.openstack.org/p/diversity -wg-agenda. Please feel free to add any topics you wish to discuss at the meeting. Thanks, Amy (spotz)

1 0

help to getting speculative merged testing fixed
by Åsmund Østvold 30 Apr '21

30 Apr '21

hi Probably a newbie mistake. I have: - job: name: build-integration-tests parent: base run: playbooks/build_and_integrationtest.yaml required-projects: - resoptima/irma - name: resoptima/tools override-checkout: main - zuul/irma-common-jobs - resoptima/irma.fileimporter In the gate pipeline I get parallel unrelated merged executions, see attached image. I am sure I did something wrong. Any help appreciated! Cheers, Åsmund [image: image.png]

2 2

questions about locally running zuul repo tests
by mikeyang＠andestech.com 27 Apr '21

27 Apr '21

Hi, As part of my eval, I'm looking at the dev process. And we're interested in a Gitea driver (it looks like there was a WIP change<726391:%20more%20work%20towards%20a%20working%20gitea%20driver%20|%20https:…> last year). When I try to run the tests locally with "tox", following the instructions in TESTING.rst, it looks like the Docker Zookeeper instance does not start up properly and the majority of tests fail with a timeout. I tried modifying "tox.ini" in order to open/map port 2181 and I can see that "tox" starts up the zookeeper container, but the tests still timeout. Looking at the code, I see a couple of issues: 1. hostname ... how does tox know the hostname/ip-address of the container (with default bridge networking) 2. certificates ... are these required and how is tox configuring this? Or am I on the wrong path? I see that tools/test-setup-docker.sh seems to have a different way of running the tests. Thanks, - mike. CONFIDENTIALITY NOTICE: This e-mail (and its attachments) may contain confidential and legally privileged information or information protected from disclosure. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein is strictly prohibited. In this case, please immediately notify the sender by return e-mail, delete the message (and any accompanying documents) and destroy all printed hard copies. Thank you for your cooperation. Copyright ANDES TECHNOLOGY CORPORATION - All Rights Reserved.

2 3

Newbie question(s) for eval
by mikeyang＠andestech.com 21 Apr '21

21 Apr '21

Hi, I’m doing an eval of Zuul-CI for my team. I’ve previously used Jenkins and internally/purpose-built CI systems. So Ansible is new to me. I’m using the quick-start tutorial so this is all in the Docker containers and images from the tutorial. I’m running this on a Ubuntu bare-metal machine. And please let me know if there is a more appropriate forum to ask this question or get eval support. So what I tried to do was use a simple job from zuul-jobs. I thought `markdownlint` would be straight-forward. Here’s the rabbit hole that I went down. 1. The quick-start tutorial is using `Ubuntu Focal` images for the `node` 2. I added a second `node` using `Ubuntu Groovy` 3. I can run `testjob` and a custom job (playbook with `shell`s) on the Groovy node 4. I tried to run the `markdownlint` Zuul job on the Groovy node but it fails on the `ensure-nodejs` role * It tries to add `https://deb.nodesource.com/node_6.x groovy main` as a repo * This fails because `dists/groovy` only exists in the 10/12/14/15.x repos * I tried many different ways to assign `node_version` to 15, but the job always fails trying to add `node_6.x` From the Ansible documentation, I see that there are many places in precedence to overwrite a role’s default value, but I have not gotten any of these to work. And I’m unclear about how the precedence extends into Zuul’s `job.vars` and `jobs.extra-vars` (are these equivalent to passing on Ansible’s command line?). Thanks, - mike. CONFIDENTIALITY NOTICE: This e-mail (and its attachments) may contain confidential and legally privileged information or information protected from disclosure. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein is strictly prohibited. In this case, please immediately notify the sender by return e-mail, delete the message (and any accompanying documents) and destroy all printed hard copies. Thank you for your cooperation. Copyright ANDES TECHNOLOGY CORPORATION - All Rights Reserved.

3 5

Nested repo structures
by James E. Blair 20 Apr '21

20 Apr '21

Hi, We were recently presented with a case of a user who uses Gerrit's support for arbitrary repo hierarchies in an unexpected way. Essentially, they have both of the following projects defined in gerrit: * project * project/subproject That is to say, project is not just a pathname, it is a full-blown Gerrit repo. Further, this has nothing to do with git submodules or anything. There are several potential issues in Zuul related to nested repos: * Stats * Secret key storage * Time database * Internal merger repos * Workspace checkouts Stats are good. We already flatten project names there (eg, translate / to _). The secret key storage we are about to move to ZK. We can handle that right now. The time database will be eliminated as part of the v5 work as well, and replaced with SQL. Not a problem. That leaves the git repos. The main issue is that git doesn't deal well with git repos that are cloned into other git repos. We need to keep these separate. The internal merger repos (that is, the ones that users never see) will need a flattened namespace. That should be relatively straightforward since we can take care of that without a user-visible impact. Finally, that leaves the subject of this mail: the user-visible workspace checkouts. Once we get Zuul to a point where it doesn't error-out on a nested repo structure, we still have to decide how to lay it down on disk in the executor workspace. I can see three options: 1) Attempt to lay the repos out in the current fashion, with project/subproject cloned on top of project. I do not think we should do this. While we might be able to do this in the limited context of a workspace checkout (as compared to the internal merger repos), it would still be fraught with problems if, for example, anyone ever did add a "subproject" directory to "project". 2) Allow users to map projects to arbitrary repo checkouts. For this we might add a new mapping to a job like this: - job: checkouts: project: project project/subproject: subproject This would allow users to write out the repos wherever it suits them, whether that's a flat structure as above, or something more complex. They could also move just a single troublesome repo. In fact, you'd really only need the second line of the map above, the first is redundant. 3) Allow users to indicate they want a flat checkout. Given that this is the first use-case for this which we have seen, it's instructive to learn how the users who prompted this discussion are actually using their repos. They want a flat checkout. So they essentially only want the last component of the path to appear in their workspace. This option is a specialized version of the above that's less flexible, but probably simpler to use for this case: - job: flat-checkout: true That would map each project to its last path component. We would need to decide what to do if *that* caused a collision. Presumably we would error out. In this case, the user would have no recourse with Zuul (they would need to change a repo name or stop using a repo in that job). Options 2 and 3 are not mutually exclusive. We could do both so that the typical use would be #3 with #2 as a backup (or for someone with really creative ideas about how to lay down repos). We could also implement one and leave the other as an idea for future expansion. I suspect the internal implementation will be about the same for both. I think I'm inclined to start with #3 just because it concisely enables a specific pattern that may be useful in other contexts. Does anyone have feedback or a preference? -Jim P.S. I suspect we're going to find cases in zuul-jobs where we have assumed project.src_dir == 'src/'+project.canonical_name, but we can and should fix those. We made that a variable for a reason.

4 4

Proposed behavior change on reconfiguration
by James E. Blair 19 Apr '21

19 Apr '21

Hi, We recently merged some changes which improve the consistency of the git repos in different jobs within the same buildset. While it's always the case that every job in a buildset has the same view of the projects involved in its shared queue, projects outside of that queue (including projects involved in providing Zuul roles, playbooks and job required-projects) have not been synchronized, meaning that if a change merged in one of those projects between two jobs in a buildset starting, the earlier job would not see the change and the later job would. It seems more natural for Zuul to fix the state of all of the repos involved in all of the jobs of a buildset, and so the change that we recently merged did this. While this seems like a straightforward change, it had an unanticipated effect on reconfiguration. When Zuul performs a reconfiguration, any jobs that are added to a project are added to currently running buildsets and started; likewise, jobs may be removed from buildsets. Any jobs which are modified but have already run will retain their previous results, but jobs which have not yet run will do so with the new configuration. It turns out that the idea of fixing all of the repos involved in a buildset and also being able to dynamically add, remove, and alter jobs are fundamentally at odds. The simplest example of this is a reconfiguration that adds a playbook to the zuul-jobs repo, while a job on an unrelated repo is running If Zuul has frozen the state of all the repos involved (including zuul-jobs) then the updated version of the job will reference a playbook which will not appear in the previously frozen repo state. In other words, we can freeze both the configuration and the repos, or neither. Earlier in Zuul's development, we felt that the ability to add, remove, or alter jobs while they were running was fairly important. However, Zuul and its usage has evolved considerably since then. I'd like to propose that we continue down the path of freezing both the repo state and the job configuration within a buildset. This means than in the future, if a job is added, removed, or altered during a reconfiguration, it will not take effect for any currently running buildsets, but will take effect for any buildsets started after the reconfiguration (that includes new buildsets created on gate resets). Given the potentially significant change to a long-standing Zuul behavior, I ask that anyone relying on this or who feels that they may be affected by this to please let us know. Thanks, Jim

2 4

[Diversity] D&I WG Office Hours at the PTG
by Amy Marrich 16 Apr '21

16 Apr '21

The Diversity and Inclusion WG has grabbed our usual PTG slot Monday at 16:00UTC and we will be holding office hours to help any project with Inclusive Naming or other D&I related topics. If you'd like us to attend any of your sessions please let me know. If you've been interested in learning more about the WG please come join us! Thanks, Amy (spotz)

1 0