October 2020 - Zuul-discuss - lists.opendev.org

install and use ansible collections in zuul v3 ansible native job?
by Marios Andreou 23 Nov '21

23 Nov '21

hi folks long story short: can we install and use ansible collections in a zuul v3/ansible native job? In the tripleo-ci squad we are trying to use the tripleo_overcloud_image_build role from the tripleo-operator-ansible collection [1]. We are already using other roles from this collection - e.g. there [2]. This is possible because in that case we have zuul calling bash calling ansible... which means we get to configure collections in the bash bit setting ANSIBLE_COLLECTIONS_PATHS [3] before invoking ansible. For the case I'm having trouble with - what i refer to as ansible native - here https://review.opendev.org/#/c/733818/33/roles/oooci-build-images/tasks/mai… we have a zuul job playbook directly trying to install and use a collection. Currently it fails because instead of collections: (doesn't work " 'collections' is not a valid attribute? [4] ) I am trying to use include_role but then it can't find the collection's modules "ERROR! no action detected in task" [5]. I tried to set it in ansible.cfg but that won't work and anyway it is surely the wrong approach. The one thing left to try is to explicitly require ansible 2.9 for the job - but wanted to reach out and see if anyone else has any problems trying to install and use ansible collections (with modules importantly) as part of a zuul v3/ansible native job. thanks for reading and for any comments, marios [1] https://github.com/openstack/tripleo-operator-ansible/tree/master/roles/tri… [2] https://opendev.org/openstack/tripleo-quickstart-extras/src/commit/55e74bb8… [3] https://opendev.org/openstack/tripleo-quickstart/src/commit/299131bcc6dca86… [4] https://23fd78c0f6b01b9c4209-59c294b8f46eb75cda2bd004e2a1c97f.ssl.cf1.rackc… [5] https://2063b7e2bb67f86f8ec5-87fe67c942382c6ee875c514bffa4f21.ssl.cf2.rackc…

3 8

new projects without restart
by Åsmund Østvold 18 Dec '20

18 Dec '20

hi, Number of projects wanting to use my Zuul setup is increasing. I am looking for a way to tell the Zuul setup that the number of 'untrusted_projects' have increased without having interrupted running builds. Can not find a solution in the documentation. I have tried the zuul German client and was not able to find a solution. Is this possible? Cheers, Åsmund

2 2

Zuul's test framework
by Matthieu Huin 05 Nov '20

05 Nov '20

Hello, Yesterday a discussion came up on IRC about using zuul's test framework in other projects [1]. Basically the framework greatly simplifies running zuul processes, and sets up projects and simulates a gerrit for testing. It is used for zuul, nodepool and zuul-client tests. The problem is that the framework lib is not available through a zuul import. That means that tests specific to nodepool, zuul-client and potentially other projects need to be defined within that framework, in the zuul project repo. This is not ideal, and in the case of zuul-client, it leads to tests being written in follow-up patches to the zuul repo, which is a bit backward [2]. Let's use this thread to suggest improvements on the situation, and weigh pros and cons. We could: 1. Copy the bootstrapping code to other projects that need it - this represents a significant amount of code, and it's going to be painful to ensure there are no divergences in the code base. 2. Split the test framework into its own package. You'll still need to invoke zuul's tox job to set it up, but at least the tests could live in their relevant project's repository. Any other alternatives? Cheers, [1] http://eavesdrop.openstack.org/irclogs/%23zuul/%23zuul.2020-09-28.log.html#… [2] see for example https://review.opendev.org/#/c/750709/ and testing in https://review.opendev.org/#/c/754103/ -- Matthieu Huin Senior Software Developer Red Hat <https://www.redhat.com> <https://red.ht/sig>

2 2

[Diversity] Diversity & Inclusion WG Meeting 11/2 - Removing Divisive Language
by Amy Marrich 30 Oct '20

30 Oct '20

The Diversity & Inclusion WG will continue working on the OSF's stance on the removal of Divisive Language within the OSF projects. We will be focusing on the creation of possible words to focus on as well as possible replacements[0] as was requested during our Forum session[1]. The WG invites members of all OSF projects to participate in this effort and to join us at our next meeting Monday, November 2nd, at 17:00 UTC which will be held at https://meetpad.opendev.org/osf-diversity-and-inclusion. <https://meetpad.opendev.org/osf-diversity-and-inclusion> If you have any questions please let me and the team know here, on #openstack-diversity on IRC, or you can email me directly. Thanks, Amy Marrich (spotz) 0 - https://etherpad.opendev.org/p/wallaby-ptg-diversity 1- https://etherpad.opendev.org/p/vSummit2020__DivisiveLanguage

1 0

[PTG] [Diversity] Extra session for Divisive Language discussions
by Amy Marrich 27 Oct '20

27 Oct '20

During the Monday session there were some questions in regards to technical writing and how things should be worded. I have invited Joanne O'Flynn, a Technical Writer from Red Hat to join us on Thursday at 17:00 UTC to help address these questions. This is not a normal PTG time slot but it was the only option due to time zones and already booked sessions for some of the participants from Monday. Thanks, Amy Marrich (spotz)

1 1

Re: [service-announce] October 20 Gerrit Outage Update
by James E. Blair 23 Oct '20

23 Oct '20

Hi, You may have seen the message to the opendev service-announce list about the recent Gerrit compromise and outage. If you haven't, I've included a copy below. If you are a Zuul maintainer or a regular contributor, you may want to subscribe to that list. As a project, we have an action item to review some commits which merged during the vulnerable window to verify that they have not been maliciously altered (see the message below for full information and context). Clark has reviewed the zuul-jobs and zuul-website repos, and I have reviewed the zuul repo and we have found nothing amiss. It would not hurt for others to review these as well. Further, if you are the author of any of these commits and still have the versions you pushed to Gerrit in a tree which you are sure has not been updated or has only been fast-forwarded, it would be useful for you to verify the commit sha is the same. All of the relevant commits can be found in individual patch files here: https://static.opendev.org/project/opendev.org/gerrit-diffs/zuul/ Barring any last-minute discoveries of something Clark or I overlooked, I think we can call this task complete. -Jim On 10/20/20 5:33 PM, Ian Wienand wrote: > As of this mail, Gerrit access has been restored. Please read on for > important information, especially around change verification. > > Background > ----------- > > On 2020-10-20 at 01:30 a user unexpectedly added a workflow approval > to a change that they were not expected to have access to. At 02:06 > UTC an alert was raised via IRC. Administrators found the account had > added themselves to a core group and made the +W vote. The account > was disabled, and removed from the groups it had added itself to by > 02:55 UTC. Administrators began to analyse the situation and Gerrit > was taken offline at 04:02 UTC to preserve state and allow for > analysis. > > From this time, administrators were working on log collection and > analysis, along with restoring backups for comparison purposes. > > By around 08:45 UTC it was clear that the privilege escalation had > been achieved by gaining control of a Launchpad SSO account with > Gerrit administrator privileges. By this time, we had ruled out > software vulnerabilities. Logs showed the first unauthorized access > of the administrator account in Gerrit on 2020-10-06. Communication > with Launchpad admins agrees with this analysis. We saw one session > opened as the administrator user to StoryBoard on this same day, but > logs show no data was modified or hidden stories viewed. > > Analysis has been performed on the Gerrit database and git trees from > October 1st, pre-dating any known unauthorized access. > > Access was restored at around 2020-10-21 00:00 UTC > > Outcomes > ----------- > > The following has been verified: > > The administrator account used has been disabled and credentials > updated > > We have verified that all group and user addition/removals since > Oct 1 are valid. The only invalid additions were made by the > compromised administrator account to add a single user account to > the Administrators group; and then that account added itself to > another known group. > > The account given administrator privilege has been removed from > the groups it added itself to and is disabled. > > There is no evidence of any unauthorized access via methods other > than Gerrit HTTP and Gerrit SSH access. > > No commits have been pushed to git trees bypassing code review. > Every git tree has been compared to the Oct 1 version and all > commits have been correctly inserted via Gerrit changes. > > The version of Gerrit we use stores HTTP API passwords in > plain-text. We know that a limited number of passwords were > gathered via the HTTP API and it is possible passwords were > gathered via the database. We thus have assumed that all HTTP API > passwords have been disclosed. This password needs to be > explicitly enabled by users, and many users do not have it > enabled. > > Remediation > ----------- > > This leaves us with the following remediation actions: > > Users should double-check their Launchpad recent activity at > https://login.launchpad.net/activity for any suspicious logins. If > found, please notify the OpenDev admins in Freenode #opendev and > Launchpad admins in #launchpad immediately. > > All HTTP API passwords have been cleared. If you push changes via > HTTPS (instead of typical SSH), are a gertty user, or run a CI > system or something else that communicates with the Gerrit HTTP > API, you will need to regenerate a password. > > Any SSH keys added to accounts since 2020-10-01 have been removed. > This affects only a limited number of accounts. This is done in > an abundance of caution, and we do not believe any accounts had > unauthorized SSH keys added > > We should audit all changes for projects since 2020-10-01. > > We have no evidence that any account had its ssh keys compromised, > thus we can rule out any unauthorized changes being uploaded via SSH. > However we can not conclusively rule out that compromised HTTP API > passwords were used to push a change through Gerrit. For example, a > change could be uploaded that looks like it came from a user, or the > API key of a core team member may have been used to approve a change > without authorization. > > Given our extensive analysis we consider it exceedingly unlikely that > this vector was used. We have had no notifications of users seeing > unexpected changes either uploaded by them, or approved by them in > projects they work on. This said, we believe it is important to > inform the community of this very unlikely, but still possible, > vulnerability of the source code. > > To this end, we have prepared a list of all changes from the known > affected period which should be audited for correctness. These are > available at > > https://static.opendev.org/project/opendev.org/gerrit-diffs/ > > Team members should browse these changes and make sure they were > correctly approved in Gerrit. If any change looks suspicious you > should notify OpenDev administrators in Freenode #opendev immediately. > > Further actions > ---------------- > > We are planning the following for the short term future: > > The Opendev administrators will be looking at alternative models > for Gerrit admin account management. > > We are already well into planning and testing a coming upgrade to > a version of Gerrit which does not store plain-text API keys. > > Longer term, we've written a spec for replacing Launchpad SSO as > our authentication provider. > > We thank you for your patience during this trying time, and we look > forward to returning to supporting the community doing what it does > best -- working together to create great things. > > > _______________________________________________ > service-announce mailing list > service-announce(a)lists.opendev.org > http://lists.opendev.org/cgi-bin/mailman/listinfo/service-announce >

2 1

[Diversity][PTG] Diversity and Inclusion WG at the PTG - Divisive Language
by Amy Marrich 23 Oct '20

23 Oct '20

The Diversity and Inclusion WG will be holding a PTG session Monday from 15:00-17:00 UTC. We will be using Meetpad[0] atleast to start but will change to Zoom if needed. Based on feedback [2] during the Forum session we will be working on providing a more comprehensive list of words and possible alternatives vs the original plan to leave this aspect within the technical groups. Thanks, Amy Marrich (spotz) 0 - https://meetpad.opendev.org/wallaby-ptg-diversity 1 - https://etherpad.opendev.org/p/wallaby-ptg-diversity 2 - https://etherpad.opendev.org/p/vSummit2020__DivisiveLanguage

1 0

client issue
by Åsmund Østvold 20 Oct '20

20 Oct '20

hi, I want to run a zuul client command and get into issues. $ zuul --version Zuul version: 3.19.1 $ zuul --zuul-url http://zuul-ci.######.net/openapi autohold-list Traceback (most recent call last): File "/home/aos/.local/share/virtualenvs/zuul-config-Lr8Y6QyF/bin/zuul", line 8, in <module> sys.exit(main()) File "/home/aos/.local/share/virtualenvs/zuul-config-Lr8Y6QyF/lib/python3.8/site-packages/zuul/cmd/client.py", line 736, in main Client().main() File "/home/aos/.local/share/virtualenvs/zuul-config-Lr8Y6QyF/lib/python3.8/site-packages/zuul/cmd/client.py", line 385, in main if self.args.func(): File "/home/aos/.local/share/virtualenvs/zuul-config-Lr8Y6QyF/lib/python3.8/site-packages/zuul/cmd/client.py", line 473, in autohold_list client = self.get_client() File "/home/aos/.local/share/virtualenvs/zuul-config-Lr8Y6QyF/lib/python3.8/site-packages/zuul/cmd/client.py", line 393, in get_client client = ZuulRESTClient(self.args.zuul_url, File "/home/aos/.local/share/virtualenvs/zuul-config-Lr8Y6QyF/lib/python3.8/site-packages/zuul/cmd/client.py", line 42, in __init__ self.session = requests.Session( TypeError: __init__() got an unexpected keyword argument 'verify' It is a http and not a https server and I suspect that it could cause the issue. Anybody know if I am doing something wrong? Cheers, Åsmund

2 5

Introducing the Open Infrastructure Foundation
by Jeremy Stanley 19 Oct '20

19 Oct '20

Hi everyone, If you’re not watching the Open Infrastructure Summit keynotes right now, you just missed some big news (and it’s not too late to tune in! https://www.youtube.com/watch?v=wNNOFfDBB0o ). Jonathan Bryce just announced the Open Infrastructure Foundation (OIF) https://openinfra.dev/ as the successor to the OpenStack Foundation (OSF). The name change reflects all of our community’s work building open infrastructure with dozens of open source components, and we could not have done it without your help. We look forward to continuing to collaborate with you over the next decade to build open source communities that write software for production. The Summit keynotes are still going, so jump on https://www.youtube.com/watch?v=wNNOFfDBB0o now to hear more about this announcement and how Zuul and other open source projects fit into the next decade of open infrastructure. You can also watch Johannes Foufas and Albin Vass from Volvo present an update tomorrow on what the community is doing and support them in the chat as they encourage Summit attendees to participate. And if you haven’t already, join as an individual member https://openinfra.dev/join/ so you can get involved! Let’s get started! -- Jeremy Stanley

1 0

Divisive Language stance final(?) draft and Forum session
by Amy Marrich 17 Oct '20

17 Oct '20

Hi everyone, Below is what we hope is the final draft of the OSF's stance on Divisive Language. If you have any comments please place them on the etherpad[0] and/or join us this Tuesday during the forum[1]. Please note that decisions related to what words should be replaced with are being left with the project's technical leadership with the knowledge that context is important to determine the best alternative. That said, the Diversity and Inclusion WG will be available to assist these groups. Thanks, Amy Marrich (spotz) 0 - https://etherpad.opendev.org/p/divisivelanguage 1- https://www.openstack.org/summit/2020/summit-schedule/events/24778/divisive… 1a - https://zoom.us/j/94295083896?pwd=VmZCTFN3eERDK1ltRHRyWDl0eG1hZz09 1b - https://etherpad.opendev.org/p/vSummit2020__DivisiveLanguage ------------------ The OpenStack Foundation (OSF) Board of Directors supports removal of wording identified as oppressive, racist and sexist by members of our communities from the software and documentation they produce. While we know there will be challenges, both technical and non-technical, this is an action we feel is important. These efforts are the responsibility of the various technical leadership groups within our communities, and we trust them to make appropriate decisions about applicability, timelines, minimizing impact to users and operators, and choosing the changes that make the most sense for their projects. Contributors will take care to make changes to software in the least disruptive way possible. While standardized wording is a laudable goal, we recognize that different implementation contexts might require different solutions. In many cases the work is also complicated by external dependencies for these projects, and their capacity to make necessary changes before we can implement ours. Terminology to which special attention should be paid includes: 1. The use of "slave," or "master" in reference to slavery-oriented relationships, as is currently found in databases, the domain name system, etc. 1. The terms "blacklist" and "whitelist" in various contexts (which might require a variety of different replacements to make sense for those contexts) 1. The use of "master" in non-slavery-related contexts such as revision control branches and documentation builds (pending feedback from our community members) We shall continue to be vigilant for other language areas that cause challenges and work with the community to evolve this policy.

1 0