- Zuul-discuss - lists.opendev.org

Prepare workspace roles
by corvus＠inaugust.com 10 Apr '20

10 Apr '20

Hi, Related to that thread we were pulling on in IRC this morning, I think there's some refactoring we can do for setting up git repos on remote nodes. We currently have the following roles: * mirror-workspace-git-repos Uses git over ssh or kubectl to push repos onto remote vms or containers. No support for cached repos. Trusted playbooks only. * prepare-workspace Starts the Zuul console daemon. Uses synchronize to push the repos onto remote vms. No support for cached repos. Can run in untrusted playbooks. * prepare-workspace-git Clones cached repos into place. Includes mirror-workspace-git-repos role to use git over ssh or kubectl to push repos onto remote vms or containers. Trusted playbooks only. OpenDev is using this role. * prepare-workspace-openshift Uses "oc rsync" to push repos onto remote containers. No support for cached repos. Can run in untrusted playbooks. It seems to me that if we set aside the requirement to be able to run in untrusted playbooks, then we effectively already have a single role that can perform the necessary tasks in all conditions. My understanding is that the untrusted forms are used to support a split base job construction, where a tenant's default parent job is actually defined in an untrusted repo and performs many setup tasks in that job, inheriting very few (or no) tasks from a parent in a trusted repo. That makes it easier to test changes to the default parent job speculatively. The question I have for the community is whether at this point that is still worth the trade off in complexity of using rsync to set up git repos? We now have much better testing of changes to roles used in base jobs, including a test which currently covers prepare-workspace-git[1]. Also, these roles change a lot less often than they used to. If we're okay dropping that requirement, I'd like to suggest that we incorporate 'mirror-workspace-git-repos' into 'prepare-workspace-git', and then deprecate all the other roles, and eventually remove them. That would mean we could have a very clear suggested base job, and we'd all be using the same role. What do folks think about that? [1] https://opendev.org/zuul/zuul-jobs/src/branch/master/zuul-tests.d/general-r… -Jim

2 1

Synchronizing to containers
by corvus＠inaugust.com 10 Apr '20

10 Apr '20

Hi, Earlier today in IRC several of us started pulling on a thread related to base pre- and post-run playbooks and containers. In short, we use the synchronize module a lot, and it only works on VMs. The fetch-output role is designed to address this partially, by reducing the number of indepedent synchronize tasks, but in order to synchronize files from a remote container, it needs to run "oc rsync" on the executor, which means that trick will only work in a trusted playbook. That may be okay for that role, but it still leaves any other role or playbook that needs to synchronize something without an important tool. We came up with a plan to improve the situation which should fit within the following constraints: * Not a Zuul-specific solution (playbooks can still run outside of Zuul). * Usable in untrusted playbooks. * Makes VM or container transparent to the role (works the same on both). The general idea is to make a new Ansible module using the collections framework which behaves the same as the current 'synchronize' module, but transparently supports containers by using "oc rsync". This module will be available automatically in Zuul, and users will be able to install the module on their own, external to Zuul, using the Ansible collections feature. Here are the steps to achieve this: 1) Create a new module in Zuul which is a fork of synchronize[1] with Tristan's oc rsync patch[2]. This module will be named "zuul.posix.synchronize" to indicate that it is related to the upstream "ansible.posix.synchronize" module. 2) Start using that in zuul-jobs roles that use synchronize. 3) Create a new repo named "zuul/ansible-collection-posix" and copy the module into that. 4) Publish that repo as an Ansible collection. 5) Install that collection in the Zuul Ansible virtualenvs for versions which support collections and use that in favor of the built-in module. 6) When all Zuul Ansible versions support collections, remove the built-in module. I don't think it's our job to make *everything* about VMs and containers transparent in Zuul, but for something so basic as getting data to and from remote nodes, I think it should be easy to handle both cases, and will make it easier for us to keep a lot of roles simple. How does this sound to folks? [1] https://github.com/ansible-collections/ansible.posix/blob/master/plugins/mo… [2] https://review.opendev.org/681553 -Jim P.S. Monty also suggests that we can similarly move the Zuul-specific modules such as zuul-console and zuul-return into standalone collections repo. That seems like a good idea too, as it may aid local development and testing of playbooks.

1 0

[OSF Community Meeting] Community Meeting Recap
by Jimmy McArthur 07 Apr '20

07 Apr '20

Hi everyone, Thank you to those who attended any of the community meetings last week! If you missed one of the calls last week, below are links to the presentation [1], recording in English [2], and recording in Mandarin [3]. A lot of links were discussed and you can find those in the meeting section of the notes as well as the chat from the recordings. We are planning to do this format with project and community updates and OSF event updates quarterly. Stay tuned for the next meeting, and let me know if you have any questions in the meantime. In addition to the quarterly meetings, we would like to hear from the project communities on topics you would like to cover in community meetings or have covered. If you have an idea you would like to share, please drop it in this etherpad [4]. If you see a topic already shared that you would like to collaborate on, please share your name, email and IRC nick next to the topic. Cheers, Jimmy [1] https://docs.google.com/presentation/d/1l05skj_BCfF8fgYWu4n0b1rQmbNhHp8sMeY… [2] https://zoom.us/rec/share/7vVXdIvopzxIYbPztF7SVpAKXYnbX6a82iMaqfZfmEl1b0Fqb… [3] https://zoom.us/rec/share/zudNaIrQ6mNLR9Lvr2Xxf5QQIL7geaa8hCcf_vsIxEwoHkn0b… [4] https://etherpad.openstack.org/p/OSF_Community_Meetings

1 0

New OpenDev Communication Channels
by Clark Boylan 07 Apr '20

07 Apr '20

Hello All, Recently, we've transitioned to using #opendev on Freenode for our synchronous IRC communications. Since then we've spun up a new service-discuss(a)lists.opendev.org mailing list [1] where we'll plan changes to services, notify of meetings, answer questions about services and usage, and otherwise communicate about OpenDev. service-announce(a)lists.opendev.org [2] will remain for important announcements. If you're interested in the developer infrastructure we run please join us on these mailing lists. We encourage every developer to at least subscribe to the service-announce[2] list. It will be very low traffic, and used only for important announcements with wide impact. We are also going to start having our weekly team meeting in #opendev-meeting. The time will continue to be 19:00 UTC Tuesdays. See you there, Clark [1] http://lists.opendev.org/cgi-bin/mailman/listinfo/service-discuss [2] http://lists.opendev.org/cgi-bin/mailman/listinfo/service-announce

1 0

Renaming install-$foo roles to ensure-$foo
by albin vass 04 Apr '20

04 Apr '20

Hi, There were some discussion on irc yesterday about getting some better role name consistency, more specific the install-$foo vs ensure-$foo. The idea is to be more consistent when naming roles that makes sure some software is installed on a node. A question from me was if there could be any difference between the install-$foo vs ensure-$foo, my guess is that there isn't since they should be idempotent. I've pushed a couple of changes with the topic role-naming-consistency that renames all roles to the ensure-$foo naming standard and should mostly work by now. https://review.opendev.org/#/q/status:open+project:zuul/zuul -jobs+branch:master+topic:role-naming-consistency At the moment the old install-$foo roles are kept and include the new ensure-$roles to make sure nothing breaks. Regards Albin

2 1

Fwd: OpenStack Foundation Community Meetings
by Allison Price 02 Apr '20

02 Apr '20

Hi everyone - In case you didn’t see the post on the Foundation mailing list, we have a community meeting tomorrow where we will be talking about updates to OSF events as well as OSF project updates. One meeting will be in English and one in Mandarin. Bring your questions and see you then! Cheers, Allison > Begin forwarded message: > > From: Allison Price <allison(a)openstack.org> > Subject: OpenStack Foundation Community Meetings > Date: March 26, 2020 at 10:21:15 AM CDT > To: foundation(a)lists.openstack.org > > Hi everyone, > > Next week we are going to have two community meetings to discuss the OpenStack 10th anniversary planning, current community projects, and an update on OSF events. Please join if you would like to hear updates or if you have questions for the OpenStack Foundation team. > > Join us: > Thursday, April 2 at 10am CT / 1500 UTC <https://zoom.us/j/913456204> > Friday, April 3 in Mandarin at 10am China Standard Time <https://zoom.us/j/389363434> > > If you are unable to join one of the above times, I will share a recording to the mailing list after the meetings. > > Cheers, > Allison > > > Allison Price > OpenStack Foundation > allison(a)openstack.org <mailto:allison@openstack.org> > > > >

1 1

[Zuul Community] Virtual OpenDev Update
by Jimmy McArthur 01 Apr '20

01 Apr '20

Hi everyone, As we continue to gather feedback and discuss what a virtual OpenDev will look like, we wanted to provide you with an update around the event. While we are still attempting to solve some of the riddles of virtual event world, we did come up with some decisions that give us something to build on, which are based on input and experience from members of the Programming Committee. Decisions: - OpenDev will happen after the Virtual PTG (exact format and timing TBD) - Will not be simultaneous/parallel Tracks so everyone has the opportunity to participate in more than one - 3 Tracks, each on different date: - Hardware Automation - Large Scale Ops - Containers in Production We also discussed the original intent and goals of OpenDev to make sure we’re still heading in a direction that’s beneficial for the community, especially since it will no longer directly precede the upcoming PTG. Similar to past OpenDev events, the goal is to identify the questions we don’t have the answers for, and to use that time to dive in and determine the work ahead of us, working together, sharing use cases, and learning from other people asking those same questions. High-level decisions, goals, and some areas we’re looking for feedback are in the etherpad below. As you continue to experience this new amazing world of virtual events, please add any feedback or ideas to the etherpad below. We’re looking forward to creating virtual OpenDev with you all! https://etherpad.openstack.org/p/Virtual_OpenDev_Planning Ashlee Ashlee Ferguson Community & Events Coordinator OpenStack Foundation

1 0

Keeping linear Git history for Zuul-gated projects
by Vitaliy Lotorev 01 Apr '20

01 Apr '20

Hi, I'm evaluating Zuul for project gating. Currently my team is using Fast Forward Only submit strategy [1] with Gerrit Autosubmitter plugin [2]. Code-Review labels are copied if change is rebased trivially (no code change). We use Fast Forward to keep Git history without merges and I wouldn't like to loose this feature while switching to Zuul. I see that all projects gated with Zuul (e.g. Zuul itself, OpenStack) are using Merge If Necessary strategy. My confusion is: Gerrit has six different merge strategies and Zuul is responsible for merging all the code. Is there any combination of Gerrit/Zuul settings to preserve linear Git history? I'm trying to understand what is wrong with Rebase If Necessary and Cherry Pick strategies. Why these strategies are not preferred for Zuul-gated projects? Will Zuul fetch random commits from A-B-C chain while using Cherry Pick or it will try to preserve the order? [1] https://gerrit-review.googlesource.com/Documentation/concept-changes.html#s… [2] https://gerrit.googlesource.com/plugins/autosubmitter/ Regards, Vitaliy

5 6

Fwd: [all] Virtualized PTG Brainstorming Community Meetings
by Kendall Nelson 31 Mar '20

31 Mar '20

Hello Everyone! So! After Marks initial email[1] that announced we would be virtualizing the PTG and collected volunteers[2] to help brainstorm the best way to hold the PTG virtually, we polled the volunteers for the best times to meet. In trying to be the most inclusive we have settled on three meetings to brainstorm together as a community. They are as follows: - April 2nd at 12:00 UTC - April 6th at 17:00 UTC - April 7th at 2:00 UTC We will be using my zoom[3] and continue using the same etherpad [2] that we collected volunteers in for all the notes from all three meetings. All are welcome! And if you can attend more than one meeting, please do! The more overlap we have the more cohesive discussions we will have. Hope to see you all there! -Kendall Nelson (diablo_rojo) [1] http://lists.openstack.org/pipermail/foundation/2020-March/002854.html [2] https://etherpad.openstack.org/p/Virtual_PTG_Planning [3] https://zoom.us/my/diablorojo

1 0

User Survey for Zuul
by Jimmy McArthur 28 Mar '20

28 Mar '20

The OSF would like to help put together a User Survey with the Zuul Community to see if we can find out how your users are using Zuul in production. This would be an anonymized survey, but would hopefully give the project greater insight into ways that people are using the software and how to make improvements/changes moving forward. If this sounds like something people are interested in, let me know and I'll start an etherpad with some basic questions. If you'd like to see an example of a survey we worked with the Kata community on, you can take a look at a PDF here: https://drive.google.com/file/d/1gZd2rh2jG2IkKBmhBeqFyB-KZCBrHLni/view?usp=… If this seems like a horrible idea, please don't hesitate to tell me that as well :) Cheers, Jimmy

8 26