From cboylan at sapwetik.org Tue Mar 2 00:54:21 2021 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 01 Mar 2021 16:54:21 -0800 Subject: Meeting Agenda for March 2, 2021 Message-ID: <18a552d9-9a75-4b9c-94f7-f70b7827969e@www.fastmail.com> We will meet on March 2, 2021 at 19:00 UTC in #opendev-meeting with this agenda: == Agenda for next meeting == * Announcements * Actions from last meeting * Specs approval * Priority Efforts (Standing meeting agenda items. Please expand if you have subtopics.) ** [http://specs.openstack.org/openstack-infra/infra-specs/specs/update-config-management.html Update Config Management] *** topic:update-cfg-mgmt *** Zuul as CD engine ** OpenDev *** Gerrit account and group inconsistencies **** https://etherpad.opendev.org/p/gerrit-user-consistency-2021 High level notes. **** We have 17 accounts with preferred email addresses that don't have a matching external id **** Need to correct the ~642 external id issues before we can push updates to refs/meta/external-ids with Gerrit online. **** Workaround is we can stop Gerrit, push to external ids directly, reindex accounts (and groups?), start gerrit, then clear accounts caches (and groups caches?) **** Next steps ***** Identify accounts that are inactive and can be more forcefully retired. Retire these to fix those errors. ***** Identify accounts that are unlikely to be used anymore based on activity and more forcefully retire those to fix these errors. (We can always undo specific updates to these accounts if necessary) ***** Work with remaining accounts to figure out how to best resolve the account conflicts. This may take some time. ***** https://review.opendev.org/c/opendev/system-config/+/777846 Collecting scripting efforts here *** Configuration tuning **** Using strong refs for jgit caches **** Batch user groups and threads * General topics ** OpenAFS cluster status (clarkb 20210302) *** Upgrading servers to Bionic then Focal next. *** New third db server for proper quorum. ** Bup and Borg Backups (clarkb 20210302) *** gitea sql db backup issues. ** Picking up steam on Puppet -> Ansible rewrites (clarkb 20210302) *** Enable Xenial -> Bionic/Focal system upgrades *** https://etherpad.opendev.org/p/infra-puppet-conversions-and-xenial-upgrades Start capturing TODO list here *** Zuul service host updates in progress now. Mergers are done. Executors in progress. ** Deploy a new refstack.openstack.org server (kopecmartin 20210302) *** Ready for testing? ** Bridge disk space (clarkb 20210302) *** This appears at least partially related to ansible and python caching. Should we just clear those caches then profile them? * Open discussion From ssbarnea at redhat.com Fri Mar 5 09:16:24 2021 From: ssbarnea at redhat.com (Sorin Sbarnea) Date: Fri, 5 Mar 2021 01:16:24 -0800 Subject: Can some projects configure auto addition of reviewers to changes? Message-ID: As I observed that proposed changes can easily be ignored for months if the original submitter did not manually added anyone or missed to do the footwork of chasing cores on irc, I wonder if it would not be possible for *some* projects to enable a feature to auto-add reviewers. On the dark side of the world I seen the "codeowners " features being quite successful in assigning reviewers, as a simple file can define the pool of potential reviewers. Based on how the repository is configured it does randomly pick a limited number of candidates. Github mentions in their release notes from 2017 that their inspiration for that feature was the OWNERS file used by chromium team (gerrit users). I also seen that Wikimedia experimented something in that area, apparently there is a wiki page where people can mark them as available to be picked. The watched project feature in Gerrit is useless for this purpose as it does not randomly pick some of a pool of possible candidates. It may be practical to use if there are projects with only 2-3 core reviewers. There are two aspects I am looking to address, at least partially: A) Improve random external contributors first-contribution experience - right now, even if they propose a perfectly valid change, there is high chance it will not get noticed. As you can imagine this lowers the chance they would try again, or to become an active contributor. B) Spread the review burden on cores and avoid "punishing" those that are more active from being assaulted by review requests. The current secret source for getting reviews on a project where you do not know anyone involves looking at recent changes to spot people that did reviews or at least have core powers and try to contact those. Even if this works or not, it puts extra pressure on active people. That is why I would see an automatic round-robin approach more appropriate. As some may no longer be involved with the project, this could prove to be an opportunity to revise the core list, or to gently remove yourself from the list of cores. C) How to deter people from adding reviewers before CI jobs report green? I have no idea how to educate people not to add reviewers to changes that are not yet green. I know there are valid cases where this is needed, but likely is more like on less than one 1:5. As no hard limit would work, maybe there is a way to display some kind of message in the UI, advising user not to assign reviewers until CI reports green. The same issue applies even to automatic reviewer assignment, which should not happen right after a review is created. Ideally this should happen when it passed CI, but it could also be ok if it would happen when a review is moved from WIP to ready-for-review. I personally use a default where any change I create is marked as WIP and this default can also be changed at server level if we want. -- /zbr -------------- next part -------------- An HTML attachment was scrubbed... URL: From hashar at free.fr Fri Mar 5 10:42:41 2021 From: hashar at free.fr (Antoine Musso) Date: Fri, 5 Mar 2021 11:42:41 +0100 Subject: Can some projects configure auto addition of reviewers to changes? In-Reply-To: References: Message-ID: Le 05/03/2021 à 10:16, Sorin Sbarnea a écrit : > > I also seen that Wikimedia experimented something > > in that area, apparently there is a wiki page where people can mark > them as available to be picked. Hello, I wrote that Wikimedia blog post! Let me side track a bit to give a bit more information. The context is Wikimedia has been struggling with code review for as long as we had more than a handful of people.  Code review is always a hot topic and is causing pain all across the community with various degrees of experience.  Some repositories are carefully watched by their maintainers, others have mostly inner team reviews and it might be hard to get a review if you are an outsider. *git blame* An idea we had was to automatically add code reviewer based on `git blame` (which I often relies on). That is how we have added the reviewers by blame plugin. Cause I was super ambitious I went with the default setting and it got enabled on every single repositories.  The day after, people complained about being added to reviewers to a random set of patches that touched code that they once touched, most often unrelated to their review capabilities.  The huge spam of emails caused us to roll back that deployment and forget about automatically adding reviewers based on blame. It just don't fit within our ecosystem. I have updated the blog post to highlight we have disabled the reviewer by blame plugin almost immediately. *homemade tooling* The primary system we use is Reviewer bot. People register their interest on a wiki page https://www.mediawiki.org/wiki/Git/Reviewers for example I can get added for any changes made to a `tox.ini` file on any repo or to any changes made to `ci/*` gerrit repositories.  I can't remember the details of the implementation, I think all changes are send to a mailing list, the bot fetches the email, processes them, fetch the wiki page and apply the rules. The equivalent as a Gerrit plugin would be the reviewers plugin: https://gerrit.googlesource.com/plugins/reviewers/+/refs/heads/master/src/main/resources/Documentation/config.md We have it installed but I don't think it is used. Last time I checked there is no UI for it and people have to add themselves via a reviewers.config file under refs/meta/config. Needless to say you loose 99% of the audience at that point. Editing a wiki page is way simpler for us. *attention set* Google is well aware of the issue related to find reviewers, or at least highlighting when a change should be acted on among the list of changes one is a reviewer for. Gerrit 3.3 introduces a featured named attention set which is a bit of a turn based, when one does an action they can bring up to the attention of some of the reviewers while others are not. So if Jane asks me to rewrite a test, she will bring my attention, I will amend my code send a patch and bring attention back to her.  After a while once we both agree the change is now fine, we will bring it to the attention of the others. I really like that turn based review among a subset of the reviewers, that should cut a bit of spam or at least make the dashboard slightly more helpful. The feature is better described at https://gerrit-review.googlesource.com/Documentation/user-attention-set.html I have been exposed to it while contributing to Gerrit upstream and it is really an enjoyable system. And all the above do not address the root causes: * finding who can review code * burden put on the shoulder of the few that can actually review, leading to a huge backlog which is most probably ignored as a result It is a social problem really :-\ -- Antoine "hashar" Musso Wikimedia Release Engineering -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Fri Mar 5 13:05:06 2021 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 5 Mar 2021 13:05:06 +0000 Subject: Can some projects configure auto addition of reviewers to changes? In-Reply-To: References: Message-ID: <20210305130506.hy6whwid3vz3b6ty@yuggoth.org> On 2021-03-05 01:16:24 -0800 (-0800), Sorin Sbarnea wrote: > As I observed that proposed changes can easily be ignored for > months if the original submitter did not manually added anyone or > missed to do the footwork of chasing cores on irc, I wonder if it > would not be possible for *some* projects to enable a feature to > auto-add reviewers. [...] The StarlingX community asked last year for us to add the reviewers plugin. I have WIP changes proposed for it at https://review.opendev.org/724913 and https://review.opendev.org/724914 but we were waiting for the dust to settle on the Gerrit 2.13->3.2 upgrade before moving forward on those. This is probably a good time to revisit them. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From cboylan at sapwetik.org Mon Mar 8 23:54:54 2021 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 08 Mar 2021 15:54:54 -0800 Subject: Team Meeting Agenda for March 9, 2021 Message-ID: We will meet March 9, 2021 at 19:00 UTC in #opendev-meeting with this agenda: == Agenda for next meeting == * Announcements ** Clarkb out March 23. Will need volunteer to run the meeting or we can skip it. ** DST change happens for those of us in North America this weekend. EU and others follow in a few weeks. * Actions from last meeting * Specs approval * Priority Efforts (Standing meeting agenda items. Please expand if you have subtopics.) ** [http://specs.openstack.org/openstack-infra/infra-specs/specs/update-config-management.html Update Config Management] *** topic:update-cfg-mgmt *** Zuul as CD engine ** OpenDev *** Gerrit account and group inconsistencies **** https://etherpad.opendev.org/p/gerrit-user-consistency-2021 High level notes. **** Workaround is we can stop Gerrit, push to external ids directly, reindex accounts (and groups?), start gerrit, then clear accounts caches (and groups caches?) **** Next steps ***** Identify accounts that cannot be used because they have no valid method of login (no username, and no openids) ***** Identify accounts that are unlikely to be used anymore based on activity and more forcefully retire those to fix these errors. (We can always undo specific updates to these accounts if necessary) ***** Work with remaining accounts to figure out how to best resolve the account conflicts. This may take some time. ***** https://review.opendev.org/c/opendev/system-config/+/777846 Collecting scripting efforts here ***** Inactive accounts have been cleaned up. *** Configuration tuning **** Using strong refs for jgit caches **** Batch user groups and threads * General topics ** OpenAFS cluster status (clarkb 20210309) *** Upgrading servers to Bionic then Focal next. *** New third db server for proper quorum. ** Bup and Borg Backups (clarkb 20210309) *** gitea sql db backup issues. ** Picking up steam on Puppet -> Ansible rewrites (clarkb 20210309) *** Enable Xenial -> Bionic/Focal system upgrades *** https://etherpad.opendev.org/p/infra-puppet-conversions-and-xenial-upgrades Start capturing TODO list here *** Zuul service host updates in progress now. Mergers and executors largely done. Next up Nodepool launchers. ** Deploy a new refstack.openstack.org server (kopecmartin 20210309) *** Ready for testing? ** Bridge disk space (clarkb 20210309) *** This is related to stevedore caching entrypoints *** We have upgraded stevedore and set config to prevent it from creating new cache files. ** PTG Planning (clarkb 20210309) *** Next PTG April 19-23 *** Please let me know if you are interested so I can fill out the team survey (or not). * Open discussion From cboylan at sapwetik.org Mon Mar 15 22:36:13 2021 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 15 Mar 2021 15:36:13 -0700 Subject: Team Meeting Agenda for March 16, 2021 Message-ID: We will meet on March 16, 2021 at 19:00 UTC (note DST changes if you live in North America) in #opendev-meeting with this agenda: == Agenda for next meeting == * Announcements ** Clarkb out March 23. Ianw has volunteered to chair the meeting on the 23rd. ** DST change has happened in North America. EU and others follow in a few weeks. * Actions from last meeting * Specs approval * Priority Efforts (Standing meeting agenda items. Please expand if you have subtopics.) ** [http://specs.openstack.org/openstack-infra/infra-specs/specs/update-config-management.html Update Config Management] *** topic:update-cfg-mgmt *** Zuul as CD engine ** OpenDev *** Gerrit account inconsistencies **** https://etherpad.opendev.org/p/gerrit-user-consistency-2021 High level notes. **** All preferred emails lack external ids issues have been corrected. All group loops have been corrected. **** Workaround is we can stop Gerrit, push to external ids directly, reindex accounts (and groups?), start gerrit, then clear accounts caches (and groups caches?) **** Next steps ***** Review groups in review:~clarkb/gerrit_user_cleanups/notes.20210315 and determine if they can be safely cleaned up like previous groups. ***** https://review.opendev.org/c/opendev/system-config/+/780663 more user auditing improvements. *** Configuration tuning **** Using strong refs for jgit caches **** Batch user groups and threads * General topics ** Picking up steam on Puppet -> Ansible rewrites (clarkb 20210316) *** Enable Xenial -> Bionic/Focal system upgrades *** https://etherpad.opendev.org/p/infra-puppet-conversions-and-xenial-upgrades Start capturing TODO list here *** Zuul service host updates in progress now. Mergers and executors done. Nodepool launchers in progress now. *** Ianw working on kerberos server updates. ** Deploy a new refstack.openstack.org server (kopecmartin 20210316) *** Ready for testing? ** PTG Planning (clarkb 20210316) *** Next PTG April 19-23 *** Please let me know if you are interested so I can fill out the team survey (or not). ** review server upgrade (ianw 20210316) *** Initial spec https://review.opendev.org/c/opendev/infra-specs/+/780478 * Open discussion From iwienand at redhat.com Mon Mar 22 22:36:49 2021 From: iwienand at redhat.com (Ian Wienand) Date: Tue, 23 Mar 2021 09:36:49 +1100 Subject: Team Meeting Agenda for March 23, 2021 Message-ID: We will meet on March 16, 2021 at 19:00 UTC in #opendev-meeting with this agenda: == Agenda for next meeting == * Announcements * Actions from last meeting * Specs approval * Priority Efforts (Standing meeting agenda items. Please expand if you have subtopics.) ** [http://specs.openstack.org/openstack-infra/infra-specs/specs/update-config-management.html Update Config Management] *** topic:update-cfg-mgmt *** Zuul as CD engine ** OpenDev *** Gerrit account inconsistencies **** https://etherpad.opendev.org/p/gerrit-user-consistency-2021 High level notes. **** All preferred emails lack external ids issues have been corrected. All group loops have been corrected. **** Workaround is we can stop Gerrit, push to external ids directly, reindex accounts (and groups?), start gerrit, then clear accounts caches (and groups caches?) **** Next steps ***** Review groups in review:~clarkb/gerrit_user_cleanups/notes.20210315 and determine if they can be safely cleaned up like previous groups. ***** https://review.opendev.org/c/opendev/system-config/+/780663 more user auditing improvements. *** Configuration tuning **** Using strong refs for jgit caches **** Batch user groups and threads * General topics ** Picking up steam on Puppet -> Ansible rewrites (clarkb 20210316) *** Enable Xenial -> Bionic/Focal system upgrades *** https://etherpad.opendev.org/p/infra-puppet-conversions-and-xenial-upgrades Start capturing TODO list here *** Zuul service host updates in progress now. Mergers, executors and launchers done. Only remaining is scheduler? ** Deploy a new refstack.openstack.org server (ianw 20210323) *** https://review.opendev.org/c/opendev/system-config/+/781593 ** PTG Planning (ianw 20210323) *** Next PTG April 19-23 *** Please let me know if you are interested so I can fill out the team survey (or not). * Open discussion From iwienand at redhat.com Wed Mar 24 04:28:57 2021 From: iwienand at redhat.com (Ian Wienand) Date: Wed, 24 Mar 2021 15:28:57 +1100 Subject: Retirement of the Openstack/Opendev Asterisk service Message-ID: Hello, As part of ongoing service upgrades, we are proposing the retirement of the Asterisk-based SIP/voice conferencing tool [1]. The meetpad instance at https://meetpad.opendev.org/ has proven popular for recent events and provides better services for collaborative meetings. Please reach out if this service is important to you. If it is something you would like to maintain, it will require porting to our current Ansible and container-based infrastructure (there will be plenty of admins happy to help guide you). Otherwise we'll propose changes to remove the configuration soon. Thanks, -i [1] https://docs.opendev.org/opendev/system-config/latest/asterisk.html From cboylan at sapwetik.org Mon Mar 29 21:10:31 2021 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 29 Mar 2021 14:10:31 -0700 Subject: Team Meeting Agenda for March 30, 2021 Message-ID: <2281c0c7-caa3-45f8-9577-2afc412ee81e@www.fastmail.com> We will meet with this agenda March 30, 2021 in #opendev at 19:00UTC: == Agenda for next meeting == * Announcements * Actions from last meeting * Specs approval * Priority Efforts (Standing meeting agenda items. Please expand if you have subtopics.) ** [http://specs.openstack.org/openstack-infra/infra-specs/specs/update-config-management.html Update Config Management] *** topic:update-cfg-mgmt *** Zuul as CD engine ** OpenDev *** Gerrit account inconsistencies **** https://etherpad.opendev.org/p/gerrit-user-consistency-2021 High level notes. **** All preferred emails lack external ids issues have been corrected. All group loops have been corrected. **** Workaround is we can stop Gerrit, push to external ids directly, reindex accounts (and groups?), start gerrit, then clear accounts caches (and groups caches?) **** Next steps ***** Review groups in review:~clarkb/gerrit_user_cleanups/notes.20210315 and determine if they can be safely cleaned up like previous groups. ***** https://review.opendev.org/c/opendev/system-config/+/780663 more user auditing improvements. *** Configuration tuning **** Using strong refs for jgit caches **** Batch user groups and threads * General topics ** Picking up steam on Puppet -> Ansible rewrites (clarkb 20210330) *** Enable Xenial -> Bionic/Focal system upgrades *** https://etherpad.opendev.org/p/infra-puppet-conversions-and-xenial-upgrades Start capturing TODO list here *** Zuul service host updates in progress now. Scheduler and Zookeeper cluster remaining. Will focus on ZK first. ** Deploy a new refstack.openstack.org server (ianw 20210330) *** https://review.opendev.org/c/opendev/system-config/+/781593 ** PTG Planning (ianw 20210330) *** Next PTG April 19-23 *** Clarkb filled out the survey and requested a few hours for us. Likely to be spent in more office hours type setup. **** Thursday April 22 1400-1600UTC and 2200-0000UTC ** docs-old volume cleanup (ianw 20210330) *** anyone think this is necessary? *** docs-dev? ** planet.openstack.org (ianw 20210330) *** there is no maintained aggregators any more *** stick current venus/planet in a python 2.7 container and forget about it *** retire it ** tarballs ORD replication (ianw 20210330) *** only things across regions are docs & airship *** tarballs is probably the only other thing worth having a out-of-region copy for *** https://etherpad.opendev.org/p/gjzssFmxw48Nn3_SBVo6 * Open discussion