- service-discuss - lists.opendev.org

Team Meeting Agenda for December 5, 2023
by Clark Boylan 05 Dec '23

05 Dec '23

We will meet with this agenda on December 5, 2023 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements ** clarkb out next Tuesday and will miss our regularly scheduled meeting time. ** Proposal: have meetings December 19 and January 9. Skip December 12, 26, and January 2. * Actions from last meeting * Specs Review * Topics ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** Tonyb is adding new mirror servers. *** New mirror servers should be ready. Is it time to update CNAMEs and plan for old server cleanup? ** DIB bionic support (ianw 20231206) *** py36 tox unittesting broken -- proposal to drop https://review.opendev.org/c/openstack/diskimage-builder/+/901093 *** unit testing has been kept this long to ensure in-chroot tools are python3.6 clean -- bionic only platform still supported *** one suggestion is to pause bionic builds - see comment in https://review.opendev.org/c/openstack/project-config/+/901692 and comments in PS1 **** clarkb points out this is painful if cloud providers change, and dib _probably_ won't break the build anyway *** second option is to drop tox 36 testing, but leave bionic test in dib-functests. Probably enough coverage for basic support. Should probably do a release first. ** Python container updates (tonyb 20230718) *** https://review.opendev.org/q/(topic:bookworm-python3.11+OR+hashtag:bookworm… Next round of image rebuilds onto bookworm. *** zuul-operator is the last hold out on python3.10. Working through failures in CI there. **** https://review.opendev.org/c/zuul/zuul-operator/+/881245 is the change we need to get landed. ** Gitea 1.21.1 Upgrade (clarkb 20230926) *** https://review.opendev.org/c/opendev/system-config/+/902490 Configure Gerrit to use new SSH key *** After Gerrit is restarted and using the new key we can remove the old key from Gitea. At that point we should be ready to plan the Gitea upgrade. *** https://review.opendev.org/c/opendev/system-config/+/897679 Upgrade to Gitea 1.21.0 ** Updating Zuul's database server (clarkb 20231121) *** Currently this is an older mysql 5.7 trove instance *** We can move it to a self hosted instance (maybe on a dedicated host?) running out of docker like many of our other services and get it more up to date. *** Are there other services we should consider this for as well? *** Research/Planning questions: https://etherpad.opendev.org/p/opendev-zuul-mysql-upgrade ** Annual Report Season (clarkb 20231128) *** OpenDev's 2023 Annual Report Draft will live here: https://etherpad.opendev.org/p/2023-opendev-annual-report ** Mailing lists: Gmail has become more demanding (fungi 20231205) *** https://review.opendev.org/q/topic:%22argh-gmail%22 Add an SPF record for the listserv * Open discussion ** (tonyb 20231128) [If time permits]. Could we enable via roles/jobs or with an additional nodepool driver. The ability for [OpenStack] project teams to run unit tests with the python images we already build eg https://review.opendev.org/c/opendev/system-config/+/898756.

1 0

Team Meeting Agenda for November 28, 2023
by Clark Boylan 28 Nov '23

28 Nov '23

We will meet with this agenda on November 28, 2023 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements * Actions from last meeting * Specs Review * Topics ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** Tonyb is adding new mirror servers. ** Python container updates (tonyb 20230718) *** https://review.opendev.org/q/(topic:bookworm-python3.11+OR+hashtag:bookworm… Next round of image rebuilds onto bookworm. *** zuul-operator is the last hold out on python3.10. Working through failures in CI there. **** https://review.opendev.org/c/zuul/zuul-operator/+/881245 is the change we need to get landed. ** Gitea 1.21.1 Upgrade (clarkb 20230926) *** https://review.opendev.org/c/opendev/system-config/+/901082 Support Gitea ssh key rotation **** This change should noop if landed today. Then we can build upon it to rotate the key out both in gitea and gerrit for replication *** https://review.opendev.org/c/opendev/system-config/+/897679 Upgrade to Gitea 1.21.0 ** Updating Zuul's database server (clarkb 20231121) *** Currently this is an older mysql 5.7 trove instance *** We can move it to a self hosted instance (maybe on a dedicated host?) running out of docker like many of our other services and get it more up to date. *** Are there other services we should consider this for as well? *** Research/Planning questions: https://etherpad.opendev.org/p/opendev-zuul-mysql-upgrade ** Annual Report Season (clarkb 20231128) *** OpenDev's 2023 Annual Report Draft will live here: https://etherpad.opendev.org/p/2023-opendev-annual-report * Open discussion ** (tonyb 20231128) [If time permits]. Could we enable via roles/jobs or with an additional nodepool driver. The ability for [OpenStack] project teams to run unit tests with the python images we already build eg https://review.opendev.org/c/opendev/system-config/+/898756.

1 0

Team Meeting Agenda for November 21, 2023
by Clark Boylan 21 Nov '23

21 Nov '23

We will meet with this agenda on November 21, 2023 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements ** US Holiday this week. * Actions from last meeting * Specs Review * Topics ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** Tonyb is adding new mirror servers. ** Python container updates (tonyb 20230718) *** https://review.opendev.org/q/(topic:bookworm-python3.11+OR+hashtag:bookworm… Next round of image rebuilds onto bookworm. *** zuul-operator is the last hold out on python3.10. Working through failures in CI there. **** https://review.opendev.org/c/zuul/zuul-operator/+/881245 is the change we need to get landed. ** Gitea 1.21.0 Upgrade (clarkb 20230926) *** https://review.opendev.org/c/opendev/system-config/+/901082 Support Gitea ssh key rotation **** This change should noop if landed today. Then we can build upon it to rotate the key out both in gitea and gerrit for replication *** https://review.opendev.org/c/opendev/system-config/+/897679 Upgrade to Gitea 1.21.0 ** Gerrit 3.8 Upgrade Planning (clarkb 20231024) *** https://etherpad.opendev.org/p/gerrit-upgrade-3.8 *** Upgrade complete *** One issue with stale js/html content for the web ui file editor. Hard refresh seems to correct this. *** https://review.opendev.org/q/topic:%22gerrit-3.7-cleanup%22 *** We can drop the 3.7 images and add 3.9 images when we are convinced a rollback is unlikely. ** Updating Zuul's database server (clarkb 20231121) *** Currently this is an older mysql 5.7 trove instance *** We can move it to a self hosted instance (maybe on a dedicated host?) running out of docker like many of our other services and get it more up to date. *** Are there other services we should consider this for as well? * Open discussion

1 0

Team Meeting Agenda for November 14, 2023
by Clark Boylan 14 Nov '23

14 Nov '23

We will meet with this agenda on November 14, 2023 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements ** US Holiday next week. I expect to hold a team meeting, but the second half of the week may be quiet. * Actions from last meeting * Specs Review * Topics ** Mailman 3 (clarkb 20230627) *** https://etherpad.opendev.org/p/mm3migration *** All lists are hosted on mm3 server, mm3 server is upgraded to the latest versions of components, and the old servers have been cleaned up. *** https://paste.opendev.org/show/bc7jfeZCt97fZm0dCPKw/ is the last remaining concern. A minor error that doesn't appear to actually create problems for us. ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** Tonyb is adding new mirror servers. ** Python container updates (tonyb 20230718) *** https://review.opendev.org/q/(topic:bookworm-python3.11+OR+hashtag:bookworm… Next round of image rebuilds onto bookworm. *** zuul-operator is the last hold out on python3.10. Working through failures in CI there. **** https://review.opendev.org/c/zuul/zuul-operator/+/881245 is the change we need to get landed. ** Gitea 1.21.0 Upgrade (clarkb 20230926) *** https://review.opendev.org/c/opendev/system-config/+/897679 *** Gitea 1.21.0 has been released *** https://github.com/go-gitea/gitea/blob/v1.21.0/CHANGELOG.md Changelog here *** SSH RSA keys must be 3072 bits long. Shorter keys are rejected by gitea. Currently the change disables the length check entirely to workaround this. ** Gerrit 3.8 Upgrade Planning (clarkb 20231024) *** https://etherpad.opendev.org/p/gerrit-upgrade-3.8 *** November 17, 2023 at 15:30 UTC *** https://lists.opendev.org/archives/list/service-announce@lists.opendev.org/… ** Running Ironic bug dashboard *** https://github.com/dtantsur/ironic-bug-dashboard *** (JayF/Dtantsur) Ironic team wants to revive an old bug dashboard; curious if infra would help us run it. Simple python app, no dependencies. We can help if needed and will run "outside" of infra if you all do not have the time. ** Backup server pruning (clarkb 20231114) *** One of the backup servers is warning about space constraints and will need to be pruned. * Open discussion

1 0

Team Meeting Agenda for November 7, 2023
by Clark Boylan 07 Nov '23

07 Nov '23

We will meet with this agenda on November 7, 2023 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements ** North America is back to standard time. Double check your meeting schedule. * Actions from last meeting * Specs Review * Topics ** Mailman 3 (clarkb 20230627) *** https://etherpad.opendev.org/p/mm3migration *** All lists are hosted on mm3 server, mm3 server is upgraded to the latest versions of components, and the old servers have been cleaned up. *** https://paste.opendev.org/show/bc7jfeZCt97fZm0dCPKw/ is the last remaining concern. A minor error that doesn't appear to actually create problems for us. ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** Next up mirrors and meetpad. ** Python container updates (tonyb 20230718) *** https://review.opendev.org/q/(topic:bookworm-python3.11+OR+hashtag:bookworm… Next round of image rebuilds onto bookworm. *** zuul-operator is the last hold out on python3.10. Working through failures in CI there. **** https://review.opendev.org/c/zuul/zuul-operator/+/881245 is the change we need to get landed. ** Gitea 1.21.0 Upgrade (clarkb 20230926) *** https://review.opendev.org/c/opendev/system-config/+/897679 *** Still don't have a changelog. *** SSH RSA keys must be 3072 bits long. Shorter keys are rejected by gitea. Currently the change disables the length check entirely to workaround this. ** Gerrit 3.8 Upgrade Planning (clarkb 20231024) *** https://etherpad.opendev.org/p/gerrit-upgrade-3.8 *** November 17, 2023 at 15:30 UTC *** https://lists.opendev.org/archives/list/service-announce@lists.opendev.org/… ** Adding tonyb to infra-root (clarkb 20231107) *** https://review.opendev.org/c/opendev/system-config/+/900220 * Open discussion

1 0

Team Meeting Agenda for October 31, 2023
by Clark Boylan 31 Oct '23

31 Oct '23

We will meet with this agenda on October 31, 2023 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements * Actions from last meeting * Specs Review * Topics ** Mailman 3 (clarkb 20230627) *** https://etherpad.opendev.org/p/mm3migration *** https://review.opendev.org/c/opendev/system-config/+/899300 Upgrade to latest mm3 suite of tools *** https://review.opendev.org/c/opendev/system-config/+/899304 cleanup our mm3 config management *** https://review.opendev.org/c/opendev/system-config/+/899305 cleanup mm2 config management *** Deleting the old server *** Do we want to add MX records? Doesn't seem necessary currently but some people claimed they have had to do it for their mail servers in order to get mail from some locations. ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** Next up mirrors and meetpad. ** Python container updates (tonyb 20230718) *** https://review.opendev.org/q/(topic:bookworm-python3.11+OR+hashtag:bookworm… Next round of image rebuilds onto bookworm. *** zuul-operator and openstackclient are the last holdouts on python3.10. Change is up to drop python3.9 builds. ** Gitea 1.21.0 Upgrade (clarkb 20230926) *** https://review.opendev.org/c/opendev/system-config/+/897679 *** Still don't have a changelog. *** SSH RSA keys must be 3072 bits long. Shorter keys are rejected by gitea. Currently the change disables the length check entirely to workaround this. ** Gerrit 3.8 Upgrade Planning (clarkb 20231024) *** https://etherpad.opendev.org/p/gerrit-upgrade-3.8 *** November 17, 2023 at 15:30 UTC *** https://lists.opendev.org/archives/list/service-announce@lists.opendev.org/… ** Etherpad 1.9.4 Upgrade (clarkb 20231031) *** https://review.opendev.org/c/opendev/system-config/+/896454 Upgrade etherpad to 1.9.4 * Open discussion

1 0

Team Meeting Agenda for October 24, 2023
by Clark Boylan 24 Oct '23

24 Oct '23

We will meet with this agenda on October 24, 2023 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements ** The PTG is happening this week * Actions from last meeting * Specs Review * Topics ** Mailman 3 (clarkb 20230627) *** https://etherpad.opendev.org/p/mm3migration *** All lists are migrated. *** Last remaining migration todo is cleaning up the old server. We should uncompress the latest kernel before snapshotting it. *** Do we want to add MX records? Doesn't seem necessary currently but some people claimed they have had to do it for their mail servers in order to get mail from some locations. ** Letsencrypt certcheck failures in Ansible (clarkb 20231017) *** Getting `'ansible.vars.hostvars.HostVarsVars object' has no attribute 'letsencrypt_certcheck_domains'` *** https://review.opendev.org/c/opendev/system-config/+/898475 Changes to LE roles to improve debugging ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** Next up mirrors and meetpad. ** Python container updates (tonyb 20230718) *** https://review.opendev.org/q/(topic:bookworm-python3.11+OR+hashtag:bookworm… Next round of image rebuilds onto bookworm. *** zuul-operator and openstackclient are the last holdouts on python3.10. Change is up to drop python3.9 builds. ** Gitea 1.21.0 Upgrade (clarkb 20230926) *** https://review.opendev.org/c/opendev/system-config/+/897679 *** Still don't have a changelog. *** SSH RSA keys must be 3072 bits long. Shorter keys are rejected by gitea. Currently the change disables the length check entirely to workaround this. ** Linaro cloud SSL cert updates (clarkb 20231024) *** We can provision these via execution of acme.sh and running a kolla-ansible deploy. ** Gerrit 3.8 Upgrade Planning (clarkb 20231024) *** https://etherpad.opendev.org/p/gerrit-upgrade-3.8 *** Need to work through the breaking changes *** November 17 or December 1 for upgrade date. *** In theory this should be an easy one compared to the last upgrade. * Open discussion

1 0

Team Meeting Agenda for October 17, 2023
by Clark Boylan 17 Oct '23

17 Oct '23

We will meet on October 17, 2023 at 19:00 UTC in #opendev-meeting with this agenda: == Agenda for next meeting == * Announcements ** The PTG is being held October 23-27 (less than a week away) * Actions from last meeting * Specs Review * Topics ** Mailman 3 (clarkb 20230627) *** https://etherpad.opendev.org/p/mm3migration *** All lists are migrated. *** Local copies of openstack-discuss emails were broken on lists01. This resulted in people sending mail getting notices that their emails could not be delivered. We have since removed the attempt at making local copies. *** Need to clean up the old server. ** Letsencrypt certcheck failures in Ansible (clarkb 20231017) *** Getting `'ansible.vars.hostvars.HostVarsVars object' has no attribute 'letsencrypt_certcheck_domains'` *** https://review.opendev.org/c/opendev/system-config/+/898475 Changes to LE roles to improve debugging ** Zuul not properly caching new branches on creation (clarkb 20231017) *** openstack/puppet-placement created a new 2023.2 branch but Zuul didn't see it leading to no jobs running against changes to that branch *** Current workaround appears to be doing a tenant reconfiguration for the affected tenant. **** docker exec zuul-scheduler_scheduler_1 zuul-scheduler tenant-reconfigure openstack ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** Next up mirrors and meetpad. ** Updating the InMotion/OpenMetal Cloud Deployment (clarkb 20230919) *** https://etherpad.opendev.org/p/redeploying-opendevs-openmetal-cloud Notes from discussion with Yuriy here *** Appear to have three options. Redeploy now, redeploy in a month or two, or redeploy after ~February. The later the redeployment the better the returns. Need to balance that with how many times we feel we can redeploy. ** Python container updates (tonyb 20230718) *** https://review.opendev.org/q/(topic:bookworm-python3.11+OR+hashtag:bookworm… Next round of image rebuilds onto bookworm. *** zuul-operator and openstackclient are the last holdouts on python3.10. Change is up to drop python3.9 builds. ** Gitea 1.21.0 Upgrade (clarkb 20230926) *** https://review.opendev.org/c/opendev/system-config/+/897679 *** Still don't have a changelog. *** SSH RSA keys must be 3072 bits long. Shorter keys are rejected by gitea. Currently the change disables the length check entirely to workaround this. * Open discussion

1 0

Team Meeting Agenda for October 10, 2023
by Clark Boylan 10 Oct '23

10 Oct '23

We will meet on October 10, 2023 at 19:00 UTC in #opendev-meeting with this agenda: == Agenda for next meeting == * Announcements ** The PTG is being held October 23-27 * Actions from last meeting * Specs Review * Topics ** Mailman 3 (clarkb 20230627) *** https://etherpad.opendev.org/p/mm3migration *** Need to schedule migrations for list domains in blocks **** lists.openstack.org Thursday October 12, 2023 ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** Next up mirrors and meetpad. ** Updating the InMotion/OpenMetal Cloud Deployment (clarkb 20230919) *** https://etherpad.opendev.org/p/redeploying-opendevs-openmetal-cloud Notes from discussion with Yuriy here *** Appear to have three options. Redeploy now, redeploy in a month or two, or redeploy after ~February. The later the redeployment the better the returns. Need to balance that with how many times we feel we can redeploy. ** Python container updates (tonyb 20230718) *** https://review.opendev.org/q/(topic:bookworm-python3.11+OR+hashtag:bookworm… Next round of image rebuilds onto bookworm. *** Gerrit is done now *** Lodgeit and gear are remaining OpenDev images. Several images in zuul need updates as well. Currently planning to keep zuul-registry on bullseye + python3.11 but we should be able to clean up bullseye for older python versions. ** Etherpad 1.9.3 Upgrade (clarkb 20230926) *** https://review.opendev.org/c/opendev/system-config/+/896454 *** Hold off until after the PTG as there appears to be a cache problem between versions that end users may notice. ** Gitea 1.21.0 Upgrade (clarkb 20230926) *** https://review.opendev.org/c/opendev/system-config/+/897679 *** Still don't have a changelog. *** SSH RSA keys must be 3072 bits long. Shorter keys are rejected by gitea. Currently the change disables the length check entirely to workaround this. ** Gerrit replication plugin leaked task files (clarkb 20230926) *** https://gerrit-review.googlesource.com/c/plugins/replication/+/387314 Clarkb wrote and pushed a fix upstream *** Currently waiting for more feedback as I'm stuck on understanding how to refactor things as requested. In particular getting logs out of the test suite would be helpful. ** Re-evaluate meeting timeslot (frickler 20231010) *** Sadly ianw is no longer regularly active in the weekly meeting, on the positive side this may be a chance to make the timeslot a bit more EU friendly *** One option is 17:00-18:00UTC Tuesdays * Open discussion

1 0

Team Meeting Agenda for October 3, 2023
by Clark Boylan 03 Oct '23

03 Oct '23

We will meet with this agenda on October 3, 2023 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements ** OpenStack release should be finalized this week ** The PTG is being held October 23-27 * Actions from last meeting * Specs Review * Topics ** Mailman 3 (clarkb 20230627) *** https://etherpad.opendev.org/p/mm3migration *** Need to schedule migrations for list domains in blocks **** lists.openstack.org Thursday October 12, 2023 ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** Next up mirrors and meetpad. ** Updating the InMotion/OpenMetal Cloud Deployment (clarkb 20230919) *** Heard back from Yuriy. Need to figure out how the OpenMetal update planning impacts our timelines for action. ** Zuul PCRE regex support is deprecated (clarkb 20230905) *** Zuul is moving all regexes to re2 instead of PCRE. We'll need to migrate job configs within OpenDev projects. ** Python container updates (tonyb 20230718) *** https://review.opendev.org/q/(topic:bookworm-python3.11+OR+hashtag:bookworm… Next round of image rebuilds onto bookworm. *** Deferring Gerrit until after the OpenStack release *** Other images queued up for updates. Note zuul-registry depends on openssl 1.something which isn't available on Bookworm. We'll need to figure that out. ** Etherpad 1.9.3 Upgrade (clarkb 20230926) *** https://review.opendev.org/c/opendev/system-config/+/896454 ** Gitea 1.21.0 Upgrade (clarkb 20230926) *** Gitea has released a 1.21.0-rc0. No Changelog yet but we can start preparing for this upgrade whenever we are ready. ** Gerrit replication plugin leaked task files (clarkb 20230926) *** https://gerrit-review.googlesource.com/c/plugins/replication/+/387314 Clarkb wrote and pushed a fix upstream * Open discussion

1 0