March 2024 - service-discuss - lists.opendev.org

Team Meeting Agenda for March 26, 2024
by Clark Boylan 26 Mar '24

26 Mar '24

We will meet with this agenda on March 26, 2024 (today) at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements ** OpenStack Release next week then PTG the week after ** Put your PTG agenda items on the etherpad: https://etherpad.opendev.org/p/apr2024-ptg-opendev * Actions from last meeting * Specs Review * Topics ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** https://review.opendev.org/q/topic:jitsi_meet-jammy-update *** Started looking at the wiki there are rough notes at: https://etherpad.opendev.org/p/opendev-bionic-server-upgrades#L58 ** MariaDB Upgrades (clarkb 20240220) *** Relying on the container image MARIADB_AUTO_UPGRADE flag *** Etherpad, Gitea, Gerrit, and Mailman could use upgrades. *** https://review.opendev.org/c/opendev/system-config/+/911000 Upgrade etherpad mariadb to 10.11 ** AFS Mirror cleanups (clarkb 20240220) *** Ubuntu Xenial is next but currently busy with PTG, Release, and other tasks. *** Can followup with webserver log processing to determine which other mirrors may be dead. ** Rebuilding Gerrit Images (clarkb 20240312) *** Gerrit 3.9.2 has been released finally. *** https://review.opendev.org/c/opendev/system-config/+/912470 Update our 3.9 image to 3.9.2 **** This will also rebuild our 3.8.4 image so we should try and restart prod gerrit on the new 3.8.4 image when available. *** Sounds like there are a number of bugfixes that a rebuild will get us. May be worth doing this just after the openstack release completes? ** Review02 had an oops last night (clarkb 20240326) *** Found the server was shutdown. After giving it a few minutes to potentially resolve itself (mostly worried about cloud action) clarkb proceeded to manually start the instance then start the containers. *** mnaser reports it may have been an OOM event on the hosting side. ** Rackspace MFA Requirement (clarkb 20240312) *** MFA is enabled. Enforcement day is today. Please lookout for any issues. ** Project Renames (clarkb 20240227) *** https://review.opendev.org/c/opendev/system-config/+/911622 Move gerrit replication queue aside during project renames. *** Penciled in April 19, 2024 submit your rename requests now. ** Nodepool image delete after upload (clarkb 20240319) *** Nodepool now has the ability to delete on disk files for images after they are uploaded. We could potentially keep only small qcow2s using this functionality to save disk space. * Open discussion

1 0

Team Meeting Agenda for March 19, 2024
by Clark Boylan 18 Mar '24

18 Mar '24

We will meet with this agenda on March 19, 2024 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements ** Put your PTG agenda items on the etherpad: https://etherpad.opendev.org/p/apr2024-ptg-opendev * Actions from last meeting * Specs Review * Topics ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** https://review.opendev.org/q/topic:jitsi_meet-jammy-update *** Started looking at the wiki there are rough notes at: https://etherpad.opendev.org/p/opendev-bionic-server-upgrades#L58 ** MariaDB Upgrades (clarkb 20240220) *** Relying on the container image MARIADB_AUTO_UPGRADE flag *** Etherpad, Gitea, Gerrit, Refstack, and Mailman could use upgrades. *** https://review.opendev.org/c/opendev/system-config/+/911000 Upgrade etherpad mariadb to 10.11 *** https://review.opendev.org/c/opendev/system-config/+/910999 Upgrade refstack mariadb to 10.11 ** AFS Mirror cleanups (clarkb 20240220) *** CentOS 7 is now gone. Next up is Ubuntu Xenial *** Can followup with webserver log processing to determine which other mirrors may be dead. ** Rebuilding Gerrit Images (clarkb 20240312) *** Gerrit 3.9.2 has been released finally. *** https://review.opendev.org/c/opendev/system-config/+/912470 Update our 3.9 image to 3.9.2 **** This will also rebuild our 3.8.4 image so we should try and restart prod gerrit on the new 3.8.4 image when available. ** Rackspace MFA Requirement (clarkb 20240312) *** We intend on manually enabling MFA on March 20, 2024 before the March 26, 2024 deadline. ** Project Renames (clarkb 20240227) *** https://review.opendev.org/c/opendev/system-config/+/911622 Move gerrit replication queue aside during project renames. *** Penciled in April 19, 2024 submit your rename requests now. ** Linaro cloud cert renewal (clarkb 20240319) *** The cert for the cloud APIs expires on April 15 and will need to be manually rotated before then. ** Nodepool image delete after upload (clarkb 20240319) *** Nodepool now has the ability to delete on disk files for images after they are uploaded. We could potentially keep only small qcow2s using this functionality to save disk space. * Open discussion

1 0

Team Meeting Agenda for March 12, 2024
by Clark Boylan 11 Mar '24

11 Mar '24

We will meet with this agenda on March 12, 2024 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements ** DST changes are starting to occur for some of us. Double check your timezone hasn't moved relative to the 19:00 UTC meeting time. * Actions from last meeting * Specs Review * Topics ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** https://review.opendev.org/q/topic:jitsi_meet-jammy-update *** Started looking at the wiki there are rough notes at: https://etherpad.opendev.org/p/opendev-bionic-server-upgrades#L58 ** MariaDB Upgrades (clarkb 20240220) *** Relying on the container image MARIADB_AUTO_UPGRADE flag *** Etherpad, Gitea, Gerrit, Refstack, and Mailman could use upgrades. *** https://review.opendev.org/c/opendev/system-config/+/911000 Upgrade etherpad mariadb to 10.11 *** https://review.opendev.org/c/opendev/system-config/+/910999 Upgrade refstack mariadb to 10.11 ** AFS Mirror cleanups (clarkb 20240220) *** CentOS 7 removal is in progress: topic: drop-centos-7 **** Announced that we would finalize CentOS 7 removal on March 15 to give OpenStack time to clean stuff up and avoid unexpected release problems *** When removing Debian mirror content afs01.dfw.openstack.org lost a volume backing /dev/vicepa and had a big sad. Not sure if related or coincidence. *** Can followup with webserver log processing to determine which other mirrors may be dead. ** Rebuilding Gerrit Images (clarkb 20240312) *** Gerrit 3.9.2 has been released finally. *** https://review.opendev.org/c/opendev/system-config/+/912470 Update our 3.9 image to 3.9.2 **** This will also rebuild our 3.8.4 image so we should try and restart prod gerrit on the new 3.8.4 image when available. ** Rackspace MFA Requirement (clarkb 20240312) *** Rackspace is requiring all accounts switch to multi factor auth March 26, 2024 *** We moved our clouds.yaml auth over to `rackspaceauth` using an `api_key` as normal auth through the API will no longer work after MFA is enabled *** Still an open question if swift log uploads for zuul jobs need changing, but indications are they already use dedicated credentials which are unlikely to be affected. *** Should we go ahead and opt into MFA now ahead of the deadline so that we can control when our API interactions will be impacted? ** Project Renames (clarkb 20240227) *** https://review.opendev.org/c/opendev/system-config/+/911622 Move gerrit replication queue aside during project renames. *** Penciled in April 19, 2024 submit your rename requests now. ** PTG Planning (clarkb 20240305) *** https://ptg.opendev.org/ptg.html *** Times have been selected. Feel free to add agenda items here: https://etherpad.opendev.org/p/apr2024-ptg-opendev * Open discussion

1 0

Team Meeting Agenda for March 5, 2024
by Clark Boylan 04 Mar '24

04 Mar '24

We will meet with this Agenda on March 5, 2024 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements * Actions from last meeting * Specs Review * Topics ** Upgrading Bionic servers to Focal/Jammy (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades *** https://review.opendev.org/q/topic:jitsi_meet-jammy-update *** Started looking at the wiki there are rough notes at: https://etherpad.opendev.org/p/opendev-bionic-server-upgrades#L58 ** MariaDB Upgrades (clarkb 20240220) *** Relying on the container image MARIADB_AUTO_UPGRADE flag *** Etherpad, Gitea, Gerrit, Refstack, and Mailman could use upgrades. *** https://review.opendev.org/c/opendev/system-config/+/911000 Upgrade etherpad mariadb to 10.11 *** https://review.opendev.org/c/opendev/system-config/+/910999 Upgrade refstack mariadb to 10.11 ** AFS Mirror cleanups (clarkb 20240220) *** OpenSUSE Leap and Debian Buster are gone now *** Have started with CentOS 7 removal: topic: drop-centos-7 **** Announced that we would finalize CentOS 7 removal on March 15 to give OpenStack time to clean stuff up and avoid unexpected release problems *** When removing Debian mirror content afs01.dfw.openstack.org lost a volume backing /dev/vicepa and had a big sad. Not sure if related or coincidence. *** Can followup with webserver log processing to determine which other mirrors may be dead. ** OpenDev email hosting (clarkb 20240220) *** Would be good to come to a consensus on whether or not we want ot host this ourselves or just pay someone to do it. ** Project Renames (clarkb 20240227) *** Do we need to update the tooling to move the replication waiting queue aside during the rename process? *** Penciled in April 19, 2024 submit your rename requests now. ** PTG Planning (clarkb 20240305) *** https://ptg.opendev.org/ptg.html *** Very little schedule content is in place at this point. Clarkb proposes we do something like Wednesday 0400-0600 and Thursday 1400-1600UTC. Gives enough time between blocks to catch up on sleep. * Open discussion

1 0