From cboylan at sapwetik.org Mon Aug 2 22:12:25 2021 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 02 Aug 2021 15:12:25 -0700 Subject: Team Meeting Agenda for August 3, 2021 Message-ID: <5e3bf99b-37c8-4c91-a90d-ad43f1edaaa7@www.fastmail.com> Hello, We will meet with this agenda on August 3, 2021 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements * Actions from last meeting * Topics ** Service Coordinator election (clarkb 20210803) *** Nominations end 23:59 UTC on August 10, 2021. ** review upgrade (ianw 20210803) *** Upgrade has been done *** https://etherpad.opendev.org/p/gerrit-upgrade-2021 *** Followups **** Tuning memory limits and cache settings **** Old server removals ** Gerrit User Cleanups (clarkb 20210803) *** Next batch of cleanups proposed in files on review02 *** Hoping this is the second to last batch with last batch getting direct cleanups that we can push. ** Project Renames (clarkb 20210803) *** Rename went well *** Need to update docs *** Need to manage ssh known_hosts for review02 users https://review.opendev.org/c/opendev/system-config/+/802922 *** Further testing improvements https://review.opendev.org/c/opendev/system-config/+/803266 ** Matrix eavesdrop and gerritbot bots (clarkb 20210803) *** Deployments are in progress ** Gitea01 backups to one of two servers are unhappy (clarkb 20210727) *** Reboots didn't help ** Gitea 1.15.0 prep *** Upstream no longer compatible with nodejs 16 to build the front end https://review.opendev.org/c/opendev/system-config/+/803265 *** Working through changes to gitea in https://review.opendev.org/c/opendev/system-config/+/803231 ** Mailman Ansible and Server Upgrades (clarkb 20210803) *** Need to fix newlist command invocation for the prod servers https://review.opendev.org/c/opendev/system-config/+/803263 *** Should we use lists.kc.io as the snapshot image and new test booted node? *** Need to pick this work up again. * Open discussion From cboylan at sapwetik.org Mon Aug 9 23:35:38 2021 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 09 Aug 2021 16:35:38 -0700 Subject: Team meeting agenda for August 10, 2021 Message-ID: <808f74f5-9963-4cb0-88ee-6b81718f0d15@www.fastmail.com> We will meet August 10, 2021 at 19:00UTC in #opendev-meeting with this agenda: == Agenda for next meeting == * Announcements * Actions from last meeting * Topics ** Service Coordinator election (clarkb 20210810) *** Nominations end 23:59 UTC on August 10, 2021. *** That is about 4 hours after the meeting ends. ** review upgrade (ianw 20210810) *** Upgrade has been done *** https://etherpad.opendev.org/p/gerrit-upgrade-2021 *** Followups **** Tuning memory limits and cache settings **** Old server removals ** Project Renames (clarkb 20210810) *** Need to update docs ** Matrix eavesdrop and gerritbot bots (clarkb 20210810) *** Debugging matrix gerritbot connection problems to review02. *** What are the next steps? ** Gitea01 backups to one of two servers are unhappy (clarkb 20210810) *** Reboots didn't help *** Now lists has the same problem? ** Gitea 1.15.0 prep *** Updating our Gitea management to only use the REST API https://review.opendev.org/c/opendev/system-config/+/803367/ ** Mailman Ansible and Server Upgrades (clarkb 20210810) *** Should we use lists.kc.io as the snapshot image and new test booted node? *** Need to pick this work up again. * Open discussion From cboylan at sapwetik.org Tue Aug 10 23:19:55 2021 From: cboylan at sapwetik.org (Clark Boylan) Date: Tue, 10 Aug 2021 16:19:55 -0700 Subject: Service Coordinator Nominations and Election In-Reply-To: <7c379706-8671-402e-9d2d-0a3ce8ff3a18@www.fastmail.com> References: <7c379706-8671-402e-9d2d-0a3ce8ff3a18@www.fastmail.com> Message-ID: On Tue, Jul 27, 2021, at 1:52 PM, Clark Boylan wrote: > Hello everyone, > > It has been about six months since the previous election for the > OpenDev Service Coordinator spot. When we did the last round I said we > could do about 2 weeks of nomination period then follow that with a > week long election if necessary. Considering this the official open > call for nominations which will stay open until 23:59 UTC on August 10, > 2021. If we need an election that will run from 00:01 UTC August 11, > 2021 to 23:59 UTC August 17, 2021 UTC. > > If anyone else is remotely interested please feel free to step up. I've > been doing this for a bit now and would be happy to have someone else > take on the role. Let me know if you have questions about what is > required. I am also more than happy to help in any transition that > happens. > I'm still happy for someone else to take this on, but we haven't had any nominations yet and time is running out. As mentioned in the meeting today I'm willing to give it another go this time around. Consider this my self nomination. At this point I think you can expect more of the same from me: * Focus on existing services and making them sustainable * Mixing in new services where it makes sense for functionality purposes if we can run them sustainably. * Deprecating and removing services that are not sustainable for us to keep running. If you're happy with how things have gone the last couple of years you should continue to be happy as I don't expect my approach to change too much. If you'd like something different please feel free to run as well (though time is running out). Thank you for your consideration, Clark From rosmaita.fossdev at gmail.com Wed Aug 11 13:26:52 2021 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Wed, 11 Aug 2021 09:26:52 -0400 Subject: Service Coordinator Nominations and Election In-Reply-To: References: <7c379706-8671-402e-9d2d-0a3ce8ff3a18@www.fastmail.com> Message-ID: On 8/10/21 7:19 PM, Clark Boylan wrote: > On Tue, Jul 27, 2021, at 1:52 PM, Clark Boylan wrote: >> Hello everyone, >> >> It has been about six months since the previous election for the >> OpenDev Service Coordinator spot. When we did the last round I said we >> could do about 2 weeks of nomination period then follow that with a >> week long election if necessary. Considering this the official open >> call for nominations which will stay open until 23:59 UTC on August 10, >> 2021. If we need an election that will run from 00:01 UTC August 11, >> 2021 to 23:59 UTC August 17, 2021 UTC. >> >> If anyone else is remotely interested please feel free to step up. I've >> been doing this for a bit now and would be happy to have someone else >> take on the role. Let me know if you have questions about what is >> required. I am also more than happy to help in any transition that >> happens. >> > > I'm still happy for someone else to take this on, but we haven't had any nominations yet and time is running out. As mentioned in the meeting today I'm willing to give it another go this time around. Consider this my self nomination. > > At this point I think you can expect more of the same from me: > > * Focus on existing services and making them sustainable > * Mixing in new services where it makes sense for functionality purposes if we can run them sustainably. > * Deprecating and removing services that are not sustainable for us to keep running. More of the same sounds good to me! Really appreciate that you're willing to continue as OpenDev Service Coordinator. I second your nomination! > > If you're happy with how things have gone the last couple of years you should continue to be happy as I don't expect my approach to change too much. If you'd like something different please feel free to run as well (though time is running out). > > Thank you for your consideration, > Clark > From radoslaw.piliszek at gmail.com Wed Aug 11 14:11:41 2021 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Wed, 11 Aug 2021 16:11:41 +0200 Subject: Service Coordinator Nominations and Election In-Reply-To: References: <7c379706-8671-402e-9d2d-0a3ce8ff3a18@www.fastmail.com> Message-ID: On Wed, Aug 11, 2021 at 3:27 PM Brian Rosmaita wrote: > > On 8/10/21 7:19 PM, Clark Boylan wrote: > > On Tue, Jul 27, 2021, at 1:52 PM, Clark Boylan wrote: > >> Hello everyone, > >> > >> It has been about six months since the previous election for the > >> OpenDev Service Coordinator spot. When we did the last round I said we > >> could do about 2 weeks of nomination period then follow that with a > >> week long election if necessary. Considering this the official open > >> call for nominations which will stay open until 23:59 UTC on August 10, > >> 2021. If we need an election that will run from 00:01 UTC August 11, > >> 2021 to 23:59 UTC August 17, 2021 UTC. > >> > >> If anyone else is remotely interested please feel free to step up. I've > >> been doing this for a bit now and would be happy to have someone else > >> take on the role. Let me know if you have questions about what is > >> required. I am also more than happy to help in any transition that > >> happens. > >> > > > > I'm still happy for someone else to take this on, but we haven't had any nominations yet and time is running out. As mentioned in the meeting today I'm willing to give it another go this time around. Consider this my self nomination. > > > > At this point I think you can expect more of the same from me: > > > > * Focus on existing services and making them sustainable > > * Mixing in new services where it makes sense for functionality purposes if we can run them sustainably. > > * Deprecating and removing services that are not sustainable for us to keep running. > > More of the same sounds good to me! Really appreciate that you're > willing to continue as OpenDev Service Coordinator. I second your > nomination! Seconding Brian on this! Hmm, does it mean I'm "thirding" Clark's nomination? :D Anyhow, I value Clark's work very highly and endorse his nomination. -yoctozepto From cboylan at sapwetik.org Mon Aug 16 23:55:39 2021 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 16 Aug 2021 16:55:39 -0700 Subject: Team Meeting Agenda for August 17, 2021 Message-ID: We will meet with this agenda on August 17, 2021 at 19:00 UTC in #opendev-meeting on OFTC: == Agenda for next meeting == * Announcements * Actions from last meeting * Topics ** Service Coordinator election (clarkb 20210817) *** Clark was the only nomination. Takes the seat by default. ** Matrix eavesdrop and gerritbot bots (clarkb 20210817) *** Zuul's next steps ** Gitea01 backups to one of two servers are unhappy (clarkb 20210817) *** Reboots didn't help *** On Friday we had a set of servers that backup at the same time with similar issues but to the other backup server. Doesn't seem to be a persistent issue. ** Mailman Ansible and Server Upgrades (clarkb 20210817) *** lists.kc.io has been snapshotted and upgraded to focal. Testing is happening now. *** Would like to do an upgrade of this list server first then see what we learn before doing lists.o.o ** Improving OpenDev's CD throughput (clarkb 20210817) *** The hourly deploy buildset takes about 55 minutes to complete. This means it starves regular deploy pipeline buildsets. *** There are some easy simple wins we can make removing jobs from the hourly pipeline if they don't need to be there. **** https://review.opendev.org/c/opendev/system-config/+/804795 Run cloud launcher daily not hourly. *** We can run many of our jobs in parallel in all of our CD pipelines. But this requires we properly document/address dependencies **** Need to understand our job dependencies and properly note them in Zuul config or address them by combining jobs. ***** Example 1: Combine service-gitea-lb and service-gitea jobs. ***** Example 2: Combine letsencrypt and nameserver jobs ***** Example 3: Have all jobs with webserver config express a dependency on the letsencrypt job **** Suggest we document the known job dependencies in a human readable format, then encode this into zuul, then we can switch to parallel runs. * Open discussion From cboylan at sapwetik.org Mon Aug 23 21:26:53 2021 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 23 Aug 2021 14:26:53 -0700 Subject: Infra Team Meeting Agenda for August 24, 2021 Message-ID: <1b0ac45b-a60b-4c64-b9e1-21cca7459cb7@www.fastmail.com> We will meet on August 24, 2021 at 19:00 UTC in #opendev-meeting with this agenda: == Agenda for next meeting == * Announcements ** Zuul is now on our matrix homeserver at #zuul:opendev.org ** OpenStack Feature Freeze starts next week * Actions from last meeting * Topics ** Mailman Ansible and Server Upgrades (clarkb 20210824) *** lists.kc.io has been snapshotted and upgraded to focal. Testing is happening now. *** Would like to do an upgrade of this list server first then see what we learn before doing lists.o.o *** Will coordinate with Kata to do the upgrade this week hopefully. ** Improving OpenDev's CD throughput (clarkb 20210824) *** The hourly deploy buildset takes about 55 minutes to complete. This means it starves regular deploy pipeline buildsets. *** There are some easy simple wins we can make removing jobs from the hourly pipeline if they don't need to be there. **** https://review.opendev.org/c/opendev/system-config/+/805487 Be more specific about when inventory changes force jobs to run. *** We can run many of our jobs in parallel in all of our CD pipelines. But this requires we properly document/address dependencies **** Need to understand our job dependencies and properly note them in Zuul config or address them by combining jobs. ***** Example 1: Combine service-gitea-lb and service-gitea jobs. ***** Example 2: Combine letsencrypt and nameserver jobs ***** Example 3: Have all jobs with webserver config express a dependency on the letsencrypt job **** Suggest we document the known job dependencies in a human readable format, then encode this into zuul, then we can switch to parallel runs. ** Gerrit Account cleanups (clarkb 20210824) *** Need to delete external ids for the recent batch of acount retirements. ** Gitea 1.15.0 upgrade (clarkb 20210824) *** This release exists now *** How do we want to handle the opendev logo image hosting for the gerrit theme? * Open discussion From cboylan at sapwetik.org Tue Aug 31 15:38:57 2021 From: cboylan at sapwetik.org (Clark Boylan) Date: Tue, 31 Aug 2021 08:38:57 -0700 Subject: Team Meeting Agenda for August 31, 2021 Message-ID: <3e332d3e-d144-4aaa-9729-2fedcfb6d394@www.fastmail.com> Sorry, this is going out a little late this week. We will meet on August 31, 2021 (today) at 19:00UTC in #opendev-meeting with this agenda: == Agenda for next meeting == * Announcements ** OpenStack Feature Freeze has begun. * Actions from last meeting * Topics ** Mailman Ansible and Server Upgrades (clarkb 20210831) *** lists.kc.io has been upgraded to Focal. Seems to be happy. *** Do resource consumption numbers look different compared to pre upgrade state? *** Time to schedule a lists.o.o upgrade? ** Improving OpenDev's CD throughput (clarkb 20210831) *** Quick check on how things look after our quick and easy updates. *** We can run many of our jobs in parallel in all of our CD pipelines. But this requires we properly document/address dependencies **** Need to understand our job dependencies and properly note them in Zuul config or address them by combining jobs. ***** Example 1: Combine service-gitea-lb and service-gitea jobs. ***** Example 2: Combine letsencrypt and nameserver jobs ***** Example 3: Have all jobs with webserver config express a dependency on the letsencrypt job **** Suggest we document the known job dependencies in a human readable format, then encode this into zuul, then we can switch to parallel runs. ** Gerrit Account cleanups (clarkb 20210831) *** Need to delete external ids for the recent batch of acount retirements. ** OpenDev logo hosting (clarkb 20210831) *** https://etherpad.opendev.org/p/logo-locations *** Are we happy with an approach yet? * Open discussion