January 2025 - service-discuss

Asking permission to scrape OpenStack docs for external search
by Jay Faulkner 29 Jan '25

29 Jan '25

Hello OpenDev admins, Mehdi, one of our developer relations engineers here at GR-OSS, and I were talking about the state of OpenStack docs, and how a repeated complaint we hear is that search is not extremely useful. He noted that for https://consuldot.net GR-OSS uses a service called https://algolia.com to provide search (top right corner of consuldot.net <http://consuldot.net>). We were hoping to pilot this for OpenStack and see how well the search worked against our documentation, but when going through the process they specifically request you have permission from the website owners before they crawl ( https://docsearch.algolia.com/apply/ ) -- so I'm reaching out to you to try and get this permission. Right now our ONLY goal is to get a prototype working for this to demo to the larger OpenStack community -- honestly I just want to see if it's worth the value before we have the larger discussion around externally-hosted search. So basically, we're looking for permission to point their crawler at OpenStack documentation (we may start with just Ironic for simplicity). We'll create a larger discussion including the foundation, TC, and larger community if it appears to be a worthwhile pursuit. Let me know what you think! Thanks, Jay Faulkner

2 1

Team Meeting Agenda for January 28, 2025
by Clark Boylan 28 Jan '25

28 Jan '25

== Agenda for next meeting == * Announcements * Actions from last meeting * Specs Review * Topics ** Zuul-launcher image builds (corvus 20240910) *** https://review.opendev.org/q/hashtag:niz+status:open is next set of work happening in Zuul ** Upgrading Old Servers (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades **** wiki.openstack.org: https://etherpad.opendev.org/p/opendev-mediawiki-upgrade **** tonyb looking at cacti after wiki *** https://etherpad.opendev.org/p/opendev-focal-server-upgrades *** Thoughts on using Noble **** Rely on borg 1.2.8 backups to existing servers until we have new backup servers running borg 1.4.x **** Some docker compose setups may need to be edited to set a restart policy of always **** Podman package updates don't appear to require container restarts. But we should observe this behavior if possible. ** Switch to quay.io/opendevmirror images where possible (clarkb 20250128) *** Mariadb is an easy switch but updating the image from one location to another (even if the hash is the same) causes a service restart *** Other changes to consider: Pulling our python images from quay.io. We lose speculative testing of the base image but that may be a reasonable compromise. ** Unpinning our Grafana deployment (clarkb 20250128) *** https://review.opendev.org/c/opendev/system-config/+/940073 *** Testing shows this may still exhibit the problem we saw that caused us to pin *** Trying to use a held node to test directly. ** Increasing Exim's Acceptable Line Length (clarkb 20250128) *** https://review.opendev.org/c/opendev/system-config/+/940248 *** Bounces were being generated by our own Exim system on mailman due to line lengths being too long. This is configurable and we can avoid the problem entirely. ** Running certcheck on bridge (clarkb 20250114) *** Any concern running this service in the bastion? *** Would better match how testing works. *** We would need to disable it on the cacti server. ** Service Coordinator Election (clarkb 20250107) *** Nominations Open From February 4, 2025 to February 18, 2025 *** Voting February 19, 2025 to February 26, 2025 *** All times and dates will be UTC based. *** https://lists.opendev.org/archives/list/service-discuss@lists.opendev.org/t… ** Beginning of the Year (Virtual) Meetup Recap (clarkb 20250107) *** https://etherpad.opendev.org/p/opendev-january-2025-meetup *** Tried to condense the notes down into a TODO list for the next $TIME * Open discussion

1 0

February 2025 OpenDev Service Coordinator Election
by Clark Boylan 22 Jan '25

22 Jan '25

It is that time of year again where we need to elect an OpenDev Service Coordinator [0] for the next six months. Nominations will be open from February 4, 2025 to February 18, 2025. If we have more than one candidate we will vote for our preferred candidate from February 19, 2025 to February 26, 2025. All dates are based on a UTC clock. Please send your self nominations to this mailing list during the nomination time frame. You are eligible to run and vote if you have contributed to OpenDev git repositories in the last 12 months (this period will be February 4, 2024 to February 3, 2025). I continue to encourage anyone who may be interested to reach out and ask questions. I am happy to support someone new in the role. [0] https://docs.opendev.org/opendev/system-config/latest/project.html#governan… Clark

1 0

Team Meeting Agenda for January 21, 2025
by Clark Boylan 20 Jan '25

20 Jan '25

== Agenda for next meeting == * Announcements * Actions from last meeting * Specs Review * Topics ** Zuul-launcher image builds (corvus 20240910) *** https://review.opendev.org/q/hashtag:niz+status:open is next set of work happening in Zuul ** Deploying new Noble Servers (clarkb 20250107) *** New paste server is up and running under podman and docker compose *** Ran into problems with our pinned version of borg on Noble with python3.12 *** Updated borg for noble servers does seem to work ok. **** https://review.opendev.org/c/opendev/system-config/+/939667 fixup for warnings treated as errors with new borg spamming email ** Deploying Lodgeit entirely without Docker Hub (clarkb 20250121) *** In theory, now that we're using podman as the container runtime for paste, we can get speculative image testing without hosting on Docker Hub. *** https://review.opendev.org/c/opendev/lodgeit/+/939385 Publish lodgeit image to quay.io ** Upgrading Old Servers (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades **** wiki.openstack.org: https://etherpad.opendev.org/p/opendev-mediawiki-upgrade **** tonyb looking at cacti after wiki *** https://etherpad.opendev.org/p/opendev-focal-server-upgrades ** Gerrit 3.10.4 Update (clarkb 20250114) *** https://review.opendev.org/c/opendev/system-config/+/939167 New bugfix releases available *** Good opportunity to bundle up the h2 compaction change with a version update. **** https://review.opendev.org/c/opendev/system-config/+/938000 ** Running certcheck on bridge (clarkb 20250114) *** Any concern running this service in the bastion? *** Would better match how testing works. *** We would need to disable it on the cacti server. ** Service Coordinator Election (clarkb 20250107) *** Proposing this schedule: **** Nominations Open From February 4, 2025 to February 18, 2025 **** Voting February 19, 2025 to February 26, 2025 **** All times and dates will be UTC based. ** Beginning of the Year (Virtual) Meetup (clarkb 20250107) *** Consider this a spiritual successor to last year's Pre PTG. *** Idea is to take some time, sync up on needs and priorities, and maybe even do some hackathon type work to flush out the backlog *** Starts not long after this meeting see you there. *** https://etherpad.opendev.org/p/opendev-january-2025-meetup * Open discussion

1 0

Team Meeting Agenda for January 14, 2025
by Clark Boylan 14 Jan '25

14 Jan '25

== Agenda for next meeting == * Announcements * Actions from last meeting * Specs Review * Topics ** Zuul-launcher image builds (corvus 20240910) *** https://review.opendev.org/q/hashtag:niz+status:open is next set of work happening in Zuul ** Deploying new Noble Servers (clarkb 20250107) *** Replacement paste server is getting deployed as we speak. *** Discovered that restart: always is the only container restart policy podman will respect on reboot. Others result in no containers on boot. ** Upgrading Old Servers (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades **** wiki.openstack.org: https://etherpad.opendev.org/p/opendev-mediawiki-upgrade **** tonyb looking at cacti after wiki *** https://etherpad.opendev.org/p/opendev-focal-server-upgrades ** Mirroring Useful Container Images (clarkb 20250107) *** Initial set of images have been mirrored. *** We rediscovered that quay images pushed via docker image protocol are private by default but jobs have been updated to mark new images public *** Time to start adding useful images and tags **** https://review.opendev.org/c/opendev/system-config/+/939169 ** Gerrit 3.10.4 Update (clarkb 20250114) *** https://review.opendev.org/c/opendev/system-config/+/939167 New bugfix releases available *** Good opportunity to bundle up the h2 compaction change with a version update. **** https://review.opendev.org/c/opendev/system-config/+/938000 ** Running certcheck on bridge (clarkb 20250114) *** Any concern running this service in the bastion? *** Would better match how testing works. *** We would need to disable it on the cacti server. ** Service Coordinator Election (clarkb 20250107) *** Proposing this schedule: **** Nominations Open From February 4, 2025 to February 18, 2025 **** Voting February 19, 2025 to February 26, 2025 **** All times and dates will be UTC based. ** Beginning of the Year (Virtual) Meetup (clarkb 20250107) *** Consider this a spiritual successor to last year's Pre PTG. *** Idea is to take some time, sync up on needs and priorities, and maybe even do some hackathon type work to flush out the backlog *** January 21-23. Need to decide on specific times and days. *** https://etherpad.opendev.org/p/opendev-january-2025-meetup * Open discussion

1 0

Team Meeting Agenda for January 7, 2025
by Clark Boylan 07 Jan '25

07 Jan '25

== Agenda for next meeting == * Announcements * Actions from last meeting * Specs Review * Topics ** Zuul-launcher image builds (corvus 20240910) *** Configs are going in opendev/zuul-jobs *** Raw image handling is being sped up through the use of zlib *** Can still use additional image build jobs for the other images we care about. ** Deploying new Noble Servers (clarkb 20250107) *** Changes to manage containers with podman and docker compose on Noble have landed *** Next step is deploying a Noble server for a straightforward service (paste most likely) and observing how that goes. *** Ultimately would like to deploy a new review server on Noble using this system. ** Upgrading Old Servers (clarkb 20230627) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades **** wiki.openstack.org: https://etherpad.opendev.org/p/opendev-mediawiki-upgrade **** tonyb looking at cacti after wiki *** https://etherpad.opendev.org/p/opendev-focal-server-upgrades ** Mirroring Useful Container Images (clarkb 20250107) *** One approach to mitigating Docker Hub rate limits is to mirror useful images to another registry like quay.io. *** https://review.opendev.org/c/opendev/system-config/+/938508 Initial change to mirror images that are generically useful ** Gerrit H2 Cache File Growth (clarkb 20250107) *** Gerrit's git_file_diff and gerrit_file_diff caches are implemented as H2 databases on disk. *** These H2 databases are backed with files on disk that only compact when Gerrit is shutdown. *** This leads to files growing quite large which impacts Gerrit startup *** https://review.opendev.org/c/opendev/system-config/+/938000 Suggested workaround from Hashar improves compaction when we do shutdown *** Should we also revert our change to increase the logical size of the cache? This may slow down growth on disk as daily pruning would trim things back regularly (but not compact). ** Rax-ord Noble Nodes With 1 VCPU (clarkb 20241210) *** Occasionally Ubuntu Noble nodes booted in rax-ord have a single vcpu *** Best I can tell nodepool didn't use the wrong flavor. Instead it seems possibly related to an older Xen release being used to boot Noble *** Not sure where the bug is (Xen or Linux) but maybe we can get Rax to do a hypervisor audit and upgrade those behind or remove them from being able to schedule things? *** On our side of things we could have an early base job check for >1 vcpu if Xen is in use. ** Service Coordinator Election (clarkb 20250107) *** Proposing this schedule: **** Nominations Open From February 4, 2025 to February 18, 2025 **** Voting February 19, 2025 to February 26, 2025 **** All times and dates will be UTC based. ** Beginning of the Year (Virtual) Meetup (clarkb 20250107 *** Consider this a spiritual successor to last year's Pre PTG. *** Idea is to take some time, sync up on needs and priorities, and maybe even do some hackathon type work to flush out the backlog *** January 21-23? Open to ideas on timing. * Open discussion

1 0