November 2022 - service-discuss

Storyboard Needs and Alternatives
by Clark Boylan 01 Dec '22

01 Dec '22

Hello, There have apparently been disparate discussions among different projects on whether or not they want to continue using Storyboard and what alternatives they should consider. Having these discussion is healthy, but I think we should try to have them in the open with a broad audience to ensure we've considered all the various moving parts involved. I'll do my best to kick start that discussion here in this thread. The Problem(s) * Storyboard is slow. * Storyboard's deployment is aging and needs to be updated. * Storyboard is largely unmaintained at this point. * Web crawlers/indexers struggle to index Storyboard content because it is rendered client side via API calls. These first two problems are related. There are apparently fixes for known performance issues and other long-standing bugs that have yet to be deployed as we need to update the deployment platform to a newer version of python. Options * Update the Storyboard deployment and see if that improves things enough to make people happy. * Migrate projects onto one or more different issue trackers. The first option would almost certainly need someone to step up and help out. But before we do that it is probably worthwhile to solicit feedback indicating people would want to continue to use storyboard if its performance and other usability issues are addressed. The second option opens us to a tremendous number of different possibilities. Launchpad, Gitea, GitHub, Bugzilla, Trac, Redmine, etc. I think for this discussion to be productive we should avoid digging too deeply into these options now. And instead determine if Storyboard is something that we as a community want to try and keep alive. Once we've answered that question we can shift gears to this option if necessary. We would love to hear your feedback. Additional user issues with storyboard, interest in updating/maintaining Storyboard (or the opposite), and thoughts on whether or not addressing known problems with the service would be sufficient to make people want to keep using it are all helpful. The goal here isn't to try and convince anyone they should use Storyboard. Instead we're trying to understand if there are those that would like to keep using Storyboard, and if so, what effort would be necessary to make that possible. Once we've sorted this out we can have a followup discussion on what would be necessary to support projects that don't want to use Storyboard any longer. Clark

6 9

Team Meeting Agenda for November 29, 2022
by Clark Boylan 29 Nov '22

29 Nov '22

Hello, We will meet November 29, 2022 at 19:00 UTC in #opendev-meeting with this agenda: == Agenda for next meeting == * Announcements * Actions from last meeting * Specs Review * Topics ** Bastion host (ianw 20221129) *** https://review.opendev.org/q/topic:prod-bastion-group *** https://review.opendev.org/q/topic:bridge-ansible-venv ** Upgrading Bionic servers to Focal/Jammy (clarkb 20221129) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades ** Mailman 3 (clarkb 20221129) *** https://etherpad.opendev.org/p/mm3migration *** Ready to add a new jammy node to our inventory and let Ansible do its thing. **** https://review.opendev.org/c/opendev/system-config/+/865361 ** Quo vadis storyboard (frickler 20221129) *** ML thread https://lists.opendev.org/pipermail/service-discuss/2022-October/000370.html *** Followup sent to hash out options for OpenDev. ** Nova server rescue behavior in vexxhost (clarkb 20221129) *** Should we consider building our own rescue image for non BFV instances? ** Gerrit 3.6 (ianw 20221129) *** https://etherpad.opendev.org/p/gerrit-upgrade-3.6 ** acme.sh failures (clarkb 20221129) *** Recently we've had a number of certs fail to renew and our checker is complaining about them. *** acme.sh did do a recent release and has since updated their default from rsa to ecdsa. * Open discussion

1 0

Team Meeting Agenda for November 22, 2022
by Clark Boylan 22 Nov '22

22 Nov '22

We will meet with this agenda on November 22, 2022 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements * Actions from last meeting * Specs Review * Topics ** Bastion host (ianw 20221115) *** https://review.opendev.org/q/topic:prod-bastion-group *** https://review.opendev.org/q/topic:bridge-ansible-venv ** Upgrading Bionic servers to Focal/Jammy (clarkb 20221115) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades ** Mailman 3 (clarkb 20221108) *** https://review.opendev.org/c/opendev/system-config/+/851248 Worthy of review at this point *** https://etherpad.opendev.org/p/mm3migration *** Testing updated images with updated packages is underway. ** Updating base python docker images to use `pip wheel` (clarkb 20221115) *** The change to modify this has landed and new images have been promoted. We don't expect problems but please call them out if seen. ** Quo vadis storyboard (frickler 20221115) *** ML thread https://lists.opendev.org/pipermail/service-discuss/2022-October/000370.html *** It has been two weeks since we asked for additional feedback and haven't received any. We should begin making plans with the feedback that we were given. ** Nova server rescue behavior in vexxhost (clarkb 20221115) *** Should we consider building our own rescue image for non BFV instances? * Open discussion

1 0

Team Meeting Agenda for November 15, 2022
by Clark Boylan 16 Nov '22

16 Nov '22

We will meet with this agenda on Tuesday, November 15, 2022 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements * Actions from last meeting * Specs Review * Topics ** Bastion host (ianw 20221115) *** https://review.opendev.org/q/topic:prod-bastion-group *** https://review.opendev.org/q/topic:bridge-ansible-venv ** Upgrading Bionic servers to Focal/Jammy (clarkb 20221115) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades ** Mailman 3 (clarkb 20221108) *** https://review.opendev.org/c/opendev/system-config/+/851248 Worthy of review at this point *** https://etherpad.opendev.org/p/mm3migration *** Currently debugging why updated images aren't working as expected. Appears to be some problem with django. ** Updating base python docker images to use `pip wheel` (clarkb 20221115) *** Will aim to land this change soon, then double check images built with new assemble process are happy. ** Etherpad docker container logs growth (clarkb 20221115) *** https://review.opendev.org/c/opendev/system-config/+/864060 ** Quo vadis storyboard (frickler 20221115) *** ML thread https://lists.opendev.org/pipermail/service-discuss/2022-October/000370.html *** No new movement on the mailing list. Maybe give it another week then we can consider feedback collected? ** Nova server rescue behavior in vexxhost (clarkb 20221115) *** There are image settings that need to be set for rescue images. A dedicated rescue image would also be able to set a non colliding disk label. ** Goodbye birdsite, hello tooters? (frickler 20221115) *** Twitter seems to be on a downward spiral and we should consider stopping using it *** OIF already has an account on fosstodon.org, also ianw and frickler *** Do we want to move the opendevinfra acc there, too? *** How much work is needed in terms of tooling? Any volunteers? * Open discussion

1 0

Team Meeting Agenda for November 8, 2022
by Clark Boylan 08 Nov '22

08 Nov '22

Hello, We will meet with this agenda on November 8, 2022 at 19:00 UTC in #opendev-meeting: == Agenda for next meeting == * Announcements * Actions from last meeting * Specs Review * Topics ** Bastion host (ianw 20221108) *** https://review.opendev.org/q/topic:prod-bastion-group *** https://review.opendev.org/q/topic:bridge-ansible-venv *** https://review.opendev.org/c/opendev/system-config/+/863564 *** https://review.opendev.org/c/opendev/system-config/+/863568 ** Upgrading Bionic servers to Focal/Jammy (clarkb 20221101) *** https://etherpad.opendev.org/p/opendev-bionic-server-upgrades ** Mailman 3 (clarkb 20221108) *** https://review.opendev.org/c/opendev/system-config/+/851248 Worthy of review at this point *** https://etherpad.opendev.org/p/mm3migration *** https://review.opendev.org/c/opendev/system-config/+/860157 Forking the upstream images due to lack of attention on issues and PRs we've filed ** Updating base python docker images to use `pip wheel` (clarkb 20221108) *** Pip changed how it addresses wheels in its wheel cache which broke our assemble script's ability to build packages on our image builds. *** I've filed a bug against pip for this and pushed a PR to fix it. However, upstream pip says we shouldn't rely on the wheel cache like this and should use `pip wheel` instead. *** https://github.com/pypa/pip/issues/11527 *** https://github.com/pypa/pip/pull/11538 *** https://review.opendev.org/c/opendev/system-config/+/862152 ** Etherpad docker container logs growth (clarkb 20221101) *** Need change to log to syslog instead similar to other services. ** Quo vadis storyboard (frickler 20221107) *** ML thread https://lists.opendev.org/pipermail/service-discuss/2022-October/000370.html *** Many people interested in moving away from storyboard *** No volunteers to update our installation *** How long can we/do we want to commit to running an outdated installation on an EOL OS? ** Nova server rescue behavior in vexxhost (clarkb 20221108) *** Clarkb has tested this and the behaviors are surprising *** Rescuing a normal disk instance results in the instance boot with the rescue image kernel and the rescued instance's root device mounted on / instead of the rescue image mounted to /. *** Rescuing a BFV instance fails. First with the default microversion because old Nova API doesn't support this. When using microversion 2.88 it also fails, but only after attempting to rescue. **** The instance goes into an error state due to "cannot be rescued: Driver Error: Cannot access storage file" **** Attempting to unrescue the instance also fails because you cannot unrescue an instance that is in an error state * Open discussion

1 0