- service-announce - lists.opendev.org

Etherpad outage April 19, 2023 at 22:00 UTC
by Clark Boylan 18 Apr '23

18 Apr '23

The OpenDev team will be taking etherpad services at https://etherpad.opendev.org offline for approximately 90 minutes beginning at 22:00 UTC on Wednesday April 19, 2023. This is necessary for us to replace the server hosting etherpad and perform necessary data migrations. If you have any questions or concerns please reach out to us and let us know. Clark

1 0

review.opendev.org Downtime for Gerrit Upgrade and Project Renames April 6, 2023
by Clark Boylan 15 Mar '23

15 Mar '23

Hello, The OpenDev team intends to have a Gerrit outage of approximately two hours on April 6, 2023 beginning at 22:00 UTC. The purpose of this outage is to upgrade Gerrit from version 3.6 to 3.7. We will also take advantage of the downtime to perform project renames. For more information on the new version of Gerrit you can refer to the upstream release notes [0]. If you would like to rename a project please get it onto our TODO list on the wiki [1] and push changes according to the documented process [2]. We've chosen this time and date because we expect it to be a quiet period, and it occurs after known releases and events like the PTG. We are happy for you to raise and concerns, questions, comments or feedback with this plan; the earlier the better. [0] https://www.gerritcodereview.com/3.7.html [1] https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Upcoming_Project_… [2] https://docs.opendev.org/opendev/infra-manual/latest/creators.html#project-…

1 0

Intent to grant control of x/virtualpdu to OpenStack community
by Jeremy Stanley 02 Mar '23

02 Mar '23

The OpenStack Ironic project relies on VirtualPDU, which is no longer actively developed. Ironic contributors reached out to the VirtualPDU maintainers with an offer to officially assume maintenance responsibilities in order to avoid forking it, and received a response from one (Mathieu Mitchell) who indicated support for that plan. Unfortunately, communication died off shortly thereafter, and the Ironic team has been unable to raise any of the current maintainers to actually add their access to the x/virtualpdu project in OpenDev's Gerrit. We don't have a documented process since this is the first time it's really come up, but I'm officially announcing that I intend to use my administrative permissions as an OpenDev sysadmin to add membership of an OpenStack Ironic team representative to the following Gerrit groups if no objections are raised before Wednesday, March 8: * virtualpdu-core * virtualpdu-release This will effectively grant full control of the x/virtualpdu repository to OpenStack Ironic contributors. Please follow up to the service-discuss(a)lists.opendev.org or openstack-discuss(a)lists.openstack.org mailing list with any concerns as soon as possible, or you can feel free to reach out to me directly by email or in IRC (fungi in the #opendev channel on the OFTC network). -- Jeremy Stanley

1 0

December 2022 OpenDev Activity Update
by Clark Boylan 14 Dec '22

14 Dec '22

I had meant to send one of these every couple of months, but it has been about six months since I sent the last update email. Apologies for the delay. It has been a busy six months; let's dive in. We bootstrapped the ability to deploy new control plane nodes on Ubuntu Jammy. All newly created servers are built on Jammy unless we have a very good reason to use something older. Additionally, we have updated our base Python Docker images to include support for Python 3.11. Python 3.11 is only a few weeks old, but we are able to deploy software on top of it taking advantage of the performance benefits it provides. Gerrit has been upgraded to 3.6.3. This release is just behind 3.7.0 which was released about a month ago. I believe this is the closest we've been to the latest Gerrit release in about a decade. We do plan to upgrade to 3.7.x, but this upgrade appears to be a bit more involved than the last few. In particular, the upgrade itself needs offline re-indexing, we need to port our label configs to the new submit requirements system, and the web UI is now based on Lit instead of polymer. The Gitea cluster is now running 1.17. This update didn't bring any major user facing changes. The next update to 1.18.x should correct a problem with Gitea misidentifying certain files as being "vendored". Prior to the October PTG event our Meetpad deployment was updated to use the latest stable releases of Jitsi Meet, and we applied a modern configuration to our installation. One major feature this brought is a landing page for meetings where users can configure their name and settings. Importantly, this landing page should prevent browsers' auto mute behavior for auto playing video+audio from breaking calls for our users. We believe this was a major source of users not being able to hear others in calls. Mailing lists for lists.opendev.org and lists.zuul-ci.org have been migrated to a new Mailman 3 server. We are using these two list domains to work out issues while we prepare for migrating the rest of our mailing lists to the new server. This should give us a much more friendly web interface for browsing archives with built in search capability as well as easier to use moderation tools for list moderators. Our Zuul deployment has seen a number of improvements. The ZooKeeper cluster backing Zuul has been upgraded to ZooKeeper 3.7 which brings performance improvements. Zuul supports Ansible 6 now, and all of our tenants are running jobs with this version of Ansible. This version of Ansible fixes pipelining which should provide some improvements to job runtime. Finally, we are upgrading Zuul weekly using automated zero downtime rolling restarts. This ensures we're getting the latest and greatest Zuul features and can provide quick and early feedback to Zuul. Looking ahead to 2023, I expect to see a lot more of what we've done over the last six months. Gerrit will be upgraded to 3.7, Gitea will be upgraded to 1.18, more mailing lists will be migrated to Mailman 3, and more services will be deployed on Jammy running with Python 3.11. See you next year! Clark

1 0

review.opendev.org Gerrit 3.6 upgrade December 12th, 20:00UTC
by Ian Wienand 07 Dec '22

07 Dec '22

Hello, We are planning upgrade review.opendev.org to the Gerrit 3.6 release on 12th December, 2022 at around 20:00UTC. This will involve an outage of around 1 hour. There should not be much user-visible change after the upgrade; the major items do not affect our usage. The full release notes are at [1] Thank you, -i [1] https://www.gerritcodereview.com/3.6.html

1 0

Mailman v3 migration for OpenDev and Zuul mailing lists 2022-12-05
by Jeremy Stanley 06 Dec '22

06 Dec '22

On Monday, December 5 at 20:00 UTC, the OpenDev Collaboratory systems administrators will be migrating the lists.opendev.org and lists.zuul-ci.org mailing list sites from their existing Mailman 2.1.29 server to a shiny new Mailman 3.3.7 deployment. This maintenance window is expected to last no more than one hour. The key takeaways are as follows: * There will be a brief outage for the affected sites while DNS is updated and configuration, subscriber lists and message archives are imported, but incoming messages should correctly queue at the sender's end and arrive at the conclusion of our work * Because this is on a new server, there are new IP addresses from which list mail will be originating, 162.209.78.70 and 2001:4800:7813:516:be76:4eff:fe04:5423 * Moderation queues will not be copied to the new server, so moderators are encouraged to process any held messages prior to the migration time in order to avoid losing them * Anyone wishing to adjust their list subscriptions, or handle moderation or list configuration after the migration needs to use the Sign Up button on the Web page to create a new account; it will be linked to your imported roles as soon as you confirm by following a URL from an E-mail message the server sends you, and is global for all sites on the server so you only need to do this once * The software providing Web front-ends for list management and archive browsing is entirely new in Mailman v3 and therefore has a much different appearance, though we've added appropriate redirects and frozen copies of old archives in order to accommodate existing hyperlinks Once we smooth out any unanticipated rough patches with the new service, we'll schedule additional maintenance to migrate mailing list sites for other domains (Airship, Kata, OpenInfra, OpenStack, StarlingX). If you have any questions or concerns, feel free to follow up on the service-discuss mailing list or find us in the #opendev channel on the OFTC IRC network. -- Jeremy Stanley

1 1

Discussing Storyboard Needs on service-discuss
by Clark Boylan 08 Nov '22

08 Nov '22

I wanted to make sure that everyone subscribed to this list is aware that we are discussing the Storyboard service, its maintenance needs, and user needs on a thread on service-discuss(a)lists.opendev.org [0]. I have been told that not everyone is aware of the service-discuss list and may have missed that thread. Please follow up on that thread if you have any feedback for (not) wanting to use storyboard or desire to help maintain the service. This is a good opportunity to remind everyone that OpenDev runs three mailing lists for user interaction: 1. service-announce(a)lists.opendev.org [1] This mailing list. Intended to be kept low volume and limited to the most important email notices. 2. service-discuss(a)lists.opendev.org [2] Where we discuss the service needs, meeting agendas, and other topics with more volume. The bulk of our mailing list discussion will be here. All are welcome to join and participate. -discuss and -announce are separate to give people a lower volume option with -announce. 3. service-incident(a)lists.opendev.org This is a private mailing list that OpenDev admins and other trusted individuals may be subscribed to. Users can email this list with sensitive messages like vulnerability disclosures. [0] https://lists.opendev.org/pipermail/service-discuss/2022-October/000370.html [1] https://lists.opendev.org/cgi-bin/mailman/listinfo/service-announce [2] https://lists.opendev.org/cgi-bin/mailman/listinfo/service-discuss

1 0

Changes to our base job nodeset and Ansible 5 removal
by Clark Boylan 12 Oct '22

12 Oct '22

As mentioned in the previous email [0] about updating our default Ansible version to Ansible 6, Zuul will be removing support for Ansible 5 soon. Based on Zuul's currently estimated release schedule we expect this to happen on or near October 18. This change won't take effect until OpenDev restarts Zuul. We will try to make that happen quickly after Zuul lands this change in order to catch any issues quickly. Additionally, it has been called out that we are still defaulting our base job nodeset to ubuntu-focal. Ubuntu Jammy has been released for about 6 months now and has a .1 release as well. To catch up we will update the default nodeset in our base jobs to ubuntu-jammy on October 25, 2022. If you would like to test that your jobs run against Ubuntu Jammy you can modify your jobs to set `nodeset: ubuntu-jammy` today and test them. Note, any jobs that expect a specific platform should already be explicitly setting the nodeset they rely on. There are a number of jobs already running on Jammy which gives us confidence in the newer images. [0] https://lists.opendev.org/pipermail/service-announce/2022-October/000046.ht… Clark

1 0

Updating Zuul tenants' default-ansible-version to 6
by Clark Boylan 05 Oct '22

05 Oct '22

Hello, The OpenDev team is planning to update the default-ansible-version to 6 in the near future. Zuul is planning to drop support for Ansible 5, and switching OpenDev to Ansible 6 early allows us to provide valuable feedback if there are problems. There are other reasons for the switch as well. Ansible 5 broke pipelining, but we expect Ansible 6 to correct this problem. Having pipelining should improve the runtime of jobs. We had been holding off on making this change as StarlingX and OpenStack have both recently worked through some fairly major releases. Those releases exist now making this a good time for a change like this. As far as testing goes, Zuul's Zuul tenant has been running with an Ansible 6 default for several weeks now. I've also tested this change with Devstack specifically [0] as Devstack seems to have a good representative set of complicated jobs. The change to update our defaults is up for review [1]. Feel free to leave any comments there or on this email thread. Note that you can override this change if necessary using the same mechanism I used to test Devstack [0]. However, if you do this please communicate that change to us as Zuul does plan to remove Ansible 5 support which means any problems with Ansible 6 need to be communicated back to Zuul. [0] https://review.opendev.org/c/openstack/devstack/+/858436 [1] https://review.opendev.org/c/openstack/project-config/+/860474

1 0

Dropping support for Ansible 2.x in our Zuul
by Jeremy Stanley 22 Sep '22

22 Sep '22

Following the earlier 2.8/2.9 deprecation in June and change of our default to Ansible 5 in July, we will soon be restarting onto a version of Zuul which no longer supports 2.x versions at all. While we've looked for any lingering overrides to old Ansible versions in master branch jobs and given projects a heads up individually, we lack insight into whether the same may be occurring on non-default branches of projects. If you are aware of any jobs on any of your branches which pin back to an earlier Ansible version, we strongly recommend you remove those overrides at your earliest opportunity, otherwise those jobs will break in the coming days. Here is the change to remove support for Ansible 2.8 and 2.9 in Zuul (we generally restart our Zuul services every weekend, so that's when it's most likely to take effect): https://review.opendev.org/857796 For some example changes which clean up old Ansible version overrides, see this review topic list: https://review.opendev.org/q/topic:cleanup-old-ansible -- Jeremy Stanley

1 0