- service-announce - lists.opendev.org

Planned outage for mailing lists on Monday 2024-12-02 15:00-17:00 UTC
by Jeremy Stanley 28 Nov '24

28 Nov '24

The OpenDev Sysadmins will be performing maintenance on our mailing list server between 15:00 and 17:00 UTC on Monday, 2024-12-02. During this time, messages to and from the lists for all our sites will be held temporarily while we make adjustments in the backing database for renaming one of the domains in use. Web archives may be unavailable for part of the time as well. The outage should start some time around 15:00 UTC and is not expected to last more than two hours (but will hopefully conclude far more quickly). Please follow up (either to the service-discuss(a)lists.opendev.org mailing list or to me personally) with any questions you may have on this activity. -- Jeremy Stanley, OpenDev Systems Administrator

1 0

Upgrading OpenDev's Gerrit Server on December 6, 2024 at 16:00 UTC
by Clark Boylan 14 Nov '24

14 Nov '24

The OpenDev team will be upgrading the Gerrit server on review.opendev.org on December 6, 2024 at 16:00 UTC. This will require a short downtime of the service if everything goes well and a longer downtime if we need to revert. We are allocating an hour (16:00 - 17:00 UTC) for this work. We will upgrade from Gerrit 3.9 to 3.10 remaining approximately one release behind the latest Gerrit release as long as the Gerrit 3.11 release happens on time this month. Gerrit 3.10 brings a number of changes including many performance improvements and fewer email notifications. The upstream release notes [0] cover these updates in much more detail if you are interested. Let us know if you have any questions or concerns. [0] https://www.gerritcodereview.com/3.10.html Clark

1 0

Pruning OpenDev's Intermediate Registry on insecure-ci-registry.opendev.org November 15, 2024
by Clark Boylan 06 Nov '24

06 Nov '24

The OpenDev team will begin some long overdue pruning of the storage system behind the intermediate container registry running on insecure-ci-registry.opendev.org on November 15, 2024. The last time we tried this we believe container image client errors were due to the pruning. Unfortunately, we don't have enough information to know if this will happen again. For this reason we will be prepared to replace the entire Swift container on the backend and start over fresh if necessary. We ask that you report any unexpected client behaviors with the intermediate registry on or after November 15, 2024. Additionally, if we do end up replacing the entire backend storage container we will lose all intermediate container image state. If this happens, jobs relying on state in the intermediate registry may fail. In those cases you may need to recheck jobs that build the images or merge additional changes to refresh the content in the intermediate registry. Clark

1 0

Switching OpenDev's Default Zuul Nodeset to ubuntu-noble
by Clark Boylan 15 Aug '24

15 Aug '24

The OpenDev team plans to switch the default nodeset in our Zuul base jobs from ubuntu-jammy to ubuntu-noble on August 21, 2024. The new release has been available for several months now and we have jobs relying on it. One of the last steps in the Noble rollout is to switch the default over. This change will impact any jobs that don't explicitly override the nodeset as they will start running on the newer Ubuntu release after this change. If this creates problems for your jobs you can either override the nodeset to ubuntu-jammy forcing the job to run on the nodes used before this change, or you can update your job to accommodate whatever changes the newer version of Ubuntu requires. If you have any questions or concerns feel free to respond to this email or reach out via IRC. For OpenStack in particular, we've been told that they expect most jobs to already override the nodeset and this change will largely be a noop for them. Clark

1 0

Removal of Inactive Projects from Zuul's Tenant Configuration
by Clark Boylan 10 Jul '24

10 Jul '24

The OpenDev team is going to start removing inactive x/* namespace projects from Zuul's tenant configuration. The two primary motivations behind this are to remove projects that already have broken configurations within Zuul (see the Zuul config errors page [0]), and to avoid projects entering this state in the future as we clean up test nodes like Ubuntu Xenial or CentOS 8 Stream or otherwise modernize the CI system. Removal from Zuul's tenant configuration is a lightweight method of ensuring we're addressing problems with active projects that can be fixed without the added noise from projects that are unlikely to fix things on their own. A first pass at this sort of cleanup is being done with this change [1]. Feel free to chime in if you know there is already work underway to correct x/* projects' config errors, or if you know of any additional projects that should be removed from Zuul. We aim to merge this change on July 16, 2024. It should be noted that if projects do become active again and have volunteers willing to correct the CI configuration state, it is straightforward to add the project back into Zuul enabling that effort to progress. Projects will simply add themselves back to the tenant configuration and then push changes to fix the CI config. [0] https://zuul.opendev.org/t/openstack/config-errors [1] https://review.opendev.org/c/openstack/project-config/+/923509 Clark

1 0

CentOS 8 Stream End of Life and Removal from OpenDev CI
by Clark Boylan 20 Jun '24

20 Jun '24

Hello everyone, I wanted to give you a heads up that CentOS 8 Stream has reached its end of life and the upstream package repos have been removed. Our mirrors have faithfully replicated the empty mirror content (there is only a readme file now [0]). The end result of which is failing CI jobs if the jobs run on CentOS 8 Stream. The OpenDev team intends on removing these nodes from Zuul and Nodepool in the near future as jobs can no longer successfully run on CentOS 8 Stream. Projects should check their CI jobs (on older branches too) and either remove jobs that rely on CentOS 8 Stream or update those jobs to run on a different platform. [0] https://mirror.dfw.rax.opendev.org/centos/8-stream/readme Clark

1 0

Ubuntu Noble Test Nodes Now Available in Zuul
by Clark Boylan 25 May '24

25 May '24

The OpenDev team has added Ubuntu Noble test nodes to Zuul and Nodepool. This means you can start running jobs on the latest Ubuntu release using `nodeset: ubuntu-noble` or `nodeset: ubuntu-noble-arm64`. If you build your own custom nodesets the labels you will want to use are `ubuntu-noble` and `ubuntu-noble-arm64`. We expect these to generally work and have already successfully run a number of builds on the new nodes. That said they are new, and you may need updates to bindep files and/or Ansible roles and playbooks. Let us know if you have any questions or find problems. Clark

1 0

Gerrit Downtime on May 31, 2024 at 16:00 UTC
by Clark Boylan 22 May '24

22 May '24

Hello, The OpenDev team is planning to upgrade Gerrit on review.opendev.org to version 3.9 on May 31, 2024 beginning at 16:00 UTC. In order to perform this upgrade we require a short service outage. We are allocating an hour for this outage, from 16:00 to 17:00 UTC, as that should give us plenty of time to implement the rollback procedure if that becomes necessary. The Gerrit 3.9 release notes can be found here: https://www.gerritcodereview.com/3.9.html. One major feature our users may be interested in is the ability to suggest and apply updates to files via review comments. If you have any questions about the upgrade or the new version of Gerrit, feel free to reach out to the OpenDev team via this email thread or on IRC in #opendev on OFTC. Clark

1 0

Gerrit Outage for Project Renames Monday April 22, 2024 at 20:00 UTC
by Clark Boylan 17 Apr '24

17 Apr '24

The OpenDev team is planning a scheduled Gerrit service outage on review.opendev.org in order to rename projects. This outage will occur on Monday April 22, 2024 at 20:00 UTC until 21:00 UTC. If you have a project you would like to rename please get in touch with the OpenDev team soon (you can respond to this email or find us in #opendev on the OFTC IRC network). Please reach out if you have any questions or concerns. The same methods described above work for that too. Clark

1 0

CVE-2024-31497 and SSH key statistics
by Jeremy Stanley 17 Apr '24

17 Apr '24

For those of you who haven't seen, CVE-2024-31497 in the PuTTY tool was announced in the past day. If you're using a NIST P521 ECDSA key and have ever used it with affected versions of PuTTY, you should remove its access everywhere and consider it potentially disclosed. Details here: https://www.chiark.greenend.org.uk/~sgtatham/putty/wishlist/vuln-p521-bias.… The key type is generally listed as the first field in the corresponding public key file, or in each line of an authorized_keys entry, including those you may have set for your account in our Gerrit server (review.opendev.org) If the type is listed as ecdsa-sha2-nistp521 or sk-ecdsa-sha2-nistp521(a)openssh.com then it's been put at risk if ever used with PuTTY versions 0.68 through 0.80. On a related note, since I wanted to see how popular that key type is among our Gerrit users, I ran some quick statistics. Of the 36976 accounts created there to date, 19888 of them have SSH keys authorized. There are 35057 total authorized keys, which means accounts with SSH keys average 1.76 keys per account. Out of those keys, 44 claim to be NIST P521 type (0.13%) used by 41 different accounts (0.20% of accounts with keys, 0.11% of all accounts). Here's a breakdown of the most popular types for keys authorized in our Gerrit: 95% ssh-rsa 3.2% ssh-ed25519 1.4% ssh-dss 0.38% ecdsa-sha2-nistp256 0.25% ssh-dsa 0.13% ecdsa-sha2-nistp521 0.023% ecdsa-sha2-nistp384 So as you can see, the key type potentially impacted by CVE-2024-31497 isn't all that popular in our circles, but it's definitely nonzero. -- Jeremy Stanley, OpenDev SysAdmin

1 0