[service-announce] review.opendev.org Gerrit outage and upgrade 15:00UTC November 20 to 01:00UTC November 23, 2020

[Clark Boylan]

Hello everyone,

By now you've probably noticed that the Gerrit upgrade appears complete. And for most users it probably is. Gerrit 3.2 is up and running, Zuul is talking to it and running jobs, and you can push/review changes. On the OpenDev side of things we're continuing to identify issues and fix them as we can. We are also still in the process of returning our Zuul driven continuous deployment to normal operations.

I hesitate to call this completely done as we may need to restart Gerrit periodically to apply config updates to address issues that we find. During our announced upgrade period (which ends at 01:00 UTC November 23) we will do these without announcements. Don't be surprised if you find Gerrit is offline during this period. Once this block of time has passed we may still need to do restarts, but we'll do our best to send notice of them on IRC and in our status log.

The only major unexpected functional issue we've found is in the git hosting for x/foo-project repos in Gerrit. The new Gerrit UI web endpoints conflict with our use of the x/ namespace, meaning "git clone http://review.opendev.org/x/foo-project" is being incorrectly interpreted as a UI request, rather than as a git clone. We've worked around that and have started to engage upstream to see if we can find a permanent fix for these projects. But I want to call this out early because there is a chance we may need to rename the entirety of x/* to notx/* in the future. We recognize this is not ideal, and we didn't catch it in testing. We'll do our best to communicate and plan for this with you if it becomes necessary.

The crowd sourcing of feedback has begun at:
    https://etherpad.opendev.org/p/gerrit-3.2-post-upgrade-notes
Feel free to add things you notice as well.

Thank you for your patience.
Clark on behalf of the OpenDev team