OpenSSH 8.8, RSA keys, and Gerrit

[Clark Boylan]

On Thu, Oct 14, 2021, at 9:04 AM, Clark Boylan wrote:
> Hello,
>
> About a year ago Fedora 33 released and gave us a preview of OpenSSH's 
> sha1 + RSA key deprecation fallout. Fedora 33 users noticed they could 
> no longer use SSH RSA keys to connect to our Gerrit at 
> review.opendev.org. This happens because Fedora 33's OpenSSH packaging 
> has deprecated sha1 hashes for RSA, and despite both the client and 
> server supporting rsa-sha2-* variants they couldn't negotiate their use 
> between them. OpenSSH 8.8 released recently and did similar in the 
> upstream software which means users with up to date OpenSSH 
> installations are noticing similar problems (Arch Linux for example).
>
> There are a couple of workarounds that you can use. Probably the best 
> option is to use an ed25519 or ecdsa key with our Gerrit. Modern 
> clients and our Gerrit SSHD negotiate these keys without issue. Less 
> optimal is to manually re-enable the use of the ssh-rsa hash, but we 
> recommend against this as your software providers have decided this is 
> no longer secure enough.
>
> On our end we've brought this up with the MINA SSHD devs [0] with the 
> hope that the SSH implementation that Gerrit uses can be updated to 
> negotiate the sha2 hashes properly. Also, the rsa-sha2 RFC indicates 
> [1] clients may fallback to a sha2 variant instead of the sha1 variant 
> which would workaround MINA's lack of support for negotiation in the 
> protocol. If you are an OpenSSH>=8.8 or Fedora>=33 user you might 
> consider filing bugs against your ssh clients to change the default 
> fallback to a sha2 variant on your platforms.

And now exactly a year after I wrote this email I get to respond to myself saying we've deployed a corrected version of Gerrit. We had actually managed to correct Gerrit 3.6 earlier this year, but the fix required MINA 2.8.0 which was much newer than what was found in Gerrit 3.5. Recently, MINA 2.8.0 was backported into Gerrit 3.5 allowing us to backport the RSA key fix into Gerrit 3.5 as well. And now we've deployed updated images based on those updates.

What this means is that you should be able to use a modern openssh client and an RSA key to communicate to review.opendev.org without any config overrides. If you've already converted to a different key type there is no reason to convert back to RSA. The main impact here is those that have config overrides can remove them and use more secure RSA + SHA2 key negotiation.

>
> [0] https://issues.apache.org/jira/browse/SSHD-1141
> [1] https://datatracker.ietf.org/doc/html/rfc8332#section-3.3
>
> Hopefully I've put enough keywords in this email that the various 
> search engines will index it, and the next time someone runs into these 
> problems they'll find this explanation.
>
> Clark