review.opendev.org SSH key change?
I have clned the python-jenkins repo few days ago and today I am getting this:
git remote update
Fetching upstream @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the RSA key sent by the remote host is SHA256:RXNl/GKyDaKiIQ93BoDvrNSKUPFvA1PNeAO9QiirYZU. Please contact your system administrator. Update the SSHFP RR in DNS with the new host key to get rid of this message.
git remote -v
upstream ssh://saperski@review.opendev.org:29418/jjb/python-jenkins (fetch) upstream ssh://saperski@review.opendev.org:29418/jjb/python-jenkins (push)
My cached entries are:
[review.opendev.org]:29418 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfsIj/jqpI+2CFdjCL6kOiqdORWvxQ2sQbCzSzzmLXic8yVhCCbwarkvEpfUOHG4eyB0vqVZfMffxf0Yy3qjURrsroBCiuJ8GdiAcGdfYwHNfBI0cR6kydBZL537YDasIk0Z3ILzhwf7474LmkVzS7V2tMTb4ZiBS/jUeiHsVp88FZhIBkyhlb/awAGcUxT5U4QBXCAmerYXeB47FPuz9JFOVyF08LzH9JRe9tfXtqaCNhlSdRe/2pPRvn2EIhn5uHWwATACG9MBdrK8xv8LqPOik2w1JkgLWyBj11vDd5I3IjrmREGw8dqImqp0r6MD8rxqADlc1elfDIXYsy+TVH review.opendev.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBjjFqbkqLfVKn5vJnKh/LfGGo0gXp/vWlXRs0H91E0X8ce4r8DHLB8PMScHrX/n7c29/DY3FQqSaYsY6o13dFA=
DNS sshfp records:
review.opendev.org. 193 IN CNAME review01.opendev.org. review01.opendev.org. 276 IN SSHFP 1 1 4A38DD28E379EA443F8A722D8330EF41754F0D5E review01.opendev.org. 276 IN SSHFP 1 2 4234515B3D59F8BF7E7095BD881F39DB21B97A15511C4B10B7FD0240 25AD93FC review01.opendev.org. 276 IN SSHFP 3 1 382C41E6FFC60CF1939B292FA62879B1918145D4 review01.opendev.org. 276 IN SSHFP 3 2 52A81E8DD662F92D903199DBC5068280D33D21D3A8E5BD023FAADD47 99AC1DDF review01.opendev.org. 276 IN SSHFP 4 1 DE5FAA47C38E616ECB2CCC4C30C7E3F788C0927A review01.opendev.org. 276 IN SSHFP 4 2 4BE301BEEC8DCC06C1084BEF1DB1D136686022B7026F678D958E548E 4B7D2FC7
Is everything ok?
Marcin
Hi Marin,
I did a bit of digging and I'm not sure why you're seeing that message (perhaps the SSHFP records point to the _system_ SSH keys and not Gerrit), anyhow:
`ssh-keyscan -p29418 review.opendev.org` matches the key cached in my system (and yours). `ssh-keyscan -p29418 review.opendev.org | ssh-keygen -lf -` matches the fingerprint you're being provided.. have you updated your SSH client recently?
Thanks Mohammed
On Sat, Aug 1, 2020 at 3:43 AM Marcin Cieslak saper@saper.info wrote:
I have clned the python-jenkins repo few days ago and today I am getting this:
git remote update
Fetching upstream @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the RSA key sent by the remote host is SHA256:RXNl/GKyDaKiIQ93BoDvrNSKUPFvA1PNeAO9QiirYZU. Please contact your system administrator. Update the SSHFP RR in DNS with the new host key to get rid of this message.
git remote -v
upstream ssh://saperski@review.opendev.org:29418/jjb/python-jenkins (fetch) upstream ssh://saperski@review.opendev.org:29418/jjb/python-jenkins (push)
My cached entries are:
[review.opendev.org]:29418 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfsIj/jqpI+2CFdjCL6kOiqdORWvxQ2sQbCzSzzmLXic8yVhCCbwarkvEpfUOHG4eyB0vqVZfMffxf0Yy3qjURrsroBCiuJ8GdiAcGdfYwHNfBI0cR6kydBZL537YDasIk0Z3ILzhwf7474LmkVzS7V2tMTb4ZiBS/jUeiHsVp88FZhIBkyhlb/awAGcUxT5U4QBXCAmerYXeB47FPuz9JFOVyF08LzH9JRe9tfXtqaCNhlSdRe/2pPRvn2EIhn5uHWwATACG9MBdrK8xv8LqPOik2w1JkgLWyBj11vDd5I3IjrmREGw8dqImqp0r6MD8rxqADlc1elfDIXYsy+TVH review.opendev.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBjjFqbkqLfVKn5vJnKh/LfGGo0gXp/vWlXRs0H91E0X8ce4r8DHLB8PMScHrX/n7c29/DY3FQqSaYsY6o13dFA=
DNS sshfp records:
review.opendev.org. 193 IN CNAME review01.opendev.org. review01.opendev.org. 276 IN SSHFP 1 1 4A38DD28E379EA443F8A722D8330EF41754F0D5E review01.opendev.org. 276 IN SSHFP 1 2 4234515B3D59F8BF7E7095BD881F39DB21B97A15511C4B10B7FD0240 25AD93FC review01.opendev.org. 276 IN SSHFP 3 1 382C41E6FFC60CF1939B292FA62879B1918145D4 review01.opendev.org. 276 IN SSHFP 3 2 52A81E8DD662F92D903199DBC5068280D33D21D3A8E5BD023FAADD47 99AC1DDF review01.opendev.org. 276 IN SSHFP 4 1 DE5FAA47C38E616ECB2CCC4C30C7E3F788C0927A review01.opendev.org. 276 IN SSHFP 4 2 4BE301BEEC8DCC06C1084BEF1DB1D136686022B7026F678D958E548E 4B7D2FC7
Is everything ok?
Marcin
Am Sa., 1. Aug. 2020 um 09:43 Uhr schrieb Marcin Cieslak saper@saper.info:
I have clned the python-jenkins repo few days ago and today I am getting this:
git remote update
Fetching upstream @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the RSA key sent by the remote host is SHA256:RXNl/GKyDaKiIQ93BoDvrNSKUPFvA1PNeAO9QiirYZU. Please contact your system administrator. Update the SSHFP RR in DNS with the new host key to get rid of this message.
git remote -v
upstream ssh://saperski@review.opendev.org:29418/jjb/python-jenkins (fetch) upstream ssh://saperski@review.opendev.org:29418/jjb/python-jenkins (push)
My cached entries are:
[review.opendev.org]:29418 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfsIj/jqpI+2CFdjCL6kOiqdORWvxQ2sQbCzSzzmLXic8yVhCCbwarkvEpfUOHG4eyB0vqVZfMffxf0Yy3qjURrsroBCiuJ8GdiAcGdfYwHNfBI0cR6kydBZL537YDasIk0Z3ILzhwf7474LmkVzS7V2tMTb4ZiBS/jUeiHsVp88FZhIBkyhlb/awAGcUxT5U4QBXCAmerYXeB47FPuz9JFOVyF08LzH9JRe9tfXtqaCNhlSdRe/2pPRvn2EIhn5uHWwATACG9MBdrK8xv8LqPOik2w1JkgLWyBj11vDd5I3IjrmREGw8dqImqp0r6MD8rxqADlc1elfDIXYsy+TVH review.opendev.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBjjFqbkqLfVKn5vJnKh/LfGGo0gXp/vWlXRs0H91E0X8ce4r8DHLB8PMScHrX/n7c29/DY3FQqSaYsY6o13dFA=
DNS sshfp records:
review.opendev.org. 193 IN CNAME review01.opendev.org. review01.opendev.org. 276 IN SSHFP 1 1 4A38DD28E379EA443F8A722D8330EF41754F0D5E review01.opendev.org. 276 IN SSHFP 1 2 4234515B3D59F8BF7E7095BD881F39DB21B97A15511C4B10B7FD0240 25AD93FC review01.opendev.org. 276 IN SSHFP 3 1 382C41E6FFC60CF1939B292FA62879B1918145D4 review01.opendev.org. 276 IN SSHFP 3 2 52A81E8DD662F92D903199DBC5068280D33D21D3A8E5BD023FAADD47 99AC1DDF review01.opendev.org. 276 IN SSHFP 4 1 DE5FAA47C38E616ECB2CCC4C30C7E3F788C0927A review01.opendev.org. 276 IN SSHFP 4 2 4BE301BEEC8DCC06C1084BEF1DB1D136686022B7026F678D958E548E 4B7D2FC7
Is everything ok?
The SSHFP records document the keys for the SSH daemon listening on port 22 used for administrative access to the server, not the keys used by gerrit. AFAICT there is no way to specify keys for different ports in DNS, so when accessing gerrit via ssh, you will have to disable DNS verification in order to get rid of this warning. For openssh this would mean to set VerifyHostKeyDNS=no (which is also the default, so likely you must have overridden this somewhere), but I do get a similar error to yours if I set the option to "yes".
Side note: Please don't clone from review.opendev.org directly but use our git farm at opendev.org for this, the review site should only be used for gerrit things, like submitting patches. At opendev.org we have multiple gitea servers behind a load balancer, allowing us to scale performance as needed, while we have gerrit is only a single instance with limited resources.
Yours Jens
On 2020-08-01 15:44:21 +0200 (+0200), Dr. Jens Harbott wrote: [...]
The SSHFP records document the keys for the SSH daemon listening on port 22 used for administrative access to the server, not the keys used by gerrit. AFAICT there is no way to specify keys for different ports in DNS, so when accessing gerrit via ssh, you will have to disable DNS verification in order to get rid of this warning. For openssh this would mean to set VerifyHostKeyDNS=no (which is also the default, so likely you must have overridden this somewhere), but I do get a similar error to yours if I set the option to "yes".
[...]
This is going to be challenging to work around, I think.
The cleanest solution is probably going to be separating the review.opendev.org service name from the system's FQDN in DNS. This way we could avoid publishing SSHFP RRs for the service name (or better still, publish different SSHFP RRs), but that means we'll need to separate out the ACME glue for DNS based X.509 cert renewals. That would likely not be too hard if we can just stop putting review01.opendev.org as one of the subject altnames.
An alternative would be to sync the Gerrit mina-sshd API and system OpenSSH host keys, though that could present a degradation of security for the base system (maybe effectively not one we care about though?).
Another alternative would be to just drop the SSHFP RRs for the Gerrit server, though that makes it inconsistent from the rest of our servers if we do.
On 2020-08-01 14:09:53 +0000 (+0000), Jeremy Stanley wrote: [...]
The cleanest solution is probably going to be separating the review.opendev.org service name from the system's FQDN in DNS. This way we could avoid publishing SSHFP RRs for the service name (or better still, publish different SSHFP RRs), but that means we'll need to separate out the ACME glue for DNS based X.509 cert renewals. That would likely not be too hard if we can just stop putting review01.opendev.org as one of the subject altnames.
[...]
Clark just reminded me in the #opendev IRC channel that we already serve separate _acme-challenge.review and _acme-challenge.review01 CNAMEs to our acme zone, so nothing actually needs to change with SSL cert renewal verification. We can just replace the review CNAME with A/AAAA, copy the two CAA RRs from review01 to review, and generate the six new SSHFP RRs for the Gerrit API associated with the review hostname.
On 2020-08-03 20:48:22 +0000 (+0000), Jeremy Stanley wrote:
On 2020-08-01 14:09:53 +0000 (+0000), Jeremy Stanley wrote:
[...]
Clark just reminded me in the #opendev IRC channel that we already serve separate _acme-challenge.review and _acme-challenge.review01 CNAMEs to our acme zone, so nothing actually needs to change with SSL cert renewal verification. We can just replace the review CNAME with A/AAAA, copy the two CAA RRs from review01 to review, and generate the six new SSHFP RRs for the Gerrit API associated with the review hostname.
In fact, as a stop gap, we can omit the SSHFP records for review.o.o initially, which will restore the prior situation for folks connecting to the API port. I'll push that now:
https://review.opendev.org/744557
I am in favor of doing a key sync as this would be the least intrusive for the user point of view.
Having the SSHFP is a good measure and improves security far more than the issue of having the key synchronized.
On 1 Aug 2020, at 15:09, Jeremy Stanley fungi@yuggoth.org wrote:
An alternative would be to sync the Gerrit mina-sshd API and system OpenSSH host keys, though that could present a degradation of security for the base system (maybe effectively not one we care about though?).
On 2020-09-14 13:00:58 +0100 (+0100), Sorin Sbarnea wrote:
I am in favor of doing a key sync as this would be the least intrusive for the user point of view.
Having the SSHFP is a good measure and improves security far more than the issue of having the key synchronized.
[...]
We ended up solving it by having the SSHFP records for review.opendev.org reflect the mina-ssh based Gerrit API listener on 29418/tcp, while the SSHFP records for review01.opendev.org (the server's canonical name) correspond to the OpenSSH listener for the operating system itself on 22/tcp. It meant dropping the CNAME indirection for the service name and just giving it A/AAAA records directly, but that's not particularly inconvenient for us to manage (they're effectively adjacent lines in the same zone file anyway).
participants (5)
-
Dr. Jens Harbott
-
Jeremy Stanley
-
Marcin Cieslak
-
Mohammed Naser
-
Sorin Sbarnea