----- Original message -----
From: Ian Wienand <iwienand@redhat.com>
To: Michael Turek <mjturek@us.ibm.com>
Cc: service-discuss@lists.opendev.org
Subject: [EXTERNAL] powerkvm CI nova cloning
Date: Thu, Dec 10, 2020 10:15 PM
 

Hello,

We've identified that powerkvm CI seems to be in a loop attempting to
clone nova.  It would be helpful if we could get this to stop :)

It's just a guess, but we found a similar problem in OpenDev where the
executors were attempting to cache the repo, but their default timeout
of 300 seconds wasn't long enough to complete the clone.  They would
kill their clone attempt and start over again immediately.  In the
Zuul executor logs you'll see something like

  2020-12-10 02:02:08,578 ERROR zuul.ExecutorServer: [e: ...]
  [build:...] Got exception while updating repo gerrit/openstack/nova

with a full traceback above.

Our solution was to increase the git_timeout [1] to long enough for
the clone to complete.  The recent upgrade may be to blame.  We're
experimenting with different options for gc/repack, but cloning the
repo from gerrit directly it close to 2gb.  Another option is to
convert this to cloning to https://opendev.org .

Please reach out if there is anything we can do to help.

Thanks,

-i

[1] https://review.opendev.org/c/opendev/system-config/+/766400