Container registry at OpenDev
Hi OpenDevers,
Are there any plans to have a container registry at OpenDev? Is there any thread that could be followed?
Kind regards, Radek -yoctozepto
On Mon, Jan 9, 2023, at 10:02 AM, Radosław Piliszek wrote:
Hi OpenDevers,
Are there any plans to have a container registry at OpenDev? Is there any thread that could be followed?
We've had several depending on how you look at it for quite some time now. There are the caching proxies for quay and docker hub. There is the insecure CI registry used for intermediate image artifacts. And there are buildset registries that live for the lifetime of a buildset providing locally cached images for jobs within a buildset.
You will probably need to be more specific about what you are asking for.
Kind regards, Radek -yoctozepto
On Mon, 9 Jan 2023 at 22:10, Clark Boylan cboylan@sapwetik.org wrote:
On Mon, Jan 9, 2023, at 10:02 AM, Radosław Piliszek wrote:
Hi OpenDevers,
Are there any plans to have a container registry at OpenDev? Is there any thread that could be followed?
We've had several depending on how you look at it for quite some time now. There are the caching proxies for quay and docker hub. There is the insecure CI registry used for intermediate image artifacts. And there are buildset registries that live for the lifetime of a buildset providing locally cached images for jobs within a buildset.
You will probably need to be more specific about what you are asking for.
Yeah, my bad. I obviously had it in mind when writing. ;-) I meant a permanent, publicly-available-for-download storage of versioned container images. Unless you somehow omitted that one option from your answer, I think you have answered that it is not supported at the moment which would agree with my upfront knowledge. On that note, are there any plans to add that support? Or is the best recommendation to use Quay (as DockerHub's limits are hurtful).
On Tue, 2023-01-10 at 10:28 +0100, Radosław Piliszek wrote:
On Mon, 9 Jan 2023 at 22:10, Clark Boylan cboylan@sapwetik.org wrote:
On Mon, Jan 9, 2023, at 10:02 AM, Radosław Piliszek wrote:
Hi OpenDevers,
Are there any plans to have a container registry at OpenDev? Is there any thread that could be followed?
We've had several depending on how you look at it for quite some time now. There are the caching proxies for quay and docker hub. There is the insecure CI registry used for intermediate image artifacts. And there are buildset registries that live for the lifetime of a buildset providing locally cached images for jobs within a buildset.
You will probably need to be more specific about what you are asking for.
Yeah, my bad. I obviously had it in mind when writing. ;-) I meant a permanent, publicly-available-for-download storage of versioned container images.
packaging of openstack has generally been out of scope fo the foundation. while there are some spec file repose that are hostsed as part of opendev binary packanges like rpms and debs have generally been considerd out of scope. the ci obvioulsy does produce tarballs of the distirbution and we push to pypi when appropriate but contaienr images has previously not been a artifact generated by opendev or openstack.
on the openstack side thats partly because there are 3 compteing container image project for openstack that i am aware of. kolla, loci and the zull project provided a third way to build images.
while i agree that it might be nice for openinfra to provide a generic registry instance similar to the one provide by github for integration with there ci i would be concerned by how much load that registry would have and the resocues both human and machine that would be required to run it.
Unless you somehow omitted that one option from your answer, I think you have answered that it is not supported at the moment which would agree with my upfront knowledge. On that note, are there any plans to add that support? Or is the best recommendation to use Quay (as DockerHub's limits are hurtful).
it would certenly be nice to have a opendev quay instance or similar registry aviable that would not not have the same rate limits. kolla if i am not mistaken has mostly swapped to quay.io as has tripleo/rdo due to the dockerhub limits. if we had an opendev registry we would still proably want to keep the iamge mirror to other registries. having the ablity to provide a contaienr registry like the github contaiern registry would bring more parity to the opendev hosting plathform vs github. had you a partical usage model in mind? i.e. a post merge job that woudl push on each commit or a perodic that woudl push some iamge nightly which would be enabled by the reslevent projects? which registry would be the athoritive default source and which would be the mirror? opendev? quay? dockerhub? the other question i woudl have is would the registry be open to project not hosted on opendev for development? presumably not but im just trying to gague how you expect it to be used.
while i agree that it might be nice for openinfra to provide a generic registry instance similar to the one provide by github for integration with there ci i would be concerned by how much load that registry would have and the resocues both human and machine that would be required to run it.
I also believe this would be the biggest roadblocker.
Unless you somehow omitted that one option from your answer, I think you have answered that it is not supported at the moment which would agree with my upfront knowledge. On that note, are there any plans to add that support? Or is the best recommendation to use Quay (as DockerHub's limits are hurtful).
it would certenly be nice to have a opendev quay instance or similar registry aviable that would not not have the same rate limits. kolla if i am not mistaken has mostly swapped to quay.io as has tripleo/rdo due to the dockerhub limits.
Yes, Kolla has moved to quay.io with the use of OpenDev's proxy for CI activities.
if we had an opendev registry we would still proably want to keep the iamge mirror to other registries. having the ablity to provide a contaienr registry like the github contaiern registry would bring more parity to the opendev hosting plathform vs github. had you a partical usage model in mind? i.e. a post merge job that woudl push on each commit or a perodic that woudl push some iamge nightly which would be enabled by the reslevent projects? which registry would be the athoritive default source and which would be the mirror? opendev? quay? dockerhub? the other question i woudl have is would the registry be open to project not hosted on opendev for development? presumably not but im just trying to gague how you expect it to be used.
This is related to the other questions about OpenDev I had recently. I mean it for a project hosted on OpenDev so that OpenDev can continue the policy of providing extra services to projects hosted by OpenDev and not externally. For these public-facing images, for now, I thought about both nightly (kept for the last 30 days let's say) and the supported releases (once 2-3 months) images to be kept available. However, this is very early for me to say anything more as we don't have any size estimates (not even on the image quantity). The authoritative would be OpenDev of course. As mentioned indirectly earlier, the current fallback I have written down is quay.io With GitHub we would use the "everything GitHub" approach.
On 2023-01-10 13:32:05 +0100 (+0100), Radosław Piliszek wrote: [...]
For these public-facing images, for now, I thought about both nightly (kept for the last 30 days let's say) and the supported releases (once 2-3 months) images to be kept available. However, this is very early for me to say anything more as we don't have any size estimates (not even on the image quantity).
[...]
Yes, knowing storage needs is one important aspect, but estimating bandwidth utilization is probably an even bigger concern for a potential service like this. Hard drives are cheap, relatively speaking, while capacity on Internet uplinks tends to be much harder to scale.
On 2023-01-10 10:28:21 +0100 (+0100), Radosław Piliszek wrote: [...]
I meant a permanent, publicly-available-for-download storage of versioned container images. Unless you somehow omitted that one option from your answer, I think you have answered that it is not supported at the moment which would agree with my upfront knowledge. On that note, are there any plans to add that support? Or is the best recommendation to use Quay (as DockerHub's limits are hurtful).
It's still not clear from your description whether you intend to only publish images for the software hosted in OpenDev, or copy images from other container repositories into it. Is it failing uploads into DockerHub which are causing pain, or downloading the dependency images from DockerHub in CI jobs, or some other limits?
If it's that DockerHub's download limits are impacting outside users of your published images (not just CI jobs, where we can try to do a better job of proxying/caching those), then that suggests the bandwidth needed to host these is probably astronomical, and that would be my biggest concern. DockerHub is clearly imposing limits for a reason, so if it's not cheap/easy for them to serve the content then I don't expect it to be any easier for a volunteer community relying on donated services to do so.
On Tue, 10 Jan 2023 at 14:13, Jeremy Stanley fungi@yuggoth.org wrote:
On 2023-01-10 10:28:21 +0100 (+0100), Radosław Piliszek wrote: [...]
I meant a permanent, publicly-available-for-download storage of versioned container images. Unless you somehow omitted that one option from your answer, I think you have answered that it is not supported at the moment which would agree with my upfront knowledge. On that note, are there any plans to add that support? Or is the best recommendation to use Quay (as DockerHub's limits are hurtful).
It's still not clear from your description whether you intend to only publish images for the software hosted in OpenDev, or copy images from other container repositories into it. Is it failing uploads into DockerHub which are causing pain, or downloading the dependency images from DockerHub in CI jobs, or some other limits?
I think this got answered by another mail - it will be OpenDev-hosted only. As for what is failing - nothing, because there is nothing. :D I was scoping the interest/vision on the side of OpenDev to be able to characterise it properly for stakeholders.
If it's that DockerHub's download limits are impacting outside users of your published images (not just CI jobs, where we can try to do a better job of proxying/caching those), then that suggests the bandwidth needed to host these is probably astronomical, and that would be my biggest concern. DockerHub is clearly imposing limits for a reason, so if it's not cheap/easy for them to serve the content then I don't expect it to be any easier for a volunteer community relying on donated services to do so.
(continued)
Yes, knowing storage needs is one important aspect, but estimating bandwidth utilization is probably an even bigger concern for a potential service like this. Hard drives are cheap, relatively speaking, while capacity on Internet uplinks tends to be much harder to scale.
That is a very good argument and it has indeed crossed my mind before. That said, it is close to impossible to estimate for a not-yet-existent project how often it will be downloaded.
Anyways, thank you all for your answers. I know all I wanted to know.
Summarising:
* CI aspects are well-covered (proxies, shared insecure registry, buildset registry) * public hosting of container image artifacts is not in the current scope and unlikely to be due to the impact it might have on the overall infrastructure
participants (4)
-
Clark Boylan
-
Jeremy Stanley
-
Radosław Piliszek
-
Sean Mooney