Hello, I would like to propose a solution to trigger jobs based on the state of an external resource accessible at an URL. The use case is quite common and actually Jenkins propose this kind of feature via the URLTrigger plugin [0]. I proposed two implementations of an URLTrigger driver for Zuul: - A trigger driver [1]: It is quite similar to the timer driver. The CronTrigger is used to provide the resource check rate. At each loop it checks the resource state and keep the state in a cache. When the cache state has changed the event is sent into the scheduler. - A require driver [2]: Idea was to rely on the timer driver to provide the resource check rate. Then attach a requirement in the pipeline via the require driver. The requirement is a RefFilter that match when the resource state change. Let me know what do you think about this URLTrigger idea. Regards, Fabien [0]: https://wiki.jenkins.io/display/JENKINS/URLTrigger+Plugin [1]: https://review.openstack.org/#/c/635567/ [2]: https://review.openstack.org/#/c/635554/
On Fri, Feb 8, 2019 at 5:48 AM Fabien Boucher <fboucher@redhat.com> wrote:
Hello,
I would like to propose a solution to trigger jobs based on the state of an external resource accessible at an URL. The use case is quite common and actually Jenkins propose this kind of feature via the URLTrigger plugin [0].
How would this be different than a periodic pipeline which checks a hash in the job and exits out if it doesn't match or performs the work if it does?
I proposed two implementations of an URLTrigger driver for Zuul:
- A trigger driver [1]: It is quite similar to the timer driver. The CronTrigger is used to provide the resource check rate. At each loop it checks the resource state and keep the state in a cache. When the cache state has changed the event is sent into the scheduler.
- A require driver [2]: Idea was to rely on the timer driver to provide the resource check rate. Then attach a requirement in the pipeline via the require driver. The requirement is a RefFilter that match when the resource state change.
Let me know what do you think about this URLTrigger idea.
Regards, Fabien
[0]: https://wiki.jenkins.io/display/JENKINS/URLTrigger+Plugin [1]: https://review.openstack.org/#/c/635567/ [2]: https://review.openstack.org/#/c/635554/ _______________________________________________ Zuul-discuss mailing list Zuul-discuss@lists.zuul-ci.org http://lists.zuul-ci.org/cgi-bin/mailman/listinfo/zuul-discuss
-- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser@vexxhost.com W. http://vexxhost.com
On Mon, Feb 11, 2019 at 4:01 PM Mohammed Naser <mnaser@vexxhost.com> wrote:
On Fri, Feb 8, 2019 at 5:48 AM Fabien Boucher <fboucher@redhat.com> wrote:
Hello,
I would like to propose a solution to trigger jobs based on the state of
an external
resource accessible at an URL. The use case is quite common and actually Jenkins propose this kind of feature via the URLTrigger plugin [0].
How would this be different than a periodic pipeline which checks a hash in the job and exits out if it doesn't match or performs the work if it does?
A periodic job to do that will have some issues: - Need to be set at a reasonable run frequency to not flood the SQL db and the build page - Where to store the previous state of the artifact, how to fetch it Such a job is not trivial to write and will probably be flawed. Better to provide a standard solution for every Zuul user with that use case I think.
Fabien Boucher <fboucher@redhat.com> writes:
On Mon, Feb 11, 2019 at 4:01 PM Mohammed Naser <mnaser@vexxhost.com> wrote:
On Fri, Feb 8, 2019 at 5:48 AM Fabien Boucher <fboucher@redhat.com> wrote:
Hello,
I would like to propose a solution to trigger jobs based on the state of
an external
resource accessible at an URL. The use case is quite common and actually Jenkins propose this kind of feature via the URLTrigger plugin [0].
How would this be different than a periodic pipeline which checks a hash in the job and exits out if it doesn't match or performs the work if it does?
A periodic job to do that will have some issues: - Need to be set at a reasonable run frequency to not flood the SQL db and the build page - Where to store the previous state of the artifact, how to fetch it
Such a job is not trivial to write and will probably be flawed. Better to provide a standard solution for every Zuul user with that use case I think.
This sounds like a good idea, and hopefully we can have it inherit from the timer trigger to reuse some code. Basically, it would be a timer trigger that only enqueued an event if the url changed. One thing we might anticipate people wanting is to trigger on different URLs for different projects. That may necessitate developing project-pipeline arguments to triggers (which we then might backport to the timer trigger so projects could specify their own schedules). Right now, we'd need to make a new pipeline for each non-overlapping set of URLs, which might be fine for a zuul tenant dedicated to a project, but less so for a general-purpose tenant (like openstack-infra). Mohammed's idea, however, isn't subject to that restriction. Using the child_jobs feature of zuul_return, we could write a zero-node job which checks a URL and clears child_jobs so that nothing else runs. That's something we could put in zuul-jobs for anyone to use, and we can do that today. It will keep resource costs down since it's a zero-node job, and won't run any real jobs unless the URL changes. We would still need to figure out where to store the previous URL state, and it would, as you say, flood the DB with instances of the zero-node job. -Jim
participants (3)
-
corvus＠inaugust.com
-
Fabien Boucher
-
Mohammed Naser