May 2019 - Zuul-discuss - lists.opendev.org

Correct way to trigger a periodic job manually
by Ian Wienand 08 May '19

08 May '19

Hello, The manual says to trigger a periodic job against master you'd do [1] $ zuul enqueue-ref --tenant openstack --trigger timer --pipeline periodic --project openstack/example_project --ref refs/heads/master Which looks about right to me, but this sets a newrev of 00..000 which is not correct; e.g. [2]. Tracing back that job to find the cause of the RETRY_LIMIT on ze06 I see: --- 2019-05-06 07:20:50,875 DEBUG zuul.AnsibleJob: [build: b1387c9741134f83a6f11e1cad0386f0] Checking out master 2019-05-06 07:20:50,877 ERROR zuul.AnsibleJob: [build: b1387c9741134f83a6f11e1cad0386f0] Exception while executing job Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/git/refs/symbolic.py", line 305, in set_reference obj = self.repo.rev_parse(ref + "^{}") # optionally deref tags File "/usr/local/lib/python3.5/dist-packages/git/repo/fun.py", line 213, in rev_parse obj = name_to_object(repo, rev[:start]) File "/usr/local/lib/python3.5/dist-packages/git/repo/fun.py", line 147, in name_to_object raise BadName(name) gitdb.exc.BadName: Ref 'master' did not resolve to an object During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/zuul/executor/server.py", line 790, in execute self._execute() File "/usr/local/lib/python3.5/dist-packages/zuul/executor/server.py", line 914, in _execute repo.checkout(selected_ref) File "/usr/local/lib/python3.5/dist-packages/zuul/merger/merger.py", line 369, in checkout File "/usr/local/lib/python3.5/dist-packages/git/refs/symbolic.py", line 308, in set_reference raise ValueError("Could not extract object from %s" % ref) ValueError: Could not extract object from master --- I set newref manually to the git master HEAD change-id, and it did work -- but it looks different to jobs actually triggered by the periodic driver itself. For example [3] is the job I triggered, and [4] is one from the periodic driver -- [4] does not have a newref. I feel like this used to work, probably because when I look back I wrote that bit of documentation [5] ;) I guess something changed; perhaps it is the client setting newref to 0000.000 by default instead of something like None? Any tips on what should I update the instructions to say now? -i [1] https://zuul-ci.org/docs/zuul/admin/client.html#manual-enqueue-examples [2] http://zuul.openstack.org/build/b1387c9741134f83a6f11e1cad0386f0 [3] http://zuul.openstack.org/build/8781cd38097b437d90ba5a9a5505633c [4] http://zuul.openstack.org/build/d30f960f7ca04b33af1eb5b7513b1b66 [5] https://review.opendev.org/#/c/518662/

2 2

exclude-unprotected-branches equivalent for Gerrit connection
by Brendan Jackman 08 May '19

08 May '19

Hi all, In my org's Gerrit setup, refs under personal/$username/ are a free-for-all. I have seen the same in other organisations too. This leads to the problem that is solved for Github by exclude-protected-branches; those branches eventually end up with broken Zuul configs (actually case that brought this to my attention is that I cannot delete a job that is no longer used, because the config in a personal/ branch refers to it). It seems like a simple solution would be to add an option under tenant.untrusted-projects.<project> that can be used to filter the branches considered by Zuul, by their name. I think the most obvious interface for that would be like job.files & job.irrelevant-files, i.e. a regex or list thereof, with a negative complement option alongside it. I think it could then behave much like exclude-unprotected-branches in that it works at the source connection level. Although I've mentioned Gerrit, I don't see why it shouldn't be independent of the actual source connection driver. This seems suspiciously obvious so I suspect I may be rather naive in suggesting it! Has it been discussed before, does it present more complications than I can see? I'm not too sure how this would related to tags. What does Zuul do with those? I didn't give the proposed option a name because I don't know if what I really want is a ref filter or a branch filter. In fact, I face a certain degree of pain because a couple the projects my Zuul deployment checks out have a lot of tags[1] so maybe this could kill two birds with one stone. Cheers, Brendan [1] $ git ls-remote --tags --refs git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git | wc -l 2763 This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed (‘intended recipient’). Accordingly the dissemination, distribution, copying or other use of this email or any of its content by any person other than the intended recipient may constitute a breach of civil or criminal law and is strictly prohibited. If you are not the intended recipient, please let us know as soon as possible and remove this email from your system. Company Reg No: 06855929 | Blu Wireless Technology Ltd, 5th Floor, 1 Temple Way, Bristol, BS2 0BY | Registered in England & Wales

2 2

Inheritance confusion, docs needed?
by Michael Johnson 07 May '19

07 May '19

As a developer creating Zuul jobs, one of the areas i find confusing is the inheritance of variables. I am not sure if this is an enhancement style request or just a missing section in the documentation. When you use host-vars, the top level variables are "merged" with the parent job variables, however the nested variables, such as "devstack_localrc" are not merged, they are "replaced". As an example is the opendev Zuul jobs (I think this applies to non-OpenStack jobs as well): When you use host-vars, the top level variables are "merged" with the parent job variables, however the nested variables, such as "devstack_localrc" are not merged, they are "replaced". host-vars: controller: devstack_localrc: DATABASE_USER: a-user In this example snippet, the variables at the devstack_localrc level are "merged", but the variables at the "DATABASE_USER" level are replaced and not inherited from the parent. Should Zuul be recursively "merging" all the way down or should we document how/which variables are inherited from the parent and which are not? The reason I'm asking is as our job definitions get more complicated (multi-node), I find myself copying variables from the parent jobs that I would really like to just inherit. Michael

2 1