Hi,
We've run into an issue a few times where a complex job defined in a
project repo doesn't run because it can't find a parent job that matches
the branch.
In all the cases I've seen so far, it's children of OpenStack's devstack
job which are at issue. That's because the devstack job is defined in
the devstack repo (an untrusted project repo) with branches. That means
that it has implied branch matchers attached to it.
We could, of course, turn those off, however, the intention is for the
devstack job to be branched. Running the Queens release is not the same
as running Ocata, so the different branch variants should be used under
different circumstances.
The problem occurs when someone wants to inherit from the devstack job
in a different repo, and there is not a devstack branch variant which
matches the branch of the child job.
The way Zuul pieces together the final job that is run is to take the
name of the job specified in the project-pipeline config, and then
collect all the variants of that job which match the change. Each time
it finds a match, it looks at the parent of that variant, and
recursively collects all the variants of the parent which match the
change (eliminating duplicates). If it can't find at least one variant
of the parent which matches the change, the job is not run.
The first time this came to my attention was for the tempest job.
During development, the devstack job has only been defined on the master
branch of devstack. The tempest job is in a repo with no branches, so
it doesn't have an implied branch matcher. When any project invokes it,
it will match the branch, and then Zuul will look for its parent job,
'devstack', and as long as the change is on the master branch, it will
find it and run. However, if the change is on stable/ocata, and there
is no devstack job defined in the stable/ocata branch of devstack, then
it won't run.
I think the solution to this case is to backport the devstack job to
devstack's stable branches. That's the situation we will ultimately be
in after our natural branching progression anyway, and we should treat
this as a one-time cost of the transition to Zuul v3.
The other two instances where this has come up are for projects which
should run a child of the devstack job on a branch which does not, and
never will, exist in devstack.
When nodepool had a feature/zuulv3 branch, we were unable to run the
devstack integration test job on that branch because of this. Ansible's
default branch is 'devel', which means in order for us to run the shade
integration test job on changes to Ansible, Zuul looks for a devstack
variant which matches the 'devel' branch.
In all cases, these issues could technically be solved by creating
devstack job variants which matched these other branches. However, it
should not be incumbent on the devstack maintainers to anticipate all of
the branch names of other projects which might want to use the devstack
job. We could create a devstack job variant with an explicit branch
matcher which matches all branches, and treat the stable branch variants
as deltas from that, however, that is more difficult to manage and
doesn't fit our branch creation model.
I can think of two solutions to this problem:
1) Have Zuul use a default branch when searching for parents with no
matching branch variants.
Every project in Zuul has a default branch (which can be specified in
the 'project' config stanza, but the default default is 'master'). If,
when searching for the parents of a job, Zuul finds no matching
variants, it could perform the search a second time with the default
branch of the parent job's project (all variants of a job are defined in
the same repo, so we know which project the job is in, so there's only
one default branch to search for each parent).
2) Allow someone creating a child job to specify a fallback branch for
Zuul to use when searching for parent jobs.
Essentially, we would modify the algorithm in the same way as the
previous suggestion, except instead of determining the fallback branch
based on the default branch of the parent job, we would let the child
job supply the fallback branch. At each level of recursion as we walk
up the inheritance path, a new fallback branch could be provided.
I'm inclined to implement option (1) at least. That is very similar to
what already happens today with git repo checkouts. I.e., if you did
manage to run one of the jobs in the examples I described, you would get
the master branch of devstack checked out in the job because of the
existing fallback behavior. It makes sense that the job matching
algorithm would be similar. And I think in many of the cases where this
issue comes up, it would do the right thing automatically (certainly it
would have in all the examples thus far).
I'm on the fence about option (2). I see no reason not to add it other
than perhaps it's unnecessary complexity. Maybe we should avoid
implementing it until we find a compelling case for it? Or do folks
anticipate a need for that now?
Thanks,
Jim