- Zuul-discuss - lists.opendev.org

Projects config introspection
by Tristan Cacqueray 12 Dec '18

12 Dec '18

Hi all, I'd like to start the discussion around introspecting projects configs. The goal is to be able to efficiently answer the following questions: * Which projects use a specific job, * Which jobs use a specific nodeset, * Which nodesets or jobs use a specific label, and * Which jobs, labels or nodesets are not used. Putting aside the job variants model (which isn't well exposed in the current REST APIs), the easiest solution would be to add new endpoints, such as /api/job-per-project/{job_name}, or /api/label-per-project/{label_name}. But this feels sub-optimal and we need a new route for each type of query. Another solution would be to add a global toDict() method for the whole Tenant configuration object, which can be exposed at /api/config (or simply dumped using a zuul-scheduler command line argument). This would enable custom search to be done client-side. Another solution would be to implement the query using GraphQL custom resolver as started in https://review.openstack.org/574625 Regards, -Tristan

8 14

Adding an unconditional cleanup job phase
by Brendan Jackman 11 Dec '18

11 Dec '18

Hi all, Looking, for the moment, for lower-hanging fruit in my quest to reduce test resource pressure with Zuul. One unfortunate hack in my use-case is that we currently lock and release test resources which are shared with another system. This is handled by an ansible role that sends API calls to that other system. This means summarily killing a running job can prevent a resource from getting unlocked. To get around this I've had to disable pipeline.dequeue-on-new-patchset. I'd like to add stage where playbooks can be added that still gets run even if the job gets stopped. My guess is that this is also useful to other people who are using the static nodepool driver and would like to unconditionally clean up files left behind by jobs - does that sound correct? It generally seems that there would be value in having and unconditional cleanup stage, anyway. If you agree so far, can we talk semantics & implementation? I think it would be sensible that these playbooks are _never_ killed, but that obviously poses a risk that users can get stuck with an unkillable job running. I guess a simple solution is to have a distinct timeout from the job timeout, which covers just the running of these playbooks. We recommend setting this pretty short (default 30 secs, say), and strictly minimising the logic in these playbooks. If I'm still making sense then from what I've read so far, it seems all the complexity here can just go in AnsibleJob - the gearman communication would already support this. The client would just send the executor:stop TextJob as normal, and then the server can return its response asynchronously when the cleanup playbooks have finished? How does that sound? Cheers, Brendan

2 3

Roadmap for running jobs on OpenShift
by Tristan Cacqueray 10 Dec '18

10 Dec '18

Hello everyone, Please find below a list of topics I think we should address in order to land the OpenShift driver. Kube config integration in zuul-executor ---------------------------------------- As of now, the k8s and the openshift drivers need a setup function in the exector module in order to be usable: https://review.openstack.org/570668 The Openshift drivers --------------------- https://review.openstack.org/570667 and https://review.openstack.org/590335 are 2 variants to support different kinds of cluster accounts: self-provisioner and user. The code may be refactored in a single driver. Inventory update by job ----------------------- We should consider a pre-run phase to create new resources and update the job inventory using zuul_return to enable the test phase to uses "hosts: pod", as proposed in: https://review.openstack.org/590092 Roles and parent jobs --------------------- The containers use-cases need a lot of love to manage builds and to enable new workflows, for example, to promote an image from one cluster to a central registry and trigger updates in others clusters. Here is an initial build&deploy Openshift pod as well as a prepare-workspace implementation using oc rsync proposed here: https://review.openstack.org/570669 Even though this doesn't have to be part of Zuul, I think we should build new roles in zuul-jobs as a community. Define the security context --------------------------- It seems like opinions diverge on the security requirements for running untrusted jobs in pods. At the summit, I was told that we should keep on using the root user so that bindep and other service operations stay the same. I strongly disagree with this approach because containers that run as root have far more permissions than their workload requires and this could be abused by untrusted playbooks. Nodepool accesses to a k8s or Openshift cluster shall be restricted to non-root users to mitigate privilege escallation. Even though this can be discussed later, I think it's important to set the expectation immediately and keep users safe. How many users are interested in these topics and what are your review bandwidths? Can we get an estimated release date for OpenShift users? Regards, -Tristan

7 16

[Newsletter] What's Happening in the Open Infrastructure Community
by Claire Massey 06 Dec '18

06 Dec '18

Hi everyone, This week, the OSF team distributed the first Open Infrastructure Community Newsletter. Our goal with the bi-weekly newsletter is to provide a digest of the latest developments and activities across open infrastructure projects, events and users. This week, the newsletter highlighted the Berlin Summit as well as brief updates from Zuul and other OSF projects - OpenStack, Airship, Kata Containers and StarlingX. You can checkout the full newsletter on Superuser [1], and if you are interested in receiving the upcoming newsletters, you can subscribe here [2]. If you would like to contribute to a future newsletter or have feedback, please reach out to community(a)openstack.org <mailto:community@openstack.org>. Thanks, Claire [1] http://superuser.openstack.org/articles/inside-open-infrastructure-newslett… <http://superuser.openstack.org/articles/inside-open-infrastructure-newslett…> [2] https://www.openstack.org/community/email-signup <https://www.openstack.org/community/email-signup>

1 0

Software Factory 3.2 released
by Tristan Cacqueray 05 Dec '18

05 Dec '18

We are pleased to announce the release of Software Factory version 3.2: https://www.softwarefactory-project.io/releases/3.2 This new release consolidates the 3.1 version with updated packages, minor fixes and some new features such as an improved update mechanism. Software Factory version 3.2 integrates Zuul and Nodepool version 3.3.1. Here is the list of features we picked in the Software Factory version of Zuul and Nodepool that are still under review upstream. We are confident these features will eventually land upstream, but they might evolve by then. They should therefore be considered a "tech preview", although we will do our best to integrate these in the least disruptive way possible: * Improved dashboard: https://review.openstack.org/#/q/topic:react-zuul * Openshift driver: https://review.openstack.org/#/q/topic:container-build-resources * RunC driver: https://review.openstack.org/535556 The Nodepool package also contains some backports to mitigate stability issues found in the (upstream) version 3.3.1: * Set pool for error'ed instances: https://review.openstack.org/621681 * Set type for error'ed instances: https://review.openstack.org/622101 * Log exceptions deleting ZK nodes: https://review.openstack.org/621301 * Fix race when deleting Node znodes: https://review.openstack.org/622403 * Count leaked nodes in unmanaged quota: https://review.openstack.org/621040 * Cleanup routine to delete empty nodes: https://review.openstack.org/622616 Best regards, The Software Factory Team

2 1

Zuul project update 2018-12-03
by corvus＠inaugust.com 04 Dec '18

04 Dec '18

Hi, It's been a while since I've sent one of these -- mostly due to all the conferences and holidays which seemed to all happen at once. There's a lot going on: I'm working on support for dynamically adjusting node request priorities. Because of the current heavy load on OpenStack's Zuul system, this should help us get results faster on changes to Zuul. I also have changes in progress to add support for storing information about artifacts (but not the artifacts themselves) in the SQL database. That will make it easier for jobs to store artifacts in arbitrary locations and have later jobs use those artifacts. In particular, this is in service of the idea of a "promotion" pipeline. Finally, I'll be proposing a formal governance structure for the project this week -- something I've been drafting for a while. Paul has been adding support for executor zones -- a long-standing feature request. This allows admins to locate executors near to the node resources they use. Shrews added support for generic node attributes via config file to nodepool (mainly supporting the executor zone feature). Monty is continuing work on refactoring of nodepool TaskManager across openstacksdk and keystoneauth libraries, and is also working with tristanC towards supporting both statsd and prometheus for metrics collection. There is a new project "promstats" as part of this effort. Tobias has been working on zk caching in nodepool, and per project and tenant usage statistics, and a spec for a scale-out scheduler. Tristan has proposed a roadmap for Openshift support, started discussion about config introspection and worked on a web ui to handle generic API errors. Clint is working on producing reliable unit test(s) for AWS/EC2 driver. Thanks everyone! -Jim

1 0

Road to fully scale out Zuul
by Tobias.Henkel＠bmw.de 03 Dec '18

03 Dec '18

Hi Zuulers, currently the zuul-scheduler is the only component of zuul that is not scale out ready and thus a single point of failure. We always had the goal to make the scheduler scale out ready at some point in time. In order to make this more concrete so we can start to work on this I tried to transform my thoughts on this into a spec: https://review.openstack.org/621479 I'd be interested in what you think about this. Cheers, Tobias

1 0

Testing multiple commits in one job with Gerrit
by Brendan Jackman 28 Nov '18

28 Nov '18

Hi all, I'm gradually setting up Zuul for my org, and basically I have just one impedance mismatch between Zuul's behaviour and my requirements: Due to pressure on our test hardware I don't think running full end-to-end tests on every individual commit will scale. What I'd ultimately like is a check/gate setup where jobs in the check pipeline runs on every single commit, but jobs in the gate pipeline only run once for an entire "stack" of commits, and then that entire stack can be merged at once. IIUC this is basically the behaviour you get already with GitHub, where the zuul notion of "change" maps to a whole Pull Request rather than an individual commit. I started looking into ways something similar can be achieved with Gerrit. After some comments from #zuul and starting at the code a bit this strategy is the one I'm considering: enable one Zuul change to encapsulate multiple Gerrit changes. (Gerrit and Zuul both use the word "change", which works because Gerrit changes and Zuul changes are currently 1:1. That wouldn't be the caes any more with my proposal so I'll always say "Gerrit change" or "Zuul change"). I haven't figured out exactly how the Gerrit changes should be grouped together, and how this should be configured in the Gerrit trigger, but I think the basic idea would be to inject extra events into the Gerrit event stream (somewhere around GerritEventConnector._handleEvent). One example might be that we detect when an entire stack of Gerrit changes has received some common set of approvals, we inject an event called (say) "zuul-change-stack-approved". The user can then have a pipeline.trigger.gerrit entry with this event, and that trigger would create a Zuul change encapsulating the entire stack of Gerrit changes. These wouldn't need to actually be treated in the code like events coming from Gerrit, but this basic idea would allow expressing this behaviour with the existing structure of the Gerrit trigger configuration. I think it would then be a matter of finding the places where the Scheduler and pipeline Managers interact with the Gerrit driver using references to Zuul changes, and ensuring that those parts of the Gerrit driver can handling being passed the new class of Zuul change. I think most of these would involve checking for if the Zuul change is of the new type and then iterating over the constituent Gerrit changes to performe some action against each (e.g. GerritConnection.canMerge, GerritRefFilter.matches, and, with a little more complication, GerritReporter.report). Does that seem like a reasonable approach to the problem? Cheers, Brendan

3 6

Multi-repo merge
by Mike Pagel 27 Nov '18

27 Nov '18

Zuul is great in gating projects consisting of many repositories. It even supports dependencies of changes within a single or across multiple repositories. Our project is making heavy use of this. Every now and then we are facing dependencies that unfortunately are not unidirectional. I know, we should have better thought about clean repo structures in the first place, but one not too artificial example is a situation where repositories are structured to express ownership, visibility and write access. In Github for instance, all this is configured at the repository level. If you like see the sample how such a situation could arise. --- Start of sample --- Assume company A is the owner of repository A, holding operating system and a top-level application framework (the "main" function in C). Further assume company B is the owner of repository B, implementing a module on top of the operating system, and the module is invoked by A's top level application. At the build level the dependencies are clear and well defined, but at the repo level they are bidirectional. Some changes in A and B will depend on each other. It turns out that a clean, backward compatible PR design, leading to unidirectional changes as required by zuul is almost always possible, but not always easy. E.g. think of a change where A replaces a BSP in their operating system, which requires adaptation in B. It might not be possible to temporarily have two BSPs in A, the the change to A will break the build. Nor can B simply switch its usage of APIs, as that, by itself would break the build as well. The only chance to get this merged would be to temporarily disable some checks, allow for the first merge, then reenable and merge the second change. Ugly. --- End of sample --- It would very much simplify our workflows if an "atomic" merge (from zuul's point of view) would be possible. I can think of some ways to sort of realize this with zuul, but none of them is really safe, even if zuul is the only party executing merges to protected branches. What would be your suggestion to accommodate such a requirement? Thanks a lot for any insight and ideas you might have. Best, Mike

6 12

Zuul CI and Gitlab
by Алексей Степаненко 27 Nov '18

27 Nov '18

Hi. At the summit in Berlin was interested in how the Zuul CI system. GitHub support has been announced, but we're using GitLab. Will there be added support for Networks in the future ? -- Aleksei Stepanenko, Selectel Team

1 0

Semaphores and resources
by Tobias.Henkel＠bmw.de 26 Nov '18

26 Nov '18

Hi Zuulers, We just discovered an (in our case) unpleasant behaviour of zuul and semaphores. Scenario: We have a job ‘foo’ with a semaphore of 5. Now the queue grows and we have 30 items with job ‘foo’ in the queue. The current behaviour of zuul is that it first gets the nodes and then locks the semaphore. This means that 5 jobs run and 25 wait for the semaphore but already got and locked their nodes. In case the job is expensive and long running this can become quite a big resource waste problem if we’re in a quota constrained situation. After some discussion on IRC this would be a proposal to solve this problem: We could make the order of ‘request nodes’ and ‘lock semaphore’ configurable per job with a default to the safe behaviour of ‘lock semaphore’ then ‘request nodes’. Jobs would still have the possibility to do the fast path by requesting the nodes first. This makes sense if you have lightweight jobs that should run in a fast consecutive sequence and you know that some waste of resources is ok. What do the other semaphore users out there think about this? Kind regards Tobias

5 7

Dequeuing buildset on first error
by Tobias.Henkel＠bmw.de 19 Nov '18

19 Nov '18

Hi Zuulers, We have several projects that have buildsets with fast (few minutes, e.g. linters) and longer running (> 40 minutes, e.g. big fat c++ compilation) jobs. We often see the case that one of the linter jobs failed and already could give feedback. However zuul waits for the completion of the fat compile job before reporting. Generally this is the desired behaviour because we want to give as much feedback as possible. However some projects may want to decide that quick feedback is more important than the most complete feedback possible. So I would propose to add a ‘report-on-first-failure’ flag to the project-pipeline. The default would be false in order to retain the current behaviour. If the deque happens the aborted jobs will have the ABORTED state so it can be clearly seen in the reporting that these were aborted. Enabling this for certain projects also could help to reduce resource consumption in quota-constrained periods. What do you think about this? Kind regards Tobias

7 6

Slides from Tungsten talk on Berlin Summit
by Jarek Łukow 16 Nov '18

16 Nov '18

Hi all, As promised, I'm sending the link to our slide deck: https://wiki.tungsten.io/download/attachments/1409104/Zuul_as_a_build_syste… Regards, -- Jarosław Łukow DevOps Engineer jaroslaw.lukow(a)codilime.com -- ------------------------------- The information in this email is confidential and may be legally privileged, it may contain information that is confidential in CodiLime Sp. z o. o. It is intended solely for the addressee. Any access to this email by third parties is unauthorized. If you are not the intended recipient of this message, any disclosure, copying, distribution or any action undertaken or neglected in reliance thereon is prohibited and may result in your liability for damages.

2 1

OpenDev, the future of OpenStack Infra
by Clark Boylan 09 Nov '18

09 Nov '18

Hello everyone, Sorry for another cross post so soon. In the land before time we had Stackforge. Stackforge gave non-OpenStack projects a place to live with their own clearly defined "not OpenStack" namespacing. As the wheel of time spun we realized that many Stackforge projects were becoming OpenStack projects and we would have to migrate them. This involved Gerrit downtimes to rename things safely. To ease the pain of this, the TC decided that all projects developed in the OpenStack Infrastructure could live under the OpenStack git namespace to simplify migrations. Unfortunately this had the effect of creating confusion over which projects were officially a part of OpenStack, and whether or not projects that were not OpenStack could use our project hosting. Stackforge lived on under a different name, "unofficial project hosting", but many potential infrastructure users either didn't understand this or didn't want that strong association to OpenStack for their project hosting [0]. Turns out that we want to be able to host OpenStack and non-OpenStack projects together without confusion in a way that makes all of the projects involved happy. In an effort to make this a reality the OpenStack Infra team has been working through a process to rename itself to make it clear that our awesome project infrastructure and open collaboration tooling is community run, not just for OpenStack, but for others that want to be involved. To this end we've acquired the opendev.org domain which will allow us to host services under a neutral name as the OpenDev Infrastructure team. The OpenStack community will continue to be the largest and a primary user for the OpenDev Infrastructure team, but our hope in making our infrastructure services more inclusive is that we'll also attract new contributors, which will ultimately benefit OpenStack and other open infrastructure projects. Our goals for OpenDev are to: * Encourage additional active infrastructure contributors to help us scale. Make it clear that this is community-run tooling & infrastructure and everyone can get involved. * Make open source collaboration tools and project infrastructure more accessible to those that want it. * Have exposure to and dogfooding of OpenStack clouds as viable open source cloud providers. * Enable more projects to take advantage of the OpenStack-pioneered model of development and collaboration, including recommended practices like code review and gating. * Help build relationships with new and adjacent open source projects and create an inclusive space for collaboration and open source development. Much of this is still in the early planning stages. This is the infrastructure team's current thinking on the subject, but understand we have an existing community from which we'd like to see buy-in and involvement. To that end we have tried to compile a list of expected FAQ/Q&A information below, but feel free to followup either on this thread or with myself for anything we haven't considered already. Any transition will be slow and considered so don't expect everything to change overnight. But don't be surprised if you run into some new HTTP redirects as we reorganize the names under which services run. We'll also be sure to keep you informed on any major (and probably minor) transition steps so that they won't surprise you. Thank you, Clark [0] It should be noted that some projects did not mind this and hosted with OpenStack Infra anyway. ARA is an excellent example of this. FAQ * What is OpenDev? OpenDev is community-run tools and infrastructure services for collaboratively developing open source software. The OpenDev infrastruture team is the community of people who operate the infrastructure under the umbrella of the OpenStack Foundation. * What services are you offering? What is the expected timeline? In the near-term we expect to transition simple services like etherpad hosting to the OpenDev domain. It wil take us months and potentially up to a year to transition key infrastructure pieces like Git and Gerrit. Example services managed by the team today include etherpad, wiki, the zuul and nodepool CI system, git and gerrit, and other minor systems like pbx conferencing and survey tools. * Where will these services live? We've acquired opendev.org and are planning to set up DNS hosting very soon. We will post a simple information page and FAQ on the website and build it out as necessary over time. * Why are you changing from the OpenStack infrastructure team to the OpenDev infrastructure team? In the same way we want to signal that our services are not strictly for OpenStack projects, and that not every project using our services is an official part of OpenStack, we want to make it clear that our team also serves this larger community. * Who should use OpenDev services? Does it have to be projects related to OpenStack, or any open source projects? In short, open source contributors who share our community values, especially those who might want to help contribute to improving and maintaining OpenDev infrastructure over time. Projects using OpenDev hosted git and gerrit services should have an OSI-approved license. * Will the OpenStack projects live at git.opendev.org? All projects hosted with OpenDev will live at git.opendev.org. For backwards compatibility reasons, at the very least git.openstack.org will be an alias for git.opendev.org for the forseeable future. The same is true of the other existing whitelabel git domains such as git.starlingx.io and git.zuul-ci.org. Whether or not other 'whitelabel' domains are created is an open question. Given a neutral domain name, the desire for such sites may not seem as necessary. * Does this mean the infrastruture team will be spending less time on OpenStack? OpenStack will continue to be the largest and a primary user for the OpenDev Infrastructure team, and we expect that our work will benefit all users. There will be additional effort required as we transition to the new namespace and reorganize, but over the long term we hope this inclusive approach will help us attract new contributors and ultimately benefit OpenStack. * Are OpenStack cloud test resources the only resources that will be used? At the present time all of the donated resources come to us from a combination of OpenStack Public and Private clouds. Nobody from any of the proprietary clouds has asked to donate resources to us. It is conceivable that the shift to OpenDev could open the door to those cloud providers wanting to donate some cloud resources. Assuming nodepool supports talking to those clouds, it is certainly a possibility, but at the moment it's all speculation. * Is this name associated with the OpenDev Conferences (opendevconf.com) that OpenStack Foundation has previously organized? Yes! They are related, albeit indirectly. To quote from the conference's promotional site, "the focus is on bringing together composable open infrastructure technologies across communities and industries." The possibility of cross-promotional tie-ins could prove synergistic, since the conference and the collaboratory we're building share a lot of similar values and goals, and are ultimately supported by the same donors, community and foundation. * How will OpenDev be governed? Will the OpenStack TC retain oversight over it? The OpenDev governance discussion is just getting started, but like all OSF-supported initiatives, OpenDev follows the Four Opens, so it will ultimately be directly governed by OpenDev contributors. While it won't be under the sole oversight of the OpenStack Technical Committee anymore, OpenDev users (in particular OpenStack) should be represented in the governance model so that they can feed back their requirements to the OpenDev team.

1 0

Community Infrastructure Berlin Summit Onboarding Session
by Clark Boylan 05 Nov '18

05 Nov '18

Hello everyone, My apologies for cross posting but wanted to make sure the various developer groups saw this. Rather than use the Infrastructure Onboarding session in Berlin [0] for infrastructure sysadmin/developer onboarding, I thought we could use the time for user onboarding. We've got quite a few new groups interacting with us recently, and it would probably be useful to have a session on what we do, how people can take advantage of this, and so on. I've been brainstorming ideas on this etherpad [1]. If you think you'll attend the session and find any of these subjects to be useful please +1 them. Also feel free to add additional topics. I expect this will be an informal session that directly targets the interests of those attending. Please do drop by if you have any interest in using this infrastructure at all. This is your chance to better understand Zuul job configuration, the test environments themselves, the metrics and data we collect, and basically anything else related to the community developer infrastructure. [0] https://www.openstack.org/summit/berlin-2018/summit-schedule/events/22950/i… [1] https://etherpad.openstack.org/p/openstack-infra-berlin-onboarding Hope to see you there, Clark

1 0

Node request priorities
by corvus＠inaugust.com 31 Oct '18

31 Oct '18

Hi, In OpenStack, we occasionally have periods where, for various reasons, one project, or a group of projects, uses a sufficiently large number of node resources that it significantly delays results for other projects. We happen to be in one of those periods, so I've had some time to think about ways to mitigate this. First, let me say that one of the potential causes is long and unreliable tests. There are efforts underway to communicate that and hopefully address it. But that topic isn't really for this list. Our concern is, when a system gets into this state, can or should we do anything in Zuul to mitigate it? I would like to avoid (for now) the use of project-based quotas. I could certainly see us adopting tenant quotas in the future, but that doesn't help us in this case where all of the contention is within a single tenant. Within a tenant, project quotas just seem to be too much work. I wouldn't even begin to know how to set them fairly. And what we really want to do is use as many node resources are available, but use them in a fair manner. I think we can accomplish this by augmenting our use of node request priority. Currently, we use a priority queue with very coarse granularity (high, normal, low) set by Zuul according to pipeline. I'd like to adjust the system so that each subsequent node request from a given project has a lower priority. In other words, at any given time, the first change from project A in gate has the highest priority, and the same is true for project B. The second change from A has a slightly lower priority, and so forth. The problem with this is that the node request priorities much change over time as the contents of the pipeline queues change. Once the first change from A merges, the second change from A should now have the highest priority. That makes this incompatible with our current priority queue system. Fortunately, Tobias has been working on adding caching to Nodepool's use of Zookeeper. If we start having node requests cached in-memory in the launchers, the in-memory records can be automatically updated by Zookeeper callbacks. This means that we can encode our additional priority information on an attribute in the Znode, updating it as necessary, and combine it with the already existing information from the priority queue to determine the order in which to fulfill requests. To summarize, I propose the following: * The current priority queue system for node requests will remain in place. I'll call that "precedence" to distinguish it from the new bits. * Add a "priority" field to the node request znodes in Zookeeper. This is an integer which indicates the priority of this request relative to others at the same pipeline precedence level. * Zuul will update the priority field after each time a pipeline queue processor runs, according to internal heuristics. I propose that initially we decrement the priority for each subsequent change from the same project, though we may also add an option to do so by project-group, as determined by membership in a shared change queue. * We add temporary feature flags to Zuul so that we can try out these new features (including different options for how to determine the priority) before committing to future support. Whatever we decide to keep, we may also want to make optional, so these flags may eventually turn into permanent configuration flags. * We implement a TreeCache for node requests in nodepool. * When nodepool begins to iterate over all node requests in order to handle them, it first sorts by precedence, then priority. Items with the same precedence and priority will remain in their chronological order of submission. I think that will help make node distribution more fair, and it builds on top of the current system without changing it so it should be forward/backward compatible, and even optional. -Jim

5 8

Discussion between OSF projects in Berlin
by Thierry Carrez 31 Oct '18

31 Oct '18

Hi! For those who will be in Berlin for the Summit, please mark your calendars for the following Forum session: https://www.openstack.org/summit/berlin-2018/summit-schedule/events/22815/c… This session is an opportunity for the technical leadership of all OSF projects to meet, discuss pain points, compare experiences and explore synergies. We expect to have members of the OpenStack, Kata Containers, StarlingX, Airship, and Zuul communities around. It will be the occasion, among other things, to push for Zuul adoption all around! If you can make it, please join us ! -- Thierry Carrez (ttx)

1 0

Announcing the First Release of StarlingX, an open source edge computing platform
by Ildiko Vancsa 25 Oct '18

25 Oct '18

Hi, We would like to draw your attention to StarlingX[1], a new independent, top-level, open source pilot project that's supported by the OpenStack Foundation. StarlingX joins other pilot projects hosted at OpenStack Foundation[2], including Airship, Kata Containers and Zuul. Today the first release of StarlingX is here! We invite you to participate in getting the word out that the release is ready and that we’re eager to welcome more contributors to this project. Learn more about it: • Read more about the project at starlingx.io • Listen to a recording of the onboarding webinar[3] • On-boarding slide deck[4] • Overview document[5] Some things you can share: • A blog on starlingx.io[6] • Social sharing: Announcements on Twitter[7] Want to get involved in the community? • Mailing Lists[8] • Weekly Calls[9] • Freenode IRC: #starlingx channel[10] Ready to dive into the code? • You can get download the first release at git.starlingx.io • StarlingX Install Guide[11] • StarlingX Developer Guide[12] If you’re at the Berlin Summit November 13-15[13]: Tuesday 11/13 • StarlingX – Project update – 6 months in the life of a new Open Source project with Brent Rowsell & Dean Troyer • StarlingX CI, from zero to Zuul with Hazzim Anaya & Elio Martinez Wednesday 11/14 • Keynote spotlight on the main stage with Ian Jolliffe & Dean Troyer • MVP (Minimum Viable Product) architecture for edge - Forum session • "Ask Me Anything" about StarlingX - Forum session • StarlingX Enhancements for Edge Networking presentation with Kailun Qin, Ruijing Guo & Dan Chen • Project Onboarding session with Greg Waines • Integrating IOT Device Management with the Edge Cloud - Forum session Thursday 11/15 • Containerized Applications' Requirements on Kubernetes Cluster at the Edge - Forum session Check out the materials to learn about the project, try out the software and join the community! We hope to see many of you in Berlin! Ildikó [1] https://www.starlingx.io/ [2] https://www.openstack.org/foundation/ [3] https://www.youtube.com/watch?v=G9uwGnKD6tM&t=232s [4] https://www.starlingx.io/collateral/StarlingX-Onboarding-Deck-Web.pdf [5] https://www.starlingx.io/collateral/StarlingX_OnePager_Web-102318pdf/ [6] https://www.starlingx.io/blog/starlingx-initial-release.html [7] https://twitter.com/starlingx [8] http://lists.starlingx.io/cgi-bin/mailman/listinfo [9] https://wiki.openstack.org/wiki/Starlingx/Meetings [10] https://freenode.net/ [11] https://docs.starlingx.io/installation_guide/index.html [12] https://docs.starlingx.io/developer_guide/index.html [13] https://www.openstack.org/summit/berlin-2018

1 0

Zuul webinar recording
by Jarek Łukow 17 Oct '18

17 Oct '18

Hi, I want to let you know that on Tuesday, me and my friend Lukas conducted an introductory webinar on Zuul. You can find the recording here: https://crowdcast.io/e/zuul (There will be also an upload without the registration wall, once our media team cuts the video a bit for publishing on YouTube). After a short presentation we've made a tutorial/demo that demonstrated integration with GitHub, Zuul status UI, multiproject jobs and "Depends-On" functionality between two projects on GH. >From the technical side, we prepared an all-in-one Zuul install (scheduler, executor, merger, web, Gearman, Zookeeper, nodepool-launcher and Apache for logs) using Docker compose, very similar to the one from https://review.openstack.org/#/c/608344/. Then we connected nodepool to a Vexxhost project and created an GitHub app attached to three repos: config project and two untrusted projects for the demo. The demo includes also the new refreshed UI - which we've built right before going live, with fingers crossed, but it went smoothly ;). We also have the tutorial scenario available here: https://jluk-codi.github.io/zuul-webinar/. Our small Zuul setup serving pull requests for the sample GitHub project will be available through the weekend (we’ll remove it Monday CEST when we get to work). This was our first webinar and a kind of a "test drive", but I expect we'll have more opportunities to help bring some attention to Zuul. Best, Jarek -- ------------------------------- The information in this email is confidential and may be legally privileged, it may contain information that is confidential in CodiLime Sp. z o. o. It is intended solely for the addressee. Any access to this email by third parties is unauthorized. If you are not the intended recipient of this message, any disclosure, copying, distribution or any action undertaken or neglected in reliance thereon is prohibited and may result in your liability for damages.

3 3

Oct 24 OSF Community Mtg: StarlingX First Release
by Claire Massey 15 Oct '18

15 Oct '18

Hey Zuul team, For the upcoming OpenStack Foundation Community Meeting[1] we will focus on the StarlingX project and edge computing. Please join us on *October 24 at 8:00am PDT* for a StarlingX community on-boarding webinar that's scheduled to coincide with the first release for the StarlingX project. Ian Jolliffe and Bruce Jones will lead the project on-boarding presentation followed by Q&A. Please mark your calendars and plan to join us. Meeting link: https://zoom.us/j/112003649. More information and news around the StarlingX first release will be shared on October 24. In the meantime you can learn more and get involved with StarlingX though theses channels: • Website: starlingx.io • Git: git.starlingx.io • Docs: docs.starlingx.io • Freenode IRC: #starlingx • Mailing Lists: lists.starlingx.io Thanks, Claire [1] https://etherpad.openstack.org/p/openstack-community-meeting <https://etherpad.openstack.org/p/openstack-community-meeting>

1 0

Zuul Project Update 2018-10-08
by corvus＠inaugust.com 15 Oct '18

15 Oct '18

Hi, We've had a number of events that were nearly back-to-back, so it's been a while since I've sent one of these. I think it's a good time to catch up. Several of us just returned from AnsibleFest where the Zuul project had a community sponsor booth, and Clint, Paul, and Ricardo gave presentations. Our presence at the event was an enormous success, exceeding everyone's expectations. Ashlee and Claire from the Foundation set up the booth and Monty, Clark, Paul, Clint and I spent as much time with them there as possible. We spent the entire two days of the event talking to people -- our voices were hoarse by the end. When we find the right audience, there's a huge amount of interest in Zuul. Unfortunately, a lot of those people are about to go try out Zuul and follow our setup instructions, which we all know aren't complete. There was a lot of interest (between potential users and the Zuul developers at AnsibleFest) in having a container-based quickstart. This is also something we've discussed in the past. The idea is to bootstrap a complete Zuul system in a sandbox environment with one command, and walk users through the process of setting up initial jobs. I think all of the pieces are in place for that now, so I've begun working on that. That's my highest priority for this week -- I'd like to finish that and land it ASAP so that any potential new users find that and have a good experience. I have a WIP patch at https://review.openstack.org/608344 and would appreciate it if folks would start looking at it and trying it out. Other news from AnsibleFest: Jesse is finishing up work on adding support for the GitHub checks api to github3.py; once that is complete, we should be able to support it in Zuul. We met a large number of folks who are interested in GitLab, one of whom has volunteered to work on implementing support for it. He's still working on getting up to speed on Zuul development, but hopefully will be joining us soon. I'm sure I'm forgetting other very important things from AnsibleFest, but other folks were there and can chime in. :) Monty and I found (via reports from new users at AnsibleFest and my work on the container quickstart) that we were not including the built javascript in either our pypi uploads or our container images. I believe Monty has either fixed those, or has patches in progress to fix them. We're ready to land the patch to convert the dashboard to react; I expect us to do that soon now that more of us are around again. David and Tristan are working on testing for the Nodepool kubernetes and OpenShift patches. The patches themselves look good; once we have testing in place I think we'll be comfortable merging them. David and Clint have been working on the ec2 driver for Nodepool. Once it is a little more complete and has some testing (likely something fake or mock-based) we can merge it. There was, as you might expect, some interest in that at AnsibleFest as well. Matthieu has a spec and code up regarding using JWT for authenticated actions in the web interface. I'm hoping that after we clear the quickstart docs and container drivers from our plate, we'll have time to take that up in earnest. I'm sure I'm missing things, but this is pretty long already. Feel free to reply with more. I plan to send out updates regularly again, so if you have a moment to update https://etherpad.openstack.org/p/zuul-update-email please do so. -Jim

10 13

Toggle gzip in upload-logs
by Paul Belanger 13 Oct '18

13 Oct '18

Greetings, I've proposed a patch to upload-logs[1] that will toggle the compressing of our console logs files. The main change is, by default we no longer compress them. Jobs would need to set zuul_log_compress: true to work as we do today. The main reason for this, is to reduce the amount of extra work a user needs to view our console logs in apache / nginx. Today, if compressed, the browser will try to download the file not render it. I think we could add documentation explaining how to enable compression and which settings would be needed for apache / nginx but in this case, seems like an optimization over necessity. Thoughts? [1] https://review.openstack.org/567696/

3 4

OpenStack Summit Forum Submission Process Extended
by Jimmy McArthur 28 Sep '18

28 Sep '18

Hello Everyone We are extended the Forum Submission process through September 30, 11:59pm Pacific (6:59am GMT). We've already gotten a ton of great submissions, but we want to leave the door open through the weekend in case we have any stragglers. Please submit your topics here: https://www.openstack.org/summit/berlin-2018/call-for-presentations If you'd like to review the submissions to date, you can go to https://www.openstack.org/summit/berlin-2018/vote-for-speakers. There is no voting period, this is just so Forum attendees can review the submissions to date. Thank you! Jimmy

1 0

Spec for a tenant-scoped admin web API
by Matthieu Huin 27 Sep '18

27 Sep '18

Hello, Following Jim's example with the containers spec, I've opened a spec review in the zuul repository: https://review.openstack.org/562321 I've been toying around with adding a few "admin" endpoints to zuul and nodepool for some time now [1][2]. Using Software Factory's authorization engine to limit access to these endpoints to specific users, and consuming germqtt's event feed to react to comments on a review, I demonstrated PoCs for features that I think would be of interest to Zuul users in general: * Ability to re-enqueue builds from zuul's web UI * Ability to trigger a autohold by commenting "autohold <job> on <tenant>" on gerrit review The aforementioned patches were meant as experiments; now I would like to go further and work to have this integrated into Zuul in a way that would be tenant-scoped. Before I rework these extensively I seek your guidance and opinions on how to proceed with this; hence this spec review. There are surely things I have overlooked, and my proposal to use groups to implement policies might need discussing. Please let me know what you think, also if it is the right place for the spec (the review on infra-specs is still open) MHU [1] https://review.openstack.org/536319 [2] https://review.openstack.org/539004

1 1

Feedback on Zuul Overview One Pager
by Claire Massey 26 Sep '18

26 Sep '18

Hi Zuul team, We'll have a Zuul branded booth at AnsibleFest in Austin, October 2-3. More information about our booth presence and how you can participate will be shared in the coming weeks. For now we’re working on creating all of the branded materials needed for the booth, including a one page piece of collateral that gives a nice overview of Zuul. A working draft of the one pager is available here: https://etherpad.openstack.org/p/Zuul_one_pager <https://etherpad.openstack.org/p/Zuul_one_pager>. If you have any feedback or edits on the copy, please provide it in the etherpad by next Monday, August 20. The completed document will end up looking polished like the Kata Containers overview one pager shown at https://katacontainers.io/media/uploads/katacontainers/uploads/katacontaine… <https://katacontainers.io/media/uploads/katacontainers/uploads/katacontaine…>. Thanks! Claire

7 12