From iwienand at redhat.com Wed Dec 2 11:14:23 2020 From: iwienand at redhat.com (Ian Wienand) Date: Wed, 2 Dec 2020 22:14:23 +1100 Subject: [all][infra] CI test result table in the new gerrit review UI In-Reply-To: <87lfemubxy.tristanC@fedora> References: <20201126143706.ejhc5u2qhtzg2qnr@yuggoth.org> <87tutctar2.tristanC@fedora> <20201126151906.2573xhh35dhblmnh@yuggoth.org> <716a66de7a2cb45ded010d01f6c344ba74dbd7de.camel@redhat.com> <87r1ogt49n.tristanC@fedora> <20201126174517.akyuykwmlwc2z6ei@yuggoth.org> <87o8jkt2ed.tristanC@fedora> <20201127032004.GC522326@fedora19.localdomain> <87lfemubxy.tristanC@fedora> Message-ID: <20201202111423.GA819047@fedora19.localdomain> On Fri, Nov 27, 2020 at 02:11:21PM +0000, Tristan Cacqueray wrote: > From what I understand, we can either use java and the polymer template > system, or the javascript api to implement the zuul results table. This was discussed today in the infra meeting @ [1] There was a couple of broad conclusions. There's no dispute that it works, for now. The maintenace of this in perpituity is the major concern. Some points: There are already two Zuul plugins in the upstream repos zuul : https://gerrit.googlesource.com/plugins/zuul -- This is rather badly namespaced, and is focused on showing the status of Depends-On: changes a cyclic dependencies zuul-status : https://gerrit.googlesource.com/plugins/zuul-status/ -- This is also using a broad namespace, and appears to be related to showing the ongoing job status in the UI directly, but does not show final results. An example I found at [2] from an old wikimedia thread Upstream have shown to take some interest in these plugins when undertaking upgrades to polymer, etc. and thus integrating there is seen as the best practice. Unless we have a strong reason not to, we would like to consume any plugins from upstream for this reason. It would be in our best interest to work with these existing plugins to clarify what they do more directly and come up with some better namespacing to make it more obvious as potential number of plugins grows. The current proposed implementation does not look like any of the upstream plugins. Admittedly, [3] shows this is not a strong eco-system; the documentation is mostly TODO and examples are thin. Implementing this with Reason [4] adds one more significant hurdle; we have no track record of maintaing this javascript dialect and it is not used upstream in any way. It also doesn't use the same build and test frameworks; this may not be a blocker but again creates hurdles by being different to upstream. One more concrete comment is that I'm pretty sure walking the shadow DOM to update things is [5] not quite as intended and we should be using endpoints. In trying to find the best place to start, I've pulled out bits of the checks plugin, image-diff plugin and codeeditor plugin into the simplest thing I could get to a tab with a table in it @ [6]. You can see this for now @ [7]. I think we're going to get more potential eyes using polymer components and making things look like the existing plugins. It's a matter of taste but I think the blank canvas of a separate tab serves the purpose well. I think with the Bazel build bits, we can ultimately write an upstream Zuul job to build this for us in gerrit's Zuul (corvus might know if that's right?) Tristan -- maybe as step one, we could try integrating what you have into that framework? Maybe we rewrite the ReasonML ... but having it something is step 1. Overall I think this is the broad strokes of what we would look at sending upstream. If they think it's too specific, Zuul tenant seems a logical home. We would also like to expand the gate testing to better handle testing plugins. This will involve us automatically adding a repo, sample change, and Zuul user comments during the gate test. A logical extension of this would be to take samples using a headless browser for reporting as artifacts; held nodes can also be used for advanced debugging. This will give us better confidence as we keep our Gerrit up-to-date. I'd like to solict some feedback on the checks plugin/API, which Zuul added support for with [8]. My understanding is that this has never really consolidated upstream and is under redevelopment [9]. I don't think there's much there for us at this point; even still seeing as we are so Zuul centric we might be able to do things with a specific plugin this API will never do. -i [1] http://eavesdrop.openstack.org/meetings/infra/2020/infra.2020-12-01-19.01.log.html#l-62 [2] https://imgur.com/a/uBk2oxQ [3] https://gerrit-review.googlesource.com/Documentation/pg-plugin-dev.html [4] https://reasonml.github.io/ [5] https://github.com/softwarefactory-project/zuul-results-gerrit-plugin/blob/master/src/ZuulResultsPlugin.re#L25 [6] https://github.com/ianw/gerrit-zuul-summary-status [7] https://104.130.172.52/c/openstack/diskimage-builder/+/554002 [8] https://opendev.org/zuul/zuul/commit/e78e948284392477d385d493fc9ec194d544483f [9] https://www.gerritcodereview.com/design-docs/ci-reboot.html From tdecacqu at redhat.com Wed Dec 2 19:15:33 2020 From: tdecacqu at redhat.com (Tristan Cacqueray) Date: Wed, 02 Dec 2020 19:15:33 +0000 Subject: [all][infra] CI test result table in the new gerrit review UI In-Reply-To: <20201202111423.GA819047@fedora19.localdomain> References: <20201126143706.ejhc5u2qhtzg2qnr@yuggoth.org> <87tutctar2.tristanC@fedora> <20201126151906.2573xhh35dhblmnh@yuggoth.org> <716a66de7a2cb45ded010d01f6c344ba74dbd7de.camel@redhat.com> <87r1ogt49n.tristanC@fedora> <20201126174517.akyuykwmlwc2z6ei@yuggoth.org> <87o8jkt2ed.tristanC@fedora> <20201127032004.GC522326@fedora19.localdomain> <87lfemubxy.tristanC@fedora> <20201202111423.GA819047@fedora19.localdomain> Message-ID: <87a6uwt3xm.tristanC@fedora> On Wed, Dec 02, 2020 at 22:14 Ian Wienand wrote: > On Fri, Nov 27, 2020 at 02:11:21PM +0000, Tristan Cacqueray wrote: >> From what I understand, we can either use java and the polymer template >> system, or the javascript api to implement the zuul results table. > > This was discussed today in the infra meeting @ [1] > > There was a couple of broad conclusions. There's no dispute that it > works, for now. The maintenace of this in perpituity is the major > concern. > > Some points: > > There are already two Zuul plugins in the upstream repos > > zuul : https://gerrit.googlesource.com/plugins/zuul -- > This is rather badly namespaced, and is focused on showing the > status of Depends-On: changes a cyclic dependencies > > zuul-status : https://gerrit.googlesource.com/plugins/zuul-status/ -- > This is also using a broad namespace, and appears to be related to > showing the ongoing job status in the UI directly, but does not show > final results. An example I found at [2] from an old wikimedia > thread > Note that my proposal goal is just to restore the missing table from the new review.opendev.org user interface. I agree that it would be better to follow how upstream does plugins. > Upstream have shown to take some interest in these plugins when > undertaking upgrades to polymer, etc. and thus integrating there is > seen as the best practice. Unless we have a strong reason not to, we > would like to consume any plugins from upstream for this reason. > > It would be in our best interest to work with these existing plugins > to clarify what they do more directly and come up with some better > namespacing to make it more obvious as potential number of plugins > grows. > > The current proposed implementation does not look like any of the > upstream plugins. Admittedly, [3] shows this is not a strong > eco-system; the documentation is mostly TODO and examples are thin. > Implementing this with Reason [4] adds one more significant hurdle; we > have no track record of maintaing this javascript dialect and it is > not used upstream in any way. It also doesn't use the same build and > test frameworks; this may not be a blocker but again creates hurdles > by being different to upstream. One more concrete comment is that I'm > pretty sure walking the shadow DOM to update things is [5] not quite > as intended and we should be using endpoints. > > In trying to find the best place to start, I've pulled out bits of the > checks plugin, image-diff plugin and codeeditor plugin into the > simplest thing I could get to a tab with a table in it @ [6]. You can > see this for now @ [7]. I think we're going to get more potential > eyes using polymer components and making things look like the existing > plugins. It's a matter of taste but I think the blank canvas of a > separate tab serves the purpose well. I think with the Bazel build > bits, we can ultimately write an upstream Zuul job to build this for > us in gerrit's Zuul (corvus might know if that's right?) > > Tristan -- maybe as step one, we could try integrating what you have > into that framework? Maybe we rewrite the ReasonML ... but having it > something is step 1. Overall I think this is the broad strokes of > what we would look at sending upstream. If they think it's too > specific, Zuul tenant seems a logical home. > This is looking great, thank you for looking into that. I guess we can directly re-use the showResult function in-place of the gr-zuul-summary-status-view_html.js content. If using polymer template is required, then we can still re-use the the `CI.fromMessages` function exported by the re-gerrit library to count the rechecks and extract build results from the message array. > We would also like to expand the gate testing to better handle testing > plugins. This will involve us automatically adding a repo, sample > change, and Zuul user comments during the gate test. A logical > extension of this would be to take samples using a headless browser > for reporting as artifacts; held nodes can also be used for advanced > debugging. This will give us better confidence as we keep our Gerrit > up-to-date. > > I'd like to solict some feedback on the checks plugin/API, which Zuul > added support for with [8]. My understanding is that this has never > really consolidated upstream and is under redevelopment [9]. I don't > think there's much there for us at this point; even still seeing as we > are so Zuul centric we might be able to do things with a specific > plugin this API will never do. > It seems like the checks API enables a similar user interface where the CI build details are integrated in the tablist along with a chip-view under the commit message. It's unclear what are the requirements but that looks like a good long term solution since we wouldn't need a custom plugin for Zuul. -Tristan > > [1] http://eavesdrop.openstack.org/meetings/infra/2020/infra.2020-12-01-19.01.log.html#l-62 > [2] https://imgur.com/a/uBk2oxQ > [3] https://gerrit-review.googlesource.com/Documentation/pg-plugin-dev.html > [4] https://reasonml.github.io/ > [5] https://github.com/softwarefactory-project/zuul-results-gerrit-plugin/blob/master/src/ZuulResultsPlugin.re#L25 > [6] https://github.com/ianw/gerrit-zuul-summary-status > [7] https://104.130.172.52/c/openstack/diskimage-builder/+/554002 > [8] https://opendev.org/zuul/zuul/commit/e78e948284392477d385d493fc9ec194d544483f > [9] https://www.gerritcodereview.com/design-docs/ci-reboot.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 515 bytes Desc: not available URL: From radoslaw.piliszek at gmail.com Thu Dec 3 09:22:29 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Thu, 3 Dec 2020 10:22:29 +0100 Subject: [all] Dynamic Zuul results table in Gerrit 3 Message-ID: Hello Fellow OpenStack and OpenDev Folks! TL;DR click on [3] and enjoy. I am starting this thread to not hijack the discussion happening on [1]. First of all, I would like to thank gibi (Balazs Gibizer) for hacking a way to get the place to render the table in the first place (pun intended). I have been a long-time-now user of [2]. I have improved and customised it for myself but never really got to share back the changes I made. The new Gerrit obviously broke the whole script so it was of no use to share at that particular state. However, inspired by gibi's work, I decided to finally sit down and fix it to work with Gerrit 3 and here it comes: [3]. Works well on Chrome with Tampermonkey. Not tested others. I hope you will enjoy this little helper (I do). I know the script looks super fugly but it generally boils down to a mix of styles of 3 people and Gerrit having funky UI rendering. Finally, I'd also like to thank hrw (Marcin Juszkiewicz) for linking me to the original Michel's script in 2019. [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/019051.html [2] https://opendev.org/x/coats/src/commit/444c95738677593dcfed0cfd9667d4c4f0d596a3/coats/openstack_gerrit_zuul_status.user.js [3] https://gist.github.com/yoctozepto/7ea1271c299d143388b7c1b1802ee75e Kind regards, -yoctozepto From marcin.juszkiewicz at linaro.org Thu Dec 3 09:33:01 2020 From: marcin.juszkiewicz at linaro.org (Marcin Juszkiewicz) Date: Thu, 3 Dec 2020 10:33:01 +0100 Subject: [all] Dynamic Zuul results table in Gerrit 3 In-Reply-To: References: Message-ID: W dniu 03.12.2020 o 10:22, Radosław Piliszek pisze: > Hello Fellow OpenStack and OpenDev Folks! > > TL;DR click on [3] and enjoy. Rename to *.user.js so Tampermonkey will recognize it as userscript and add window around with Install button. > However, inspired by gibi's work, I decided to finally sit down and > fix it to work with Gerrit 3 and here it comes: [3]. > Works well on Chrome with Tampermonkey. Not tested others. Firefox with Tampermonkey - script works. > I hope you will enjoy this little helper (I do). Nice job! It does not catch all Zuul jobs. For Kolla patches AArch64 jobs are not listed. > I know the script looks super fugly but it generally boils down to a > mix of styles of 3 people and Gerrit having funky UI rendering. > Finally, I'd also like to thank hrw (Marcin Juszkiewicz) for linking > me to the original Michel's script in 2019. YW. From radoslaw.piliszek at gmail.com Thu Dec 3 09:46:37 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Thu, 3 Dec 2020 10:46:37 +0100 Subject: [all] Dynamic Zuul results table in Gerrit 3 In-Reply-To: References: Message-ID: On Thu, Dec 3, 2020 at 10:33 AM Marcin Juszkiewicz wrote: > > W dniu 03.12.2020 o 10:22, Radosław Piliszek pisze: > > Hello Fellow OpenStack and OpenDev Folks! > > > > TL;DR click on [3] and enjoy. > > Rename to *.user.js so Tampermonkey will recognize it as userscript and > add window around with Install button. > I, personally, hate this "feature" of Tampermonkey but sure - if it makes it easier for you. ;-) Done. Anyone - just click on "Raw" and you will be pestered by Tampermonkey to install the script. > > However, inspired by gibi's work, I decided to finally sit down and > > fix it to work with Gerrit 3 and here it comes: [3]. > > Works well on Chrome with Tampermonkey. Not tested others. > > Firefox with Tampermonkey - script works. > Yay, thanks for confirming! > > I hope you will enjoy this little helper (I do). > > Nice job! > Well, thank you! :-) > It does not catch all Zuul jobs. For Kolla patches AArch64 jobs are not > listed. I have already replied on IRC but will repeat here for others. The dynamic part includes all the running queues. However, the static part parses only a single comment and AArch64 is in a different one. I am obviously biased (being x86-only folk) but I find it more problematic than worth it to hack it any further. I hope a proper solution arrives sooner. :-) > > I know the script looks super fugly but it generally boils down to a > > mix of styles of 3 people and Gerrit having funky UI rendering. > > > Finally, I'd also like to thank hrw (Marcin Juszkiewicz) for linking > > me to the original Michel's script in 2019. > > YW. > -yoctozepto From radoslaw.piliszek at gmail.com Thu Dec 3 10:03:42 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Thu, 3 Dec 2020 11:03:42 +0100 Subject: [all] Dynamic Zuul results table in Gerrit 3 In-Reply-To: References: Message-ID: On Thu, Dec 3, 2020 at 10:44 AM Yatin Karel wrote: > > Hi, > > On Thu, Dec 3, 2020 at 2:57 PM Radosław Piliszek > wrote: > > > > Hello Fellow OpenStack and OpenDev Folks! > > > > TL;DR click on [3] and enjoy. > > > > I am starting this thread to not hijack the discussion happening on [1]. > > > > First of all, I would like to thank gibi (Balazs Gibizer) for hacking > > a way to get the place to render the table in the first place (pun > > intended). > > > > I have been a long-time-now user of [2]. > > I have improved and customised it for myself but never really got to > > share back the changes I made. > > The new Gerrit obviously broke the whole script so it was of no use to > > share at that particular state. > > However, inspired by gibi's work, I decided to finally sit down and > > fix it to work with Gerrit 3 and here it comes: [3]. > > Works well on Chrome with Tampermonkey. Not tested others. > > Thanks, Works for me in firefox with Greasemonkey. The only difference > I noticed wrt previous gerrit one is that the current one displays > both already available zuul results(from previous run) and the current > running one, which is fine. Yay, thanks for confirming. Yes, I obviously forgot all the little changes I made but this is one of them. I somehow like it better like this. The styling is not perfect now to differentiate between the two tables but it can be figured out based on the contents. > > > > I hope you will enjoy this little helper (I do). > > > > > I know the script looks super fugly but it generally boils down to a > > mix of styles of 3 people and Gerrit having funky UI rendering. > > > > Finally, I'd also like to thank hrw (Marcin Juszkiewicz) for linking > > me to the original Michel's script in 2019. > > > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/019051.html > > [2] https://opendev.org/x/coats/src/commit/444c95738677593dcfed0cfd9667d4c4f0d596a3/coats/openstack_gerrit_zuul_status.user.js > > [3] https://gist.github.com/yoctozepto/7ea1271c299d143388b7c1b1802ee75e > > > > Kind regards, > > -yoctozepto > > > > Thanks and Regards > Yatin Karel > -yoctozepto From radoslaw.piliszek at gmail.com Thu Dec 3 12:16:46 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Thu, 3 Dec 2020 13:16:46 +0100 Subject: [all] Dynamic Zuul results table in Gerrit 3 In-Reply-To: References: Message-ID: On Thu, Dec 3, 2020 at 11:03 AM Radosław Piliszek wrote: > ... I obviously forgot all the little changes I made ... Just remembered one so writing down for posterity. One of the enhancements is that it supports multiple tenants, i.e. it works for non-openstack/ namespaces, e.g. x/, opendev/ or zuul/ too. -yoctozepto From mnaser at vexxhost.com Thu Dec 3 13:23:20 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Thu, 3 Dec 2020 08:23:20 -0500 Subject: [all] Dynamic Zuul results table in Gerrit 3 In-Reply-To: References: Message-ID: On Thu, Dec 3, 2020 at 4:22 AM Radosław Piliszek wrote: > > Hello Fellow OpenStack and OpenDev Folks! > > TL;DR click on [3] and enjoy. I just set it up and it's awesome. Perhaps this could somehow make it's way into a Gerrit plugin so it can be available for all users. > I am starting this thread to not hijack the discussion happening on [1]. > > First of all, I would like to thank gibi (Balazs Gibizer) for hacking > a way to get the place to render the table in the first place (pun > intended). > > I have been a long-time-now user of [2]. > I have improved and customised it for myself but never really got to > share back the changes I made. > The new Gerrit obviously broke the whole script so it was of no use to > share at that particular state. > However, inspired by gibi's work, I decided to finally sit down and > fix it to work with Gerrit 3 and here it comes: [3]. > Works well on Chrome with Tampermonkey. Not tested others. > > I hope you will enjoy this little helper (I do). > > I know the script looks super fugly but it generally boils down to a > mix of styles of 3 people and Gerrit having funky UI rendering. > > Finally, I'd also like to thank hrw (Marcin Juszkiewicz) for linking > me to the original Michel's script in 2019. > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/019051.html > [2] https://opendev.org/x/coats/src/commit/444c95738677593dcfed0cfd9667d4c4f0d596a3/coats/openstack_gerrit_zuul_status.user.js > [3] https://gist.github.com/yoctozepto/7ea1271c299d143388b7c1b1802ee75e > > Kind regards, > -yoctozepto > -- Mohammed Naser VEXXHOST, Inc. From smooney at redhat.com Thu Dec 3 13:29:09 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 03 Dec 2020 13:29:09 +0000 Subject: [all] Dynamic Zuul results table in Gerrit 3 In-Reply-To: References: Message-ID: <278d4f9f3c1e78a2cc9e685540fc92efa9b3ef45.camel@redhat.com> On Thu, 2020-12-03 at 13:16 +0100, Radosław Piliszek wrote: > On Thu, Dec 3, 2020 at 11:03 AM Radosław Piliszek > wrote: > > ... I obviously forgot all the little changes I made ... > > Just remembered one so writing down for posterity. > One of the enhancements is that it supports multiple tenants, i.e. it > works for non-openstack/ namespaces, e.g. x/, opendev/ or zuul/ too. the main issue i still see with this script is it only looks a zuul comments so im worried that if too many peopel install it they will not look at the thirdparty ci result becaue they are uesed to the old ui where they are there. it works on firfox with code injector too by the way. the running job bit is nice nova-multi-cell RUNNING (100%) nova-next RUNNING (65%) but really i hope we can come to a solution server side that will work for all CIs these clinet side scripts are nice and i like that the running job links go to the job console like they do on the zuul web interace but we really do need to make sure the thirdpary cis are support just as much as the first party on. > > -yoctozepto > From tdecacqu at redhat.com Thu Dec 3 13:38:48 2020 From: tdecacqu at redhat.com (Tristan Cacqueray) Date: Thu, 03 Dec 2020 13:38:48 +0000 Subject: [all] Dynamic Zuul results table in Gerrit 3 In-Reply-To: References: Message-ID: <877dpzt3fb.tristanC@fedora> On Thu, Dec 03, 2020 at 10:22 Radosław Piliszek wrote: > Hello Fellow OpenStack and OpenDev Folks! > > TL;DR click on [3] and enjoy. > Hello It seems like this script is injecting build details directly using the innerHTML attribute without filtering html entities, please see the `Security considerations` section of https://developer.mozilla.org/en-US/docs/Web/API/Element/innerHTML -Tristan > I am starting this thread to not hijack the discussion happening on [1]. > > First of all, I would like to thank gibi (Balazs Gibizer) for hacking > a way to get the place to render the table in the first place (pun > intended). > > I have been a long-time-now user of [2]. > I have improved and customised it for myself but never really got to > share back the changes I made. > The new Gerrit obviously broke the whole script so it was of no use to > share at that particular state. > However, inspired by gibi's work, I decided to finally sit down and > fix it to work with Gerrit 3 and here it comes: [3]. > Works well on Chrome with Tampermonkey. Not tested others. > > I hope you will enjoy this little helper (I do). > > I know the script looks super fugly but it generally boils down to a > mix of styles of 3 people and Gerrit having funky UI rendering. > > Finally, I'd also like to thank hrw (Marcin Juszkiewicz) for linking > me to the original Michel's script in 2019. > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/019051.html > [2] https://opendev.org/x/coats/src/commit/444c95738677593dcfed0cfd9667d4c4f0d596a3/coats/openstack_gerrit_zuul_status.user.js > [3] https://gist.github.com/yoctozepto/7ea1271c299d143388b7c1b1802ee75e > > Kind regards, > -yoctozepto -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 515 bytes Desc: not available URL: From radoslaw.piliszek at gmail.com Thu Dec 3 13:48:07 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Thu, 3 Dec 2020 14:48:07 +0100 Subject: [all] Dynamic Zuul results table in Gerrit 3 In-Reply-To: <278d4f9f3c1e78a2cc9e685540fc92efa9b3ef45.camel@redhat.com> References: <278d4f9f3c1e78a2cc9e685540fc92efa9b3ef45.camel@redhat.com> Message-ID: On Thu, Dec 3, 2020 at 2:29 PM Sean Mooney wrote: > > On Thu, 2020-12-03 at 13:16 +0100, Radosław Piliszek wrote: > > On Thu, Dec 3, 2020 at 11:03 AM Radosław Piliszek > > wrote: > > > ... I obviously forgot all the little changes I made ... > > > > Just remembered one so writing down for posterity. > > One of the enhancements is that it supports multiple tenants, i.e. it > > works for non-openstack/ namespaces, e.g. x/, opendev/ or zuul/ too. > > the main issue i still see with this script is it only looks a zuul comments > so im worried that if too many peopel install it they will not look at the thirdparty > ci result becaue they are uesed to the old ui where they are there. Yes, this is the original limitation. This is analogous to hrw's comment about AArch64. Open to contributions. :-) I am pretty sure it would be best to treat this as a PoC and build a proper solution on Gerrit's side. As for dynamic results - I bet it is even trickier to handle the TP solutions. > it works on firfox with code injector too by the way. ++ -yoctozepto From radoslaw.piliszek at gmail.com Thu Dec 3 13:52:45 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Thu, 3 Dec 2020 14:52:45 +0100 Subject: [all] Dynamic Zuul results table in Gerrit 3 In-Reply-To: <877dpzt3fb.tristanC@fedora> References: <877dpzt3fb.tristanC@fedora> Message-ID: On Thu, Dec 3, 2020 at 2:38 PM Tristan Cacqueray wrote: > > > On Thu, Dec 03, 2020 at 10:22 Radosław Piliszek wrote: > > Hello Fellow OpenStack and OpenDev Folks! > > > > TL;DR click on [3] and enjoy. > > > > Hello > > It seems like this script is injecting build details directly using > the innerHTML attribute without filtering html entities, > please see the `Security considerations` section of > > https://developer.mozilla.org/en-US/docs/Web/API/Element/innerHTML Yes, it is a generally valid remark but I consider both Gerrit and Zuul (both of OpenDev) to have the exact same level of trust so did not modify the approach. But yes, for anyone trying to learn best practices from this snippet - please do not, it is far from them. :-) In general this approach is very wasteful as it causes rebuilding (or rather rejoining) and reparsing of html, instead of DOM manipulations. For such a simple table it does not hurt but please do not do it at home. -yoctozepto From ykarel at redhat.com Thu Dec 3 09:43:51 2020 From: ykarel at redhat.com (Yatin Karel) Date: Thu, 3 Dec 2020 15:13:51 +0530 Subject: [all] Dynamic Zuul results table in Gerrit 3 In-Reply-To: References: Message-ID: Hi, On Thu, Dec 3, 2020 at 2:57 PM Radosław Piliszek wrote: > > Hello Fellow OpenStack and OpenDev Folks! > > TL;DR click on [3] and enjoy. > > I am starting this thread to not hijack the discussion happening on [1]. > > First of all, I would like to thank gibi (Balazs Gibizer) for hacking > a way to get the place to render the table in the first place (pun > intended). > > I have been a long-time-now user of [2]. > I have improved and customised it for myself but never really got to > share back the changes I made. > The new Gerrit obviously broke the whole script so it was of no use to > share at that particular state. > However, inspired by gibi's work, I decided to finally sit down and > fix it to work with Gerrit 3 and here it comes: [3]. > Works well on Chrome with Tampermonkey. Not tested others. Thanks, Works for me in firefox with Greasemonkey. The only difference I noticed wrt previous gerrit one is that the current one displays both already available zuul results(from previous run) and the current running one, which is fine. > > I hope you will enjoy this little helper (I do). > > I know the script looks super fugly but it generally boils down to a > mix of styles of 3 people and Gerrit having funky UI rendering. > > Finally, I'd also like to thank hrw (Marcin Juszkiewicz) for linking > me to the original Michel's script in 2019. > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/019051.html > [2] https://opendev.org/x/coats/src/commit/444c95738677593dcfed0cfd9667d4c4f0d596a3/coats/openstack_gerrit_zuul_status.user.js > [3] https://gist.github.com/yoctozepto/7ea1271c299d143388b7c1b1802ee75e > > Kind regards, > -yoctozepto > Thanks and Regards Yatin Karel From cboylan at sapwetik.org Mon Dec 7 23:47:21 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 07 Dec 2020 15:47:21 -0800 Subject: Meeting Agenda for December 8, 2020 Message-ID: <403493a1-25b9-410e-a30e-d6bd68bb3c9a@www.fastmail.com> We will meet on December 8, 2020 at 19:00 UTC in #opendev-meeting with this agenda: == Agenda for next meeting == * Announcements ** Need volunteer to run next week's meeting. Or we can cancel. * Actions from last meeting * Specs approval * Priority Efforts (Standing meeting agenda items. Please expand if you have subtopics.) ** [http://specs.openstack.org/openstack-infra/infra-specs/specs/update-config-management.html Update Config Management] *** topic:update-cfg-mgmt *** Zuul as CD engine ** OpenDev *** Post Gerrit Upgrade Updates **** Configuration tuning ***** Adding config to jgit.config ***** Using strong refs for jgit caches ***** Enabling git protocol v2 ***** Batch user groups and threads **** Zuul results table rendering (ianw 20201208) **** WIP changes (ianw 20201208) **** Zuul needs updates to properly support Gerrit 3.3.0 * General topics ** Bup and Borg Backups (clarkb 20201208) ** Docker rate limiting is being seen (at least in limestone) (clarkb 20201208) ** PTG followups (clarkb 20201208) ** meetpad was not useable for (some?) participants from China (frickler 20201208) *** clarkb trying to set up time with Horace to test a meetpad call with a participant in China. ** Splitting puppet else into specific infra-prod jobs (clarkb 20201208) *** Should be mostly mechanical *** Does it make sense to try and sprint this? Have several people work on getting it done in a short period of time? ** Trusty Upgrade Progress (clarkb 20200929) *** Wiki updates * Open discussion From fungi at yuggoth.org Thu Dec 10 01:37:01 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 10 Dec 2020 01:37:01 +0000 Subject: Opendev review crawler In-Reply-To: Message-ID: <20201210013701.r73se6qgi6yj5555@yuggoth.org> Thanks again for implementing our earlier recommendations for setting a recognizable user agent string on your queries and spacing out each request by adding a delay. Unfortunately we've needed to temporarily block connections from your IP address again. It appears that, following our upgrade from Gerrit 2.13 to 3.2 we're seeing an overwhelming increase in memory pressure which seems to coincide with the times at which your queries started again on Monday. We're going to be working on trying to improve some aspects of memory management for the service, so we'd appreciate if you could put those Gerrit queries on hold while we attempt to do so. In the meantime, we should discuss what alternative methods we have for providing the data you're seeking. Thanks for understanding, and apologies for the inconvenience. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From iwienand at redhat.com Fri Dec 11 03:15:34 2020 From: iwienand at redhat.com (Ian Wienand) Date: Fri, 11 Dec 2020 14:15:34 +1100 Subject: powerkvm CI nova cloning Message-ID: <20201211031534.GA1435536@fedora19.localdomain> Hello, We've identified that powerkvm CI seems to be in a loop attempting to clone nova. It would be helpful if we could get this to stop :) It's just a guess, but we found a similar problem in OpenDev where the executors were attempting to cache the repo, but their default timeout of 300 seconds wasn't long enough to complete the clone. They would kill their clone attempt and start over again immediately. In the Zuul executor logs you'll see something like 2020-12-10 02:02:08,578 ERROR zuul.ExecutorServer: [e: ...] [build:...] Got exception while updating repo gerrit/openstack/nova with a full traceback above. Our solution was to increase the git_timeout [1] to long enough for the clone to complete. The recent upgrade may be to blame. We're experimenting with different options for gc/repack, but cloning the repo from gerrit directly it close to 2gb. Another option is to convert this to cloning to https://opendev.org. Please reach out if there is anything we can do to help. Thanks, -i [1] https://review.opendev.org/c/opendev/system-config/+/766400 From mjturek at us.ibm.com Fri Dec 11 16:12:34 2020 From: mjturek at us.ibm.com (Michael J Turek) Date: Fri, 11 Dec 2020 16:12:34 +0000 Subject: powerkvm CI nova cloning In-Reply-To: <20201211031534.GA1435536@fedora19.localdomain> References: <20201211031534.GA1435536@fedora19.localdomain> Message-ID: An HTML attachment was scrubbed... URL: From 393958790 at qq.com Thu Dec 17 02:02:11 2020 From: 393958790 at qq.com (=?ISO-8859-1?B?UGVuZ3poaQ==?=) Date: Thu, 17 Dec 2020 10:02:11 +0800 Subject: OpenDev git error: RPC failed; result=18, HTTP code = 200iB | 57.00 KiB/s Message-ID: Hi OpenDev team, When trying to pull tripleo from opendev.org, git failed with Fatal Error 128. Can you please check and fix it? [ooo at VM-3O ~]$ git config http.postBuffer 24288000 [ooo at VM-3O ~]$ git clone https://opendev.org/openstack/tripleo-quickstart-extras/ Cloning into 'tripleo-quickstart-extras'... remote: Enumerating objects: 19996, done. remote: Counting objects: 100% (19996/19996), done. remote: Compressing objects: 100% (6835/6835), done. error: RPC failed; result=18, HTTP code = 200iB | 57.00 KiB/s     fatal: The remote end hung up unexpectedly fatal: early EOF fatal: index-pack failed The Error is reproducible with/without the following git configurations [ooo at VM-3O ~]$ git config --list core.compression=-1 http.postbuffer=24288000 Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From 393958790 at qq.com Thu Dec 17 06:23:36 2020 From: 393958790 at qq.com (=?ISO-8859-1?B?UGVuZ3poaQ==?=) Date: Thu, 17 Dec 2020 14:23:36 +0800 Subject: OpenDev git error: RPC failed; result=18, HTTP code = 200iB | 57.00 KiB/s In-Reply-To: References: Message-ID: I have to "git clone https://opendev.org/openstack/tripleo-quickstart-extras/  --depth 1" Maybe my network is too slow.. ------------------ Original ------------------ From: "Pengzhi" <393958790 at qq.com>; Date: Thu, Dec 17, 2020 10:02 AM To: "service-discuss" From 393958790 at qq.com Thu Dec 17 08:05:21 2020 From: 393958790 at qq.com (=?ISO-8859-1?B?UGVuZ3poaQ==?=) Date: Thu, 17 Dec 2020 16:05:21 +0800 Subject: OpenDev git error: RPC failed; result=18, HTTP code = 200iB | 57.00 KiB/s In-Reply-To: References: Message-ID: A successful clone: Cloning into 'tripleo-quickstart-extras'... remote: Enumerating objects: 19996, done. remote: Counting objects: 100% (19996/19996), done. remote: Compressing objects: 100% (8880/8880), done. remote: Total 19996 (delta 13092), reused 16295 (delta 9833) Receiving objects: 100% (19996/19996), 3.32 MiB | 0 bytes/s, done. Resolving deltas: 100% (13092/13092), done. Checking out files: 100% (579/579), done. real 0m6.018s user 0m0.431s sys 0m0.418s A failed clone: Cloning into 'tripleo-quickstart-extras'... remote: Enumerating objects: 19996, done. remote: Counting objects: 100% (19996/19996), done. remote: Compressing objects: 100% (8949/8949), done. error: RPC failed; result=18, HTTP code = 200B | 9.00 KiB/s        fatal: The remote end hung up unexpectedly fatal: early EOF fatal: index-pack failed ------------------ Original ------------------ From: "Pengzhi" <393958790 at qq.com>; Date: Thu, Dec 17, 2020 02:23 PM To: "service-discuss" From fungi at yuggoth.org Thu Dec 17 18:01:53 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 17 Dec 2020 18:01:53 +0000 Subject: OpenDev git error: RPC failed; result=18, HTTP code = 200iB | 57.00 KiB/s In-Reply-To: References: Message-ID: <20201217180153.kauqyq7bcar6areu@yuggoth.org> [Cc'ing you since you don't seem to be subscribed to this ML] On 2020-12-17 10:02:11 +0800 (+0800), Pengzhi wrote: > When trying to pull tripleo from opendev.org, git failed with > Fatal Error 128. Can you please check and fix it? [...] I've been unable to reproduce this, even individually testing each of our backend servers from multiple locations globally. This seems more like it's the result of a misbehaving firewall/nat/proxy between you and the servers. Do you have insight into the gateways for your local network and their configuration? You might also check for packet loss between your systems and the https://opendev.org/ frontend. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From joshua.hesketh at gmail.com Fri Dec 18 05:03:28 2020 From: joshua.hesketh at gmail.com (Joshua Hesketh) Date: Fri, 18 Dec 2020 16:03:28 +1100 Subject: OpenDev git error: RPC failed; result=18, HTTP code = 200iB | 57.00 KiB/s In-Reply-To: <20201217180153.kauqyq7bcar6areu@yuggoth.org> References: <20201217180153.kauqyq7bcar6areu@yuggoth.org> Message-ID: Hey, This could be related to upload pack sizes I debugged some years ago: https://josh.hesketh.net.au/2015/06/git-openstack-org-adventures/ Basically it might be worth checking the timeouts in HAProxy and/or Apache/etc. Unfortunately I don't have the time to take a look myself anymore :-(. Hope that helps though! Cheers, Josh On Fri, Dec 18, 2020 at 5:02 AM Jeremy Stanley wrote: > [Cc'ing you since you don't seem to be subscribed to this ML] > > On 2020-12-17 10:02:11 +0800 (+0800), Pengzhi wrote: > > When trying to pull tripleo from opendev.org, git failed with > > Fatal Error 128. Can you please check and fix it? > [...] > > I've been unable to reproduce this, even individually testing each > of our backend servers from multiple locations globally. This seems > more like it's the result of a misbehaving firewall/nat/proxy > between you and the servers. Do you have insight into the gateways > for your local network and their configuration? You might also check > for packet loss between your systems and the https://opendev.org/ > frontend. > -- > Jeremy Stanley > -------------- next part -------------- An HTML attachment was scrubbed... URL: