From dr.huilei at gmail.com Mon May 10 18:33:05 2021 From: dr.huilei at gmail.com (Hui Lei) Date: Mon, 10 May 2021 14:33:05 -0400 Subject: [Openinfralabs] Project Caerus Code Release Message-ID: Dear all, As you may know, the Caerus project of the Open Infra Labs investigates techniques such as near-data processing and semantic caching to optimize the performance of disaggregated data lakes. I am pleased to announce that an initial version of the project code is now available in the open infra lab repo (https://github.com/open-infrastructure-labs/caerus). The initial code base enables the pushing down of Spark SQL operations to the data nodes of HDFS. Attached is a preliminary evaluation of the work using TPCH benchmark. You are all welcome to check out and, better yet, contribute to the work. We will be using the project wiki ( https://github.com/open-infrastructure-labs/caerus/wiki) for discussions and questions. - Hui Lei -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Caerus NDP evaluation with Spark and HDFS - May 2021.pdf Type: application/pdf Size: 147719 bytes Desc: not available URL: From fungi at yuggoth.org Mon May 10 18:51:35 2021 From: fungi at yuggoth.org (Jeremy Stanley) Date: Mon, 10 May 2021 18:51:35 +0000 Subject: [Openinfralabs] May 11 meeting of Telemetry Working Group cancelled Message-ID: <20210510185135.zd3wxlqgfqm2zpw4@yuggoth.org> I received a notification from Jennifer Stacy at BU letting me know that this month's Telemetry Working Group meeting, originally scheduled for May 11 (tomorrow), is cancelled due to lack of availability for key participants. The June 8 meeting is still planned as usual. Feel free to follow up here on the mailing list, IRC/Slack, or GitHub issues with any important topics of discussion in the meantime. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From msd at bu.edu Mon May 10 20:08:13 2021 From: msd at bu.edu (Daitzman, Michael S) Date: Mon, 10 May 2021 20:08:13 +0000 Subject: [Openinfralabs] backing up github Message-ID: We don’t have an enterprise account with github which led to a a realization that we don’t have an explicit “backup” model for openinfralabs repositories. Historically I think we’ve depended on people having local copies in case of “outages”. I was curious, does the open infrastructure foundation have a way to do “backups” of repositories? --- Michael Daitzman (He/Him) msd at bu.edu cell:978-201-9965 -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Mon May 10 20:48:23 2021 From: fungi at yuggoth.org (Jeremy Stanley) Date: Mon, 10 May 2021 20:48:23 +0000 Subject: [Openinfralabs] backing up github In-Reply-To: References: Message-ID: <20210510204823.drlhvstymuhy663s@yuggoth.org> On 2021-05-10 20:08:13 +0000 (+0000), Daitzman, Michael S wrote: > We don’t have an enterprise account with github which led to a a > realization that we don’t have an explicit “backup” model for > openinfralabs repositories. > > Historically I think we’ve depended on people having local copies > in case of “outages”. I was curious, does the open > infrastructure foundation have a way to do “backups” of > repositories? The Open Infrastructure Foundation doesn't really host any project infrastructure at all. It helps fund and recommends use of the OpenDev Collaboratory, which backs up all repositories developed within its open source code review system (Gerrit) to two separate external systems in different service providers for added redundancy: https://docs.opendev.org/opendev/system-config/latest/sysadmin.html#backups If projects choose to source their own hosting infrastructure instead, they generally also work out their own disaster recovery solutions for those systems. If memory serves, it was decided to put these repositories within GitHub's proprietary hosting platform because that was more convenient for contributors from involved organizations who were already hosting development efforts there. What backup mechanisms are they using for their own repositories, and can the OILabs repos simply be added to that? If the people managing the GitHub organization where development takes place don't have ready access to a backup mechanism, it may be worth approaching the foundation about using project funds to cover the costs of some solution, but it would be a good idea to work up a details for the proposal. It sounds like this "GitHub Enterprise" offers the backups you require. Would upgrading the current organization to that license provide the needed features? (You'll have to forgive my ignorance, I know next to nothing about GitHub or its features, as I prefer to focus on open source software and services.) -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From mhild at redhat.com Mon May 17 11:24:21 2021 From: mhild at redhat.com (Marcel Hild) Date: Mon, 17 May 2021 13:24:21 +0200 Subject: [Openinfralabs] Project Caerus Code Release In-Reply-To: References: Message-ID: Hey Hui, good to see the project making progress Would you be interested in deploying the Caerus-Dike POC into the OpenShift environment we're running at the MOC? We have OpenDataHub and Spark available there, so it would be nice to see some long running integration work. If so, just head over to https://www.operate-first.cloud/ and follow the onboarding process. marcel On Mon, May 10, 2021 at 8:37 PM Hui Lei wrote: > Dear all, > > As you may know, the Caerus project of the Open Infra Labs investigates > techniques such as near-data processing and semantic caching to optimize > the performance of disaggregated data lakes. I am pleased to announce that > an initial version of the project code is now available in the open infra > lab repo (https://github.com/open-infrastructure-labs/caerus). The > initial code base enables the pushing down of Spark SQL operations to the > data nodes of HDFS. Attached is a preliminary evaluation of the work using > TPCH benchmark. You are all welcome to check out and, better yet, > contribute to the work. We will be using the project wiki ( > https://github.com/open-infrastructure-labs/caerus/wiki) for discussions > and questions. > > - Hui Lei > > _______________________________________________ > Openinfralabs mailing list > Openinfralabs at lists.opendev.org > http://lists.opendev.org/cgi-bin/mailman/listinfo/openinfralabs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dr.huilei at gmail.com Mon May 17 13:48:03 2021 From: dr.huilei at gmail.com (Hui Lei) Date: Mon, 17 May 2021 09:48:03 -0400 Subject: [Openinfralabs] Project Caerus Code Release In-Reply-To: References: Message-ID: Marcel, Thanks for the suggestion. We will look into that. On Mon, May 17, 2021 at 7:25 AM Marcel Hild wrote: > Hey Hui, > good to see the project making progress > Would you be interested in deploying the Caerus-Dike > POC into the > OpenShift environment we're running at the MOC? > We have OpenDataHub and Spark available there, so it would be nice to see > some long running integration work. > > If so, just head over to https://www.operate-first.cloud/ and follow the > onboarding process. > > marcel > > On Mon, May 10, 2021 at 8:37 PM Hui Lei wrote: > >> Dear all, >> >> As you may know, the Caerus project of the Open Infra Labs investigates >> techniques such as near-data processing and semantic caching to optimize >> the performance of disaggregated data lakes. I am pleased to announce that >> an initial version of the project code is now available in the open infra >> lab repo (https://github.com/open-infrastructure-labs/caerus). The >> initial code base enables the pushing down of Spark SQL operations to the >> data nodes of HDFS. Attached is a preliminary evaluation of the work using >> TPCH benchmark. You are all welcome to check out and, better yet, >> contribute to the work. We will be using the project wiki ( >> https://github.com/open-infrastructure-labs/caerus/wiki) for discussions >> and questions. >> >> - Hui Lei >> >> _______________________________________________ >> Openinfralabs mailing list >> Openinfralabs at lists.opendev.org >> http://lists.opendev.org/cgi-bin/mailman/listinfo/openinfralabs >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Fri May 21 19:59:57 2021 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 21 May 2021 19:59:57 +0000 Subject: [Openinfralabs] OpenDev Collaboratory IRC services Message-ID: <20210521195957.sm3pqjh57fgszckn@yuggoth.org> I know lots of people are discussing the recent Freenode IRC upheaval and what it means for their projects. On behalf of the OpenDev sysadmins I've started a thread on the service-discuss at lists.opendev.org mailing list to get feedback from communities currently utilizing the OpenDev Collaboratory's IRC bot services in their channels on Freenode (meeting minutes/logging, Gerrit change events, OpenDev service status information, channel operator assistance): http://lists.opendev.org/pipermail/service-discuss/2021-May/000236.html Please follow up there with your thoughts, to assist in our decision making, so we can take the needs of your project into account. Thanks! -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From fungi at yuggoth.org Wed May 26 19:35:20 2021 From: fungi at yuggoth.org (Jeremy Stanley) Date: Wed, 26 May 2021 19:35:20 +0000 Subject: [Openinfralabs] OpenDev IRC services are moving to OFTC this weekend Message-ID: <20210526193520.z7thgiipj37fyqsm@yuggoth.org> As a majority of our constituent projects have voiced a preference for enacting our long-standing evacuation plan, the OpenDev Collaboratory's IRC service bots will be switching from Freenode to the OFTC network this weekend (May 29-30, 2021). We understand this is short notice, but multiple projects have requested that we act quickly. Please expect some gaps in channel logging and notifications from our various bots over the course of the weekend. I have provided a much more detailed writeup to the service-discuss mailing list, and encourage anyone with questions to read it and follow up there if needed. Subsequent updates will be sent only to service-discuss, in order to limit noise for individual project lists and keep further discussion focused in one place as much as possible: http://lists.opendev.org/pipermail/service-discuss/2021-May/000249.html -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: