[Openinfralabs] Proposing a new OILabs sub-project on compute-storage coordination

[Jonathan Bryce]

Hi Hui,

Sounds like an interesting project.

I can help you and initial collaborators get a repo set up under https://gitlab.com/open-infrastructure-labs/ if you have an initial set of GitLab usernames to add.

Jonathan



> On May 15, 2020, at 1:49 PM, Hui Lei <dr.huilei at gmail.com> wrote:
> 
> Hi all,
> 
> I would like to propose a new sub-project under the OILabs umbrella. The proposed sub-project, dubbed Caerus, will develop techniques to improve the interactions between compute and storage servers. As you may know, there is a trend of compute and storage decoupling, where a compute engine such as Spark, Presto, and Tensorflow is physically separate and far apart from a storage system such as an object store, a file system, or even a database. There are various things that can be done to improve the end-to-end performance, e.g., pushing part of computation to the storage side, and opportunely caching source data, intermediate results or metadata on the compute side. Project Caerus will develop a framework for coordinating between compute and storage servers and for autonomously making optimization decisions such as what part of processing to be pushed down to storage and what content to be cached on the compute server. I believe this project aligns well with OpenInfraLabs' mission of integrating different layers of a cloud stack.  
> 
> I welcome your thoughts on the project idea and, more importantly, your contribution after the project is officially kicked off.
> 
> Hui Lei
> _______________________________________________
> Openinfralabs mailing list
> Openinfralabs at lists.opendev.org
> http://lists.opendev.org/cgi-bin/mailman/listinfo/openinfralabs