[Openinfralabs] Proposing a new OILabs sub-project on compute-storage coordination

[Hui Lei]

Hi all,

I would like to propose a new sub-project under the OILabs umbrella. The
proposed sub-project, dubbed Caerus, will develop techniques to improve the
interactions between compute and storage servers. As you may know, there is
a trend of compute and storage decoupling, where a compute engine such as
Spark, Presto, and Tensorflow is physically separate and far apart from a
storage system such as an object store, a file system, or even a database.
There are various things that can be done to improve the end-to-end
performance, e.g., pushing part of computation to the storage side, and
opportunely caching source data, intermediate results or metadata on the
compute side. Project Caerus will develop a framework for coordinating
between compute and storage servers and for autonomously making
optimization decisions such as what part of processing to be pushed down to
storage and what content to be cached on the compute server. I believe this
project aligns well with OpenInfraLabs' mission of integrating different
layers of a cloud stack.

I welcome your thoughts on the project idea and, more importantly, your
contribution after the project is officially kicked off.

Hui Lei
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendev.org/pipermail/openinfralabs/attachments/20200515/3933c465/attachment.html>