[Openinfralabs] Project Caerus Update

[Marcel Hild]

Hey Hui,
thanks for the update.

Have you considered to implement a running version for your POC to
the Operate First Community Cloud at https://www.operate-first.cloud/ ?
There's also a spark cluster available and JupyterNotebooks.
If the community could try out your work, that might increase the awarnes

On Wed, Aug 4, 2021 at 5:02 AM Hui Lei <dr.huilei at gmail.com> wrote:

> Dear all,
>
> I would like to take this opportunity to give you another update on
> Project Caerus. As you may remember, the project develops techniques such
> as near-data processing and semantic caching to optimize the performance of
> disaggregated data lakes. On the front of near data processing, we have
> implemented the pushdown of a wide range of SQL operators from a Spark
> cluster to a storage cluster that deploys either HDFS (CSV format) or S3.
> Our evaluation using TCPH has shown significant improvements in application
> latency, network I/O and compute-side CPU time. You can check out our design
> document
> <https://github.com/open-infrastructure-labs/caerus-dike/blob/master/doc/ndp_design.pdf>
> and latest evaluation results
> <https://github.com/open-infrastructure-labs/caerus-dike/blob/master/doc/s3_hdfs_results_6_1_2021.pdf>
> in GitHub.
>
> On the front of semantic cache, which explores opportune caching of a
> variety of data and metadata, we have the core functionality working, with
> 4x-5x improvement in execution time and CPU time. Again the design
> document
> <https://github.com/open-infrastructure-labs/caerus-semantic-cache/blob/master/Design.docx>
> and the initial evaluation results
> <https://github.com/open-infrastructure-labs/caerus-semantic-cache/blob/master/Evaluation.docx>
> are available in GitHub.
>
> As always, your comments and contributions are welcome.
>
> - Hui
> _______________________________________________
> Openinfralabs mailing list
> Openinfralabs at lists.opendev.org
> http://lists.opendev.org/cgi-bin/mailman/listinfo/openinfralabs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendev.org/pipermail/openinfralabs/attachments/20210805/9e4b25d7/attachment.html>