[Openinfralabs] Project Caerus Update

[Hui Lei]

Marcel,

This item is already on our backlog. We just haven't been able to get to
it. We certainly welcome other community members to join us and help
accelerate this item.

Hui

On Thu, Aug 5, 2021 at 9:04 AM Marcel Hild <mhild at redhat.com> wrote:

> Hey Hui,
> thanks for the update.
>
> Have you considered to implement a running version for your POC to
> the Operate First Community Cloud at https://www.operate-first.cloud/ ?
> There's also a spark cluster available and JupyterNotebooks.
> If the community could try out your work, that might increase the awarnes
>
> On Wed, Aug 4, 2021 at 5:02 AM Hui Lei <dr.huilei at gmail.com> wrote:
>
>> Dear all,
>>
>> I would like to take this opportunity to give you another update on
>> Project Caerus. As you may remember, the project develops techniques such
>> as near-data processing and semantic caching to optimize the performance of
>> disaggregated data lakes. On the front of near data processing, we have
>> implemented the pushdown of a wide range of SQL operators from a Spark
>> cluster to a storage cluster that deploys either HDFS (CSV format) or S3.
>> Our evaluation using TCPH has shown significant improvements in application
>> latency, network I/O and compute-side CPU time. You can check out our design
>> document
>> <https://github.com/open-infrastructure-labs/caerus-dike/blob/master/doc/ndp_design.pdf>
>> and latest evaluation results
>> <https://github.com/open-infrastructure-labs/caerus-dike/blob/master/doc/s3_hdfs_results_6_1_2021.pdf>
>> in GitHub.
>>
>> On the front of semantic cache, which explores opportune caching of a
>> variety of data and metadata, we have the core functionality working, with
>> 4x-5x improvement in execution time and CPU time. Again the design
>> document
>> <https://github.com/open-infrastructure-labs/caerus-semantic-cache/blob/master/Design.docx>
>> and the initial evaluation results
>> <https://github.com/open-infrastructure-labs/caerus-semantic-cache/blob/master/Evaluation.docx>
>> are available in GitHub.
>>
>> As always, your comments and contributions are welcome.
>>
>> - Hui
>> _______________________________________________
>> Openinfralabs mailing list
>> Openinfralabs at lists.opendev.org
>> http://lists.opendev.org/cgi-bin/mailman/listinfo/openinfralabs
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendev.org/pipermail/openinfralabs/attachments/20210805/a96e9505/attachment.html>