Open Stack

On Wed, Oct 31, 2018 at 10:57 AM Bogdan Dobrelya <bdobreli at redhat.com> wrote:
>
> (cross-posting openstack-dev)
>
> Hello.
> [tl;dr] I'm looking for co-author(s) to come up with "Edge clouds data
> consistency requirements and challenges" a position paper [0] (papers
> submitting deadline is Nov 8).
>
> The problem scope is synchronizing control plane and/or
> deployments-specific data (not necessary limited to OpenStack) across
> remote Edges and central Edge and management site(s). Including the same
> aspects for overclouds and undercloud(s), in terms of TripleO; and other
> deployment tools of your choice.
>
> Another problem is to not go into different solutions for Edge
> deployments management and control planes of edges. And for tenants as
> well, if we think of tenants also doing Edge deployments based on Edge
> Data Replication as a Service, say for Kubernetes/OpenShift on top of
> OpenStack.
>
> So the paper should name the outstanding problems, define data
> consistency requirements and pose possible solutions for synchronization
> and conflicts resolving. Having maximum autonomy cases supported for
> isolated sites, with a capability to eventually catch up its distributed
> state. Like global database [1], or something different perhaps (see
> causal-real-time consistency model [2],[3]), or even using git. And
> probably more than that?.. (looking for ideas)


I can offer detail on whatever aspects of the "shared  / global
database" idea.  The way we're doing it with Galera for now is all
about something simple and modestly effective for the moment, but it
doesn't have any of the hallmarks of a long-term, canonical solution,
because Galera is not well suited towards being present on many
(dozens) of endpoints.     The concept that the StarlingX folks were
talking about, that of independent databases that are synchronized
using some kind of middleware is potentially more scalable, however I
think the best approach would be API-level replication, that is, you
have a bunch of Keystone services and there is a process that is
regularly accessing the APIs of these keystone services and
cross-publishing state amongst all of them.   Clearly the big
challenge with that is how to resolve conflicts, I think the answer
would lie in the fact that the data being replicated would be of
limited scope and potentially consist of mostly or fully
non-overlapping records.

That is, I think "global database" is a cheap way to get what would be
more effective as asynchronous state synchronization between identity
services.

>
> See also the "check" list in-line, which I think also meets the data
> consistency topics well - it would be always nice to have some
> theoretical foundations at hand, when repairing some
> 1000-edges-spread-off and fully broken global database, by hand :)
>
> PS. I must admit I have yet any experience with those IEEE et al
> academic things and looking for someone who has it, to team and
> co-author that positioning paper by. That's as a start, then we can
> think of presenting it and expanding into work items for OpenStack Edge
> WG and future development plans.
>
> [0] http://conferences.computer.org/ICFC/2019/Paper_Submission.html
> [1] https://review.openstack.org/600555
> [2] https://jepsen.io/consistency
> [3] http://www.cs.cornell.edu/lorenzo/papers/cac-tr.pdf
>
> On 10/22/18 3:44 PM, Flavia Delicato wrote:
> > =================================================================================
> > IEEE International Conference on Fog Computing (ICFC 2019)
> > June 24-26, 2019
> > Prague, Czech Republic
> > http://conferences.computer.org/ICFC/2019/
> > Co-located with the IEEE International Conference on Cloud Engineering
> > (IC2E 2019)
> > ==================================================================================
> >
> > Important Dates
> > ---------------
> > Paper registration and abstract: Nov 1st, 2018
> > Full paper submission due: Nov 8th, 2018
> > Notification of paper acceptance: Jan. 20th, 2019
> > Workshop and tutorial proposals due: Nov 11, 2018
> > Notification of proposal acceptance: Nov 18, 2018
> >
> > Call for Contributions
> > ----------------------
> > Fog computing is the extension of cloud computing into its edge and
> > the physical world to meet the data volume and decision velocity
> > requirements in many emerging applications, such as augmented and
> > virtual realities (AR/VR), cyber-physical systems (CPS), intelligent
> > and autonomous systems, and mission-critical systems. The boundary
> > between centralized, powerful computing cloud and massively
> > distributed, Internet connected sensors, actuators, and things is
> > blurred in this new computing paradigm.
> >
> > The ICFC 2019 technical program will feature tutorials, workshops, and
> > research paper sessions. We solicit high-quality contributions in the
> > above categories. Details of submission is available on the conference
> > Web site. Topics of interest include but are not limited to:
> >
> > * System architecture for fog computing
>
> (check)
>
> > * Coordination between cloud, fog, and sensing/actuation endpoints
> > * Connectivity, storage, and computation in the edge
> > * Data processing and management for fog computing
>
> (check)
>
> > * Efficient and embedded AI in the fog
> > * System and network manageability
> > * Middleware and coordination platforms
> > * Power, energy, and resource management
> > * Device and hardware support for fog computing
> > * Programming models, abstractions, and software engineering for fog computing
>
> (check)
>
> > * Security, privacy, and ethics issues related to fog computing
> > * Theoretical foundations and formal methods for fog computing systems
>
> (check)
>
> > * Applications and experiences
> >
> > Organizing Committee
> > --------------------
> > General Chairs:
> > Hui Lei, IBM
> > Albert Zomaya, The University of Sydney
> >
> > PC Co-chairs:
> > Erol Gelenbe, Imperial College London
> > Jie Liu, Microsoft Research
> >
> > Tutorials and Workshops Chair:
> > David Bermbach, TU Berlin
> >
> > Publicity Co-chairs:
> > Flavia Delicato,Federal University of Rio de Janeiro
> > Mathias Fischer, University Hamburg
> >
> > Publication Chair
> > Javid Taheri, Karlstad University
> >
> > Webmaster
> > Wei Li, The University of Sydney
> >
> > Steering Committee
> > ------------------
> > Mung Chiang, Purdue University
> > Erol Gelenbe, Imperial College London
> > Christos Kozarakis, Stanford University
> > Hui Lei, IBM
> > Chenyang Lu, Washington University in St Louis
> > Beng Chin Ooi, National University of Singapore
> > Neeraj Suri, TU Darmstadt
> > Albert Zomaya, The University of Sydney
> >
> > Program Committee
> > ------------------
> >
> > Tarek Abdelzaher, UIUC
> > Anne Benoit, ENS Lyon
> > David Bermbach, TU Berlin
> > Bharat Bhargava, Purdue University
> > Olivier Brun, LAAS/CNRS Laboratory
> > Jiannong Cao, Hong Kong Polytech
> > Flavia C. Delicato, UFRJ, Brazil
> > Xiaotie Deng, Peking University, China
> > Schahram Dustdar, TU Wien, Germany
> > Maria Gorlatova, Duke University
> > Dharanipragada Janakiram, IIT Madras
> > Wenjing Luo, Virginia Tech
> > Pedro José Marrón, Universität Duisburg-Essen
> > Geyong Min, University of Exeter
> > Suman Nath, Microsoft Research
> > Vincenzo Piuri, Universita Degli Studi Di Milano
> > Yong Meng Teo, National University of Singapore
> > Guoliang Xing, Chinese University of Hong Kong
> > Yuanyuan Yang, SUNY Stony Brook
> > Xiaoyun Zhu, Cloudera
> >
>
>
> --
> Best regards,
> Bogdan Dobrelya,
> Irc #bogdando