CEP optimizer

The asset, namely the FERARI CEP optimizer, is being developed within WP5. Its design principles and current implementation details were reported in Deliverable D5.2. In FERARI, we aim at designing solutions capable of scaling at a planetary level, thus retaining the potential of being applicable to arbitrarily large businesses and data volumes.  Since in vast scale applications (event) data of interest are produced or collected at remote sources; they need to be combined to respond to global application inquiries. In this context, collecting voluminous data at a central computing point (termed site/cloud) first and then processing them is infeasible because such a solution would abuse the available network resources of the underlying infrastructure and would cause a bottleneck at that central point. Data of interest arriving at multiple, potentially geographically dispersed, sites should be efficiently processed in-situ (if possible) first and then wisely combined at an inter-cloud level to provide holistic results to final stakeholders. Efficiency in such a setting calls for reduced communication to avoid congested network links among sites without affecting the timely delivery of notifications and application reports. The above goals are in the core of our CEP optimizer which produces execution plans for application queries that orchestrate site interactions in a way that both ensures:

  1. a) optimal network utilization and b) compliance with application Quality-of-Service (QoS) requirements related to the time horizon, from the occurrence of interesting situations, in which corresponding reports should be made available to end users.

Our algorithms and design principles are generic enough to support a wide range of application query functionality. They can be employed on top of any CEP Engine being selected as the software responsible for intra-cloud data processing and query execution at the site level. Therefore, our approach can be fostered as a paradigm for any similar implementation irrespectively of the CEP Engine or specific application demands.

Having clarified that, in the context of FERARI, the currently developed software is built to support – in terms of application query operators and respective functionality – IBM Proactive Technology Online (Proton) CEP Engine and in particular its streaming cloud extension, namely ProtonOnStorm. This is not too great a restriction as Proton and ProtonOnStorm are open source platforms with an already important user base, as will be explained below. Moreover, the TEM specification which is another important FERARI asset for letting business users express inquiries in a declarative way (details see above), can be mapped to an Event Processing Network (EPN) conceptualization which is supported by ProtonOnStorm at the technical level.

Therefore, the TEM specification is commutatively supported by our approach. In addition, the CEP optimizer outcomes incorporate and exploit the FERARI assets related to in situ processing at the site level.

The CEP optimizer, as a FERARI asset, has already been pushed to the scientific community by publishing corresponding results – in close collaboration with the rest of the FERARI consortium – in the top database conference and the most well esteemed conference within the event processing community:

FERARI: A Prototype for Complex Event Processing over Streaming Multi-cloud Platforms,

  1. Flouris, V. Manikaki, N. Giatrakos, A. Deligiannakis, M. Garofalakis, M. Mock, S. Bothe, I. Skarbovsky, F. Fournier, T. Krizan, M. Stajcer, J. Yom-Tov, T. Curin

in: SIGMOD, 2016 (accepted –  to appear).

Complex Event Processing over Streaming Multi-cloud Platforms – The FERARI Approach,

  1. Flouris, V. Manikaki, N. Giatrakos, A. Deligiannakis, M. Garofalakis, M. Mock, S. Bothe, I. Skarbovsky, F. Fournier, T. Krizan, M. Stajcer, J. Yom-Tov, M. Volarevic

in: DEBS, 2016 (invited submission – under review).

ferari architecture