Skip to content

Instantly share code, notes, and snippets.

@yalp

yalp/espr_slides.md

Last active August 29, 2015 14:24
Show Gist options
  • Save yalp/600181f33cb03dc3614e to your computer and use it in GitHub Desktop.
Save yalp/600181f33cb03dc3614e to your computer and use it in GitHub Desktop.
Download ZIP
EspR4FastData Slides
Raw
espr_slides.md

Data Handling GE

by Marc Capdevielle

Data Handling GE

Goal

  • process "basic" Context Entities from NGSI-enabled sensors
  • generate higher levels of abstraction (Room, Floor, Building)
  • at the gateway level (runs on RPi)

Powerfull features

The Esper CEP (Complex Event Processing) engine provides a lot of features (aggreation, filters, rate limiting, correlation, partitioning...) with a nice SQL like syntax.

Current state

Architecture

  • Mixes roles between a Context Broker and a CEP
  • Configuration API difficult use in cluster context
  • Not compatible with other IoT GE

Code

  • Partial implementation of the NGSI 9/10 protocol
  • Alpha/prototype code quality

Delivery

  • No delivery in more than six months !

@Orange

Strong commitment

  • Orange has a lot of interest in Esper processing technology
  • committed to maintain the Data Handling GE

A new team

  • 3 developers
  • finding a dedicated IoT WP manager has proved difficult

Beginning was slow

  • Steep learning curve : NGSI, Esper CEP and the Fiware ecosystem
  • Started a cleanup of the codebase but too much technical debt

Proposals

  • simpler mapping of Context Element to Esper Events
  • better configuration API
  • separated, local Context Broker (TBD)
  • JSON support
  • tested with IoT Agent library and Orion CB
  • complete rewrite
  • support special attribute types (location, timestamps, commands)
  • first delivery : end of september

Proposal 1 : Compatiblity with other GE

With IoT Agent

  • Support JSON
  • Support lazy and commands attributes

With Orion CB

  • Support Service / ServicePath Orion headers
  • PEP proxy OAuth2 access control

Proposal 2 : simpler API

Static configuration

  • A single POST of a JSON configuration is required
    • easier cluster configuration
  • Transactional style configuration
    • rollback on error
    • hot reload when possible

Proposal 3 : local CB

Lightweight Context Broker

  • Prevents hiding NGSI sensors from context broker
  • Authorization capabilities
  • Handling pull mode (lazy attributes)

Cons

  • Makes the gateway architecture more complex
  • Which subset of NGSI 9/10 to support ?

Proposal 4 : simpler mapping

Esper is like a relational DB

  • uses a fixed schema
  • forward declaration during configuration

Consequences

  • all Context Entities must have a mandatory type
  • a type declares a list of known attributes

Thank you

Links

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment