LeandroOrdonez

LeandroOrdonez

#!/bin/bash

VERSION="1.0.2"
# Interface connect to out lan
INTERFACE="eth0"
# Interface virtual for incomming traffic
VIRTUAL="ifb0"
# set the direction (1 = outgoing only, 2 = incoming only 3 = both)
DIRECTION=3
# Speed

LeandroOrdonez

LeandroOrdonez

public class AggregateTuple extends Aggregate{
    public String key;
    public String geohash;
    public Long timestamp;

    public AggregateTuple() {
        super();
    }

    public AggregateTuple(String key, String geohash, Long timestamp, Long count, Double sum, Double avg) {

LeandroOrdonez

"u14dhq#20191101:140000:000": {"geohash": "u14dhq", "timestamp": 1572616800000, "avg": 24.32, "sum": 81788.16, "count": 3363} 
"u14dhq#20191101:150000:000": {"geohash": "u14dhq", "timestamp": 1572620400000, "avg": 32.10, "sum": 111964.80, "count": 3488} 
"u14dhq#20191101:160000:000": {"geohash": "u14dhq", "timestamp": 1572624000000, "avg": 35.13, "sum": 115015.62, "count": 3274} 
"u14dhq#20191101:170000:000": {"geohash": "u14dhq", "timestamp": 1572627600000, "avg": 30.08, "sum": 101700.48, "count": 3381} 

LeandroOrdonez

public class AggregateTuple {
    public String sensorId;
    public Long count;
    public Double sum;
    public Double avg;

    public AggregateTuple() {
    }

    public AggregateTuple(String sensorId, Long count, Double sum, Double avg) {

LeandroOrdonez

#!/usr/bin/env bash


# This script implements 3 asynchronous function
# setTimeout
# setInterval
# async
# killJob function is not asynchronous

# check the README.md for information on how to use this script

LeandroOrdonez

LeandroOrdonez

Context information management in the CoT: current challenges

1. Limited query API:

The current query API based on the FIWARE-NGSiv2 specification is limited to basic CRUD operations [1, 2], neglecting query patterns such as:

• Aggregation of a certain reading for different levels of granularity (e.g.: sliding window over the last 5-minutes, last hour, last 24h, last week, etc.) [3].
• Aggregation of a certain reading over a location or geospatial area [3].
• Metrics over the full history of the collected data [3].
• Retrieving all the data/readings related to a specific type of information (weather data, air quality data, water quality, etc.)^✻.

LeandroOrdonez

from pyspark.sql import SparkSession
from pyspark.sql.functions import hour, month

spark = SparkSession \
    .builder \
    .appName("Python Spark SQL basic example") \
    .getOrCreate()

sensor_df = spark.read.json("../../Code/ShopperAnalytics/immutable_sensor_data.json")
sensor_df = sensor_df.withColumn("report_date", sensor_df.report_date["$date"].cast("date"))