Skip to content

Instantly share code, notes, and snippets.

@jonticknor
jonticknor / strucstream.py
Last active March 31, 2021 20:24
Beacon detection using Spark Structured Streaming
from pyspark.sql import SparkSession
from pyspark.sql.functions import collect_list
from pyspark.sql.functions import split
from pyspark.sql.functions import udf
from pyspark.sql.types import FloatType
import numpy as np
# Padding allows for considerdation of log time stamp issues,
# network delay, etc.