Skip to content

Instantly share code, notes, and snippets.

Avatar

geoHeil geoHeil

View GitHub Profile
View foo.py
df['device_id'] = df['device_id'].astype(str).apply(lambda x: x.zfill(2))
df = df.sort_values(['device_id'])
d_r = df.reset_index()
a4_dims = (11.7, 8.27)
fig, ax = pyplot.subplots(figsize=a4_dims)
ax = sns.lineplot(x='index', y='metrik_0', style='device_id', hue='device_id', data=d_r, ax=ax)
import matplotlib.dates as mdates
ax.axvspan(*mdates.datestr2num(['2020-01-04 05:00:00', '2020-01-05 16:00:00']), color='gray', alpha=0.2)
ax.axvspan(*mdates.datestr2num(['2020-01-04 15:00:00', '2020-01-05 00:00:00']), color='gray', alpha=0.2)
View rapids_15_3.8.txt
# packages in environment at ~/.conda/envs/rapidstest:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge /conda-forge
_openmp_mutex 4.5 1_gnu /conda-forge
abseil-cpp 20200225.2 he1b5a44_2 /conda-forge
aiohttp 3.6.2 py38h516909a_0 /conda-forge
appdirs 1.4.3 py_1 /conda-forge
argon2-cffi 20.1.0 py38h1e0a361_1 /conda-forge
arrow-cpp 0.17.1 py38h1234567_11_cpu /conda-forge
View gist:3b71999901af4ca44c14d6d6dc5eac8d
```
(rapidstest) > $ conda list [±4-econ-graph ●●]
# packages in environment at ~/.conda/envs/rapidstest:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge /conda-forge
_openmp_mutex 4.5 1_gnu /conda-forge
aiohttp 3.6.2 py37h516909a_0 /conda-forge
appdirs 1.4.3 py_1 /conda-forge
View Foo.scala
// setup in bash
wget https://repo1.maven.org/maven2/org/apache/flink/flink-connector-kafka_2.11/1.11.1/flink-connector-kafka_2.11-1.11.1.jar -P lib/
wget https://repo1.maven.org/maven2/org/apache/flink/flink-connector-kafka-base_2.11/1.11.1/flink-connector-kafka-base_2.11-1.11.1.jar -P lib/
wget https://repo1.maven.org/maven2/org/apache/kafka/kafka-clients/0.10.2.1/kafka-clients-0.10.2.1.jar -P lib/
wget https://repo1.maven.org/maven2/org/apache/flink/flink-avro-confluent-registry/1.11.1/flink-avro-confluent-registry-1.11.1.jar -P lib/
wget https://repo1.maven.org/maven2/org/apache/flink/flink-avro/1.11.1/flink-avro-1.11.1.jar -P lib/
wget https://repo1.maven.org/maven2/org/apache/flink/force-shading/1.11.1/force-shading-1.11.1.jar -P lib/
wget https://repo1.maven.org/maven2/org/apache/avro/avro/1.8.2/avro-1.8.2.jar -P lib/
wget https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.11.1/jackson-core-2.11.1.jar -P lib/
wget https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-d
@geoHeil
geoHeil / Tweet.scala
Created Jun 29, 2020
new specific type class
View Tweet.scala
/** MACHINE-GENERATED FROM AVRO SCHEMA. DO NOT EDIT DIRECTLY */
package com.github.geoheil.streamingreference
import scala.annotation.switch
final case class Tweet(var tweet_id: Option[String], var text: Option[String], var source: Option[String], var geo: Option[String], var place: Option[String], var lang: Option[String], var created_at: Option[String], var timestamp_ms: Option[String], var coordinates: Option[String], var user_id: Option[Long], var user_name: Option[String], var screen_name: Option[String], var user_created_at: Option[String], var followers_count: Option[Long], var friends_count: Option[Long], var user_lang: Option[String], var user_location: Option[String], var hashtags: Option[Seq[String]]) extends org.apache.avro.specific.SpecificRecordBase {
def this() = this(None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None)
def get(field$: Int): AnyRef = {
(field$: @switch) match {
case 0 => {
View docker-compose.yml
version: "3"
services:
nifi:
image: apache/nifi:1.11.4
ports:
- 8080:8080 # Unsecured HTTP Web Port
environment:
- NIFI_WEB_HTTP_PORT=8080
- NIFI_CLUSTER_IS_NODE=true
- NIFI_CLUSTER_NODE_PROTOCOL_PORT=8082
@geoHeil
geoHeil / docker-compose.yml
Last active Jun 8, 2020
NiFi cannot connect to docker / fails to put record with timeout exception
View docker-compose.yml
version: "3"
services:
zookeeper: # the configuration manager
hostname: zookeeper
container_name: zookeeper
image: 'bitnami/zookeeper:3.6.1'
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
nifi:
image: apache/nifi:1.11.4
@geoHeil
geoHeil / docker-compose.yml
Created Jun 8, 2020
nifi docker volume persistence
View docker-compose.yml
version: "3"
services:
zookeeper: # the configuration manager
hostname: zookeeper
container_name: zookeeper
image: 'bitnami/zookeeper:3.6.1'
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
nifi:
image: apache/nifi:1.11.4
@geoHeil
geoHeil / docker-compose.yml
Created Jun 7, 2020
NiFi & Registry in docker with (not working) persistence
View docker-compose.yml
version: "3"
services:
nifi:
hostname: demo.nifi
image: apache/nifi:1.11.4
container_name: nifi
ports:
- 8080:8080
- 443:8443
links:
View example.py
%pylab inline
import pandas as pd
import geopandas as gp
import seaborn as sns; sns.set()
import numpy as np
from h3 import h3
You can’t perform that action at this time.