Skip to content

Instantly share code, notes, and snippets.

View geoHeil's full-sized avatar

geoHeil geoHeil

95 followers · 82 following
View GitHub Profile
@geoHeil
geoHeil / docker-compose.yml
Created June 8, 2020 16:42
nifi docker volume persistence
version: "3"
services:
zookeeper: # the configuration manager
hostname: zookeeper
container_name: zookeeper
image: 'bitnami/zookeeper:3.6.1'
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
nifi:
image: apache/nifi:1.11.4
@geoHeil
geoHeil / dbt_materialize_redpanda.yml
Last active August 8, 2022 15:12
dbt_materialize_redpanda.yml
version: '3.7'
services:
redpanda:
image: docker.vectorized.io/vectorized/redpanda:v22.1.7
container_name: redpanda
hostname: redpanda
command:
- redpanda start
- --overprovisioned
- --smp 1
@geoHeil
geoHeil / sensor.py
Created July 25, 2022 20:51
sensor not resetting example
def make_date_file_sensor_for_asset(j: JobDefinition, resource_defs_for_ssh):
@sensor(
job=j,
name=j.name + "_sensor",
default_status=DefaultSensorStatus.RUNNING,
)
def date_file_sensor(context):
with build_resources(resource_defs_for_ssh) as resources:
ssh = resources.ssh
sftp = ssh.open_sftp()
@geoHeil
geoHeil / docker-compose.yaml
Last active July 14, 2022 07:32
readpanda materialize automated CI setup with user auth
version: '3.7'
# based on
# 1. https://redpandacommunity.slack.com/archives/C01AJDUT88N/p1657529113002319?thread_ts=1657527079.070669&cid=C01AJDUT88N
# 2. https://redpandacommunity.slack.com/archives/C01AJDUT88N/p1652213796745369?thread_ts=1652195821.668589&cid=C01AJDUT88N
services:
redpanda:
image: docker.vectorized.io/vectorized/redpanda:v22.1.4
container_name: redpanda
hostname: redpanda
command:
@geoHeil
geoHeil / stacktrace-geomesa-geospark.scala
Last active February 10, 2022 16:18
geomesa-geospark scala
```
22/02/10 17:07:54 WARN SimpleFunctionRegistry: The function geospark_rs_array replaced a previously registered function.
22/02/10 17:07:54 WARN SimpleFunctionRegistry: The function geospark_rs_normalize replaced a previously registered function.
22/02/10 17:07:54 WARN SimpleFunctionRegistry: The function geospark_st_union_aggr replaced a previously registered function.
22/02/10 17:07:54 WARN SimpleFunctionRegistry: The function geospark_st_envelope_aggr replaced a previously registered function.
22/02/10 17:07:54 WARN SimpleFunctionRegistry: The function geospark_st_intersection_aggr replaced a previously registered function.
22/02/10 17:07:54 WARN SimpleFunctionRegistry: The function st_boundary replaced a previously registered function.
22/02/10 17:07:54 WARN SimpleFunctionRegistry: The function st_coorddim replaced a previously registered function.
22/02/10 17:07:54 WARN SimpleFunctionRegistry: The function st_dimension replaced a previously registered function.
22/02/10 17:07:54 WARN SimpleFunctionReg
@geoHeil
geoHeil / example.py
Last active April 12, 2021 07:54
%pylab inline
import pandas as pd
import geopandas as gp
import seaborn as sns; sns.set()
import numpy as np
from h3 import h3
@geoHeil
geoHeil / foo.py
Last active November 9, 2020 13:34
matrixprofile / stumpy / mulltiple time-series handling / pandas UDF per group (no aggregation to scalar
sure:
%pylab inline
import stumpy
import pandas as pd
import numpy as np
import random
random_seed = 47
@geoHeil
geoHeil / foo.py
Created October 15, 2020 13:20
df['device_id'] = df['device_id'].astype(str).apply(lambda x: x.zfill(2))
df = df.sort_values(['device_id'])
d_r = df.reset_index()
a4_dims = (11.7, 8.27)
fig, ax = pyplot.subplots(figsize=a4_dims)
ax = sns.lineplot(x='index', y='metrik_0', style='device_id', hue='device_id', data=d_r, ax=ax)
import matplotlib.dates as mdates
ax.axvspan(*mdates.datestr2num(['2020-01-04 05:00:00', '2020-01-05 16:00:00']), color='gray', alpha=0.2)
ax.axvspan(*mdates.datestr2num(['2020-01-04 15:00:00', '2020-01-05 00:00:00']), color='gray', alpha=0.2)
@geoHeil
geoHeil / rapids_15_3.8.txt
Created October 1, 2020 12:20
# packages in environment at ~/.conda/envs/rapidstest:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge /conda-forge
_openmp_mutex 4.5 1_gnu /conda-forge
abseil-cpp 20200225.2 he1b5a44_2 /conda-forge
aiohttp 3.6.2 py38h516909a_0 /conda-forge
appdirs 1.4.3 py_1 /conda-forge
argon2-cffi 20.1.0 py38h1e0a361_1 /conda-forge
arrow-cpp 0.17.1 py38h1234567_11_cpu /conda-forge
@geoHeil
geoHeil / gist:3b71999901af4ca44c14d6d6dc5eac8d
Created October 1, 2020 11:26
conda_list.txt
```
(rapidstest) > $ conda list [±4-econ-graph ●●]
# packages in environment at ~/.conda/envs/rapidstest:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge /conda-forge
_openmp_mutex 4.5 1_gnu /conda-forge
aiohttp 3.6.2 py37h516909a_0 /conda-forge
appdirs 1.4.3 py_1 /conda-forge