Skip to content

Instantly share code, notes, and snippets.

Avatar
💻

Alexander Reelsen spinscale

💻
View GitHub Profile
@spinscale
spinscale / autobahn-api-to-es.cr
Last active Sep 1, 2021
Visualizing the unofficial autobahn API into the Elastic Stack
View autobahn-api-to-es.cr
# autobahn-api-to-es
#
# Indexes autobahn API metadata into Elasticsearch to use it with Kibana Maps
#
# Original API description https://gist.github.com/LilithWittmann/06bd153317b635e7b622651f5cfd95ea
#
#
# MIT License
#
# Copyright (c) [2021]
@spinscale
spinscale / data.json
Created Mar 31, 2021
Daily Elastic Byte - Interesting field types
View data.json
#############################
## unsigned long
#############################
DELETE unsigned_long_test
PUT unsigned_long_test
{
"mappings": {
"properties": {
"my_counter": {
@spinscale
spinscale / data.json
Created Mar 31, 2021
Daily Elastic Byte - Tale of an aggregation
View data.json
# Show the scoreboard in the contributor app
DELETE scoreboard
PUT scoreboard/_bulk?refresh
{"index":{}}
{ "score" : 1, "@timestamp" : "2021-02-28", "email":"peter@example.org", "name" : "Peter Parker"}
{"index":{}}
{ "score" : 4, "@timestamp" : "2021-02-01", "email":"peter@example.org", "name" : "Peter MiddleName Parker"}
{"index":{}}
@spinscale
spinscale / examples.json
Created Mar 18, 2021
Elastic Bytes - rank/distance feature
View examples.json
###################################################
## Boosting scores based on numeric field values ##
###################################################
# Existing solution: function_score query
# Problem: Performance penalty
# Solution: Rescoring, only rescoring top-n documents
# Lucene optimization in Elasticsearch 7: Block MAX WAND
# New query: distance_feature, efficiently skips non-competitive hits
@spinscale
spinscale / runtime-fields.json
Created Mar 17, 2021
Elastic Bytes - Runtime fields
View runtime-fields.json
##################
# Runtime fields #
##################
GET /
DELETE orders
PUT orders
{
@spinscale
spinscale / calculate-vaccinations.sh
Created Jan 27, 2021
Download RKI data and extrapolate full first vaccination
View calculate-vaccinations.sh
#!/bin/bash
# retrieve URL
url="https://www.rki.de/DE/Content/InfAZ/N/Neuartiges_Coronavirus/Daten/Impfquotenmonitoring.xlsx?__blob=publicationFile"
wget -o /dev/null $url -O out.xlsx
# convert to csv
# run 'pip3 install xlsx2csv' (or maybe pip depending on your python installation)
xlsx2csv -a out.xlsx out.csv
@spinscale
spinscale / kibana.snippets
Created Jun 4, 2020
Elasticsearch - Hands-on Introduction (2020-06, DevClash)
View kibana.snippets
###################################
## Introduction to Elasticsearch ##
###################################
#########################
## If you do not want to run Elasticsearch yourself
## Test a free 14 day trial at
## https://www.elastic.co/cloud/
#########################
@spinscale
spinscale / sample.json
Created Mar 4, 2020
Percolator example (training MUC)
View sample.json
DELETE back_in_stock_notifications
DELETE test
PUT test/_bulk?refresh
{ "index" : { "_id" : "1"}}
{ "id": "1", "title" : "T-Shirt", "size" : "XL", "color": "red", "stock" : 0 }
{ "index" : { "_id" : 2}}
{ "id": "2", "title" : "T-Shirt", "size" : "XL", "color": "green", "stock" : 1 }
{ "index" : { "_id" : 3}}
{ "id": "3", "title" : "T-Shirt", "size" : "XL", "color": "yellow", "stock" : 2 }
@spinscale
spinscale / test.sh
Created Feb 28, 2020
Crystal webserver dumping all HTTP requests headers/bodies
View test.sh
crystal eval 'port = 8080 ; require "http/server"; server = HTTP::Server.new do |ctx| req = ctx.request; req.to_io STDOUT ; STDOUT.print "\n\n" if req.body end ; server.listen 8080'
@spinscale
spinscale / sample.json
Created Feb 18, 2020
Inference processor with language identification
View sample.json
POST _ingest/pipeline/_simulate?filter_path=**.predicted_value
{
"pipeline": {
"processors": [
{
"inference": {
"model_id": "lang_ident_model_1",
"inference_config": { "classification": {}},
"field_mappings": {}
}