Skip to content

Instantly share code, notes, and snippets.

View benwtrent's full-sized avatar
🏠
Working from home

Benjamin Trent benwtrent

🏠
Working from home
View GitHub Profile
@benwtrent
benwtrent / datafeed_config.json
Last active October 15, 2020 12:53
Calculating the percent true and using it in a job
{
"indices" : [
"kibana_sample_data_flights"
],
"query" : {
"bool" : {
"must" : [
{
"match_all" : { }
}
PUT users/_mapping
{
"properties": {
"geo.location.point": {
"type": "geo_point"
}
}
}
POST users/_update_by_query
@benwtrent
benwtrent / es-commands.es
Created June 26, 2020 11:39
helpful actions to support anomalous behavior by geo area
###
# transform definition
###
{
"id" : "count_by_grid_15",
"source" : {
"index" : [
"gtfs*"
],
"query" : {
package org.elasticsearch.benchmark.ml;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Level;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Param;
@benwtrent
benwtrent / recreation_attempt.es
Last active February 26, 2020 17:32
Trying to recreate wierd script bug
PUT programs
{
"mappings": {
"properties": {
"visit_times" : {
"type" : "date",
"null_value" : "0"
}
}
}
@benwtrent
benwtrent / pipeline_and_composite
Created February 11, 2020 14:43
pipeline agg in composite example
GET kibana_sample_data_ecommerce/_search
{
"size": 0,
"aggs": {
"user": {
"composite": {
"sources": [
{
"user": {
"terms": {
@benwtrent
benwtrent / ESModel.py
Last active February 11, 2020 17:53
model transformers
from typing import List
def add_if_exists(d: dict, k: str, v) -> dict:
"""
:param v:
:param k:
:type d: object
"""
if v is not None:
"processors": [
{
{
"drop": {
"if" : "ctx.lat == 0.0"
}
}
},
{
"set": {
@benwtrent
benwtrent / mappings
Last active November 11, 2019 17:24
building out avg price prediction on a house given ashville air bnb listing data
{
"listings-ash" : {
"aliases" : { },
"mappings" : {
"_meta" : {
"created_by" : "ml-file-data-visualizer"
},
"properties" : {
"@timestamp" : {
"type" : "date"
@benwtrent
benwtrent / LogParser.java
Created July 12, 2019 19:37
Java implementation of the Drain algorithm.
import java.util.AbstractMap;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
/**
* This class contains the parsing logic for implementing the DRAIN algorithm.