Skip to content

Instantly share code, notes, and snippets.

Avatar
🐌
Thinking....

Andrii Gakhov gakhov

🐌
Thinking....
View GitHub Profile
@gakhov
gakhov / hyperloglog-example.py
Created Jul 30, 2019
Example: How to use HyperLogLog from pdsa Python library
View hyperloglog-example.py
import json
from psda.cardinality.hyperloglog import HyperLogLog
hll = HyperLogLog(precision=10) # 2^{10} = 1024 counters
with open('visitors.txt') as f:
for line in f:
ip = json.loads(line)['ip']
hll.add(ip)
@gakhov
gakhov / gist:5406f6a54f7b286b4e4f02e72a723ac2
Created Apr 23, 2016 — forked from debasishg/gist:8172796
A collection of links for streaming algorithms and data structures
View gist:5406f6a54f7b286b4e4f02e72a723ac2
  1. General Background and Overview
@gakhov
gakhov / find_top_frequencies_using_fft.py
Created Aug 7, 2015
Apply FFT transformation to a signal and find its top frequencies.
View find_top_frequencies_using_fft.py
import heapq
def get_fft(signal, f_s=1.0):
"""Calculate FFT of the signal
"""
fft = np.abs(np.fft.fft(signal))
freq = np.fft.fftfreq(fft.size, 1.0 / f_s)
half_n = np.ceil(fft.size / 2.0)
@gakhov
gakhov / get_growing_and_lowering_intervals.py
Created Aug 7, 2015
Using a window smoothing and local extrema find intervals where curve is growing or lowering.
View get_growing_and_lowering_intervals.py
from scipy.signal import argrelextrema
def get_intervals(local_min, local_max, total):
"""Calculate growing and lowering intervals based on
local max and min. From the definition, growing intervals
start from local minimum, and lowering intervals start from
local maximum.
"""
growing = []
lowering = []
@gakhov
gakhov / find_turning_points.py
Created Aug 7, 2015
Using Ramer-Douglas-Peucker algorithm construct an approximated trajectory and find "valuable" turning points.
View find_turning_points.py
from rdp import rdp
def angle(directions):
"""Return the angle between vectors
"""
vec2 = directions[1:]
vec1 = directions[:-1]
norm1 = np.sqrt((vec1 ** 2).sum(axis=1))
norm2 = np.sqrt((vec2 ** 2).sum(axis=1))
View find_outliers.py
# Find potential outliers in values array
# and visualize them on a plot
def is_outlier(value, p25, p75):
"""Check if value is an outlier
"""
lower = p25 - 1.5 * (p75 - p25)
upper = p75 + 1.5 * (p75 - p25)
return value <= lower or value >= upper
@gakhov
gakhov / munin-apache.conf
Last active Aug 29, 2015
munin 2.0 apache conf
View munin-apache.conf
Alias /munin /var/cache/munin/www
<Directory /var/cache/munin/www>
# Rewrite rules to serve traffic from the root instead of /munin-cgi
RewriteEngine On
# Static files
RewriteRule ^/favicon.ico /var/cache/munin/www/static/favicon.ico [L]
RewriteRule ^/static/(.*) /var/cache/munin/www/static/$1 [L]
# HTML
@gakhov
gakhov / DS18B20-munin.py
Created Jan 26, 2014
munin plugin for DS18B20 sensor
View DS18B20-munin.py
#!/usr/bin/env python
from __future__ import division
import os
import sys
import time
import re
DS18B20_ID = "28-0000055f1c88"
DS18B20_TEMP_RE = re.compile(r't=(?P<temperature>\d+)', re.M)
@gakhov
gakhov / es_backupdaily.sh
Created Oct 29, 2012
Dump items from ES index to json files
View es_backupdaily.sh
#!/bin/bash
# script for backuping ES index to the filesystem
#
# This script gets all records indexed on the specified day
# on SERVER/INDEXNAME and saves them on the filesystem
# as a json file INDEXNAME/BACKUPDIR/BACKUPFILE
# Script also check if mapping and index settings are
# already saved and dump them if not.
#
# Script accept one parameter in format YYYY-MM-DD,