Skip to content

Instantly share code, notes, and snippets.

@athoune
athoune / rotatated.py
Last active Aug 29, 2015
Checking logrotate coverage
View rotatated.py
#!/usr/bin/env python
from glob import glob
from fnmatch import fnmatch
from os import walk
from os.path import join
import re
DATE_OR_VERSION = re.compile('\.(\d{4}-\d{2}-\d{2})|(\d+)$')
@athoune
athoune / split.py
Created May 17, 2014
Splitting large mongodb log with mtools.
View split.py
#!/usr/bin/env python
import sys
from mtools.util.logfile import LogFile
log = LogFile(open(sys.argv[1], 'r'))
day = None
out = None
for line in log:
d = line.datetime
@athoune
athoune / gist:5b8598e747598c2e3d29
Last active Aug 29, 2015
Implementing dhash with numpy and skimage.
View gist:5b8598e747598c2e3d29
import numpy as np
from skimage.data import imread
from skimage.transform import resize
from skimage.color import rgb2grey
"""
http://www.hackerfactor.com/blog/?/archives/529-Kind-of-Like-That.html
"""
TWOS = np.array([2 ** n for n in range(7, -1, -1)])
@athoune
athoune / gist:7e9d4eff5dd358d6dafa
Created Jun 4, 2014
Counting IP, and countries
View gist:7e9d4eff5dd358d6dafa
package main
import (
"bufio"
"fmt"
"github.com/nranchev/go-libGeoIP"
"io"
"os"
"regexp"
"time"
@athoune
athoune / Dockerfile
Created Aug 2, 2014
Dockerfile for Fipes
View Dockerfile
FROM debian:7.5
MAINTAINER <Mathieu>
RUN apt-get update
RUN apt-get -q install -y erlang make git
RUN useradd -d /opt/fipes --system fipes
ADD Makefile /opt/fipes/
@athoune
athoune / Dockerfile
Created Aug 12, 2014
Building Influxdb
View Dockerfile
FROM debian:7.5
MAINTAINER mlecarme@bearstech.com
ENV DEBIAN_FRONTEND noninteractive
# Basic needed tools
RUN apt-get -q update
RUN apt-get -q install -y mercurial bzr protobuf-compiler flex bison \
valgrind g++ make autoconf libtool libz-dev libbz2-dev curl \
rpm build-essential git wget gawk procps
View gist:e1ff5fea7d921e2e7c6b
#!/usr/bin/env python
import sys
from tvdb_api import Tvdb
from tvnamer.config_defaults import defaults
from tvnamer.utils import Config, FileParser, FileFinder
from tvnamer.tvnamer_exceptions import (ShowNotFound, SeasonNotFound, EpisodeNotFound,
View gist:97226d094d240a98a642
import json
import redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)
while True:
chan, packet = r.blpop('packetbeat')
packet = json.loads(packet)
if packet['http'] is None:
continue
@athoune
athoune / eth.py
Last active Aug 29, 2015
Finding the first active ethernet interface with ansible
View eth.py
def first_eth(v, **kw):
'''Finding the first active ethernet interface'''
for n in range(10):
name = 'ansible_eth%i' % n
if name not in v:
return
eth = v[name]
if eth['active']:
return eth
@athoune
athoune / gist:7091dbf83cdb46d5a302
Last active Aug 29, 2015
Logstash bulk export
View gist:7091dbf83cdb46d5a302

Massive import of cold data into Kibana

I want to push old logs into Kibana. The logs are classical, one line per event, splitted and gzipped by logrotate. Logstash stdin import module is ugly, and I don't need to install logstash (and java) on the source computer. Netcat and socket import module is what I need.

input {
    tcp {
        host => "my.private.ip"
        port => 4243
 }