Skip to content

Instantly share code, notes, and snippets.

View athoune's full-sized avatar

Mathieu Lecarme athoune

View GitHub Profile
@athoune
athoune / rotatated.py
Last active August 29, 2015 13:55
Checking logrotate coverage
#!/usr/bin/env python
from glob import glob
from fnmatch import fnmatch
from os import walk
from os.path import join
import re
DATE_OR_VERSION = re.compile('\.(\d{4}-\d{2}-\d{2})|(\d+)$')
@athoune
athoune / split.py
Created May 17, 2014 13:04
Splitting large mongodb log with mtools.
#!/usr/bin/env python
import sys
from mtools.util.logfile import LogFile
log = LogFile(open(sys.argv[1], 'r'))
day = None
out = None
for line in log:
d = line.datetime
@athoune
athoune / gist:5b8598e747598c2e3d29
Last active August 29, 2015 14:01
Implementing dhash with numpy and skimage.
import numpy as np
from skimage.data import imread
from skimage.transform import resize
from skimage.color import rgb2grey
"""
http://www.hackerfactor.com/blog/?/archives/529-Kind-of-Like-That.html
"""
TWOS = np.array([2 ** n for n in range(7, -1, -1)])
@athoune
athoune / gist:7e9d4eff5dd358d6dafa
Created June 4, 2014 09:18
Counting IP, and countries
package main
import (
"bufio"
"fmt"
"github.com/nranchev/go-libGeoIP"
"io"
"os"
"regexp"
"time"
@athoune
athoune / Dockerfile
Created August 2, 2014 14:01
Dockerfile for Fipes
FROM debian:7.5
MAINTAINER <Mathieu>
RUN apt-get update
RUN apt-get -q install -y erlang make git
RUN useradd -d /opt/fipes --system fipes
ADD Makefile /opt/fipes/
@athoune
athoune / Dockerfile
Created August 12, 2014 21:23
Building Influxdb
FROM debian:7.5
MAINTAINER mlecarme@bearstech.com
ENV DEBIAN_FRONTEND noninteractive
# Basic needed tools
RUN apt-get -q update
RUN apt-get -q install -y mercurial bzr protobuf-compiler flex bison \
valgrind g++ make autoconf libtool libz-dev libbz2-dev curl \
rpm build-essential git wget gawk procps
#!/usr/bin/env python
import sys
from tvdb_api import Tvdb
from tvnamer.config_defaults import defaults
from tvnamer.utils import Config, FileParser, FileFinder
from tvnamer.tvnamer_exceptions import (ShowNotFound, SeasonNotFound, EpisodeNotFound,
import json
import redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)
while True:
chan, packet = r.blpop('packetbeat')
packet = json.loads(packet)
if packet['http'] is None:
continue
@athoune
athoune / eth.py
Last active August 29, 2015 14:07
Finding the first active ethernet interface with ansible
def first_eth(v, **kw):
'''Finding the first active ethernet interface'''
for n in range(10):
name = 'ansible_eth%i' % n
if name not in v:
return
eth = v[name]
if eth['active']:
return eth
@athoune
athoune / gist:7091dbf83cdb46d5a302
Last active August 29, 2015 14:07
Logstash bulk export

Massive import of cold data into Kibana

I want to push old logs into Kibana. The logs are classical, one line per event, splitted and gzipped by logrotate. Logstash stdin import module is ugly, and I don't need to install logstash (and java) on the source computer. Netcat and socket import module is what I need.

input {
    tcp {
        host => "my.private.ip"
        port => 4243
    }