Skip to content

Instantly share code, notes, and snippets.

View timfeirg's full-sized avatar
🌴
On vacation

timfeirg timfeirg

🌴
On vacation
View GitHub Profile
@timfeirg
timfeirg / scrapyd command line program
Created October 15, 2014 04:18
scrapyd command line program, start scrapyd crawler using a python program
"""
scrapyd commandline interface, for my own project use
"""
import argparse
import requests
import json
import os
@timfeirg
timfeirg / sendcloud.py
Created December 8, 2015 04:01
simple wrapper for sendcloud
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
File: sendcloud
Author: timfeirg
Email: liuyifu@ricebook.com
Github: https://github.com/timfeirg/
Description: sendcloud python wrapper
"""
@timfeirg
timfeirg / db.py
Last active December 19, 2018 08:56
scrapy HBase cache storage
from datetime import datetime
import happybase
import pymysql
from impala.dbapi import connect as hive_connect
from impala.util import as_pandas
from pymysql.cursors import DictCursor
from retrying import retry
from thriftpy.transport import TTransportException
@timfeirg
timfeirg / json_utils.py
Last active March 25, 2018 06:52
the world's only json encoder that handles buffer message (or grpc message, I don't know)
import json
from datetime import datetime
from decimal import Decimal
from flask import Response
from functools import wraps
from google.protobuf.internal import api_implementation
if api_implementation.Type() == 'cpp':
from google.protobuf.pyext.cpp_message import GeneratedProtocolMessageType
@timfeirg
timfeirg / keybase.md
Created May 6, 2018 12:07
keybase.md

Keybase proof

I hereby claim:

  • I am timfeirg on github.
  • I am timfeirg (https://keybase.io/timfeirg) on keybase.
  • I have a public key ASC1DDtmQwwr6tlAaRIYRNTO9bJUpy156TwKlU69uhryVwo

To claim this, I am signing this object:

@timfeirg
timfeirg / delete-old-docker-images.sh
Created June 7, 2018 15:48
delete old docker images
#!/bin/env bash
IFS='
'
long_ago=`date +%Y-%m-%d --date='2 weeks ago'`
for image in $(docker images --format '{{.CreatedAt}}|{{.Repository}}:{{.Tag}}')
do
created=`echo ${image%|*} | grep -oP "^[^\s]+"`
tag=${image#*|}
if [ -z "$created" ] || [ -z "$tag" ]; then
@timfeirg
timfeirg / unused.sh
Created September 25, 2018 11:03
analyse unused class and functions (using ag)
#!/bin/sh
set -e
target_directory=${1-.}
export IFS='
'
for target in $(ag -o '((class|def)\s+(\w+))'); do
target_name=$(echo $target | ag -o '\w+$')
@timfeirg
timfeirg / ceph_alerts.yml
Last active March 18, 2024 05:27
ceph monitoring
apiVersion: 1
groups:
- folder: ceph-alerts
interval: 10s
name: cluster health
orgId: 1
rules:
- annotations:
description: The cluster state has been HEALTH_ERROR for more than 5 minutes.
Please check 'ceph health detail' for more information.