Skip to content

Instantly share code, notes, and snippets.

@DeanThompson
DeanThompson / fast_run.log
Created August 6, 2021 06:45
ClickHouse trace logs
[ym7201] 2021.08.06 14:43:46.422434 [ 467350 ] {e1544413-2398-4538-aab5-3d1348590ff3} <Debug> executeQuery: (from 172.16.24.44:59090, user: readonly, using production parser) SELECT count() FROM t_table_all WHERE (month >= '2019-06-01') AND (month <= '2021-06-01') AND category_id_3 IN ('YM-01-09-04', 'YM-01-09-07', 'YM-01-09-02', 'YM-01-09-06', 'YM-01-09-03', 'YM-02-04-02', 'YM-02-04-07', 'YM-02-04-05', 'YM-02-05-00', 'YM-01-09-00', 'YM-05-05-01', 'YM-04-03-04', 'YM-01-03-04', 'YM-02-01-07', 'YM-02-01-06', 'YM-02-01-12', 'YM-01-03-02', 'YM-04-03-09', 'YM-01-04-02', 'YM-02-01-05', 'YM-04-04-01', 'YM-01-08-05', 'YM-02-01-09', 'YM-01-03-00', 'YM-02-01-10', 'YM-02-04-04', 'YM-07-05-00', 'YM-01-02-16', 'YM-01-02-14', 'YM-01-02-13', 'YM-01-02-12', 'YM-01-02-17', 'YM-01-02-15', 'YM-01-02-02', 'YM-01-02-04', 'YM-01-02-07', 'YM-01-02-00', 'YM-01-02-03', 'YM-01-02-05', 'YM-01-02-09', 'YM-01-09-01', 'YM-01-06-09', 'YM-01-06-42', 'YM-01-06-11', 'YM-01-06-07', 'YM-01-06-21', 'YM-01-06-15', 'YM-01-06-16', 'YM-01-06-20', 'Y
This file has been truncated, but you can view the full file.
╰─➤ pip install --no-cache --verbose -U fastavro
Using pip 20.2.2 from /home/ymetl/.pyenv/versions/3.6.1/envs/weibo/lib/python3.6/site-packages/pip (python 3.6)
Non-user install because user site-packages disabled
Created temporary directory: /tmp/pip-ephem-wheel-cache-93qbizrt
Created temporary directory: /tmp/pip-req-tracker-nsdxt48r
Initialized build tracking at /tmp/pip-req-tracker-nsdxt48r
Created build tracker: /tmp/pip-req-tracker-nsdxt48r
Entered build tracker: /tmp/pip-req-tracker-nsdxt48r
Created temporary directory: /tmp/pip-install-1qqjonnv
Looking in indexes: http://mirrors.aliyun.com/pypi/simple, http://pypi.yimian.com.cn/simple
@DeanThompson
DeanThompson / migrate_clickhouse.py
Last active August 21, 2020 15:49
Simple Python script to migrate data from one ClickHouse cluster to another.
import collections
import datetime
import functools
import logging
import time
from clickhouse_driver import Client
source_conn = Client(host='source-host', user='user', password='password')
target_conn = Client(host='target-host', user='user', password='password')
#!/usr/bin/env python
import sys, os, time, atexit
from signal import SIGTERM
class Daemon:
"""
A generic daemon class.
Usage: subclass the Daemon class and override the run() method
#!/bin/sh
git filter-branch --env-filter '
OLD_EMAIL="your-old-email@example.com"
CORRECT_NAME="Your Correct Name"
CORRECT_EMAIL="your-correct-email@example.com"
if [ "$GIT_COMMITTER_EMAIL" = "$OLD_EMAIL" ]
then
@DeanThompson
DeanThompson / install_gevent.sh
Created December 23, 2015 12:51
Command to install gevent using pip
CFLAGS='-std=c99' pip install gevent
@DeanThompson
DeanThompson / setup_vim.sh
Last active August 31, 2016 03:37
Shell script to setup vim configure.
#!/bin/bash
cd ~
# Backup your own vim files.
mv .vim .vim.bak
mv .vimrc .vimrc.bak
# Clone this repository
git clone https://github.com/DeanThompson/vimfiles.git .vim
@DeanThompson
DeanThompson / haversine.py
Last active July 29, 2021 14:18
Haversine Formula in Python (Bearing and Distance between two GPS points)
from math import radians, cos, sin, asin, sqrt
def haversine(lon1, lat1, lon2, lat2):
"""
Calculate the great circle distance between two points
on the earth (specified in decimal degrees)
"""
# convert decimal degrees to radians
lon1, lat1, lon2, lat2 = map(radians, [lon1, lat1, lon2, lat2])
@DeanThompson
DeanThompson / load_outfile.sql
Created July 20, 2015 06:28
Load data from outfile in mysql server.
LOAD DATA INFILE 'path/to/file.txt' REPLACE INTO TABLE tablename CHARACTER SET UTF8 FIELDS TERMINATED BY '|' IGNORE 1 LINES;
@DeanThompson
DeanThompson / youdao.py
Last active July 30, 2023 16:36
命令行版本的有道词典
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
命令行版本的有道词典,调用有道翻译的 API 实现
用法一:
% python youdao.py search
原文: search
发音: sɜːtʃ