Skip to content

Instantly share code, notes, and snippets.

@idiomer
idiomer / termux_mxrig.md
Created April 30, 2021 03:50 — forked from 2niuhe/termux_mxrig.md
install XMRig on termux

Monero XMR mining via termux ;ubuntu

  1. install ubuntu in termux command; (ubuntu is optional)
pkg install update && upgrade
apt install git
apt install wget
apt install proot
@idiomer
idiomer / utils.sh
Created November 24, 2020 08:05
utils.sh
# 删除7天前的文件夹(和文件? [-type f]),这些文件夹以dt=开头
/usr/bin/find /tf/outside/allen.zhou/new_novel_ctr/data/doReader_bookChapterInfo/ -mtime +7 -name 'dt=*' | xargs rm -rf
# pip install sqlalchemy
# pip install pymysql
from sqlalchemy import create_engine
import pandas as pd
def get_mysql_conn(stream_results=True):
user, password, host, port, db = 'user', 'pwd', 'host', "port", 'database'
mysql_engine_str = 'mysql+pymysql://%s:%s@%s:%s/%s' % (user, password, host, port, db)
engine = create_engine(mysql_engine_str, execution_options=dict(stream_results=True))
conn = engine.connect()
@idiomer
idiomer / facets_overview_display.py
Last active August 7, 2020 06:48
用facets_overview查看dataset的概览
import base64
from IPython.core.display import display, HTML
# pip install facets_overview
from facets_overview.generic_feature_statistics_generator import GenericFeatureStatisticsGenerator
def display_overview(df_train, df_test=None):
gfsg = GenericFeatureStatisticsGenerator()
if df_test is not None:
proto = gfsg.ProtoFromDataFrames([{'name': 'train', 'table': df_train},
@idiomer
idiomer / dateUtils.py
Last active January 26, 2021 11:41
python date utils: yesterday, dateAdd, dateSub, dateRange
def get_yesterday(n_days_ago=1, FMT='%Y-%m-%d'):
import datetime
return datetime.datetime.strftime(
datetime.datetime.now() - datetime.timedelta(n_days_ago), FMT
)
def dateAdd(thedate, num, FMT='%Y-%m-%d'):
import datetime
strptime, strftime = datetime.datetime.strptime, datetime.datetime.strftime
@idiomer
idiomer / multiple_read.py
Last active July 17, 2020 03:10
多进程读取多个数据文件
from glob import glob
import multiprocessing
from tqdm import tqdm
import pandas as pd
def json_reader(fname):
df = pd.read_json(fname, lines=True)
return df
@idiomer
idiomer / dictUtils.py
Last active August 18, 2020 09:11
用点来访问dict里面的key
class dotdict(dict):
"""dot.notation access to dictionary attributes"""
__getattr__ = dict.get
__setattr__ = dict.__setitem__
__delattr__ = dict.__delitem__
class TrieTree:
""" 字典树 """
class Node:
# 启用鼠标
set -g mouse on
# 屏幕回滚最大行数
set-option -g history-limit 10000
# 在tmux中按前缀+r可重载当前配置
bind r source-file ~/.tmux.conf \; display "Reloaded!"
@idiomer
idiomer / .gitignore
Last active April 14, 2020 08:08
gitignore for python project
*~
*.swp
*.pyc
*.pyo
__pycache__/
*.bak
*.backup
bak/
backup/
*.tmp
@idiomer
idiomer / unique_everseen.py
Created March 20, 2020 04:16
元素排重,且保持原有顺序
def unique_everseen(iterable, key=None):
"List unique elements, preserving order. Remember all elements ever seen."
# unique_everseen('AAAABBBCCDAABBB') --> A B C D
# unique_everseen('ABBCcAD', str.lower) --> A B C D
from itertools import filterfalse
seen = set()
seen_add = seen.add
if key is None:
for element in filterfalse(seen.__contains__, iterable):