Skip to content

Instantly share code, notes, and snippets.

@Miopas
Miopas / source.list
Created February 5, 2018 03:05
A source.list for Ubuntu 12
deb http://mirrors.aliyun.com/ubuntu/ trusty main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ trusty-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ trusty-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ trusty-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ trusty-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty-backports main restricted universe multiverse
@Miopas
Miopas / md5.py
Created May 2, 2018 00:27
get md5 value of a string
import hashlib
a = 'test'
b = hashlib.md5(a.encode('utf8'))
print(b.hexdigest())
@Miopas
Miopas / excel.script
Created May 9, 2018 11:34
Excel 高亮行
Private Sub Worksheet_SelectionChange(ByVal Target As Range)
Cells.Interior.ColorIndex = xlNone
Rows(Target.row).Interior.ColorIndex = 36 '所在行高亮显示
End Sub
@Miopas
Miopas / spider.sh
Last active May 15, 2018 02:17
把一堆url放在文件里用curl爬取网页的超简单脚本。(以及 shell 脚本读文件模板)
set -e -x
while read line; do
read var1 var2 <<< $line # $line is spilited by '\t'
curl $var1 > tmp
iconv -f gbk -t utf8 tmp > pages/$var2 # conv encoding
done < $1
rm tmp
@Miopas
Miopas / install_java.sh
Created May 18, 2018 09:10
install jdk 8 in centos
set -e -x
mkdir /usr/java
chmod 755 /usr/java/
#将下载的JDK rpm包复制至创建的/usr/java目录下
#执行命令进行安装,安装至/usr/java/的新建文件夹jdk1.8.0_111中
cp /usr/local/src/jdk-8u162-linux-x64.rpm /usr/java/
@Miopas
Miopas / .bashrc
Created May 18, 2018 09:54
my bash config
export CLICOLOR=1
export PS1='\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\] [\D{%a} \t] \n@$(ifconfig | grep "inet " | grep -v 127.0.0. | tail -1 | sed "s/netmask.*//" | sed "s/.*inet //" )\$ '
alias ls='ls -lrt'
alias grep='grep --color'
@Miopas
Miopas / url_encoder.py
Last active May 31, 2018 08:24
url encoder
#coding=utf8
import sys
import urllib.parse
import traceback
keyword = '测试'
try:
url = "http:/127.0.0.1:8983/solr/demo/query?q=titile_t:" + urllib.parse.quote(keyword, "utf-8")
print(url)
except:
@Miopas
Miopas / hadoop_sort
Created June 6, 2018 08:53
hadoop streaming 二次排序参数
-partitioner org.apache.hadoop.mapred.lib.KeyFieldBasedPartitioner \
-jobconf stream.num.map.output.key.fields=2 \
-jobconf num.key.fields.for.partition=1
@Miopas
Miopas / hanzi.py
Last active June 25, 2018 04:49
判断字符串是否包含汉字/是否全是汉字
# for python2
def is_hanzi(text):
return all(u'\u4e00' <= char <= u'\u9fff' for char in text.decode('utf8'))
def has_hanzi(text):
return any(u'\u4e00' <= char <= u'\u9fff' for char in text.decode('utf8'))
# for python3
@Miopas
Miopas / hadoop_zip.sh
Created July 2, 2018 05:37
hadoop zip
source ~/.bashrc
input=***
output=***
hadoop fs -rmr ${output}
$HADOOP_HOME/bin/hadoop jar $HADOOP_STREAMING \
-D mapred.reduce.tasks=100 \
-D mapred.job.name="zip" \
-D mapred.job.priority=VERY_HIGH \
-D mapred.map.max.attempts="1" \
-D mapreduce.output.fileoutputformat.compress=true \