Skip to content

Instantly share code, notes, and snippets.

View lxxstc's full-sized avatar

Xiaoxu Lv lxxstc

29 followers · 24 following
  • botim.me
  • beijing
View GitHub Profile
@lxxstc
lxxstc / start my gist
Created February 17, 2013 08:51
gist从这里开始
hello this my first gist
@lxxstc
lxxstc / Python Gist
Last active December 13, 2015 20:29
My Python Gist
Just Entry
@lxxstc
lxxstc / HQL.sql
Last active December 14, 2015 01:09
Hive Job With Python Script
add file reduce.py;
FROM (
select *
from
(
select
dep_airport,
arr_airport,
dep_time,
@lxxstc
lxxstc / PyQuery_Test.py
Created February 25, 2013 06:56
PyQuery 使用测试 pyquery是一个python版的html parser，有比较好的Python Style接口
#!/usr/bin/env python
url = 'http://foo.com/bar'
d = pq(url=url)
a_tags = d('a').filter(lambda i, this: pq(this).text() == 'some pattern here'.decode('utf-8'))
urls = filter(lambda x: re.search('edocfare.travelsky.com', x), map(lambda x: x.attrib['href'], a_tags))
print urls
url = urls[0]
@lxxstc
lxxstc / cpu_performace_test.sh
Last active December 14, 2015 23:49
系统性能测试
time echo "scale=5000; a(1)*4" | bc -l > /dev/null
@lxxstc
lxxstc / network.md
Created March 29, 2013 15:47

查看DNS服务

cat /etc/resolv.conf

@lxxstc
lxxstc / Postgresql9.2_Xeon-E5-2630.md
Last active December 15, 2015 15:19

系统参数

  • CPU: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz (文档里面说最好使用gcc >= 4.6)
  • Memory: 64GB 4X
  • Kernel: 3.2.34
  • Issue: CentOS 6.2
  • Postgresql: 9.2
  • GCC: 4.8.0
  • CFLAGS: -march=native -O3 -mavx -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard
  • LDFLAGS: -Wl,--as-needed -Wl,-rpath,'/opt/pg92/lib',--enable-new-dtags
@lxxstc
lxxstc / hadoop_script
Last active December 17, 2015 05:19
hadoop streaming script
hadoop jar ./contrib/streaming/hadoop-streaming-0.20.2-cdh3u4.jar -input /home/xiaoxu.lv/publish_fare/raw_data/blacketermmergeresults_zip_2013-05-10.txt -output /home/xiaoxu.lv/publish_fare/rule_detail/ -mapper extract_raw.py -file extract_raw.py -reducer cat -numReduceTasks 41
hadoop jar ./contrib/streaming/hadoop-streaming-0.20.2-cdh3u4.jar -input /home/xiaoxu.lv/publish_fare/rule_detail/ -output /home/xiaoxu.lv/publish_fare/rule_detail_count_finger/ -mapper "cut -f 1" -reducer "uniq -c" -numReduceTasks 41
@lxxstc
lxxstc / Postgresql-HA.md
Created June 4, 2013 16:07

概念

Write Ahead Log

  • Why? [Write Ahead Log] 减少磁盘IO 做PITR
  • Location: $PGDATA/pg_xlog
  • Size: 16MB determinated at compile time
  • rotation
@lxxstc
lxxstc / Postgresql-HA.md
Last active December 18, 2015 01:58

概念

Write Ahead Log

  • Why? [Write Ahead Log] 减少磁盘IO 做PITR
  • Location: $PGDATA/pg_xlog
  • Size: 16MB determinated at compile time
  • rotation