Skip to content

Instantly share code, notes, and snippets.

View M2shad0w's full-sized avatar
🎯
Focusing

Deyong Zhu M2shad0w

🎯
Focusing
52 followers · 94 following
View GitHub Profile
@M2shad0w
M2shad0w / r.md
Created February 24, 2017 01:45
r install devtools error 
r v3.3

Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  无法载入共享目标对象‘/usr/local/lib/R/3.3/site-library/openssl/libs/openssl.so’：:
  dlopen(/usr/local/lib/R/3.3/site-library/openssl/libs/openssl.so, 6): Symbol not found: _DSA_get0_key
  Referenced from: /usr/local/lib/R/3.3/site-library/openssl/libs/openssl.so
 Expected in: flat namespace
@M2shad0w
M2shad0w / datanode-ref.md
Created February 17, 2017 07:05
hadoop start datanode error

error

27-172.16.10.16-1467200716990:blk_1074065552_324924 to 172.16.10.5:50010 got
java.net.NoRouteToHostException: 没有到主机的路由
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
	at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
	at org.apache.hadoop.hdfs.server.datanode.DataNode$DataTransfer.run(DataNode.java:2144)
@M2shad0w
M2shad0w / ambari-agent-start-error.md
Last active February 16, 2017 10:11
ambari-agent-start-error.md

error

手动起 ambari-agent 遇到如下错误

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/ambari_agent/main.py", line 306, in <module>
    main(heartbeat_stop_callback)
  File "/usr/lib/python2.6/site-packages/ambari_agent/main.py", line 297, in main
    ExitHelper.execute_cleanup()
TypeError: unbound method execute_cleanup() must be called with ExitHelper instance as first argument (got nothing instead)
@M2shad0w
M2shad0w / hive_auto.md
Created November 10, 2016 10:36
hive auto 实例 
HIVE_CMD = """
	hive -e "alter table track_event_data.track_event add partition(dt='%s');"
	"""       
# 在 hive 的 track event 中添加数据
cmd = HIVE_CMD % loop.strftime("%y-%m-%d")
log.info(cmd)
subprocess.check_call(cmd, shell=True)
@M2shad0w
M2shad0w / fix_matlab_error.md
Last active October 21, 2016 15:29
RuntimeError: Python is not installed as a framework 错误解决方案
@M2shad0w
M2shad0w / .md
Created October 12, 2016 06:02
go get package insecure 
go get -u -insecure git.hunliji.cn:3018/root/gosrc.git
@M2shad0w
M2shad0w / pyspark.sh
Created August 18, 2016 03:57
设置pyspark的启动环境
export PYSPARK_PYTHON=/Users/m2shad0w/.virtualenvs/python2env/bin/python
export PYSPARK_DRIVER_PYTHON=ipython
export PYSPARK_DRIVER_PYTHON_OPTS="notebook"
@M2shad0w
M2shad0w / autojump
Last active August 13, 2016 14:50
修复 autojump j 命令不能使用
[[ -s `brew --prefix`/etc/autojump.sh ]] && . `brew --prefix`/etc/autojump.sh
@M2shad0w
M2shad0w / howto-design-an-api.md
Created August 13, 2016 03:53 — forked from notsobad/howto-design-an-api.md

如何设计一个API服务

By @notsobad

认证

api key

url中附加一个key

/api/posts/?key=aasd2323423asdfasdf
@M2shad0w
M2shad0w / HBaseNewAPI.scala
Created August 12, 2016 07:35 — forked from wuchong/HBaseNewAPI.scala
Spark 下 操作 HBase 1.0.0 新版API
import org.apache.hadoop.hbase.util.Bytes
import org.apache.hadoop.hbase.{HColumnDescriptor, HTableDescriptor, TableName, HBaseConfiguration}
import org.apache.hadoop.hbase.client._
import org.apache.spark.SparkContext
import scala.collection.JavaConversions._
/**
* HBase 1.0.0 新版API, CRUD 的基本操作代码示例
**/
object HBaseNewAPI {