Skip to content

Instantly share code, notes, and snippets.

View M2shad0w's full-sized avatar
🎯
Focusing

Deyong Zhu M2shad0w

🎯
Focusing
52 followers · 94 following
View GitHub Profile
@M2shad0w
M2shad0w / git_config_user_name.md
Created August 2, 2016 01:25
git config user info

用户信息 第一个要配置的是你个人的用户名称和电子邮件地址。这两条配置很重要，每次 Git 提交时都会引用这两条信息，说明是谁提交了更新，所以会随更新内容一起被永久纳入历史记录：

$ git config --global user.name "John Doe"
$ git config --global user.email johndoe@example.com

如果用了 --global 选项，那么更改的配置文件就是位于你用户主目录下的那个，以后你所有的项目都会默认使用这里配置的用户信息。如果要在某个特定的项目中使用其他名字或者电邮，只要去掉 --global 选项重新配置即可，新的设定保存在当前项目的 .git/config 文件里。

@M2shad0w
M2shad0w / hdp2_4bug.md
Created August 2, 2016 01:27
Default Spark env config error in HDP2.4 installation

Default Spark env config error in HDP2.4 installation Default Spark env config error in HDP2.4 installation:

#SPARK_DRIVER_MEMORY="512 Mb"

This line include a wrong config param "512 Mb", it should be "512M" without space. This will cause spark-shell yarn client start error:

Invalid initial heap size.

Suggest to update the template.

@M2shad0w
M2shad0w / HBaseNewAPI.scala
Created August 12, 2016 07:35 — forked from wuchong/HBaseNewAPI.scala
Spark 下 操作 HBase 1.0.0 新版API
import org.apache.hadoop.hbase.util.Bytes
import org.apache.hadoop.hbase.{HColumnDescriptor, HTableDescriptor, TableName, HBaseConfiguration}
import org.apache.hadoop.hbase.client._
import org.apache.spark.SparkContext
import scala.collection.JavaConversions._
/**
* HBase 1.0.0 新版API, CRUD 的基本操作代码示例
**/
object HBaseNewAPI {
@M2shad0w
M2shad0w / howto-design-an-api.md
Created August 13, 2016 03:53 — forked from notsobad/howto-design-an-api.md

如何设计一个API服务

By @notsobad

认证

api key

url中附加一个key

/api/posts/?key=aasd2323423asdfasdf
@M2shad0w
M2shad0w / autojump
Last active August 13, 2016 14:50
修复 autojump j 命令不能使用
[[ -s `brew --prefix`/etc/autojump.sh ]] && . `brew --prefix`/etc/autojump.sh
@M2shad0w
M2shad0w / pyspark.sh
Created August 18, 2016 03:57
设置pyspark的启动环境
export PYSPARK_PYTHON=/Users/m2shad0w/.virtualenvs/python2env/bin/python
export PYSPARK_DRIVER_PYTHON=ipython
export PYSPARK_DRIVER_PYTHON_OPTS="notebook"
@M2shad0w
M2shad0w / .md
Created October 12, 2016 06:02
go get package insecure 
go get -u -insecure git.hunliji.cn:3018/root/gosrc.git
@M2shad0w
M2shad0w / fix_matlab_error.md
Last active October 21, 2016 15:29
RuntimeError: Python is not installed as a framework 错误解决方案
@M2shad0w
M2shad0w / hive_auto.md
Created November 10, 2016 10:36
hive auto 实例 
HIVE_CMD = """
	hive -e "alter table track_event_data.track_event add partition(dt='%s');"
	"""       
# 在 hive 的 track event 中添加数据
cmd = HIVE_CMD % loop.strftime("%y-%m-%d")
log.info(cmd)
subprocess.check_call(cmd, shell=True)
@M2shad0w
M2shad0w / ambari-agent-start-error.md
Last active February 16, 2017 10:11
ambari-agent-start-error.md

error

手动起 ambari-agent 遇到如下错误

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/ambari_agent/main.py", line 306, in <module>
    main(heartbeat_stop_callback)
  File "/usr/lib/python2.6/site-packages/ambari_agent/main.py", line 297, in main
    ExitHelper.execute_cleanup()
TypeError: unbound method execute_cleanup() must be called with ExitHelper instance as first argument (got nothing instead)