Skip to content

Instantly share code, notes, and snippets.

View liuminyt's full-sized avatar

liuminyt

View GitHub Profile
@liuminyt
liuminyt / protobuf-250-on-mac.txt
Created January 7, 2022 17:18 — forked from peterroth/protobuf-250-on-mac.txt
How to install protobuf 2.5.0 on Mac (10.15.7, Catalina). brew required!
brew install automake libtool wget
wget https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.bz2
tar -xvjf protobuf-2.5.0.tar.bz2
cd protobuf-2.5.0
./autogen.sh
./configure
make; make check
sudo make install
Validate the successful installation:
@liuminyt
liuminyt / sqoop
Last active January 18, 2016 15:46
sqoop export/import data between hdfs/hive/hbase and MYSQL/Oracle
sqoop // sqoop import data from MYSQL to HDFS/Hive/HBase
import
--connect
jdbc:mysql://localhost/database
--username
root
-password admin
--table mysql1
-m //specify the number of map tasks(parallel processes) to use to perform the import
1
@liuminyt
liuminyt / mysql
Created October 27, 2014 12:56
mysql 语法
show databases; | create database database_name; | drop database database_name;
use database_name;
show tables; | create table table_name(fields_name fields_type primary key/not null/auto_increment, .......); | drop table table_name;
alter table table_name add/modify/change field_name fields_type auto_increment/not null/unique /// first/after fields_name
alter table table_name drop column fields_name;
insert int table_name values("", "", "",);
@liuminyt
liuminyt / mahout
Created October 24, 2014 03:47
mahout a machine learning or stata mining lib
recommender: File-->DateModel-->CachingRecommender-->recommend-->userId-->print
cluster
classify
@liuminyt
liuminyt / gist:ee81a8fd93a6dddf4e08
Created October 11, 2014 02:55
hadoop
fs.append()
fs.create()
files-->FileInputStream-->BufferedInputStream-->DataInputStrem-->program-->
DataOutputStream-->BufferedOutputStream-->FileOutputStream-->files
mapreduce
input-->split-->map-->shuffle-->reduce
client-->jobtracker-->tasktracker-->map or reduce
mapreduce programs Configuration:
Tool ToolRunner:
@liuminyt
liuminyt / gist:94a76fa8cee8189c77b7
Last active October 18, 2015 06:13
git
git init
git add <file>
git commit -m "description"
git status
git reset --hard HEAD^ #git reset --hard commit_id
git reflog
#######################
stage master
git checkout -- <file> discard changes
@liuminyt
liuminyt / Quick Start
Last active August 29, 2015 14:06
HTTP方法（动作）: GET HEAD POST PUT DELETE OPTIONS
静态文件： CSS javascript
渲染模板：GET HEAD POST PUT DELETE OPTIONS CSS javascript
@liuminyt
liuminyt / gist:3abdee3e6f01dc296ebc
Last active August 29, 2015 14:06
flask
windows环境 安装easy_install
#配置虚拟环境
easy_install pip
pip install virtualenv
mkdir newproject
cd newproject
virtualenv nenv
#每次使用项目，必须先激活环境（linux中：$ . venv/bin/activate）
@liuminyt
liuminyt / gist:c02286e1e2ae46e505e8
Last active July 18, 2022 11:21
Unofficial Windows Binaries for Python Extension Packages
setuptools
APSW windows上python数据库操作 conn = aspw.Connection('conn.db')
Backports 升级python新版本
Basemap 2D绘图工具包
bitarry bitarry objects like lists
blist better performance for large lists
blaze next generation of NumPy 分布式数据分析包
blosc 针对二进制数据进行优化的压缩器
bokeh 基于html5的可视化库针对大数据集和流
Boost.Python c++和python 交互包