Skip to content

Instantly share code, notes, and snippets.

@ericzhong
Last active November 24, 2017 10:02
Show Gist options
  • Save ericzhong/f741dd386e989e91c734e2faab27db89 to your computer and use it in GitHub Desktop.
Save ericzhong/f741dd386e989e91c734e2faab27db89 to your computer and use it in GitHub Desktop.
Pig 安装与使用

安装

环境:

CentOS 7.4
hadoop-2.7.4

下载:

# pig-0.17.0 依赖 hadoop-2.7.x
wget http://apache.mirrors.nublue.co.uk/pig/latest/pig-0.17.0.tar.gz
tar xvf pig-0.17.0.tar.gz
cd pig-0.17.0/

环境变量:

echo "export PATH=`pwd`/bin:\$PATH" | sudo tee /etc/profile.d/pig.sh
source /etc/profile.d/pig.sh

验证是否可用:

pig –help

Pig 有多种执行模式:

pig -x local ...       # 本地模式
pig ...                # mapreduce 模式
pig -x mapreduce ...   # mapreduce 模式

样例:

pig -x local
grunt> A = load '/etc/passwd' using PigStorage(':'); 
grunt> B = foreach A generate $0 as id; 
grunt> dump B; 
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment