Skip to content

Instantly share code, notes, and snippets.

@hsiaoting

hsiaoting/stream-hadoop

Created January 14, 2013 16:59
Show Gist options
  • Save hsiaoting/4531507 to your computer and use it in GitHub Desktop.
Save hsiaoting/4531507 to your computer and use it in GitHub Desktop.
Download ZIP
hadoop stream test...
Raw
stream-hadoop
$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/hadoop-streaming.jar \
-input "/user/me/samples/cachefile/input.txt" \
-mapper "xargs cat" \
-reducer "cat" \
-output "/user/me/samples/cachefile/out" \
-cacheArchive 'hdfs://hadoop-nn1.example.com:8020/user/me/samples/cachefile/cachedir.jar#testlink' \
-jobconf mapred.map.tasks=1 \
-jobconf mapred.reduce.tasks=1 \
-jobconf mapred.job.name="Experiment"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment