Skip to content

Instantly share code, notes, and snippets.

@Kornel

Kornel/spark-mahout.sh

Created October 6, 2015 09:40
Show Gist options
  • Save Kornel/084fe23e4e961c990cc5 to your computer and use it in GitHub Desktop.
Save Kornel/084fe23e4e961c990cc5 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
spark-mahout.sh
#!/bin/bash
MAHOUT_HOME=$PWD/mahout-mahout-0.11.0
#export MAHOUT_LOCAL=$PWD/mahout-mahout-0.11.0
SPARK_HOME=/usr/lib/spark1.5/
INPUT=/projects/reco_prod/data/sessions-cf/second/
OUTPUT=/projects/reco_prod/data/sessions-cf/similarity
EXECUTOR_MEM=14g
$MAHOUT_HOME/bin/mahout spark-itemsimilarity --input $INPUT --output $OUTPUT -sem $EXECUTOR_MEM -ma yarn-client -D:spark.yarn.queue=reco_prod
#$MAHOUT_HOME/bin/mahout spark-itemsimilarity
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment