Skip to content

Instantly share code, notes, and snippets.

@apetresc
Last active April 27, 2018 19:18
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save apetresc/ee066b96464900f96927d6736b314e75 to your computer and use it in GitHub Desktop.
Save apetresc/ee066b96464900f96927d6736b314e75 to your computer and use it in GitHub Desktop.
#!/bin/bash
function join_by { local IFS="$1"; shift; echo "$*"; }
SPARK_HADOOP_VERSION=$("$(pwd)/build/mvn" help:evaluate -Dexpression=hadoop.version -Phadoop-2.7 2>/dev/null\
| grep -v "INFO"\
| grep -v "WARNING"\
| tail -n 1)
OPTIONS=""
for option in "$@"
do
echo "$option"
if [[ "$option" = *azure ]]; then
OPTIONS="$OPTIONS -Phadoop-azure"
elif [[ "$option" = *aws ]]; then
OPTIONS="$OPTIONS -Phadoop-aws"
else
echo "Unkown profile $option"
fi;
done
set -x
./dev/make-distribution.sh --name $(join_by - $SPARK_HADOOP_VERSION $@) --tgz -Pnetlib-lgpl -Pyarn -Phive -Phive-thriftserver -Phadoop-2.7 ${OPTIONS}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment