Skip to content

Instantly share code, notes, and snippets.

@Leemoonsoo
Created June 23, 2015 19:34
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Leemoonsoo/d3b255c0d3f66cb934d5 to your computer and use it in GitHub Desktop.
Save Leemoonsoo/d3b255c0d3f66cb934d5 to your computer and use it in GitHub Desktop.
Read spark default in Apache Zeppelin. place this scripts inside of your conf/zeppelin-env.sh
function readSparkConf() {
SPARK_CONF_PATH="${1}"
echo "Reading ${SPARK_CONF_PATH}"
while read line; do
echo "${line}" | grep -e "^spark[.]" > /dev/null
if [ $? -ne 0 ]; then
# skip the line not started with 'spark.'
continue;
fi
SPARK_CONF_KEY=`echo "${line}" | sed -e 's/\(^spark[^ ]*\)[ \t]*\(.*\)/\1/g'`
SPARK_CONF_VALUE=`echo "${line}" | sed -e 's/\(^spark[^ ]*\)[ \t]*\(.*\)/\2/g'`
export ZEPPELIN_JAVA_OPTS+=" -D${SPARK_CONF_KEY}=\"${SPARK_CONF_VALUE}\""
done < ${SPARK_CONF_PATH}
}
readSparkConf /your/apth/to/spark
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment