Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
set -ex
# update/upgrade and install htop
sudo apt-get update -y && sudo apt-get upgrade -y
sudo apt-get install htop
# install required python packages
python3 -m pip install --user --upgrade pip
python3 -m pip install -r requirements.txt --upgrade
# download latest postgres driver jar
if [ -f "$POSTGRES_JAR" ]; then
echo "$POSTGRES_JAR exist"
wget -nv "${POSTGRES_JAR}"
# spark-submit logging level from INFO to WARN
sudo cp /usr/local/spark/conf/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.