Skip to content

Instantly share code, notes, and snippets.

@prasincs

prasincs/qubole_snippet

Created November 16, 2017 21:21
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save prasincs/0f057e9cd27185c2163d94c267838737 to your computer and use it in GitHub Desktop.
Save prasincs/0f057e9cd27185c2163d94c267838737 to your computer and use it in GitHub Desktop.
Download ZIP
Quark Examples
Raw
qubole_snippet
[qubole-example]
deployment_environment: qubole
master: https://api.qubole.com/api/v1.2
# custom remote services are prefixed with <name>- they're only looked at
# if the deployment_environment matches the prefix
# To be filled by environment variables
qubole-auth_token: %(QUBOLE_API_TOKEN)s
qubole-cluster_id: 1111
# This needs to be changed at some point
qubole-py_files: /tmp/mist.zip
qubole-cluster_label: mist-us-east-1-spark
qubole-update_dependencies_script: s3://path/to/script/update_files.sh
qubole-config_path: /mist/spark-dev/config/config_staging.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment