Skip to content

Instantly share code, notes, and snippets.

@nmukerje

nmukerje/Point Jupyter to EMR Cluster

Last active May 23, 2020 21:04
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nmukerje/b9c9c5d2831b45d37de9e0f2b08dc598 to your computer and use it in GitHub Desktop.
Save nmukerje/b9c9c5d2831b45d37de9e0f2b08dc598 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Point Jupyter to EMR Cluster
%%sh
echo '{"kernel_python_credentials" : {"url": "http://<EMR Master node Private IP>:8998/"}, "session_configs":
{"executorMemory": "2g","executorCores": 2,"numExecutors":4}}' > ~/.sparkmagic/config.json
less ~/.sparkmagic/config.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment