Skip to content

Instantly share code, notes, and snippets.

@1ambda

1ambda/spark-jupyter.yaml

Created January 2, 2022 01:11
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save 1ambda/981ce812ed36abcb657aaa401700f30a to your computer and use it in GitHub Desktop.
Save 1ambda/981ce812ed36abcb657aaa401700f30a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
spark-jupyter.yaml
- display_name: "ML Type 1 [ON_DEMAND]"
description: "Type 1 [ON_DEMAND] - Basic (CPU 2-6, Memory 20 GiB)"
kubespawner_override:
image: ...
node_selector:
...
eks.amazonaws.com/capacityType: ON_DEMAND
cpu_limit: 6
cpu_guarantee: 2
mem_limit: '20G'
mem_guarantee: '20G'
environment:
SPARK_OPTS: "--master=local[6] --driver-memory=20g --driver-java-options=-Dlog4j.logLevel=info"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment