Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save ganeshchand/b4887f0e09e81f510952630985acef5c to your computer and use it in GitHub Desktop.
Save ganeshchand/b4887f0e09e81f510952630985acef5c to your computer and use it in GitHub Desktop.
IntelliJ Out-of-memory Troubleshooting
Check Memory usage:
println(sys.runtime.totalMemory())
println(sys.runtime.maxMemory())
println(sys.runtime.freeMemory())
Often, Scala or Spark program will throw Spark driver and executor will be entirely running inside the JVM that is running your code shown here that creates the SparkContext.
By that time, it's too late to obtain more Java heap than was allocated when the JVM started. You need to add that -Xmx1G arg to the
command IntelliJ uses to launch the JVM that runs your code.
VM Options: -Xmx4G
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment