Instantly share code, notes, and snippets.
Hi ! there seems to be some some missing imports !
e.g where does the spark object comme from ?
otherwise nice gist !
Thanks @aschmu! The variable spark is a default SparkSession. I forgot to mention you should be running Jupyter with a PySpark kernel. I put a sample script on how I do this here https://gist.github.com/BryanCutler/b7f10167c4face19e03330a07b24ce21 in case it could be of help. Thanks for the feedback!!