How to create a Spark DataFrame from Pandas or NumPy with Arrow

PySpark_createDataFrame_with_Arrow.ipynb

Hi ! there seems to be some some missing imports !
e.g where does the spark object comme from ?
otherwise nice gist !

Thanks @aschmu! The variable spark is a default SparkSession. I forgot to mention you should be running Jupyter with a PySpark kernel. I put a sample script on how I do this here https://gist.github.com/BryanCutler/b7f10167c4face19e03330a07b24ce21 in case it could be of help. Thanks for the feedback!!