Skip to content

Instantly share code, notes, and snippets.

@dvainrub

dvainrub/pyspark-hello-world.py

Last active October 8, 2022 08:11
Show Gist options
  • Save dvainrub/b6178dc0e976e56abe9caa9b72f73d4a to your computer and use it in GitHub Desktop.
Save dvainrub/b6178dc0e976e56abe9caa9b72f73d4a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
pyspark-hello-world.py
from pyspark.sql import SparkSession
def init_spark():
spark = SparkSession.builder.appName("HelloWorld").getOrCreate()
sc = spark.sparkContext
return spark,sc
def main():
spark,sc = init_spark()
nums = sc.parallelize([1,2,3,4])
print(nums.map(lambda x: x*x).collect())
if __name__ == '__main__':
main()
@VijaybabuNakkonda
Copy link

I faced the same issue. But with Python3 the code is working fine. Not sure how to manage. Please let me know if you found a solution

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment