Skip to content

Instantly share code, notes, and snippets.

@RyanRosario
Created May 23, 2019 08:02
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save RyanRosario/7ef4eef7a1c780eb541a698180876938 to your computer and use it in GitHub Desktop.
Save RyanRosario/7ef4eef7a1c780eb541a698180876938 to your computer and use it in GitHub Desktop.
from __future__ import print_function
from pyspark import SparkConf, SparkContext
from pyspark.sql import SQLContext
def main(context):
"""Main function takes a Spark SQL context."""
# YOUR CODE HERE
# YOU MAY ADD OTHER FUNCTIONS AS NEEDED
if __name__ == "__main__":
conf = SparkConf().setAppName("CS143 Project 2B")
conf = conf.setMaster("local[*]")
sc = SparkContext(conf=conf)
sqlContext = SQLContext(sc)
sc.addPyFile("cleantext.py")
main(sqlContext)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment