Skip to content

Instantly share code, notes, and snippets.

@asgillmor

asgillmor/spark_context_fixture.py

Last active March 6, 2019 11:37
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save asgillmor/4d3ea1693084336b59bf8aa360d20c5c to your computer and use it in GitHub Desktop.
Save asgillmor/4d3ea1693084336b59bf8aa360d20c5c to your computer and use it in GitHub Desktop.
Download ZIP
Raw
spark_context_fixture.py
@pytest.fixture(scope='session')
def spark_context():
conf = SparkConf().setMaster('local[2]').setAppName('pytest-pyspark-local-testing')
sc = SparkContext(conf=conf)
yield sc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment