Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
from pyspark.sql import SparkSession
from tensorflowonspark import TFCluster, TFNode
spark = SparkSession \
.builder \
.config("...")
.appName("model-training") \
.getOrCreate()
spark.sparkContext.addPyFile("/usr/local/tensorflow/tfspark-{version}.zip")
cluster = TFCluster.run(spark.sparkContext,\
map_fun,\
args, \
num_executors=num_executors, \
num_ps=num_ps,\
tensorboard=False,\
driver_ps_nodes=False,\
input_mode=TFCluster.InputMode.SPARK)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.