Skip to content

Instantly share code, notes, and snippets.

@benoitdescamps

benoitdescamps/randomSearch_sparkml

Created May 11, 2018 21:41
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save benoitdescamps/38b28c6430d7174ee902528fc10dc13a to your computer and use it in GitHub Desktop.
Save benoitdescamps/38b28c6430d7174ee902528fc10dc13a to your computer and use it in GitHub Desktop.
Download ZIP
code snippets for Hyperparameters (part II): Random Search on Spark
Raw
randomSearch_sparkml
import org.apache.spark.ml.Pipeline
import org.apache.spark.ml.tuning.CrossValidatorModel
import org.apache.spark.ml.param.ParamMap
val pipeline: Pipeline = ...
val paramGrid: Array[ParamMap] = new ParamGridBuilder().
addGrid(...).
addGrid(...).
build
val cv = new CrossValidator().
setEstimator(pipeline).
setEvaluator(...).
setEstimatorParamMaps(paramGrid).
setNumFolds(...).
setParallelism(...)
val bestModel: CrossValidatorModel = cv.fit(training)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment