Skip to content

Instantly share code, notes, and snippets.

@willprice76
Created August 27, 2020 10:42
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save willprice76/eab80ac67c5c01bf29ad055c5bda6c9b to your computer and use it in GitHub Desktop.
Save willprice76/eab80ac67c5c01bf29ad055c5bda6c9b to your computer and use it in GitHub Desktop.
controller endpoint to train sentiment model
@PostMapping("/sentiment/train")
public String train(@RequestBody List<TextData> data) throws IOException {
Instant start = Instant.now();
Dataset<Row> input = spark.createDataFrame(data, TextData.class);
LOG.debug("Running training with {} rows of text data", data.size());
Pipeline pipeline = getSentimentTrainingPipeline();
PipelineModel newPipelineModel = pipeline.fit(input);
long trainingTime = Duration.between(start, Instant.now()).toMillis();
//Overwrite the existing scoring pipeline
scoringPipeline = new LightPipeline(newPipelineModel, false);
return String.format("Training completed in %s milliseconds", trainingTime);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment