Skip to content

Instantly share code, notes, and snippets.

@tzolov
Last active May 9, 2018 07:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tzolov/ce73902f7d2db897caeddace3c8e8625 to your computer and use it in GitHub Desktop.
Save tzolov/ce73902f7d2db897caeddace3c8e8625 to your computer and use it in GitHub Desktop.
mkdir /tmp/twitter-sentiment
cd /tmp/twitter-sentiment
# Download SCDF 1.5.0.RC1 server and shell
wget https://repo.spring.io/milestone/org/springframework/cloud/spring-cloud-dataflow-server-local/1.5.0.RC1/spring-cloud-dataflow-server-local-1.5.0.RC1.jar
wget https://repo.spring.io/milestone/org/springframework/cloud/spring-cloud-dataflow-shell/1.5.0.RC1/spring-cloud-dataflow-shell-1.5.0.RC1.jar
# Download the pre-build twitter-sentiment-processor-rabbit-2.0.0.BUILD-SNAPSHOT.jar and twitter-sentiment-processor-rabbit-2.0.0.BUILD-SNAPSHOT-metadata.jar
wget https://www.dropbox.com/s/0praocvbwhqk6uq/twitter-sentiment-processor-rabbit-2.0.0.BUILD-SNAPSHOT.jar
wget https://www.dropbox.com/s/qewy9zwx1743w6z/twitter-sentiment-processor-rabbit-2.0.0.BUILD-SNAPSHOT-metadata.jar
# Start SCDF server
java -jar ./spring-cloud-dataflow-server-local-1.5.0.RC1.jar
# In annother terminal start
java -jar ./spring-cloud-dataflow-shell-1.5.0.RC1.jar
# Import latest (Darwin-RC1) spring cloud stream apps
dataflow:>app import --uri http://bit.ly/Darwin-RC1-stream-applications-rabbit-maven
# Replace the twitter-sentiment with the custom version downloaded above.
dataflow:>app unregister --type processor --name twitter-sentiment
dataflow:>app register --type processor --name twitter-sentiment --uri file://tmp/twitter-sentiment/twitter-sentiment-processor-rabbit-2.0.0.BUILD-SNAPSHOT.jar --metadata-uri file://tmp/twitter-sentiment/twitter-sentiment-processor-rabbit-2.0.0.BUILD-SNAPSHOT-metadata.jar
# Verify that the twitter-sentiment is registered and available
dataflow:>app info --id processor:twitter-sentiment
# Create stream
dataflow:>stream create --name sentiment --definition "twitterstream --access-token-secret=<your token secret> --access-token=<your access token> --consumer-secret=<your consumer secret> --consumer-key=<your consumer key> --track=java --stream-type=filter | filter --expression='#jsonPath(new String(payload),''$.lang'')==''en'' ' | twitter-sentiment --vocabulary='http://dl.bintray.com/big-data/generic/vocab.csv' --output-name='output/Softmax' --model-fetch='output/Softmax' --model='http://dl.bintray.com/big-data/generic/minimal_graph.proto' | log"
# Deploy the stream, channaling all logs to the SCDF server console.
dataflow:>stream deploy --name sentiment --properties "deployer.*.local.inheritLogging=true"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment