Skip to content

Instantly share code, notes, and snippets.

@padewitte
Last active November 27, 2018 10:28
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save padewitte/1e159475b0ae7a9b6b8e to your computer and use it in GitHub Desktop.
Save padewitte/1e159475b0ae7a9b6b8e to your computer and use it in GitHub Desktop.
Quick example of a Camel route using the DBCursor outputype with a MongoDB endpoint
/*
* Demonstration of the new MongoDB outypeType attribute in Camel 2.16.
* Priorir to launching script lower the maximum ram available for groovy by setting maximum size to 664M `export JAVA_OPTS="$JAVA_OPTS -Xmx64M"`.
* MongoDB should run localy with no security on test database otherwise change the mongoClient initialization.
* First feed the base by running the script with FEED parameter ( groovy CamelMongoDBCursorExample.groovy FEED )
* Try to list all document by running the script with LIST parameter ( groovy CamelMongoDBCursorExample.groovy LIST ). You should face a OutOfMemory error.
* Run again with CURSOR parameter ( groovy CamelMongoDBCursorExample.groovy CURSOR ) all documents are print in the console.
*
* ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
* "THE BEER-WARE LICENSE" (Revision 42):
* padewitte@gmail.com wrote this file. As long as you retain this notice you can do whatever you want with this stuff. If we meet some day, and you think this stuff is worth it, you can buy me a beer in return<;
* Pierre-Alban
* ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
*/
@Grab('org.apache.camel:camel-core:2.16.0')
@Grab('org.apache.camel:camel-mongodb:2.16.0')
@Grab('org.slf4j:slf4j-simple:1.6.6')
@Grab(group='org.apache.commons', module='commons-lang3', version='3.4')
import org.apache.camel.*
import org.apache.camel.impl.*
import org.apache.camel.builder.*
import org.apache.camel.util.jndi.*
import com.mongodb.Mongo
import com.mongodb.BasicDBObject
import groovy.transform.Field
import org.apache.commons.lang3.RandomStringUtils
@Field def COUNTER = 0;
def jndiContext = new JndiContext();
jndiContext.bind("mongoClient", new Mongo("127.0.0.1"));
def camelContext = new DefaultCamelContext(jndiContext)
def routeBuilder
//A processor printing the end of route and stopping camel context
def endOfRouteProcessor = {exchange -> println "End of route. HIt CTRL + C to exit program. ";}as Processor
//Generate a DBObject with a counter and a random string of 1000 characters
def feedingProcessor = {
exchange -> exchange.getIn().setBody(new BasicDBObject("counter", COUNTER++).append("text",RandomStringUtils.randomAlphabetic((int)1000)));
} as Processor
//Display body in console
def printDocument = { exchange -> println(exchange.getIn().getBody()); } as Processor
if (args.length != 1) {
usage()
} else if (args[0] == "FEED") {
routeBuilder = new RouteBuilder() {
def void configure() {
println "Adding 10000 documents to cameltest collection in db test."
from("timer://trigger?repeatCount=1")
.loop(30000)
.process(feedingProcessor)
.to("mongodb:mongoClient?database=test&collection=cameltest&operation=insert").end().process(endOfRouteProcessor)
}
}
} else if (args[0] == "LIST") {
routeBuilder = new RouteBuilder() {
def void configure() {
println "MongoDB findall with default (List) output type"
from("timer://trigger?repeatCount=1")
.to("mongodb:mongoClient?database=test&collection=cameltest&operation=findAll").split(body()).streaming().process(printDocument).end().process(endOfRouteProcessor)
}
}
} else if (args[0] == "CURSOR") {
routeBuilder = new RouteBuilder() {
def void configure() {
println "MongoDB findall with DBCursor output type"
from("timer://trigger?repeatCount=1").setHeader("CamelMongoDbBatchSize",constant(10))
.to("mongodb:mongoClient?database=test&collection=cameltest&operation=findAll&outputType=DBCursor").split(body()).streaming().process(printDocument).end().process(endOfRouteProcessor)
}
}
} else {
usage()
}
camelContext.addRoutes(routeBuilder)
camelContext.start()
addShutdownHook { camelContext.stop(); }
synchronized (this) {
this.wait()
}
def usage() {
println "Usage CamelMongoDBStream.groovy FEED|LIST|STREAM. http://padewitte.tumblr.com for details."
System.exit(0)
}
@DarkStar1
Copy link

It is not possible to stream from a MongoDB Cursor or iterable in it seems. I have tried using the mongoDB and MongoDB3 components to no avail. (https://stackoverflow.com/q/53467621/107301)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment