This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# -*- mode: ruby -*- | |
# vi: set ft=ruby : | |
# All Vagrant configuration is done below. The "2" in Vagrant.configure | |
# configures the configuration version (we support older styles for | |
# backwards compatibility). Please don't change it unless you know what | |
# you're doing. | |
Vagrant.configure(2) do |config| | |
# The most common configuration options are documented and commented below. | |
# For a complete reference, please see the online documentation at |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import com.mongodb.spark._ | |
import org.bson.Document | |
import com.mongodb.spark.config._ | |
import org.apache.spark.sql.SQLContext | |
import com.mongodb.spark.sql._ | |
// load the first dataframe "EVAs" | |
val evadf = sqlContext.read.mongo() | |
evadf.printSchema() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package spacewalk; | |
import com.mongodb.spark.api.java.MongoSpark; | |
import com.mongodb.spark.config.WriteConfig; | |
import com.mongodb.spark.rdd.api.java.JavaMongoRDD; | |
import org.apache.spark.SparkConf; | |
import org.apache.spark.api.java.JavaPairRDD; | |
import org.apache.spark.api.java.JavaRDD; | |
import org.apache.spark.api.java.JavaSparkContext; | |
import org.apache.spark.api.java.function.*; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// Usage: | |
// in the mongo shell, | |
// 'use' the database where the city data is loaded | |
// and execute the following lines | |
// | |
// | |
// load( "pathto/function.js"); | |
// db.cities.find().forEach( buildEdges ); | |
// findShortestPath(); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
_id: “900006:140312”, | |
location: { | |
"geometry": { | |
"type": "Point", | |
"coordinates": [125.6, 10.1] | |
} | |
} | |
data: [ | |
{ speed: NaN, time: NaN }, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
for (var i=0; i<1000; i++) { db.blog.insert( { "headline" : i, "date" : i + 1, "section" : i+2 } ); } | |
- db.blog.find( { "headline" : 200 } ) | |
- db.blog.find( { "headline" : 200, section: 50 } ) - which order should the index be in? | |
- db.blog.find( { "headline" : 200 }).sort( { "date" : -1 }) - index? |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#5 Aggregations in 5 Minutes: The Nickel Tour of the Aggregation Framework | |
###Sample Document | |
``` | |
mongos> db.logs.findOne() | |
{ | |
"_id" : ObjectId("556172a53004b760dde8a488"), | |
"d_id" : ObjectId("556172a53004b760dde8a443"), | |
"v" : 6205, | |
"timestamp" : ISODate("3129-12-13T02:03:17.906Z"), |