Skip to content

Instantly share code, notes, and snippets.

@muxueqz

muxueqz/dpark_mongo.py

Created March 1, 2013 03:08
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save muxueqz/5062200 to your computer and use it in GitHub Desktop.
Save muxueqz/5062200 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
dpark_mongo.py
import pymongo
import dpark
mongodb_host = 'x.x.x.x'
db = pymongo.Connection(host=mongodb_host)
def save_to_mongo(sdict):
result_db = db.log_result.result
result_db.insert(sdict)
db.disconnect()
N = 100000
count = dpark.accumulator(0)
def random_once(*args, **kwrgs):
sdict = {'1':'1'}
save_to_mongo(sdict)
dpark.parallelize(range(0, N), 10).foreach(random_once)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment