Skip to content

Instantly share code, notes, and snippets.

View k0emt's full-sized avatar

Bryan Nehl k0emt

24 followers · 13 following
View GitHub Profile
@k0emt
k0emt / Enron_to_Mongo.py
Last active April 26, 2018 04:05
Convert the Enron email dataset files to a MongoDB. Blog post here: http://soloso.blogspot.com/2011/07/getting-enron-mail-database-into.html Brendan McAdams @rit created a version of the code which utilizes the Python email library to produce a database with more metadata. You can see the results of his work here: http://mongodb-enron-email.s3-w…
import os
import datetime
from pymongo import MongoClient
__author__ = 'k0emt'
MAIL_DIR_PATH = '/Users/k0emt/Projects/enron/enron_mail_20110402/maildir'
PREFIX_TRIM_AMOUNT = len(MAIL_DIR_PATH) + 1
MAX_USER_RUN_LIMIT = 50
MAX_USER_EMAILS_PER_FOLDER_FILE_LIMIT = 2
counter = 1
@k0emt
k0emt / Experiment.py
Last active June 11, 2023 15:41
Basic Hello world in Python with corresponding unittest
__author__ = 'k0emt'
class Greeter:
def __init__(self):
self.message = 'Hello world'
# print self.message
@k0emt
k0emt / Arguments.py
Created September 2, 2011 20:15
Overloading constructor and variable arguments example
__author__ = 'k0emt'
class Greeter:
def __init__(self):
self.message = 'Hello world!'
# print self.message
class GreeterOverloaded:
def __init__(self, greeting=None):
if greeting is None:
@k0emt
k0emt / DarkMatterLogger.py
Created September 15, 2011 04:00
Demo code for RabbitMQ Publish/Subscribe (fanout) exchange with Python
import sys
import pika
# prerequisites are that you have RabbitMQ installed
# create a "darkmatter" named VirtualHost (VHOST)
# rabbitmqctl.bat add_vhost darkmatter
# create a user APP_USER with associated APP_PASS word
# rabbitmqctl add_user darkmatteradmin <password>
# give the APP_USER the necessary permissions
# rabbitmqctl set_permissions -p darkmatter darkmatteradmin ".*" ".*" ".*"
@k0emt
k0emt / pymongo_work_with_document.py
Last active December 12, 2015 01:18
MongoDB, PyMongo example working with returned documents.
from pymongo import MongoClient
import pymongo
import sys
# establish a connection to the database
connection = MongoClient("mongodb://localhost", safe=True)
# get a handle to the school database
db = connection.school
scores = db.scores
@k0emt
k0emt / updating_docs.py
Last active August 14, 2019 20:51
This gist shows how to loop throw and alter/update an individual uniquely identified document.
from pymongo import MongoClient
import sys
# code example to show updating individual records in a loop
# initialize the database with:
# mongoimport -d school -c scores --type json grades.js
# verify no records with "added"
# db.scores.find({"added":{$exists:true}}) // returns nothing
@k0emt
k0emt / cities.xml
Created February 15, 2013 07:44
Example of XSLT Transformation with for-each-grouping. command to create: saxon -xsl:for-each-group.xsl -s:cities.xml > grouped_cities.xml
<cities>
<state>
<name>Alabama</name>
<city>Abbeville</city>
<number>1</number>
</state>
<state>
<name>Alabama</name>
<city>Adamsville</city>
<number>1</number>
@k0emt
k0emt / mongoGetKeys.py
Last active December 13, 2015 22:18
Utility code to get the keys from a document in a specified MongoDB database and collection.
#!/usr/local/bin/python
import sys
from pymongo import MongoClient
if len(sys.argv) < 3:
print 'usage is: ' + sys.argv[0] + ' databaseName collectionName'
sys.exit()
dbName = sys.argv[1]
@k0emt
k0emt / hello_world_bottle.py
Created April 6, 2013 04:50
Example of hello world in bottle with python
import bottle
@bottle.route('/')
def home_page():
return "Hello World\n"
@bottle.route('/testpage')
def test_page():
return "this is a test page"
@k0emt
k0emt / create_replica_set.bat
Last active November 20, 2016 18:16
Helper batch file for starting up a replica set named m101. Drop the --logpath option if you want to enjoy the log text flying by in the console. The --oplogSize option is the magic sauce to avoid creation of numerous 512MB+ files up to 5% of disk space.
mkdir \data\rs1 \data\rs2 \data\rs3
start mongod --replSet m101 --logpath "1.log" --dbpath \data\rs1 --port 27017 --smallfiles --oplogSize 64
start mongod --replSet m101 --logpath "2.log" --dbpath \data\rs2 --port 27018 --smallfiles --oplogSize 64
start mongod --replSet m101 --logpath "3.log" --dbpath \data\rs3 --port 27019 --smallfiles --oplogSize 64