This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import pandas as pd | |
from pytz import datetime | |
import gzip, cPickle | |
from urlparse import parse_qs | |
f = gzip.open('user_query_tuple.data','rb') | |
data_dt = cPickle.load(f) | |
f.close() | |
f = gzip.open('query_dict.data','rb') | |
data = cPickle.load(f) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def doit(text): | |
import re | |
matches=re.findall(r'\"(.+?)\"',text) | |
#matches1= re.findall(r'(.+?)\"',text) | |
# matches is now ['String 1', 'String 2', 'String3'] | |
print matches | |
return ",".join(matches) | |
q='"Terrified, mortified, petrified, stupified by you." said John Nash, "alright"' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<!DOCTYPE html> | |
<html> | |
<head> | |
<script type="text/javascript" src="http://mbostock.github.com/d3/d3.js?2.0.0"></script> | |
<script type="text/javascript" src="http://mbostock.github.com/d3/d3.csv.js?2.0.0"></script> | |
<script type="text/javascript" src="http://mbostock.github.com/d3/d3.time.js?2.0.0"></script> | |
<style type="text/css"> | |
body { | |
font: 10px sans-serif; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# herein we backup our indexes! this script should run at like 6pm or something, after logstash | |
# rotates to a new ES index and theres no new data coming in to the old one. we grab metadatas, | |
# compress the data files, create a restore script, and push it all up to S3. | |
TODAY=`date +"%Y.%m.%d"` | |
INDEXNAME="logstash-$TODAY" # this had better match the index name in ES | |
INDEXDIR="/usr/local/elasticsearch/data/logstash/nodes/0/indices/" | |
BACKUPCMD="/usr/local/backupTools/s3cmd --config=/usr/local/backupTools/s3cfg put" | |
BACKUPDIR="/mnt/es-backups/" | |
YEARMONTH=`date +"%Y-%m"` |
NewerOlder