Skip to content

Instantly share code, notes, and snippets.

@VikParuchuri
VikParuchuri / sentiment_plot.R
Created June 18, 2012 16:12
Generate Sentiment Plot
term<-c("egypt","jordan","israel","saudi")
term_vec<-foreach(i=1:length(all_score_frames),.combine=rbind) %do%
{
score_row<-rep(0,length(term))
for(z in 1:length(score_row))
{
sel_score<-all_score_frames[[i]][all_score_frames[[i]]$term==term[z],"score"]
sel_score[is.na(sel_score)]<-0
if(length(sel_score)==0)
sel_score<-0
@VikParuchuri
VikParuchuri / gist:3820561
Created October 2, 2012 16:18
Old QuerySignatureV1Handler
class QuerySignatureV1AuthHandler(QuerySignatureHelper, AuthHandler):
"""
Provides Query Signature V1 Authentication.
"""
SignatureVersion = 1
capability = ['sign-v1', 'mturk']
def _calc_signature(self, params, *args):
boto.log.debug('using _calc_signature_1')
@VikParuchuri
VikParuchuri / gist:3820628
Created October 2, 2012 16:21
New QuerySignatureV1AuthHandler
class QuerySignatureV1AuthHandler(QuerySignatureHelper, AuthHandler):
"""
Provides Query Signature V1 Authentication.
"""
SignatureVersion = 1
capability = ['sign-v1', 'mturk']
def _calc_signature(self, params, *args):
from copy import deepcopy
def invert(X):
"""
Invert a matrix X according to gauss-jordan elimination
In gauss-jordan elimination, we perform basic row operations to turn a matrix into
row-echelon form. If we concatenate an identity matrix to our input
matrix during this process, we will turn the identity matrix into our inverse.
X - input list of lists where each list is a matrix row
output - inverse of X
"""
@VikParuchuri
VikParuchuri / textrank.py
Created October 15, 2013 20:45
Implement the textrank algorithm in Python.
import networkx as nx
from nltk.tokenize.punkt import PunktSentenceTokenizer
from sklearn.feature_extraction.text import TfidfTransformer, CountVectorizer
SENTENCES_IN_SUMMARY = 10
MIN_SENTENCE_LENGTH = 50
MAX_SENTENCE_LENGTH = 200
def f7(seq):
seen = set()
from queue.models import Submission
import queue
subs = Submission.objects.filter(queue_name="open-ended", retired=False)
queue_name="open-ended"
for sub in subs:
    qitem=str(sub.id)
    qcount = queue.producer.push_to_queue(queue_name, qitem)
* Install brew packages (brew-packages.txt)
* Setup all packages that need it so that they start at startup
* Install pip (easy_install pip)
* Install virtualenvwrapper (pip install virtualenvwrapper)
* export WORKON_HOME=~/Envs
* mkdir -p $WORKON_HOME
* source /usr/local/bin/virtualenvwrapper.sh
* mkvirtualenv turfly
* export ARCHFLAGS="-Wno-error=unused-command-line-argument-hard-error-in-future";
psql -d postgres
@VikParuchuri
VikParuchuri / gist:1a65b15cb10584568003
Created February 23, 2015 14:52
Convert fields to markdown
import os
import re
import yaml
import textwrap
import html2text
def write_to_yaml(k, v, yaml_data):
if isinstance(v, str):
v = v.strip()
if v.count("\n") == 0 and k not in ["left_text", "initial_display", "hint", "answer", "check_val", "imports", "instructions"]:
@VikParuchuri
VikParuchuri / Rescaling.ipynb
Last active August 29, 2015 14:22
Sample ipython notebook to test rescaling
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
def code_filter(code):
"""
Filters out low quality training code, with an aim to improving benchmark performance. Works best with Python.
"""
# Python keywords
keywords = ["assert", "async", "await", "break", "class", "continue", "def", "del", "elif", "else", "except",
"finally", "for", "from", "global", "if", "import", "in", "is", "lambda", "nonlocal", "not", "or",
"pass", "raise", "return", "try", "while", "with", "yield"]
# This covers single line comments in python + other common languages
comment_symbols = ["#", "//"]