Skip to content

Instantly share code, notes, and snippets.

View rohit-gupta's full-sized avatar

Rohit Gupta rohit-gupta

View GitHub Profile
@rohit-gupta
rohit-gupta / postgres upsert
Created January 31, 2015 04:40
query to carry out an upsert in postgres
begin;
lock my_table in exclusive mode;
delete from my_table where key='key';
insert into my_table (key, value) values ('key', 'value');
commit;
@rohit-gupta
rohit-gupta / gitconfig
Last active March 25, 2018 11:53
git config
[alias]
history = log --pretty=format:'%C(red) %h %C(yellow) %ad %C(green) %an %C(white) %s' --date=relative
newbranch = checkout -b
@rohit-gupta
rohit-gupta / PythonDeploy
Last active August 29, 2015 14:15
Launch process in background with STDOUT and pid logged to files, later using pid kill said process
python helloworld.py > helloworld.out & echo $! > helloworld.pid
cat helloworld.pid | xargs kill -9
@rohit-gupta
rohit-gupta / install_xkcd_terminal.sh
Created November 24, 2016 09:25
Install a command to display the latest xkcd in your terminal using braille unicode characters
sudo -H pip install drawille
wget https://raw.githubusercontent.com/asciimoo/drawille/master/examples/xkcd.py
mkdir -p ~/.terminalscripts/
mv xkcd.py ~/.terminalscripts/xkcd.py
alias xkcd="python ~/.terminalscripts/xkcd.py"
@rohit-gupta
rohit-gupta / CCES_2016_Topline.R
Created July 13, 2017 10:56
Calculate Topline Voting Intention Figure from CCES-2016
library(survey)
load("~/CCES/CCES2016_Common_FirstRelease.RData")
survObj <- svydesign(ids = ~1, data = x, weights = x$commonweight_post)
prop.table(svytable(~CC16_410a, design = survObj))
@rohit-gupta
rohit-gupta / nltk_sentiment.py
Created September 2, 2017 10:58
NLTK Sentiment Analysis Basics
from nltk.tokenize import sent_tokenize
text = """All of the MSM is fake news at this point. Their blind hatred of Trump has caused them to throw away what credibility they once had.
#DeathToTheMSM"""
sent_tokenize_list = sent_tokenize(text)
for sentence in sent_tokenize_list:
print sentence
@rohit-gupta
rohit-gupta / preprocess_video.sh
Last active September 4, 2017 09:01
Useful commands for video pre-processing pipeline for deep learning
# clip videos and delete original
ffmpeg -hide_banner -y -i clipped.avi -vf "trim=start=0:end=2,setpts=PTS-STARTPTS" -an tiny.avi && rm -rf clipped.avi
# Extract Frames
ffmpeg -i vid4.avi frames/vid4/%04d.jpg -hide_banner
# Download videos from Youtube
youtube-dl --get-filename -o '%(id)s_%(fps)s.%(ext)s' w4JM08PDEng -f 'bestvideo[height<=240]'
@rohit-gupta
rohit-gupta / keras_fit_generator.py
Created September 9, 2017 16:14
How to write a Keras data generator
def generator(features, labels, batch_size):
# Create empty arrays to contain batch of features and labels#
batch_features = np.zeros((batch_size, 64, 64, 3))
batch_labels = np.zeros((batch_size,1))
while True:
for i in range(batch_size):
# choose random index in features
@rohit-gupta
rohit-gupta / generator_model_fit.py
Created September 13, 2017 18:15
Fitting a generator to video data to train a action classification model
# Create generator
train_generator = MSRVTTSequence(train_captions, video_folder=videos_folder, fps_dict=video_fps, tag_dict=tags, batch_size=16)
validation_generator = MSRVTTSequence(validation_captions, video_folder=videos_folder, fps_dict=video_fps, tag_dict=tags, batch_size=16)
from keras.applications.resnet50 import ResNet50
from keras.layers import TimeDistributed, Bidirectional
from keras.layers import Input, LSTM, Dense
from keras.models import Model
from keras.callbacks import CSVLogger, ModelCheckpoint, ReduceLROnPlateau
@rohit-gupta
rohit-gupta / keras_encoder_decoder.py
Created September 15, 2017 06:41
Implementing Seq2Seq in Keras with State Transfer and Teacher Forcing
from keras.models import Model
from keras.layers import Input, LSTM, Dense, TimeDistributed
inputs = Input(batch_shape=(8,16,1024))
layer = LSTM(256, return_state=True, return_sequences=True)
outputs = layer(inputs)
# TODO Add Teacher Forcing
output, state = outputs[0], outputs[1:]
output = LSTM(256)(output, initial_state=state)