Skip to content

Instantly share code, notes, and snippets.

View cozek's full-sized avatar
🤕

Kaushik Amar Das cozek

🤕
View GitHub Profile
def get_gpu_with_highest_free_memory():
"""
Returns the index of the GPU with the highest free memory.
This function sorts the GPUs by PCI bus ID and then returns the index of the GPU with the highest free memory.
Returns:
list: A list containing the index and free memory of the GPU with the highest free memory.
Notes:
@cozek
cozek / emoji.tex
Last active July 8, 2020 12:58
Using Emoji in Latex
\usepackage{scalerel,xparse}
\NewDocumentCommand\emojismile{}{
\scalerel*{
\includegraphics{EmojiFolder/u1F60A.png}
}{X}
}
% Alternative
\NewDocumentCommand\emojismiley{}{
\includegraphics[scale=0.05]{EmojiFolder/u1F60A.png}
@cozek
cozek / split_df.py
Created March 24, 2020 09:16
Split dataframe into train and validation
def split_dataframe(df:pd.DataFrame, train_frac:float, shuffle: bool ):
"""
Splits DataFrame into train and val
Args:
df: DataFrame to split, note: indexes will be reset
train_frac: fraction to use for training
shuffle: Shuffles df if true
Returns:
split_df: DataFrame with splits mentioned in 'split' column
"""
@cozek
cozek / assert_keras_gpu.py
Created October 7, 2019 06:15
Check if keras is using the GPU
# https://stackoverflow.com/questions/44544766/how-do-i-check-if-keras-is-using-gpu-version-of-tensorflow/44547144
# confirm TensorFlow sees the GPU
from tensorflow.python.client import device_lib
assert 'GPU' in str(device_lib.list_local_devices())
# print(device_lib.list_local_devices())
# confirm Keras sees the GPU
from keras import backend
assert len(backend.tensorflow_backend._get_available_gpus()) > 0
@cozek
cozek / tweet_tokenizer.py
Created October 4, 2019 14:07
tweet tokenizing script
"""
Tweet tokenizing script based on the one provided by pennington GloVe coupled with NLTK tweettokenizer
Date: 4th October 2019
"""
import pickle
import sys
import re
import csv
@cozek
cozek / preprocess_twitter.py
Created October 4, 2019 11:16 — forked from ppope/preprocess_twitter.py
FORK: Python version of Ruby script to preprocess tweets for use in GloVe featurization http://nlp.stanford.edu/projects/glove/.
"""
preprocess-twitter.py
python preprocess-twitter.py "Some random text with #hashtags, @mentions and http://t.co/kdjfkdjf (links). :)"
Script for preprocessing tweets by Romain Paulus
with small modifications by Jeffrey Pennington
with translation to Python by Motoki Wu
Translation of Ruby script to create features for GloVe vectors for Twitter data.
@cozek
cozek / sent_to_emb.py
Created September 21, 2019 19:20
sentence_to_embedding
def sent_to_embedding(embedding, data, max_len=None):
'''
Creates fasttext embedding of given list of sentences
input:
embedding: fasttext.model
data: list of sentences
max_len: maximum number of words to consider
returns:
emb_matrix: numpy matrix containing word level embeddings
@cozek
cozek / basic_ann_v1.py
Created June 4, 2019 07:03
A basic python numpy implementation of a neural network
import numpy as np
import matplotlib.pyplot as plt
from pprint import pprint
from tqdm import tqdm
class myNN(object):
def __init__(self):
self.params = None
self.layer_dims = None
@cozek
cozek / build.py
Created January 19, 2019 04:02
Python 3 script for easily running hadoop mapreduce programs
#!/usr/bin/env python3
#DISCLAMER: Provided with no warranty, you are responsible for whatever may befall you
# or your property as a result of using this script.
#You implicitly agree to this by running this script.
#Feel free to improve, modify and distribute
import os
import sys
@cozek
cozek / docker-wordpress.sh
Created October 7, 2018 00:35 — forked from tatemz/docker-wordpress.sh
A quick way to get a WordPress installation up and running with Docker
#!/bin/bash
mkdir wordpress-site && cd wordpress-site
touch docker-compose.yml
cat > docker-compose.yml <<EOL
version: "2"
services:
my-wpdb: