Skip to content

Instantly share code, notes, and snippets.

@ngopal
ngopal / generic_csv_to_sqlite_writer.py
Last active December 17, 2019 04:57
quick and dirty script to convert csv to sqlite
import csv, sqlite3, sys
file_to_read = sys.argv[1]
db_name = sys.argv[2]
table_name = sys.argv[3]
manifest = sys.argv[4] # lists header names and data types
con = sqlite3.connect("./"+db_name)
cur = con.cursor()
# Read Manifest
# Make create table query using manifest
@ngopal
ngopal / loading_pretrained_embeddings.py
Created April 25, 2019 00:07
Loading pre-trained vectors into keras models
# The first step is to load the pre-trained vectors into python. The example below uses glove data.
import os
GLOVE_DIR = "/path/to/pretrained/embeddings/glove.6B/"
embeddings_index = {}
f = open(os.path.join(GLOVE_DIR, 'glove.6B.100d.txt'), "r")
for line in f:
values = line.split()
word = values[0]
coefs = np.asarray(values[1:], dtype='float32')
embeddings_index[word] = coefs
@ngopal
ngopal / README
Created October 6, 2018 23:29
lstm toy
# Toy Example of LSTM
## Relevant Links
* https://www.kaggle.com/amirrezaeian/time-series-data-analysis-using-lstm-tutorial
* https://stackoverflow.com/questions/13703720/converting-between-datetime-timestamp-and-datetime64
* https://visualstudiomagazine.com/articles/2014/01/01/how-to-standardize-data-for-neural-networks.aspx
@ngopal
ngopal / stacking_example.py
Created February 24, 2018 22:20 — forked from geffy/stacking_example.py
Stacking example
# -*- coding: utf-8 -*-
"""
Created on Mon Sep 23 23:16:44 2017
@author: Marios Michailidis
This is an example that performs stacking to improve mean squared error
This examples uses 2 bases learners (a linear regression and a random forest)
and linear regression (again) as a meta learner to achieve the best score.
The initial train data are split in 2 halves to commence the stacking.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
{
"created_at": "Mon Jul 17 21:05:24 +0000 2017",
"id": 887055659015032837,
"id_str": "887055659015032837",
"text": "RT @TEN000HOURS: The MIGOS of AAU Basketball... https:\/\/t.co\/h5lGeWHqpu",
"source": "\u003ca href=\"http:\/\/twitter.com\/download\/iphone\" rel=\"nofollow\"\u003eTwitter for iPhone\u003c\/a\u003e",
"truncated": false,
"in_reply_to_status_id": null,
"in_reply_to_status_id_str": null,
"in_reply_to_user_id": null,
####
# Eigens
#####
# How to calculate covariance matrix
# Great video: https://www.youtube.com/watch?v=9B5vEVjH2Pk
dat <- as.matrix(
cbind(c(90, 90, 60, 30, 30),
c(80, 60, 50, 40, 20),
c(40, 80, 70, 70, 70))
@ngopal
ngopal / Exploring SciKit Learn.ipynb
Created May 28, 2017 22:53
Trying AdaBoosted Decision Trees
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@ngopal
ngopal / Data+Scientist+Thinking+Process.ipynb
Created May 23, 2017 18:12
IMBD_notebook example for Tim
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.