Skip to content

Instantly share code, notes, and snippets.

Avatar
🤖
Too much to do, too little time.

Gaurav Menghani reddragon

🤖
Too much to do, too little time.
View GitHub Profile
@reddragon
reddragon / logs.sh
Created Mar 6, 2012
Use it for finding the total CPU time taken by tasks on Hadoop, after making the change.
View logs.sh
#!/bin/bash
LOGS_PATH="logs/userlogs/"
declare JOB=`ls $LOGS_PATH`;
JOB_PATH="$LOGS_PATH$JOB";
COMMAND="ls -1 --hide=job* $JOB_PATH/";
declare RET=`$COMMAND`;
`rm cputime.out`;
`touch cputime.out`;
for TASK in $RET
do
@reddragon
reddragon / libevent-demo.cpp
Last active Jun 11, 2021
A demo for libevent usage
View libevent-demo.cpp
#include <iostream>
#include <cstdio>
#include <vector>
#include <event2/event.h>
#include <glog/logging.h>
#include <cassert>
#include <string>
#include <cstring>
#include <sys/socket.h>
#include <netinet/in.h>
@reddragon
reddragon / Wordcount.java
Created Jan 29, 2012
Hadoop Word Count Example
View Wordcount.java
import java.io.IOException;
import java.util.StringTokenizer;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
@reddragon
reddragon / transfer-learning.md
Created Apr 2, 2018
Transfer Learning Papers
View transfer-learning.md

How transferable are features in deep neural networks? - Yosinski et al.

  • Transfer Learning

    • Train on a base network, try to take that network and tweak it to work for a new target network.
    • Notes from CS231N.
  • Tries to figure out how much information can we transfer between networks trained on different datasets.

  • Quantifies the transferability by layer.

  • Hypothesis:

    • First few layers are general (Gabor Filters kind of features) and can adapt well.
View l2_lambda.md

Results on MNIST

Feed Forward model with two hidden layers (300, 60).

l2_lambda Accuracy@1 after 80k iters (Two Runs)
0.00 98.15, 98.04
0.01 98.31, 98.19
0.02 98.19, 98.15
0.04 97.93, 97.92
View mnist-pytorch.py
import torch
import torchvision
import torch.nn as nn
import torch.nn.functional as F
import torchvision.transforms as transforms
import matplotlib.pyplot as plt
import numpy as np
import torch.optim as optim
from torch.autograd import Variable
@reddragon
reddragon / struct.cpp
Last active Sep 12, 2017
Set struct members inline
View struct.cpp
#include <iostream>
using namespace std;
struct Foo {
int a;
double b;
};
int main() {
const Foo f = {
@reddragon
reddragon / frozen-lake-nn.py
Created Jun 12, 2017
Frozen Lake NN Implementation
View frozen-lake-nn.py
import gym
import logging
import sys
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import cPickle as pickle
import os
@reddragon
reddragon / frozen-lake-iterative.py
Created Jun 11, 2017
Frozen Lake solved using the Q-Learning algorithm with an actual Q-value table
View frozen-lake-iterative.py
import gym
import logging
import sys
import numpy as np
from gym import wrappers
SEED = 0
NUM_EPISODES = 3000
# Hyperparams
@reddragon
reddragon / script.py
Created May 29, 2017
Get all of those graduation pics
View script.py
import os
import sys
import urllib2
def normalize_path(path):
if path[-1] == '/':
path = path[:-1]
return path
def get_dir_name(path):