Skip to content

Instantly share code, notes, and snippets.

View wchan2's full-sized avatar

Will Chan wchan2

View GitHub Profile
@wchan2
wchan2 / .gitignore
Last active May 12, 2016 02:11
Large Flask Applications
__pycache__
*.pyc
@wchan2
wchan2 / README.md
Last active May 8, 2017 08:43
GoLang HTTP Middleware

GoLang HTTP Middleware

Utility functions that can compose http.Handlers and http.HandlerFuncs to create middlewares. Note that when the status code is written to the http.ResponseWriter, the response is immediately flushed regardless whether a handlers further down the middleware chain is writing to the http.ResponseWriter.

Use cases

Some use cases where this may be useful are listed below. Please feel free to comment to add to the list.

  • Requiring authentication before allowing access to the main handler
  • Proxy to different services depending on application logic or migrations
@wchan2
wchan2 / PopularityLeague.java
Last active September 29, 2015 03:55
Filter Link Popularity using Hadoop
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.ArrayWritable;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.NullWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
@wchan2
wchan2 / OrphanPages.java
Last active September 29, 2015 03:55
Orphaned Web Pages Finder using Hadoop
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.NullWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
@wchan2
wchan2 / TopTitleStatistics.java
Last active September 29, 2015 03:55
Top Wikipedia Titles Stats using Hadoop
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.ArrayWritable;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.NullWritable;
@wchan2
wchan2 / TopWords.java
Created September 23, 2015 15:00
Top Words Map Reduce using Hadoop
import java.io.IOException;
import java.lang.Integer;
import java.util.*;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
@wchan2
wchan2 / SelectiveWordCount.java
Last active September 13, 2015 05:57
Selective Word Count
import java.io.File;
import java.io.BufferedReader;
import java.io.FileReader;
import java.lang.reflect.Array;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.*;
public class MP1 {
Random generator;
@wchan2
wchan2 / .gitignore
Last active August 29, 2015 14:26
Dictionary Tree
*.pyc
@wchan2
wchan2 / .gitignore
Last active August 29, 2015 14:26
Custom Python HashMap
*.pyc
@wchan2
wchan2 / .gitignore
Last active August 29, 2015 14:24
Word Frequency Search
*.pyc
.DS_STORE
records.txt
queries.txt