Skip to content

Instantly share code, notes, and snippets.

View suvayu's full-sized avatar

Suvayu Ali suvayu

View GitHub Profile
@alexott
alexott / minimial-cedet-config.el
Created October 22, 2012 07:30
Working configuration for CEDET from bzr & GNU Emacs (with working C++ & Java name completion)
;;; minimial-cedet-config.el --- Working configuration for CEDET from bzr
;; Copyright (C) Alex Ott
;;
;; Author: Alex Ott <alexott@gmail.com>
;; Keywords: cedet, C++, Java
;; Requirements: CEDET from bzr (http://cedet.sourceforge.net/bzr-repo.shtml)
;; Do checkout of fresh CEDET, and use this config (don't forget to change path below)
@alexpearce
alexpearce / sqlite3_example.py
Last active August 29, 2015 14:08
Creating and manipulating a run number database.
import sqlite3
import random
# Create the database
con = sqlite3.connect('/path/to/database.db')
con.execute('CREATE TABLE runs (run INTEGER PRIMARY KEY)')
# Insert many runs...
runs = range(int(1e3))
random.shuffle(runs)
@alexpearce
alexpearce / sqlite3_timing.py
Created October 27, 2014 10:44
Timing operations on an SQLite database.
import timeit
import os
DB_PATH = 'database.db'
# Number of runs to generate
NRUNS = int(1e6)
insert_setup = """import sqlite3
con = sqlite3.connect('{0}')
con.execute('CREATE TABLE runs (run INTEGER PRIMARY KEY)')
@yangchenyun
yangchenyun / fetch_kindle.js
Last active February 19, 2023 10:10
Get back my books from Kindle
/*
* @fileoverview Program to free the content in kindle books as plain HTML.
*
* This is largely based on reverse engineering kindle cloud app
* (https://read.amazon.com) to read book data from webSQL.
*
* Access to kindle library is required to download this book.
*/
// The Kindle Compression Module copied from http://read.amazon.com application
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@karpathy
karpathy / pg-pong.py
Created May 30, 2016 22:50
Training a Neural Network ATARI Pong agent with Policy Gradients from raw pixels
""" Trains an agent with (stochastic) Policy Gradients on Pong. Uses OpenAI Gym. """
import numpy as np
import cPickle as pickle
import gym
# hyperparameters
H = 200 # number of hidden layer neurons
batch_size = 10 # every how many episodes to do a param update?
learning_rate = 1e-4
gamma = 0.99 # discount factor for reward

Generating Flame Graphs for Apache Spark

Flame graphs are a nifty debugging tool to determine where CPU time is being spent. Using the Java Flight recorder, you can do this for Java processes without adding significant runtime overhead.

When are flame graphs useful?

Shivaram Venkataraman and I have found these flame recordings to be useful for diagnosing coarse-grained performance problems. We started using them at the suggestion of Josh Rosen, who quickly made one for the Spark scheduler when we were talking to him about why the scheduler caps out at a throughput of a few thousand tasks per second. Josh generated a graph similar to the one below, which illustrates that a significant amount of time is spent in serialization (if you click in the top right hand corner and search for "serialize", you can see that 78.6% of the sampled CPU time was spent in serialization). We used this insight to spee

@apehex
apehex / fetch_kindle.js
Last active July 7, 2020 14:47 — forked from yangchenyun/fetch_kindle.js
Get back my books from Kindle
/*
* @fileoverview Program to free the content in kindle books as plain HTML.
*
* This is largely based on reverse engineering kindle cloud app
* (https://read.amazon.com) to read book data from webSQL.
*
* Access to kindle library is required to download this book.
*/
// The Kindle Compression Module copied from http://read.amazon.com application
@vingkan
vingkan / activity.md
Last active July 5, 2021 17:53
Ethical CS: Quantitative Input Influence Activity

Algorithmic Audit: QII

A big moving company gets so many applications that it has started using an automated algorithm to decide who to hire. You have been called in as an independent consultant to determine if the hiring algorithm is biased against women. The algorithm is proprietary so you cannot access its source code. Instead, you will learn how to perform an algorithmic audit to measure potential biases.

In this activity, you will edit the influence.py module.

Applicant Data

Each applicant's data is stored as a list with five elements. Each element is a string representing a different attribute: