Skip to content

Instantly share code, notes, and snippets.

View daviddao's full-sized avatar
🛖
Villagers share their technological secrets. Receive 2 Eurekas.

David Dao daviddao

🛖
Villagers share their technological secrets. Receive 2 Eurekas.
View GitHub Profile
@daviddao
daviddao / spatial_transformer.py
Last active June 28, 2017 05:04
Tensorflow Spatial Transformer
import tensorflow as tf
def transformer(U, theta, downsample_factor=1, name='SpatialTransformer', **kwargs):
"""Spatial Transformer Layer
Implements a spatial transformer layer as described in [1]_.
Based on [2]_ and edited by David Dao for Tensorflow.
Parameters
----------
'''
Takes an image in numpy format as input and returns a padded image
'''
def pad_image(image, output_shape):
x_pad = (output_shape[0] - image.shape[0]) / 2
y_pad = (output_shape[1] - image.shape[1]) / 2
resized = np.pad(image, ((x_pad,x_pad),(y_pad,y_pad)), mode='constant', constant_values=0)
return resized
# Confusion Matrix (improved version with total numbers)
def plot_confusion_matrix(conf_arr, labels, title='Confusion matrix', cmap=plt.cm.Blues):
#plt.imshow(cm, interpolation='nearest', cmap=cmap)
norm_conf = []
for i in conf_arr:
a = 0
tmp_arr = []
a = sum(i, 0)
for j in i:
tmp_arr.append(float(j)/float(a))
@daviddao
daviddao / beautiful_idiomatic_python.md
Created August 1, 2017 20:10 — forked from JeffPaine/beautiful_idiomatic_python.md
Transforming Code into Beautiful, Idiomatic Python: notes from Raymond Hettinger's talk at pycon US 2013. The code examples and direct quotes are all from Raymond's talk. I've reproduced them here for my own edification and the hopes that others will find them as handy as I have!

Transforming Code into Beautiful, Idiomatic Python

Notes from Raymond Hettinger's talk at pycon US 2013 video, slides.

The code examples and direct quotes are all from Raymond's talk. I've reproduced them here for my own edification and the hopes that others will find them as handy as I have!

Looping over a range of numbers

for i in [0, 1, 2, 3, 4, 5]:

How to install Intel SGX, big picture

  1. Check if your machine has Intel SGX (https://github.com/ayeks/SGX-hardware)
  2. Enable it in BIOS (On Ubuntu, restart and press F2)
  3. Install the SGX driver (https://github.com/01org/linux-sgx-driver), take care to switch to branch SGX2
  4. Install Intel SGX SDK (https://github.com/01org/linux-sgx/). This allows you to compile programs in Simulation mode with flag make SGX_MODE=SIM
  5. Install Intel SGX PSW. For that you need to download the capability licensing client from the Intel website! (https://software.intel.com/en-us/sgx-sdk/download)
  6. If you make an SGX program, remember to always source environment!

That's it

@daviddao
daviddao / distributed_model_parallelism.py
Created January 27, 2018 18:31
Distributed model parallelism with PyTorch
def train(epoch):
model.train()
model.cuda()
optimizer = optim.SGD(model.parameters(), lr=args.lr, momentum=args.momentum)
if dist.get_rank() == 0:
input_from_part2 = torch.FloatTensor(data.size()[0], 320)
for batch_idx, (data, target) in enumerate(train_loader):
optimizer.zero_grad()
data = Variable(data.cuda())
output = model(data)
@daviddao
daviddao / schema.json
Created June 20, 2018 20:40
schemas for data markets
//Job Result Schema
{
"$id": "http://example.com/example.json",
"type": "object",
"definitions": {},
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"job_id": {
"$id": "/properties/job_id",
"type": "string",
@daviddao
daviddao / students.md
Last active June 29, 2018 04:12
Doing a thesis under my supervision? Here are some things you should know

Thesis

So you want to do a thesis under my supervision? Here's what you should know ...

What I care about ...

  1. You, obviously! I want you to get smarter, more creative and ambitious during your thesis. I fail as teacher when you haven't picked up at least one new skill after submitting your thesis.
  2. The project! My thesis offers are often quite challenging, pioneering or/and researchy, but never useless and stupid work. Research is always an exploration into the unknown.

I respect ...

[{"constant":true,"inputs":[{"name":"_id","type":"bytes32"},{"name":"_key","type":"bytes32"}],"name":"meta","outputs":[{"name":"","type":"bytes32"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":true,"inputs":[],"name":"count","outputs":[{"name":"","type":"uint256"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[{"name":"_new","type":"address"}],"name":"setOwner","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[{"name":"_id","type":"bytes32"}],"name":"unregister","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[],"name":"register","outputs":[],"payable":true,"stateMutability":"payable","type":"function"},{"constant":false,"inputs":[{"name":"_fee","type":"uint256"}],"name":"setFee","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[],"name":"owner","outputs":[{"name":"","type

Keybase proof

I hereby claim:

  • I am daviddao on github.
  • I am daviddao (https://keybase.io/daviddao) on keybase.
  • I have a public key ASDjk6i5ZIF9WqaG_paFtxVSVm2YAOoxuVxWMDs-iL6ubwo

To claim this, I am signing this object: