Skip to content

Instantly share code, notes, and snippets.

View ethancaballero's full-sized avatar

Ethan Caballero ethancaballero

View GitHub Profile
import numpy as np
import torch
import random
from torch.autograd import Variable, grad
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import numpy
import theano
import theano.tensor as tt
from theano.gradient import disconnected_grad as stop_grad
x = tt.dscalar('x')
y = x ** 2
gy = tt.grad(y, x)
f = theano.function([x], gy)
@yaroslavvb
yaroslavvb / kfac_nano_eager_test.py
Last active May 31, 2018 22:11
Small example of KFAC in Eager mode
import numpy as np
import tensorflow as tf
import scipy
from tensorflow.contrib.eager.python import tfe
tfe.enable_eager_execution()
# manual numpy example
# X = np.array(([[0., 1], [2, 3]]))
# W0 = X
# W1 = np.array(([[0., 1], [2, 3]]))/10
# Inception Score Calculator
#
# A Brock, 2017
#
# This snippet assumes you have two functions defined:
# 1. sample_net, which takes in a batch x num_latents random vector and returns batch samples,
# 2. eval_net, which takes in batch samples and returns a batch x #classes prediction vector.
num_latents = 100

A Tour of PyTorch Internals (Part I)

The fundamental unit in PyTorch is the Tensor. This post will serve as an overview for how we implement Tensors in PyTorch, such that the user can interact with it from the Python shell. In particular, we want to answer four main questions:

  1. How does PyTorch extend the Python interpreter to define a Tensor type that can be manipulated from Python code?
  2. How does PyTorch wrap the C libraries that actually define the Tensor's properties and methods?
  3. How does PyTorch cwrap work to generate code for Tensor methods?
  4. How does PyTorch's build system take all of these components to compile and generate a workable application?

Extending the Python Interpreter

PyTorch defines a new package torch. In this post we will consider the ._C module. This module is known as an "extension module" - a Python module written in C. Such modules allow us to define new built-in object types (e.g. the Tensor) and to call C/C++ functions.

@rtqichen
rtqichen / pytorch_weight_norm.py
Last active May 11, 2023 06:58
Pytorch weight normalization - works for all nn.Module (probably)
## Weight norm is now added to pytorch as a pre-hook, so use that instead :)
import torch
import torch.nn as nn
from torch.nn import Parameter
from functools import wraps
class WeightNorm(nn.Module):
append_g = '_g'
append_v = '_v'
import argparse
import numpy as np
import tensorflow as tf
from tensorflow.python.framework.errors import FailedPreconditionError
"""Code for data dependent initialization in Weight Normalization paper:
https://arxiv.org/abs/1602.07868
"""
@machinaut
machinaut / divided_icosahedron.py
Last active August 19, 2023 19:27
Subdivide Icosahedrons to make nice spheres
#!/usr/bin/env python
from itertools import combinations, chain
import numpy as np
from pyhull.convex_hull import ConvexHull
from stl import Mode
from stl.mesh import Mesh
def subdivide(shape):
''' Take a triangulated sphere and subdivide each face. '''
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@yohokuno
yohokuno / ptb_word_lm.py
Created December 22, 2016 14:13
TensorFlow implementation of "A Theoretically Grounded Application of Dropout in Recurrent Neural Networks Yarin Gal, Zoubin Ghahramani
# Copyright 2015 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,