chitchcock

Stevey's Google Platforms Rant

I was at Amazon for about six and a half years, and now I've been at Google for that long. One thing that struck me immediately about the two companies -- an impression that has been reinforced almost daily -- is that Amazon does everything wrong, and Google does everything right. Sure, it's a sweeping generalization, but a surprisingly accurate one. It's pretty crazy. There are probably a hundred or even two hundred different ways you can compare the two companies, and Google is superior in all but three of them, if I recall correctly. I actually did a spreadsheet at one point but Legal wouldn't let me show it to anyone, even though recruiting loved it.

I mean, just to give you a very brief taste: Amazon's recruiting process is fundamentally flawed by having teams hire for themselves, so their hiring bar is incredibly inconsistent across teams, despite various efforts they've made to level it out. And their operations are a mess; they don't real

cavinsmith

List of desktop Nvidia GPUS ordered by CUDA core count

I created it for those who use Neural Style

Guys, please add your hardware setups, neural-style configs and results in comments!

GPU	CUDA cores	Memory	Processor frequency
GeForce GTX TITAN Z	5760	12 GB	705 / 876

JeffBezanson

An array is a structure mapping indices to data values, where the indices have a particular structure.

An array can be thought of as a pair of an index set and a data set.

Two arrays are equal if they map the same indices to the same values.

The data

An array is iterable, and when iterated generates exactly the values of

jihunchoi

def _sequence_mask(sequence_length, max_len=None):
    if max_len is None:
        max_len = sequence_length.data.max()
    batch_size = sequence_length.size(0)
    seq_range = torch.range(0, max_len - 1).long()
    seq_range_expand = seq_range.unsqueeze(0).expand(batch_size, max_len)
    seq_range_expand = Variable(seq_range_expand)
    if sequence_length.is_cuda:
        seq_range_expand = seq_range_expand.cuda()
    seq_length_expand = (sequence_length.unsqueeze(1)

rtqichen

## Weight norm is now added to pytorch as a pre-hook, so use that instead :)

import torch
import torch.nn as nn
from torch.nn import Parameter
from functools import wraps

class WeightNorm(nn.Module):
    append_g = '_g'
    append_v = '_v'

killeent

A Tour of PyTorch Internals (Part I)

The fundamental unit in PyTorch is the Tensor. This post will serve as an overview for how we implement Tensors in PyTorch, such that the user can interact with it from the Python shell. In particular, we want to answer four main questions:

1. How does PyTorch extend the Python interpreter to define a Tensor type that can be manipulated from Python code?
2. How does PyTorch wrap the C libraries that actually define the Tensor's properties and methods?
3. How does PyTorch cwrap work to generate code for Tensor methods?
4. How does PyTorch's build system take all of these components to compile and generate a workable application?

Extending the Python Interpreter

PyTorch defines a new package torch. In this post we will consider the ._C module. This module is known as an "extension module" - a Python module written in C. Such modules allow us to define new built-in object types (e.g. the Tensor) and to call C/C++ functions.

spro

import torch
from torch import nn
from torch.autograd import Variable
import torch.nn.functional as F

class RNN(nn.Module):
    def __init__(self, input_size, hidden_size, output_size, n_layers=1):
        super(RNN, self).__init__()
        self.input_size = input_size
        self.hidden_size = hidden_size

mattjj

import autograd.numpy as np
import autograd.numpy.random as npr
from autograd import grad, jacobian, hessian, make_vjp
from autograd.core import backward_pass

# explicit dense version, for testing
def make_dense_ggnvp(f, g=lambda x: 1./2*np.dot(x, x)):
  def ggnvp_maker(x):
    J = jacobian(f)(x)
    H = hessian(g)(f(x))

apaszke

from graphviz import Digraph
import torch
from torch.autograd import Variable, Function

def iter_graph(root, callback):
    queue = [root]
    seen = set()
    while queue:
        fn = queue.pop()
        if fn in seen:

hyqneuron

import torch
import torch.nn as nn
from torch.nn import Parameter
from torch.autograd import Variable, Function
from collections import defaultdict
import graphviz

"""
This is a rather distorted implementation of graph visualization in PyTorch.