Skip to content

Instantly share code, notes, and snippets.

@pengsun
Last active April 21, 2016 03:53
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save pengsun/aa0c9ee1ad1e37f52032208e6ae28821 to your computer and use it in GitHub Desktop.
Save pengsun/aa0c9ee1ad1e37f52032208e6ae28821 to your computer and use it in GitHub Desktop.
-- nn.LookupTable vs nn.SparseLinear
require'cunn'
V = 30000 -- vocabulary size
C = 500 -- output dim
B = 100*500 -- #batches
nloop = 3
-- onehot input
input = torch.LongTensor(B):random(V):cuda()
inputTable = {}
for i = 1, B do
inputTable[i] = torch.CudaTensor(1, 2)
inputTable[i][1][1] = input[i]
inputTable[i][1][2] = 1.0
end
-- dense grad output
gOutput = torch.CudaTensor(B, C):normal()
function timing_module(input, m)
time = torch.tic()
for i = 1, nloop do
m:updateOutput(input)
end
cutorch.synchronize()
time = torch.toc(time)
print(torch.type(m) .. ' fprop time ' .. time/nloop)
time = torch.tic()
for i = 1, nloop do
m:accGradParameters(input, gOutput)
end
cutorch.synchronize()
time = torch.toc(time)
print(torch.type(m) .. ' bprop time ' .. time/nloop)
end
-- LookupTable
lt = nn.LookupTable(V, C):cuda()
timing_module(input, lt)
-- SparseLinear
sl = nn.SparseLinear(V, C):cuda()
timing_module(inputTable, sl)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment