Skip to content

Instantly share code, notes, and snippets.

@szagoruyko
Created October 25, 2016 10:05
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save szagoruyko/e5cf5e9b54661a817695c8c7b5c3dfa6 to your computer and use it in GitHub Desktop.
Save szagoruyko/e5cf5e9b54661a817695c8c7b5c3dfa6 to your computer and use it in GitHub Desktop.
require 'xlua'
require 'sys'
local batches_folder = '/opt/rocks/cifar.torch/cifar-10-batches-t7'
local data = {}
local labels = {}
for i=1,5 do
local name = paths.concat(batches_folder, 'data_batch_'..i..'.t7')
local part = torch.load(paths.concat(batches_folder, name), 'ascii')
table.insert(data, part.data:view(3,32,32,-1))
table.insert(labels, part.labels:squeeze())
end
data = torch.ByteTensor.cat(data, 4)
labels = torch.ByteTensor.cat(labels)
test_part = torch.load(paths.concat(batches_folder, 'test_batch.t7'), 'ascii')
test_labels = test_part.labels
test_data = test_part.data
local dataset = {
trainData = {
data = data:permute(4,1,2,3):clone(),
labels = labels:add(1),
size = function() return labels:numel() end,
},
testData = {
data = test_data:view(3,32,32,-1):permute(4,1,2,3):clone(),
labels = test_labels:squeeze():add(1),
size = function() return test_labels:numel() end,
}
}
print(dataset)
print(dataset.trainData.labels:max())
print(dataset.testData.labels:max())
torch.save('cifar10_original.t7', dataset)
@Nithanaroy
Copy link

Thanks sharing your code. How many training samples per epoch after data augmentation as mentioned in Experimental Results section?
https://arxiv.org/pdf/1605.07146.pdf

For experiments we chose well-known CIFAR-10, CIFAR-100, SVHN and ImageNet image
classification datasets. CIFAR-10 and CIFAR-100 datasets [17] consist of 32 × 32 color
images drawn from 10 and 100 classes split into 50,000 train and 10,000 test images. For data
augmentation we do horizontal flips and take random crops from image padded by 4 pixels on
each side, filling missing pixels with reflections of original image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment