Skip to content

Instantly share code, notes, and snippets.

@AlkindiX

AlkindiX/MEMO

Last active March 10, 2024 20:10
Show Gist options
  • Star 15 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save AlkindiX/9c54d1155ba72415f3b585e26c9df6b3 to your computer and use it in GitHub Desktop.
Save AlkindiX/9c54d1155ba72415f3b585e26c9df6b3 to your computer and use it in GitHub Desktop.
Download ZIP
Run machine learning on 7900XT and 7900XTX on PyTorch
Raw
hugging_face_bert_test.py
# %%
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import os
os.environ["HSA_OVERRIDE_GFX_VERSION"] = "11.0.0"
os.environ['ROCM_PATH'] = '/opt/rocm'
os.environ['PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION'] = 'python'
# ROCm 5.5 MIOpen workaround
if not torch.cuda.is_available():
raise Exception("CUDA/ROCm not working!")
else:
print('CUDA/ROCm installed!')
# activate
if not torch.cuda.is_initialized():
torch.cuda.init()
else:
print('CUDA/ROCm initialized!')
midevices = torch.cuda.device_count()
print('CUDA/ROCm devices:', midevices)
for i in range(midevices):
print(f'{i} CUDA/ROCm device:', torch.cuda.get_device_name(i))
# set device
torch.cuda.set_device(0)
print('CUDA/ROCm device set to:', torch.cuda.current_device())
# %%
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("asafaya/bert-large-arabic")
model = AutoModelForMaskedLM.from_pretrained("asafaya/bert-large-arabic")
# %%
from transformers import pipeline
unmasker = pipeline('fill-mask', model='asafaya/bert-large-arabic', tokenizer='asafaya/bert-large-arabic', device=0)
# %%
unmasker("من الواضح أن اللغة العربية هي [MASK] العالم العربي.", top_k=2)
Raw
MEMO
Requirements:-
1. Ubuntu 22.04
2. 7900XT or 7900XTX
Pre-requests before making any installation.
Follow step 1-3 if you installed amdgpu.
1. If you already installed redeon graphic card driver from AMD using amdgpu-install. Completely remove the driver
and connect your HDMI to motherboard. Then restart your PC
2. After restarting. Make sure to remove any remaining package that was installed using the script. Run apt-get autoremove.
3. Remove amdgpu apt list files. Run apt-get update
4. Install ROCm driver. Head to amd website and follow the instruction. If you did not find the installation webpage. Head to
ROCm GitHub source page and follow the instruction.
After trial and error from myside. You must have the following packages installed. I don't remember the order of installation
but I pulled this data from `apt list --installed | grep rocm` Also make sure to install rocm >= 5.5
To install rocm package
rocm-clang-ocl/jammy,now 0.5.0.50500-63~22.04 amd64 [installed,automatic]
rocm-cmake/jammy,now 0.8.1.50500-63~22.04 amd64 [installed,automatic]
rocm-core/jammy,now 5.5.0.50500-63~22.04 amd64 [installed,automatic]
rocm-dbgapi/jammy,now 0.70.0.50500-63~22.04 amd64 [installed,automatic]
rocm-debug-agent/jammy,now 2.0.3.50500-63~22.04 amd64 [installed,automatic]
rocm-dev/jammy,now 5.5.0.50500-63~22.04 amd64 [installed]
rocm-device-libs/jammy,now 1.0.0.50500-63~22.04 amd64 [installed,automatic]
rocm-gdb/jammy,now 12.1.50500-63~22.04 amd64 [installed,automatic]
rocm-hip-libraries/jammy,now 5.5.0.50500-63~22.04 amd64 [installed]
rocm-hip-runtime-dev/jammy,now 5.5.0.50500-63~22.04 amd64 [installed]
rocm-hip-runtime/jammy,now 5.5.0.50500-63~22.04 amd64 [installed]
rocm-hip-sdk/jammy,now 5.5.0.50500-63~22.04 amd64 [installed]
rocm-language-runtime/jammy,now 5.5.0.50500-63~22.04 amd64 [installed,automatic]
rocm-libs/jammy,now 5.5.0.50500-63~22.04 amd64 [installed]
rocm-llvm/jammy,now 16.0.0.23144.50500-63~22.04 amd64 [installed,automatic]
rocm-ocl-icd/jammy,now 2.0.0.50500-63~22.04 amd64 [installed,automatic]
rocm-opencl-dev/jammy,now 2.0.0.50500-63~22.04 amd64 [installed,automatic]
rocm-opencl/jammy,now 2.0.0.50500-63~22.04 amd64 [installed,automatic]
rocm-smi-lib/jammy,now 5.0.0.50500-63~22.04 amd64 [installed,automatic]
rocm-utils/jammy,now 5.5.0.50500-63~22.04 amd64 [installed,automatic]
rocminfo/jammy,now 1.0.0.50500-63~22.04 amd64 [installed,automatic]
Now install pytorch.
(check their website for any new update)
pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm5.5
Make sure to run this if you encounter any issue with seg faults and MIOpen errors.
export MIOPEN_USER_DB_PATH="/tmp/my-miopen-cache"
export MIOPEN_CUSTOM_CACHE_DIR=${MIOPEN_USER_DB_PATH}
rm -rf ${MIOPEN_USER_DB_PATH}
mkdir -p ${MIOPEN_USER_DB_PATH}
Run python mnist.py (time python mnist.py)
Hardware
64GB 6000MHz DDR5 XMP Enabled
AMD Ryzen 7950X3D
AMD AMD Radeon 7900 XT 7900 XT 20GB XFX MARC 10
Gigabyte Aorus Elite AX
# Note for hugging face
# install pip install transformer accelerate
To load any module into GPU
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("asafaya/bert-large-arabic")
model = AutoModelForMaskedLM.from_pretrained("asafaya/bert-large-arabic")
from transformers import pipeline
unmasker = pipeline('fill-mask', model='asafaya/bert-large-arabic', tokenizer='asafaya/bert-large-arabic', device=0)
# make sure to write device=0. Not setting this default to CPU to me 7950X3D cpu for some reason.
Raw
mnist.py
from __future__ import print_function
import argparse
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchvision import datasets, transforms
from torch.optim.lr_scheduler import StepLR
import os
# Some workarounds after trial and error. New Navi architecture uses >= 11.0.0 DSA code.
# Set the following as well using export on your terminal if this script throw some seg error
os.environ["HSA_OVERRIDE_GFX_VERSION"] = "11.0.0"
os.environ['ROCM_PATH'] = '/opt/rocm'
os.environ['PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION'] = 'python'
# ROCm 5.5 MIOpen workaround
if not torch.cuda.is_available():
raise Exception("CUDA/ROCm not working!")
else:
print('CUDA/ROCm installed!')
# activate
if not torch.cuda.is_initialized():
torch.cuda.init()
else:
print('CUDA/ROCm initialized!')
midevices = torch.cuda.device_count()
print('CUDA/ROCm devices:', midevices)
for i in range(midevices):
print(f'{i} CUDA/ROCm device:', torch.cuda.get_device_name(i))
# set device
torch.cuda.set_device(0)
print('CUDA/ROCm device set to:', torch.cuda.current_device())
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(1, 32, 3, 1)
self.conv2 = nn.Conv2d(32, 64, 3, 1)
self.dropout1 = nn.Dropout(0.25)
self.dropout2 = nn.Dropout(0.5)
self.fc1 = nn.Linear(9216, 128)
self.fc2 = nn.Linear(128, 10)
def forward(self, x):
x = self.conv1(x)
x = F.relu(x)
x = self.conv2(x)
x = F.relu(x)
x = F.max_pool2d(x, 2)
x = self.dropout1(x)
x = torch.flatten(x, 1)
x = self.fc1(x)
x = F.relu(x)
x = self.dropout2(x)
x = self.fc2(x)
output = F.log_softmax(x, dim=1)
return output
def train(args, model, device, train_loader, optimizer, epoch):
model.train()
for batch_idx, (data, target) in enumerate(train_loader):
data, target = data.to(device), target.to(device)
optimizer.zero_grad()
output = model(data)
loss = F.nll_loss(output, target)
loss.backward()
optimizer.step()
if batch_idx % args.log_interval == 0:
print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
epoch, batch_idx * len(data), len(train_loader.dataset),
100. * batch_idx / len(train_loader), loss.item()))
if args.dry_run:
break
def test(model, device, test_loader):
model.eval()
test_loss = 0
correct = 0
with torch.no_grad():
for data, target in test_loader:
data, target = data.to(device), target.to(device)
output = model(data)
test_loss += F.nll_loss(output, target, reduction='sum').item() # sum up batch loss
pred = output.argmax(dim=1, keepdim=True) # get the index of the max log-probability
correct += pred.eq(target.view_as(pred)).sum().item()
test_loss /= len(test_loader.dataset)
print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
test_loss, correct, len(test_loader.dataset),
100. * correct / len(test_loader.dataset)))
def main():
# Training settings
parser = argparse.ArgumentParser(description='PyTorch MNIST Example')
parser.add_argument('--batch-size', type=int, default=64, metavar='N',
help='input batch size for training (default: 64)')
parser.add_argument('--test-batch-size', type=int, default=1000, metavar='N',
help='input batch size for testing (default: 1000)')
parser.add_argument('--epochs', type=int, default=14, metavar='N',
help='number of epochs to train (default: 14)')
parser.add_argument('--lr', type=float, default=1.0, metavar='LR',
help='learning rate (default: 1.0)')
parser.add_argument('--gamma', type=float, default=0.7, metavar='M',
help='Learning rate step gamma (default: 0.7)')
parser.add_argument('--no-cuda', action='store_true', default=False,
help='disables CUDA training')
parser.add_argument('--no-mps', action='store_true', default=False,
help='disables macOS GPU training')
parser.add_argument('--dry-run', action='store_true', default=False,
help='quickly check a single pass')
parser.add_argument('--seed', type=int, default=1, metavar='S',
help='random seed (default: 1)')
parser.add_argument('--log-interval', type=int, default=10, metavar='N',
help='how many batches to wait before logging training status')
parser.add_argument('--save-model', action='store_true', default=False,
help='For Saving the current Model')
args = parser.parse_args()
use_cuda = not args.no_cuda and torch.cuda.is_available()
use_mps = not args.no_mps and torch.backends.mps.is_available()
torch.manual_seed(args.seed)
if use_cuda:
device = torch.device("cuda")
elif use_mps:
device = torch.device("mps")
else:
device = torch.device("cpu")
train_kwargs = {'batch_size': args.batch_size}
test_kwargs = {'batch_size': args.test_batch_size}
if use_cuda:
cuda_kwargs = {'num_workers': 1,
'pin_memory': True,
'shuffle': True}
train_kwargs.update(cuda_kwargs)
test_kwargs.update(cuda_kwargs)
transform=transforms.Compose([
transforms.ToTensor(),
transforms.Normalize((0.1307,), (0.3081,))
])
dataset1 = datasets.MNIST('../data', train=True, download=True,
transform=transform)
dataset2 = datasets.MNIST('../data', train=False,
transform=transform)
train_loader = torch.utils.data.DataLoader(dataset1,**train_kwargs)
test_loader = torch.utils.data.DataLoader(dataset2, **test_kwargs)
model = Net().to(device)
optimizer = optim.Adadelta(model.parameters(), lr=args.lr)
scheduler = StepLR(optimizer, step_size=1, gamma=args.gamma)
for epoch in range(1, args.epochs + 1):
train(args, model, device, train_loader, optimizer, epoch)
test(model, device, test_loader)
scheduler.step()
if args.save_model:
torch.save(model.state_dict(), "mnist_cnn.pt")
if __name__ == '__main__':
main()
Raw
output.txt
CUDA/ROCm installed!
CUDA/ROCm devices: 2
0 CUDA/ROCm device: Radeon RX 7900 XT
1 CUDA/ROCm device: AMD Radeon Graphics
CUDA/ROCm device set to: 0
Train Epoch: 1 [0/60000 (0%)] Loss: 2.286303
Train Epoch: 1 [640/60000 (1%)] Loss: 1.619755
Train Epoch: 1 [1280/60000 (2%)] Loss: 0.863853
Train Epoch: 1 [1920/60000 (3%)] Loss: 0.763415
Train Epoch: 1 [2560/60000 (4%)] Loss: 0.470558
Train Epoch: 1 [3200/60000 (5%)] Loss: 0.454752
Train Epoch: 1 [3840/60000 (6%)] Loss: 0.334980
Train Epoch: 1 [4480/60000 (7%)] Loss: 0.624757
Train Epoch: 1 [5120/60000 (9%)] Loss: 0.299231
Train Epoch: 1 [5760/60000 (10%)] Loss: 0.222814
Train Epoch: 1 [6400/60000 (11%)] Loss: 0.325722
Train Epoch: 1 [7040/60000 (12%)] Loss: 0.194261
Train Epoch: 1 [7680/60000 (13%)] Loss: 0.213115
Train Epoch: 1 [8320/60000 (14%)] Loss: 0.143714
Train Epoch: 1 [8960/60000 (15%)] Loss: 0.415952
Train Epoch: 1 [9600/60000 (16%)] Loss: 0.169987
Train Epoch: 1 [10240/60000 (17%)] Loss: 0.151499
Train Epoch: 1 [10880/60000 (18%)] Loss: 0.112238
Train Epoch: 1 [11520/60000 (19%)] Loss: 0.227815
Train Epoch: 1 [12160/60000 (20%)] Loss: 0.060721
Train Epoch: 1 [12800/60000 (21%)] Loss: 0.169976
Train Epoch: 1 [13440/60000 (22%)] Loss: 0.152559
Train Epoch: 1 [14080/60000 (23%)] Loss: 0.146166
Train Epoch: 1 [14720/60000 (25%)] Loss: 0.155003
Train Epoch: 1 [15360/60000 (26%)] Loss: 0.309573
Train Epoch: 1 [16000/60000 (27%)] Loss: 0.246533
Train Epoch: 1 [16640/60000 (28%)] Loss: 0.148106
Train Epoch: 1 [17280/60000 (29%)] Loss: 0.191950
Train Epoch: 1 [17920/60000 (30%)] Loss: 0.139336
Train Epoch: 1 [18560/60000 (31%)] Loss: 0.195350
Train Epoch: 1 [19200/60000 (32%)] Loss: 0.220971
Train Epoch: 1 [19840/60000 (33%)] Loss: 0.166819
Train Epoch: 1 [20480/60000 (34%)] Loss: 0.165741
Train Epoch: 1 [21120/60000 (35%)] Loss: 0.086579
Train Epoch: 1 [21760/60000 (36%)] Loss: 0.278489
Train Epoch: 1 [22400/60000 (37%)] Loss: 0.100752
Train Epoch: 1 [23040/60000 (38%)] Loss: 0.212050
Train Epoch: 1 [23680/60000 (39%)] Loss: 0.276167
Train Epoch: 1 [24320/60000 (41%)] Loss: 0.130804
Train Epoch: 1 [24960/60000 (42%)] Loss: 0.089291
Train Epoch: 1 [25600/60000 (43%)] Loss: 0.063802
Train Epoch: 1 [26240/60000 (44%)] Loss: 0.204756
Train Epoch: 1 [26880/60000 (45%)] Loss: 0.282732
Train Epoch: 1 [27520/60000 (46%)] Loss: 0.103581
Train Epoch: 1 [28160/60000 (47%)] Loss: 0.139586
Train Epoch: 1 [28800/60000 (48%)] Loss: 0.059837
Train Epoch: 1 [29440/60000 (49%)] Loss: 0.129647
Train Epoch: 1 [30080/60000 (50%)] Loss: 0.173143
Train Epoch: 1 [30720/60000 (51%)] Loss: 0.200272
Train Epoch: 1 [31360/60000 (52%)] Loss: 0.170435
Train Epoch: 1 [32000/60000 (53%)] Loss: 0.134229
Train Epoch: 1 [32640/60000 (54%)] Loss: 0.161856
Train Epoch: 1 [33280/60000 (55%)] Loss: 0.097512
Train Epoch: 1 [33920/60000 (57%)] Loss: 0.262982
Train Epoch: 1 [34560/60000 (58%)] Loss: 0.110638
Train Epoch: 1 [35200/60000 (59%)] Loss: 0.164555
Train Epoch: 1 [35840/60000 (60%)] Loss: 0.081531
Train Epoch: 1 [36480/60000 (61%)] Loss: 0.244769
Train Epoch: 1 [37120/60000 (62%)] Loss: 0.108114
Train Epoch: 1 [37760/60000 (63%)] Loss: 0.036871
Train Epoch: 1 [38400/60000 (64%)] Loss: 0.060095
Train Epoch: 1 [39040/60000 (65%)] Loss: 0.050034
Train Epoch: 1 [39680/60000 (66%)] Loss: 0.536586
Train Epoch: 1 [40320/60000 (67%)] Loss: 0.132366
Train Epoch: 1 [40960/60000 (68%)] Loss: 0.090065
Train Epoch: 1 [41600/60000 (69%)] Loss: 0.201307
Train Epoch: 1 [42240/60000 (70%)] Loss: 0.129913
Train Epoch: 1 [42880/60000 (71%)] Loss: 0.018223
Train Epoch: 1 [43520/60000 (72%)] Loss: 0.073085
Train Epoch: 1 [44160/60000 (74%)] Loss: 0.081079
Train Epoch: 1 [44800/60000 (75%)] Loss: 0.099291
Train Epoch: 1 [45440/60000 (76%)] Loss: 0.197632
Train Epoch: 1 [46080/60000 (77%)] Loss: 0.068322
Train Epoch: 1 [46720/60000 (78%)] Loss: 0.063282
Train Epoch: 1 [47360/60000 (79%)] Loss: 0.076087
Train Epoch: 1 [48000/60000 (80%)] Loss: 0.064270
Train Epoch: 1 [48640/60000 (81%)] Loss: 0.204694
Train Epoch: 1 [49280/60000 (82%)] Loss: 0.045380
Train Epoch: 1 [49920/60000 (83%)] Loss: 0.093631
Train Epoch: 1 [50560/60000 (84%)] Loss: 0.055415
Train Epoch: 1 [51200/60000 (85%)] Loss: 0.169766
Train Epoch: 1 [51840/60000 (86%)] Loss: 0.093971
Train Epoch: 1 [52480/60000 (87%)] Loss: 0.076500
Train Epoch: 1 [53120/60000 (88%)] Loss: 0.037874
Train Epoch: 1 [53760/60000 (90%)] Loss: 0.109982
Train Epoch: 1 [54400/60000 (91%)] Loss: 0.069943
Train Epoch: 1 [55040/60000 (92%)] Loss: 0.146147
Train Epoch: 1 [55680/60000 (93%)] Loss: 0.071112
Train Epoch: 1 [56320/60000 (94%)] Loss: 0.186059
Train Epoch: 1 [56960/60000 (95%)] Loss: 0.096106
Train Epoch: 1 [57600/60000 (96%)] Loss: 0.107285
Train Epoch: 1 [58240/60000 (97%)] Loss: 0.094049
Train Epoch: 1 [58880/60000 (98%)] Loss: 0.065223
Train Epoch: 1 [59520/60000 (99%)] Loss: 0.062499
Test set: Average loss: 0.0477, Accuracy: 9841/10000 (98%)
Train Epoch: 2 [0/60000 (0%)] Loss: 0.111675
Train Epoch: 2 [640/60000 (1%)] Loss: 0.015048
Train Epoch: 2 [1280/60000 (2%)] Loss: 0.020823
Train Epoch: 2 [1920/60000 (3%)] Loss: 0.169394
Train Epoch: 2 [2560/60000 (4%)] Loss: 0.036462
Train Epoch: 2 [3200/60000 (5%)] Loss: 0.051407
Train Epoch: 2 [3840/60000 (6%)] Loss: 0.094461
Train Epoch: 2 [4480/60000 (7%)] Loss: 0.048435
Train Epoch: 2 [5120/60000 (9%)] Loss: 0.105586
Train Epoch: 2 [5760/60000 (10%)] Loss: 0.072193
Train Epoch: 2 [6400/60000 (11%)] Loss: 0.041041
Train Epoch: 2 [7040/60000 (12%)] Loss: 0.082491
Train Epoch: 2 [7680/60000 (13%)] Loss: 0.006681
Train Epoch: 2 [8320/60000 (14%)] Loss: 0.058887
Train Epoch: 2 [8960/60000 (15%)] Loss: 0.043568
Train Epoch: 2 [9600/60000 (16%)] Loss: 0.077991
Train Epoch: 2 [10240/60000 (17%)] Loss: 0.135082
Train Epoch: 2 [10880/60000 (18%)] Loss: 0.040199
Train Epoch: 2 [11520/60000 (19%)] Loss: 0.060972
Train Epoch: 2 [12160/60000 (20%)] Loss: 0.038435
Train Epoch: 2 [12800/60000 (21%)] Loss: 0.047128
Train Epoch: 2 [13440/60000 (22%)] Loss: 0.034678
Train Epoch: 2 [14080/60000 (23%)] Loss: 0.215021
Train Epoch: 2 [14720/60000 (25%)] Loss: 0.045986
Train Epoch: 2 [15360/60000 (26%)] Loss: 0.157211
Train Epoch: 2 [16000/60000 (27%)] Loss: 0.007970
Train Epoch: 2 [16640/60000 (28%)] Loss: 0.113235
Train Epoch: 2 [17280/60000 (29%)] Loss: 0.144736
Train Epoch: 2 [17920/60000 (30%)] Loss: 0.016518
Train Epoch: 2 [18560/60000 (31%)] Loss: 0.133768
Train Epoch: 2 [19200/60000 (32%)] Loss: 0.032983
Train Epoch: 2 [19840/60000 (33%)] Loss: 0.104494
Train Epoch: 2 [20480/60000 (34%)] Loss: 0.104596
Train Epoch: 2 [21120/60000 (35%)] Loss: 0.042851
Train Epoch: 2 [21760/60000 (36%)] Loss: 0.013128
Train Epoch: 2 [22400/60000 (37%)] Loss: 0.059972
Train Epoch: 2 [23040/60000 (38%)] Loss: 0.039612
Train Epoch: 2 [23680/60000 (39%)] Loss: 0.117994
Train Epoch: 2 [24320/60000 (41%)] Loss: 0.016756
Train Epoch: 2 [24960/60000 (42%)] Loss: 0.126752
Train Epoch: 2 [25600/60000 (43%)] Loss: 0.098190
Train Epoch: 2 [26240/60000 (44%)] Loss: 0.047053
Train Epoch: 2 [26880/60000 (45%)] Loss: 0.118982
Train Epoch: 2 [27520/60000 (46%)] Loss: 0.132032
Train Epoch: 2 [28160/60000 (47%)] Loss: 0.017717
Train Epoch: 2 [28800/60000 (48%)] Loss: 0.319930
Train Epoch: 2 [29440/60000 (49%)] Loss: 0.114873
Train Epoch: 2 [30080/60000 (50%)] Loss: 0.263901
Train Epoch: 2 [30720/60000 (51%)] Loss: 0.051616
Train Epoch: 2 [31360/60000 (52%)] Loss: 0.051622
Train Epoch: 2 [32000/60000 (53%)] Loss: 0.010319
Train Epoch: 2 [32640/60000 (54%)] Loss: 0.038780
Train Epoch: 2 [33280/60000 (55%)] Loss: 0.021453
Train Epoch: 2 [33920/60000 (57%)] Loss: 0.037479
Train Epoch: 2 [34560/60000 (58%)] Loss: 0.133600
Train Epoch: 2 [35200/60000 (59%)] Loss: 0.054258
Train Epoch: 2 [35840/60000 (60%)] Loss: 0.002056
Train Epoch: 2 [36480/60000 (61%)] Loss: 0.053813
Train Epoch: 2 [37120/60000 (62%)] Loss: 0.045620
Train Epoch: 2 [37760/60000 (63%)] Loss: 0.052088
Train Epoch: 2 [38400/60000 (64%)] Loss: 0.015002
Train Epoch: 2 [39040/60000 (65%)] Loss: 0.051094
Train Epoch: 2 [39680/60000 (66%)] Loss: 0.015119
Train Epoch: 2 [40320/60000 (67%)] Loss: 0.041180
Train Epoch: 2 [40960/60000 (68%)] Loss: 0.046142
Train Epoch: 2 [41600/60000 (69%)] Loss: 0.059244
Train Epoch: 2 [42240/60000 (70%)] Loss: 0.031860
Train Epoch: 2 [42880/60000 (71%)] Loss: 0.048548
Train Epoch: 2 [43520/60000 (72%)] Loss: 0.050572
Train Epoch: 2 [44160/60000 (74%)] Loss: 0.014749
Train Epoch: 2 [44800/60000 (75%)] Loss: 0.056841
Train Epoch: 2 [45440/60000 (76%)] Loss: 0.056511
Train Epoch: 2 [46080/60000 (77%)] Loss: 0.057224
Train Epoch: 2 [46720/60000 (78%)] Loss: 0.054993
Train Epoch: 2 [47360/60000 (79%)] Loss: 0.106901
Train Epoch: 2 [48000/60000 (80%)] Loss: 0.035149
Train Epoch: 2 [48640/60000 (81%)] Loss: 0.049964
Train Epoch: 2 [49280/60000 (82%)] Loss: 0.028029
Train Epoch: 2 [49920/60000 (83%)] Loss: 0.041599
Train Epoch: 2 [50560/60000 (84%)] Loss: 0.071151
Train Epoch: 2 [51200/60000 (85%)] Loss: 0.013265
Train Epoch: 2 [51840/60000 (86%)] Loss: 0.028597
Train Epoch: 2 [52480/60000 (87%)] Loss: 0.186898
Train Epoch: 2 [53120/60000 (88%)] Loss: 0.264790
Train Epoch: 2 [53760/60000 (90%)] Loss: 0.137415
Train Epoch: 2 [54400/60000 (91%)] Loss: 0.017394
Train Epoch: 2 [55040/60000 (92%)] Loss: 0.006440
Train Epoch: 2 [55680/60000 (93%)] Loss: 0.129567
Train Epoch: 2 [56320/60000 (94%)] Loss: 0.009801
Train Epoch: 2 [56960/60000 (95%)] Loss: 0.134191
Train Epoch: 2 [57600/60000 (96%)] Loss: 0.014789
Train Epoch: 2 [58240/60000 (97%)] Loss: 0.025339
Train Epoch: 2 [58880/60000 (98%)] Loss: 0.023878
Train Epoch: 2 [59520/60000 (99%)] Loss: 0.166601
Test set: Average loss: 0.0354, Accuracy: 9878/10000 (99%)
Train Epoch: 3 [0/60000 (0%)] Loss: 0.031954
Train Epoch: 3 [640/60000 (1%)] Loss: 0.098245
Train Epoch: 3 [1280/60000 (2%)] Loss: 0.033908
Train Epoch: 3 [1920/60000 (3%)] Loss: 0.030587
Train Epoch: 3 [2560/60000 (4%)] Loss: 0.007363
Train Epoch: 3 [3200/60000 (5%)] Loss: 0.070951
Train Epoch: 3 [3840/60000 (6%)] Loss: 0.014102
Train Epoch: 3 [4480/60000 (7%)] Loss: 0.016992
Train Epoch: 3 [5120/60000 (9%)] Loss: 0.031029
Train Epoch: 3 [5760/60000 (10%)] Loss: 0.009184
Train Epoch: 3 [6400/60000 (11%)] Loss: 0.045750
Train Epoch: 3 [7040/60000 (12%)] Loss: 0.035073
Train Epoch: 3 [7680/60000 (13%)] Loss: 0.054077
Train Epoch: 3 [8320/60000 (14%)] Loss: 0.008315
Train Epoch: 3 [8960/60000 (15%)] Loss: 0.014970
Train Epoch: 3 [9600/60000 (16%)] Loss: 0.095580
Train Epoch: 3 [10240/60000 (17%)] Loss: 0.006922
Train Epoch: 3 [10880/60000 (18%)] Loss: 0.203542
Train Epoch: 3 [11520/60000 (19%)] Loss: 0.055540
Train Epoch: 3 [12160/60000 (20%)] Loss: 0.043469
Train Epoch: 3 [12800/60000 (21%)] Loss: 0.036822
Train Epoch: 3 [13440/60000 (22%)] Loss: 0.018697
Train Epoch: 3 [14080/60000 (23%)] Loss: 0.105894
Train Epoch: 3 [14720/60000 (25%)] Loss: 0.017616
Train Epoch: 3 [15360/60000 (26%)] Loss: 0.028660
Train Epoch: 3 [16000/60000 (27%)] Loss: 0.255229
Train Epoch: 3 [16640/60000 (28%)] Loss: 0.017552
Train Epoch: 3 [17280/60000 (29%)] Loss: 0.071355
Train Epoch: 3 [17920/60000 (30%)] Loss: 0.216176
Train Epoch: 3 [18560/60000 (31%)] Loss: 0.043601
Train Epoch: 3 [19200/60000 (32%)] Loss: 0.063606
Train Epoch: 3 [19840/60000 (33%)] Loss: 0.100486
Train Epoch: 3 [20480/60000 (34%)] Loss: 0.005702
Train Epoch: 3 [21120/60000 (35%)] Loss: 0.003965
Train Epoch: 3 [21760/60000 (36%)] Loss: 0.083673
Train Epoch: 3 [22400/60000 (37%)] Loss: 0.187438
Train Epoch: 3 [23040/60000 (38%)] Loss: 0.214030
Train Epoch: 3 [23680/60000 (39%)] Loss: 0.043855
Train Epoch: 3 [24320/60000 (41%)] Loss: 0.011205
Train Epoch: 3 [24960/60000 (42%)] Loss: 0.099398
Train Epoch: 3 [25600/60000 (43%)] Loss: 0.008929
Train Epoch: 3 [26240/60000 (44%)] Loss: 0.015417
Train Epoch: 3 [26880/60000 (45%)] Loss: 0.003308
Train Epoch: 3 [27520/60000 (46%)] Loss: 0.142496
Train Epoch: 3 [28160/60000 (47%)] Loss: 0.009433
Train Epoch: 3 [28800/60000 (48%)] Loss: 0.179714
Train Epoch: 3 [29440/60000 (49%)] Loss: 0.004949
Train Epoch: 3 [30080/60000 (50%)] Loss: 0.121293
Train Epoch: 3 [30720/60000 (51%)] Loss: 0.107995
Train Epoch: 3 [31360/60000 (52%)] Loss: 0.052988
Train Epoch: 3 [32000/60000 (53%)] Loss: 0.125709
Train Epoch: 3 [32640/60000 (54%)] Loss: 0.016742
Train Epoch: 3 [33280/60000 (55%)] Loss: 0.007047
Train Epoch: 3 [33920/60000 (57%)] Loss: 0.003665
Train Epoch: 3 [34560/60000 (58%)] Loss: 0.004129
Train Epoch: 3 [35200/60000 (59%)] Loss: 0.080460
Train Epoch: 3 [35840/60000 (60%)] Loss: 0.290621
Train Epoch: 3 [36480/60000 (61%)] Loss: 0.069194
Train Epoch: 3 [37120/60000 (62%)] Loss: 0.121097
Train Epoch: 3 [37760/60000 (63%)] Loss: 0.206893
Train Epoch: 3 [38400/60000 (64%)] Loss: 0.025053
Train Epoch: 3 [39040/60000 (65%)] Loss: 0.120507
Train Epoch: 3 [39680/60000 (66%)] Loss: 0.048921
Train Epoch: 3 [40320/60000 (67%)] Loss: 0.035273
Train Epoch: 3 [40960/60000 (68%)] Loss: 0.080340
Train Epoch: 3 [41600/60000 (69%)] Loss: 0.121503
Train Epoch: 3 [42240/60000 (70%)] Loss: 0.054043
Train Epoch: 3 [42880/60000 (71%)] Loss: 0.123253
Train Epoch: 3 [43520/60000 (72%)] Loss: 0.025251
Train Epoch: 3 [44160/60000 (74%)] Loss: 0.007584
Train Epoch: 3 [44800/60000 (75%)] Loss: 0.006557
Train Epoch: 3 [45440/60000 (76%)] Loss: 0.031720
Train Epoch: 3 [46080/60000 (77%)] Loss: 0.014270
Train Epoch: 3 [46720/60000 (78%)] Loss: 0.015007
Train Epoch: 3 [47360/60000 (79%)] Loss: 0.034181
Train Epoch: 3 [48000/60000 (80%)] Loss: 0.022434
Train Epoch: 3 [48640/60000 (81%)] Loss: 0.068090
Train Epoch: 3 [49280/60000 (82%)] Loss: 0.039806
Train Epoch: 3 [49920/60000 (83%)] Loss: 0.035164
Train Epoch: 3 [50560/60000 (84%)] Loss: 0.019233
Train Epoch: 3 [51200/60000 (85%)] Loss: 0.086994
Train Epoch: 3 [51840/60000 (86%)] Loss: 0.002837
Train Epoch: 3 [52480/60000 (87%)] Loss: 0.022604
Train Epoch: 3 [53120/60000 (88%)] Loss: 0.358101
Train Epoch: 3 [53760/60000 (90%)] Loss: 0.100417
Train Epoch: 3 [54400/60000 (91%)] Loss: 0.060839
Train Epoch: 3 [55040/60000 (92%)] Loss: 0.081137
Train Epoch: 3 [55680/60000 (93%)] Loss: 0.026246
Train Epoch: 3 [56320/60000 (94%)] Loss: 0.189262
Train Epoch: 3 [56960/60000 (95%)] Loss: 0.009371
Train Epoch: 3 [57600/60000 (96%)] Loss: 0.005513
Train Epoch: 3 [58240/60000 (97%)] Loss: 0.087663
Train Epoch: 3 [58880/60000 (98%)] Loss: 0.069246
Train Epoch: 3 [59520/60000 (99%)] Loss: 0.062751
Test set: Average loss: 0.0350, Accuracy: 9881/10000 (99%)
Train Epoch: 4 [0/60000 (0%)] Loss: 0.021462
Train Epoch: 4 [640/60000 (1%)] Loss: 0.021030
Train Epoch: 4 [1280/60000 (2%)] Loss: 0.146503
Train Epoch: 4 [1920/60000 (3%)] Loss: 0.009287
Train Epoch: 4 [2560/60000 (4%)] Loss: 0.020573
Train Epoch: 4 [3200/60000 (5%)] Loss: 0.072001
Train Epoch: 4 [3840/60000 (6%)] Loss: 0.031971
Train Epoch: 4 [4480/60000 (7%)] Loss: 0.014062
Train Epoch: 4 [5120/60000 (9%)] Loss: 0.036964
Train Epoch: 4 [5760/60000 (10%)] Loss: 0.011551
Train Epoch: 4 [6400/60000 (11%)] Loss: 0.004171
Train Epoch: 4 [7040/60000 (12%)] Loss: 0.129366
Train Epoch: 4 [7680/60000 (13%)] Loss: 0.049621
Train Epoch: 4 [8320/60000 (14%)] Loss: 0.130965
Train Epoch: 4 [8960/60000 (15%)] Loss: 0.026810
Train Epoch: 4 [9600/60000 (16%)] Loss: 0.012270
Train Epoch: 4 [10240/60000 (17%)] Loss: 0.045667
Train Epoch: 4 [10880/60000 (18%)] Loss: 0.043227
Train Epoch: 4 [11520/60000 (19%)] Loss: 0.029779
Train Epoch: 4 [12160/60000 (20%)] Loss: 0.113995
Train Epoch: 4 [12800/60000 (21%)] Loss: 0.120887
Train Epoch: 4 [13440/60000 (22%)] Loss: 0.014474
Train Epoch: 4 [14080/60000 (23%)] Loss: 0.037020
Train Epoch: 4 [14720/60000 (25%)] Loss: 0.011506
Train Epoch: 4 [15360/60000 (26%)] Loss: 0.006775
Train Epoch: 4 [16000/60000 (27%)] Loss: 0.013694
Train Epoch: 4 [16640/60000 (28%)] Loss: 0.005319
Train Epoch: 4 [17280/60000 (29%)] Loss: 0.002370
Train Epoch: 4 [17920/60000 (30%)] Loss: 0.017354
Train Epoch: 4 [18560/60000 (31%)] Loss: 0.084097
Train Epoch: 4 [19200/60000 (32%)] Loss: 0.007217
Train Epoch: 4 [19840/60000 (33%)] Loss: 0.042848
Train Epoch: 4 [20480/60000 (34%)] Loss: 0.068850
Train Epoch: 4 [21120/60000 (35%)] Loss: 0.011194
Train Epoch: 4 [21760/60000 (36%)] Loss: 0.015206
Train Epoch: 4 [22400/60000 (37%)] Loss: 0.035048
Train Epoch: 4 [23040/60000 (38%)] Loss: 0.003624
Train Epoch: 4 [23680/60000 (39%)] Loss: 0.102620
Train Epoch: 4 [24320/60000 (41%)] Loss: 0.015614
Train Epoch: 4 [24960/60000 (42%)] Loss: 0.093455
Train Epoch: 4 [25600/60000 (43%)] Loss: 0.010973
Train Epoch: 4 [26240/60000 (44%)] Loss: 0.063876
Train Epoch: 4 [26880/60000 (45%)] Loss: 0.014338
Train Epoch: 4 [27520/60000 (46%)] Loss: 0.148824
Train Epoch: 4 [28160/60000 (47%)] Loss: 0.023257
Train Epoch: 4 [28800/60000 (48%)] Loss: 0.046206
Train Epoch: 4 [29440/60000 (49%)] Loss: 0.031617
Train Epoch: 4 [30080/60000 (50%)] Loss: 0.042372
Train Epoch: 4 [30720/60000 (51%)] Loss: 0.161496
Train Epoch: 4 [31360/60000 (52%)] Loss: 0.003014
Train Epoch: 4 [32000/60000 (53%)] Loss: 0.003436
Train Epoch: 4 [32640/60000 (54%)] Loss: 0.028811
Train Epoch: 4 [33280/60000 (55%)] Loss: 0.111324
Train Epoch: 4 [33920/60000 (57%)] Loss: 0.007646
Train Epoch: 4 [34560/60000 (58%)] Loss: 0.012681
Train Epoch: 4 [35200/60000 (59%)] Loss: 0.045081
Train Epoch: 4 [35840/60000 (60%)] Loss: 0.009693
Train Epoch: 4 [36480/60000 (61%)] Loss: 0.029159
Train Epoch: 4 [37120/60000 (62%)] Loss: 0.005575
Train Epoch: 4 [37760/60000 (63%)] Loss: 0.027660
Train Epoch: 4 [38400/60000 (64%)] Loss: 0.008116
Train Epoch: 4 [39040/60000 (65%)] Loss: 0.076387
Train Epoch: 4 [39680/60000 (66%)] Loss: 0.004709
Train Epoch: 4 [40320/60000 (67%)] Loss: 0.017319
Train Epoch: 4 [40960/60000 (68%)] Loss: 0.015712
Train Epoch: 4 [41600/60000 (69%)] Loss: 0.007456
Train Epoch: 4 [42240/60000 (70%)] Loss: 0.006268
Train Epoch: 4 [42880/60000 (71%)] Loss: 0.034743
Train Epoch: 4 [43520/60000 (72%)] Loss: 0.014964
Train Epoch: 4 [44160/60000 (74%)] Loss: 0.003976
Train Epoch: 4 [44800/60000 (75%)] Loss: 0.000111
Train Epoch: 4 [45440/60000 (76%)] Loss: 0.015429
Train Epoch: 4 [46080/60000 (77%)] Loss: 0.090143
Train Epoch: 4 [46720/60000 (78%)] Loss: 0.005872
Train Epoch: 4 [47360/60000 (79%)] Loss: 0.064455
Train Epoch: 4 [48000/60000 (80%)] Loss: 0.014810
Train Epoch: 4 [48640/60000 (81%)] Loss: 0.011236
Train Epoch: 4 [49280/60000 (82%)] Loss: 0.025830
Train Epoch: 4 [49920/60000 (83%)] Loss: 0.004559
Train Epoch: 4 [50560/60000 (84%)] Loss: 0.009476
Train Epoch: 4 [51200/60000 (85%)] Loss: 0.004826
Train Epoch: 4 [51840/60000 (86%)] Loss: 0.057912
Train Epoch: 4 [52480/60000 (87%)] Loss: 0.068570
Train Epoch: 4 [53120/60000 (88%)] Loss: 0.004383
Train Epoch: 4 [53760/60000 (90%)] Loss: 0.002811
Train Epoch: 4 [54400/60000 (91%)] Loss: 0.005478
Train Epoch: 4 [55040/60000 (92%)] Loss: 0.004250
Train Epoch: 4 [55680/60000 (93%)] Loss: 0.006073
Train Epoch: 4 [56320/60000 (94%)] Loss: 0.034624
Train Epoch: 4 [56960/60000 (95%)] Loss: 0.024132
Train Epoch: 4 [57600/60000 (96%)] Loss: 0.043976
Train Epoch: 4 [58240/60000 (97%)] Loss: 0.004622
Train Epoch: 4 [58880/60000 (98%)] Loss: 0.018018
Train Epoch: 4 [59520/60000 (99%)] Loss: 0.067030
Test set: Average loss: 0.0292, Accuracy: 9899/10000 (99%)
Train Epoch: 5 [0/60000 (0%)] Loss: 0.016778
Train Epoch: 5 [640/60000 (1%)] Loss: 0.034248
Train Epoch: 5 [1280/60000 (2%)] Loss: 0.012869
Train Epoch: 5 [1920/60000 (3%)] Loss: 0.013974
Train Epoch: 5 [2560/60000 (4%)] Loss: 0.002261
Train Epoch: 5 [3200/60000 (5%)] Loss: 0.149539
Train Epoch: 5 [3840/60000 (6%)] Loss: 0.107843
Train Epoch: 5 [4480/60000 (7%)] Loss: 0.046836
Train Epoch: 5 [5120/60000 (9%)] Loss: 0.056937
Train Epoch: 5 [5760/60000 (10%)] Loss: 0.040649
Train Epoch: 5 [6400/60000 (11%)] Loss: 0.002701
Train Epoch: 5 [7040/60000 (12%)] Loss: 0.000842
Train Epoch: 5 [7680/60000 (13%)] Loss: 0.010104
Train Epoch: 5 [8320/60000 (14%)] Loss: 0.003840
Train Epoch: 5 [8960/60000 (15%)] Loss: 0.026385
Train Epoch: 5 [9600/60000 (16%)] Loss: 0.017698
Train Epoch: 5 [10240/60000 (17%)] Loss: 0.003605
Train Epoch: 5 [10880/60000 (18%)] Loss: 0.059263
Train Epoch: 5 [11520/60000 (19%)] Loss: 0.066392
Train Epoch: 5 [12160/60000 (20%)] Loss: 0.011516
Train Epoch: 5 [12800/60000 (21%)] Loss: 0.007131
Train Epoch: 5 [13440/60000 (22%)] Loss: 0.035553
Train Epoch: 5 [14080/60000 (23%)] Loss: 0.004083
Train Epoch: 5 [14720/60000 (25%)] Loss: 0.002642
Train Epoch: 5 [15360/60000 (26%)] Loss: 0.257820
Train Epoch: 5 [16000/60000 (27%)] Loss: 0.083252
Train Epoch: 5 [16640/60000 (28%)] Loss: 0.090598
Train Epoch: 5 [17280/60000 (29%)] Loss: 0.011982
Train Epoch: 5 [17920/60000 (30%)] Loss: 0.023577
Train Epoch: 5 [18560/60000 (31%)] Loss: 0.049298
Train Epoch: 5 [19200/60000 (32%)] Loss: 0.016358
Train Epoch: 5 [19840/60000 (33%)] Loss: 0.017522
Train Epoch: 5 [20480/60000 (34%)] Loss: 0.012936
Train Epoch: 5 [21120/60000 (35%)] Loss: 0.010544
Train Epoch: 5 [21760/60000 (36%)] Loss: 0.017033
Train Epoch: 5 [22400/60000 (37%)] Loss: 0.039964
Train Epoch: 5 [23040/60000 (38%)] Loss: 0.072129
Train Epoch: 5 [23680/60000 (39%)] Loss: 0.022347
Train Epoch: 5 [24320/60000 (41%)] Loss: 0.013183
Train Epoch: 5 [24960/60000 (42%)] Loss: 0.000919
Train Epoch: 5 [25600/60000 (43%)] Loss: 0.002910
Train Epoch: 5 [26240/60000 (44%)] Loss: 0.059928
Train Epoch: 5 [26880/60000 (45%)] Loss: 0.263032
Train Epoch: 5 [27520/60000 (46%)] Loss: 0.101784
Train Epoch: 5 [28160/60000 (47%)] Loss: 0.034156
Train Epoch: 5 [28800/60000 (48%)] Loss: 0.043765
Train Epoch: 5 [29440/60000 (49%)] Loss: 0.021913
Train Epoch: 5 [30080/60000 (50%)] Loss: 0.047016
Train Epoch: 5 [30720/60000 (51%)] Loss: 0.001282
Train Epoch: 5 [31360/60000 (52%)] Loss: 0.040013
Train Epoch: 5 [32000/60000 (53%)] Loss: 0.155510
Train Epoch: 5 [32640/60000 (54%)] Loss: 0.025113
Train Epoch: 5 [33280/60000 (55%)] Loss: 0.047155
Train Epoch: 5 [33920/60000 (57%)] Loss: 0.005715
Train Epoch: 5 [34560/60000 (58%)] Loss: 0.037122
Train Epoch: 5 [35200/60000 (59%)] Loss: 0.023239
Train Epoch: 5 [35840/60000 (60%)] Loss: 0.013034
Train Epoch: 5 [36480/60000 (61%)] Loss: 0.001897
Train Epoch: 5 [37120/60000 (62%)] Loss: 0.002091
Train Epoch: 5 [37760/60000 (63%)] Loss: 0.089172
Train Epoch: 5 [38400/60000 (64%)] Loss: 0.006358
Train Epoch: 5 [39040/60000 (65%)] Loss: 0.013635
Train Epoch: 5 [39680/60000 (66%)] Loss: 0.063116
Train Epoch: 5 [40320/60000 (67%)] Loss: 0.036646
Train Epoch: 5 [40960/60000 (68%)] Loss: 0.045822
Train Epoch: 5 [41600/60000 (69%)] Loss: 0.007596
Train Epoch: 5 [42240/60000 (70%)] Loss: 0.023370
Train Epoch: 5 [42880/60000 (71%)] Loss: 0.077922
Train Epoch: 5 [43520/60000 (72%)] Loss: 0.080046
Train Epoch: 5 [44160/60000 (74%)] Loss: 0.007909
Train Epoch: 5 [44800/60000 (75%)] Loss: 0.011852
Train Epoch: 5 [45440/60000 (76%)] Loss: 0.023438
Train Epoch: 5 [46080/60000 (77%)] Loss: 0.021004
Train Epoch: 5 [46720/60000 (78%)] Loss: 0.012173
Train Epoch: 5 [47360/60000 (79%)] Loss: 0.017805
Train Epoch: 5 [48000/60000 (80%)] Loss: 0.057742
Train Epoch: 5 [48640/60000 (81%)] Loss: 0.136364
Train Epoch: 5 [49280/60000 (82%)] Loss: 0.083794
Train Epoch: 5 [49920/60000 (83%)] Loss: 0.096774
Train Epoch: 5 [50560/60000 (84%)] Loss: 0.038170
Train Epoch: 5 [51200/60000 (85%)] Loss: 0.003804
Train Epoch: 5 [51840/60000 (86%)] Loss: 0.103958
Train Epoch: 5 [52480/60000 (87%)] Loss: 0.096511
Train Epoch: 5 [53120/60000 (88%)] Loss: 0.019337
Train Epoch: 5 [53760/60000 (90%)] Loss: 0.002442
Train Epoch: 5 [54400/60000 (91%)] Loss: 0.028027
Train Epoch: 5 [55040/60000 (92%)] Loss: 0.002171
Train Epoch: 5 [55680/60000 (93%)] Loss: 0.038744
Train Epoch: 5 [56320/60000 (94%)] Loss: 0.004720
Train Epoch: 5 [56960/60000 (95%)] Loss: 0.030969
Train Epoch: 5 [57600/60000 (96%)] Loss: 0.008049
Train Epoch: 5 [58240/60000 (97%)] Loss: 0.141733
Train Epoch: 5 [58880/60000 (98%)] Loss: 0.000649
Train Epoch: 5 [59520/60000 (99%)] Loss: 0.002005
Test set: Average loss: 0.0290, Accuracy: 9902/10000 (99%)
Train Epoch: 6 [0/60000 (0%)] Loss: 0.035874
Train Epoch: 6 [640/60000 (1%)] Loss: 0.015234
Train Epoch: 6 [1280/60000 (2%)] Loss: 0.106802
Train Epoch: 6 [1920/60000 (3%)] Loss: 0.232045
Train Epoch: 6 [2560/60000 (4%)] Loss: 0.057787
Train Epoch: 6 [3200/60000 (5%)] Loss: 0.013683
Train Epoch: 6 [3840/60000 (6%)] Loss: 0.022014
Train Epoch: 6 [4480/60000 (7%)] Loss: 0.023238
Train Epoch: 6 [5120/60000 (9%)] Loss: 0.025405
Train Epoch: 6 [5760/60000 (10%)] Loss: 0.029373
Train Epoch: 6 [6400/60000 (11%)] Loss: 0.018778
Train Epoch: 6 [7040/60000 (12%)] Loss: 0.005019
Train Epoch: 6 [7680/60000 (13%)] Loss: 0.002476
Train Epoch: 6 [8320/60000 (14%)] Loss: 0.017168
Train Epoch: 6 [8960/60000 (15%)] Loss: 0.025131
Train Epoch: 6 [9600/60000 (16%)] Loss: 0.035988
Train Epoch: 6 [10240/60000 (17%)] Loss: 0.007058
Train Epoch: 6 [10880/60000 (18%)] Loss: 0.005135
Train Epoch: 6 [11520/60000 (19%)] Loss: 0.013429
Train Epoch: 6 [12160/60000 (20%)] Loss: 0.006595
Train Epoch: 6 [12800/60000 (21%)] Loss: 0.060380
Train Epoch: 6 [13440/60000 (22%)] Loss: 0.007521
Train Epoch: 6 [14080/60000 (23%)] Loss: 0.050194
Train Epoch: 6 [14720/60000 (25%)] Loss: 0.007325
Train Epoch: 6 [15360/60000 (26%)] Loss: 0.078446
Train Epoch: 6 [16000/60000 (27%)] Loss: 0.016103
Train Epoch: 6 [16640/60000 (28%)] Loss: 0.011763
Train Epoch: 6 [17280/60000 (29%)] Loss: 0.003992
Train Epoch: 6 [17920/60000 (30%)] Loss: 0.073336
Train Epoch: 6 [18560/60000 (31%)] Loss: 0.004513
Train Epoch: 6 [19200/60000 (32%)] Loss: 0.046354
Train Epoch: 6 [19840/60000 (33%)] Loss: 0.013252
Train Epoch: 6 [20480/60000 (34%)] Loss: 0.084399
Train Epoch: 6 [21120/60000 (35%)] Loss: 0.010710
Train Epoch: 6 [21760/60000 (36%)] Loss: 0.012121
Train Epoch: 6 [22400/60000 (37%)] Loss: 0.005189
Train Epoch: 6 [23040/60000 (38%)] Loss: 0.023924
Train Epoch: 6 [23680/60000 (39%)] Loss: 0.011361
Train Epoch: 6 [24320/60000 (41%)] Loss: 0.019906
Train Epoch: 6 [24960/60000 (42%)] Loss: 0.020395
Train Epoch: 6 [25600/60000 (43%)] Loss: 0.004499
Train Epoch: 6 [26240/60000 (44%)] Loss: 0.075138
Train Epoch: 6 [26880/60000 (45%)] Loss: 0.006783
Train Epoch: 6 [27520/60000 (46%)] Loss: 0.005234
Train Epoch: 6 [28160/60000 (47%)] Loss: 0.007840
Train Epoch: 6 [28800/60000 (48%)] Loss: 0.032227
Train Epoch: 6 [29440/60000 (49%)] Loss: 0.025816
Train Epoch: 6 [30080/60000 (50%)] Loss: 0.079187
Train Epoch: 6 [30720/60000 (51%)] Loss: 0.008943
Train Epoch: 6 [31360/60000 (52%)] Loss: 0.052444
Train Epoch: 6 [32000/60000 (53%)] Loss: 0.011039
Train Epoch: 6 [32640/60000 (54%)] Loss: 0.235901
Train Epoch: 6 [33280/60000 (55%)] Loss: 0.013927
Train Epoch: 6 [33920/60000 (57%)] Loss: 0.015742
Train Epoch: 6 [34560/60000 (58%)] Loss: 0.080295
Train Epoch: 6 [35200/60000 (59%)] Loss: 0.022024
Train Epoch: 6 [35840/60000 (60%)] Loss: 0.039387
Train Epoch: 6 [36480/60000 (61%)] Loss: 0.012625
Train Epoch: 6 [37120/60000 (62%)] Loss: 0.058724
Train Epoch: 6 [37760/60000 (63%)] Loss: 0.001260
Train Epoch: 6 [38400/60000 (64%)] Loss: 0.005475
Train Epoch: 6 [39040/60000 (65%)] Loss: 0.027949
Train Epoch: 6 [39680/60000 (66%)] Loss: 0.012357
Train Epoch: 6 [40320/60000 (67%)] Loss: 0.015415
Train Epoch: 6 [40960/60000 (68%)] Loss: 0.017720
Train Epoch: 6 [41600/60000 (69%)] Loss: 0.127825
Train Epoch: 6 [42240/60000 (70%)] Loss: 0.033114
Train Epoch: 6 [42880/60000 (71%)] Loss: 0.154449
Train Epoch: 6 [43520/60000 (72%)] Loss: 0.060201
Train Epoch: 6 [44160/60000 (74%)] Loss: 0.013477
Train Epoch: 6 [44800/60000 (75%)] Loss: 0.013566
Train Epoch: 6 [45440/60000 (76%)] Loss: 0.032102
Train Epoch: 6 [46080/60000 (77%)] Loss: 0.173121
Train Epoch: 6 [46720/60000 (78%)] Loss: 0.003324
Train Epoch: 6 [47360/60000 (79%)] Loss: 0.030368
Train Epoch: 6 [48000/60000 (80%)] Loss: 0.014489
Train Epoch: 6 [48640/60000 (81%)] Loss: 0.016031
Train Epoch: 6 [49280/60000 (82%)] Loss: 0.018323
Train Epoch: 6 [49920/60000 (83%)] Loss: 0.071169
Train Epoch: 6 [50560/60000 (84%)] Loss: 0.049739
Train Epoch: 6 [51200/60000 (85%)] Loss: 0.008135
Train Epoch: 6 [51840/60000 (86%)] Loss: 0.003430
Train Epoch: 6 [52480/60000 (87%)] Loss: 0.004954
Train Epoch: 6 [53120/60000 (88%)] Loss: 0.016208
Train Epoch: 6 [53760/60000 (90%)] Loss: 0.078187
Train Epoch: 6 [54400/60000 (91%)] Loss: 0.077117
Train Epoch: 6 [55040/60000 (92%)] Loss: 0.029709
Train Epoch: 6 [55680/60000 (93%)] Loss: 0.023949
Train Epoch: 6 [56320/60000 (94%)] Loss: 0.025149
Train Epoch: 6 [56960/60000 (95%)] Loss: 0.004128
Train Epoch: 6 [57600/60000 (96%)] Loss: 0.039843
Train Epoch: 6 [58240/60000 (97%)] Loss: 0.009665
Train Epoch: 6 [58880/60000 (98%)] Loss: 0.031347
Train Epoch: 6 [59520/60000 (99%)] Loss: 0.007405
Test set: Average loss: 0.0265, Accuracy: 9916/10000 (99%)
Train Epoch: 7 [0/60000 (0%)] Loss: 0.018384
Train Epoch: 7 [640/60000 (1%)] Loss: 0.496064
Train Epoch: 7 [1280/60000 (2%)] Loss: 0.002627
Train Epoch: 7 [1920/60000 (3%)] Loss: 0.002680
Train Epoch: 7 [2560/60000 (4%)] Loss: 0.049651
Train Epoch: 7 [3200/60000 (5%)] Loss: 0.094052
Train Epoch: 7 [3840/60000 (6%)] Loss: 0.004991
Train Epoch: 7 [4480/60000 (7%)] Loss: 0.003527
Train Epoch: 7 [5120/60000 (9%)] Loss: 0.027425
Train Epoch: 7 [5760/60000 (10%)] Loss: 0.077459
Train Epoch: 7 [6400/60000 (11%)] Loss: 0.011377
Train Epoch: 7 [7040/60000 (12%)] Loss: 0.016814
Train Epoch: 7 [7680/60000 (13%)] Loss: 0.019747
Train Epoch: 7 [8320/60000 (14%)] Loss: 0.032541
Train Epoch: 7 [8960/60000 (15%)] Loss: 0.003562
Train Epoch: 7 [9600/60000 (16%)] Loss: 0.000970
Train Epoch: 7 [10240/60000 (17%)] Loss: 0.015149
Train Epoch: 7 [10880/60000 (18%)] Loss: 0.029264
Train Epoch: 7 [11520/60000 (19%)] Loss: 0.052042
Train Epoch: 7 [12160/60000 (20%)] Loss: 0.191418
Train Epoch: 7 [12800/60000 (21%)] Loss: 0.021183
Train Epoch: 7 [13440/60000 (22%)] Loss: 0.003795
Train Epoch: 7 [14080/60000 (23%)] Loss: 0.020874
Train Epoch: 7 [14720/60000 (25%)] Loss: 0.017544
Train Epoch: 7 [15360/60000 (26%)] Loss: 0.001321
Train Epoch: 7 [16000/60000 (27%)] Loss: 0.004796
Train Epoch: 7 [16640/60000 (28%)] Loss: 0.009283
Train Epoch: 7 [17280/60000 (29%)] Loss: 0.047032
Train Epoch: 7 [17920/60000 (30%)] Loss: 0.107188
Train Epoch: 7 [18560/60000 (31%)] Loss: 0.026144
Train Epoch: 7 [19200/60000 (32%)] Loss: 0.069030
Train Epoch: 7 [19840/60000 (33%)] Loss: 0.019213
Train Epoch: 7 [20480/60000 (34%)] Loss: 0.008449
Train Epoch: 7 [21120/60000 (35%)] Loss: 0.003225
Train Epoch: 7 [21760/60000 (36%)] Loss: 0.015505
Train Epoch: 7 [22400/60000 (37%)] Loss: 0.010573
Train Epoch: 7 [23040/60000 (38%)] Loss: 0.037231
Train Epoch: 7 [23680/60000 (39%)] Loss: 0.020584
Train Epoch: 7 [24320/60000 (41%)] Loss: 0.004110
Train Epoch: 7 [24960/60000 (42%)] Loss: 0.052341
Train Epoch: 7 [25600/60000 (43%)] Loss: 0.034880
Train Epoch: 7 [26240/60000 (44%)] Loss: 0.022722
Train Epoch: 7 [26880/60000 (45%)] Loss: 0.022698
Train Epoch: 7 [27520/60000 (46%)] Loss: 0.000951
Train Epoch: 7 [28160/60000 (47%)] Loss: 0.007782
Train Epoch: 7 [28800/60000 (48%)] Loss: 0.018998
Train Epoch: 7 [29440/60000 (49%)] Loss: 0.005576
Train Epoch: 7 [30080/60000 (50%)] Loss: 0.074849
Train Epoch: 7 [30720/60000 (51%)] Loss: 0.072354
Train Epoch: 7 [31360/60000 (52%)] Loss: 0.001896
Train Epoch: 7 [32000/60000 (53%)] Loss: 0.033901
Train Epoch: 7 [32640/60000 (54%)] Loss: 0.114391
Train Epoch: 7 [33280/60000 (55%)] Loss: 0.051415
Train Epoch: 7 [33920/60000 (57%)] Loss: 0.009272
Train Epoch: 7 [34560/60000 (58%)] Loss: 0.001477
Train Epoch: 7 [35200/60000 (59%)] Loss: 0.003052
Train Epoch: 7 [35840/60000 (60%)] Loss: 0.010101
Train Epoch: 7 [36480/60000 (61%)] Loss: 0.010736
Train Epoch: 7 [37120/60000 (62%)] Loss: 0.001869
Train Epoch: 7 [37760/60000 (63%)] Loss: 0.075402
Train Epoch: 7 [38400/60000 (64%)] Loss: 0.006963
Train Epoch: 7 [39040/60000 (65%)] Loss: 0.055510
Train Epoch: 7 [39680/60000 (66%)] Loss: 0.051980
Train Epoch: 7 [40320/60000 (67%)] Loss: 0.018207
Train Epoch: 7 [40960/60000 (68%)] Loss: 0.013079
Train Epoch: 7 [41600/60000 (69%)] Loss: 0.006033
Train Epoch: 7 [42240/60000 (70%)] Loss: 0.003060
Train Epoch: 7 [42880/60000 (71%)] Loss: 0.025844
Train Epoch: 7 [43520/60000 (72%)] Loss: 0.106046
Train Epoch: 7 [44160/60000 (74%)] Loss: 0.006455
Train Epoch: 7 [44800/60000 (75%)] Loss: 0.076891
Train Epoch: 7 [45440/60000 (76%)] Loss: 0.088425
Train Epoch: 7 [46080/60000 (77%)] Loss: 0.007998
Train Epoch: 7 [46720/60000 (78%)] Loss: 0.037086
Train Epoch: 7 [47360/60000 (79%)] Loss: 0.061713
Train Epoch: 7 [48000/60000 (80%)] Loss: 0.009340
Train Epoch: 7 [48640/60000 (81%)] Loss: 0.052292
Train Epoch: 7 [49280/60000 (82%)] Loss: 0.078367
Train Epoch: 7 [49920/60000 (83%)] Loss: 0.011505
Train Epoch: 7 [50560/60000 (84%)] Loss: 0.032816
Train Epoch: 7 [51200/60000 (85%)] Loss: 0.034384
Train Epoch: 7 [51840/60000 (86%)] Loss: 0.054743
Train Epoch: 7 [52480/60000 (87%)] Loss: 0.041077
Train Epoch: 7 [53120/60000 (88%)] Loss: 0.001048
Train Epoch: 7 [53760/60000 (90%)] Loss: 0.000958
Train Epoch: 7 [54400/60000 (91%)] Loss: 0.006497
Train Epoch: 7 [55040/60000 (92%)] Loss: 0.064084
Train Epoch: 7 [55680/60000 (93%)] Loss: 0.004225
Train Epoch: 7 [56320/60000 (94%)] Loss: 0.008345
Train Epoch: 7 [56960/60000 (95%)] Loss: 0.035556
Train Epoch: 7 [57600/60000 (96%)] Loss: 0.006555
Train Epoch: 7 [58240/60000 (97%)] Loss: 0.011516
Train Epoch: 7 [58880/60000 (98%)] Loss: 0.063590
Train Epoch: 7 [59520/60000 (99%)] Loss: 0.083176
Test set: Average loss: 0.0261, Accuracy: 9919/10000 (99%)
Train Epoch: 8 [0/60000 (0%)] Loss: 0.026821
Train Epoch: 8 [640/60000 (1%)] Loss: 0.017867
Train Epoch: 8 [1280/60000 (2%)] Loss: 0.016649
Train Epoch: 8 [1920/60000 (3%)] Loss: 0.006549
Train Epoch: 8 [2560/60000 (4%)] Loss: 0.057640
Train Epoch: 8 [3200/60000 (5%)] Loss: 0.001056
Train Epoch: 8 [3840/60000 (6%)] Loss: 0.004022
Train Epoch: 8 [4480/60000 (7%)] Loss: 0.000680
Train Epoch: 8 [5120/60000 (9%)] Loss: 0.018457
Train Epoch: 8 [5760/60000 (10%)] Loss: 0.073352
Train Epoch: 8 [6400/60000 (11%)] Loss: 0.009088
Train Epoch: 8 [7040/60000 (12%)] Loss: 0.025987
Train Epoch: 8 [7680/60000 (13%)] Loss: 0.022370
Train Epoch: 8 [8320/60000 (14%)] Loss: 0.023301
Train Epoch: 8 [8960/60000 (15%)] Loss: 0.004061
Train Epoch: 8 [9600/60000 (16%)] Loss: 0.001865
Train Epoch: 8 [10240/60000 (17%)] Loss: 0.024155
Train Epoch: 8 [10880/60000 (18%)] Loss: 0.024145
Train Epoch: 8 [11520/60000 (19%)] Loss: 0.002700
Train Epoch: 8 [12160/60000 (20%)] Loss: 0.007374
Train Epoch: 8 [12800/60000 (21%)] Loss: 0.083919
Train Epoch: 8 [13440/60000 (22%)] Loss: 0.084770
Train Epoch: 8 [14080/60000 (23%)] Loss: 0.008005
Train Epoch: 8 [14720/60000 (25%)] Loss: 0.004969
Train Epoch: 8 [15360/60000 (26%)] Loss: 0.016590
Train Epoch: 8 [16000/60000 (27%)] Loss: 0.004098
Train Epoch: 8 [16640/60000 (28%)] Loss: 0.057866
Train Epoch: 8 [17280/60000 (29%)] Loss: 0.106917
Train Epoch: 8 [17920/60000 (30%)] Loss: 0.005900
Train Epoch: 8 [18560/60000 (31%)] Loss: 0.000704
Train Epoch: 8 [19200/60000 (32%)] Loss: 0.003374
Train Epoch: 8 [19840/60000 (33%)] Loss: 0.004536
Train Epoch: 8 [20480/60000 (34%)] Loss: 0.013632
Train Epoch: 8 [21120/60000 (35%)] Loss: 0.000729
Train Epoch: 8 [21760/60000 (36%)] Loss: 0.009512
Train Epoch: 8 [22400/60000 (37%)] Loss: 0.004544
Train Epoch: 8 [23040/60000 (38%)] Loss: 0.007051
Train Epoch: 8 [23680/60000 (39%)] Loss: 0.001681
Train Epoch: 8 [24320/60000 (41%)] Loss: 0.199115
Train Epoch: 8 [24960/60000 (42%)] Loss: 0.072970
Train Epoch: 8 [25600/60000 (43%)] Loss: 0.017623
Train Epoch: 8 [26240/60000 (44%)] Loss: 0.037379
Train Epoch: 8 [26880/60000 (45%)] Loss: 0.114546
Train Epoch: 8 [27520/60000 (46%)] Loss: 0.120858
Train Epoch: 8 [28160/60000 (47%)] Loss: 0.026314
Train Epoch: 8 [28800/60000 (48%)] Loss: 0.014088
Train Epoch: 8 [29440/60000 (49%)] Loss: 0.032632
Train Epoch: 8 [30080/60000 (50%)] Loss: 0.090679
Train Epoch: 8 [30720/60000 (51%)] Loss: 0.006700
Train Epoch: 8 [31360/60000 (52%)] Loss: 0.013988
Train Epoch: 8 [32000/60000 (53%)] Loss: 0.003472
Train Epoch: 8 [32640/60000 (54%)] Loss: 0.042974
Train Epoch: 8 [33280/60000 (55%)] Loss: 0.015423
Train Epoch: 8 [33920/60000 (57%)] Loss: 0.035945
Train Epoch: 8 [34560/60000 (58%)] Loss: 0.002197
Train Epoch: 8 [35200/60000 (59%)] Loss: 0.005037
Train Epoch: 8 [35840/60000 (60%)] Loss: 0.000486
Train Epoch: 8 [36480/60000 (61%)] Loss: 0.004266
Train Epoch: 8 [37120/60000 (62%)] Loss: 0.028284
Train Epoch: 8 [37760/60000 (63%)] Loss: 0.006714
Train Epoch: 8 [38400/60000 (64%)] Loss: 0.000692
Train Epoch: 8 [39040/60000 (65%)] Loss: 0.005868
Train Epoch: 8 [39680/60000 (66%)] Loss: 0.111093
Train Epoch: 8 [40320/60000 (67%)] Loss: 0.046122
Train Epoch: 8 [40960/60000 (68%)] Loss: 0.103231
Train Epoch: 8 [41600/60000 (69%)] Loss: 0.012917
Train Epoch: 8 [42240/60000 (70%)] Loss: 0.311769
Train Epoch: 8 [42880/60000 (71%)] Loss: 0.001156
Train Epoch: 8 [43520/60000 (72%)] Loss: 0.018721
Train Epoch: 8 [44160/60000 (74%)] Loss: 0.088779
Train Epoch: 8 [44800/60000 (75%)] Loss: 0.000345
Train Epoch: 8 [45440/60000 (76%)] Loss: 0.081547
Train Epoch: 8 [46080/60000 (77%)] Loss: 0.226403
Train Epoch: 8 [46720/60000 (78%)] Loss: 0.006353
Train Epoch: 8 [47360/60000 (79%)] Loss: 0.003092
Train Epoch: 8 [48000/60000 (80%)] Loss: 0.009313
Train Epoch: 8 [48640/60000 (81%)] Loss: 0.002984
Train Epoch: 8 [49280/60000 (82%)] Loss: 0.030551
Train Epoch: 8 [49920/60000 (83%)] Loss: 0.004961
Train Epoch: 8 [50560/60000 (84%)] Loss: 0.044408
Train Epoch: 8 [51200/60000 (85%)] Loss: 0.011096
Train Epoch: 8 [51840/60000 (86%)] Loss: 0.024380
Train Epoch: 8 [52480/60000 (87%)] Loss: 0.002835
Train Epoch: 8 [53120/60000 (88%)] Loss: 0.018776
Train Epoch: 8 [53760/60000 (90%)] Loss: 0.250061
Train Epoch: 8 [54400/60000 (91%)] Loss: 0.009001
Train Epoch: 8 [55040/60000 (92%)] Loss: 0.009098
Train Epoch: 8 [55680/60000 (93%)] Loss: 0.005339
Train Epoch: 8 [56320/60000 (94%)] Loss: 0.007335
Train Epoch: 8 [56960/60000 (95%)] Loss: 0.007425
Train Epoch: 8 [57600/60000 (96%)] Loss: 0.011311
Train Epoch: 8 [58240/60000 (97%)] Loss: 0.068839
Train Epoch: 8 [58880/60000 (98%)] Loss: 0.012238
Train Epoch: 8 [59520/60000 (99%)] Loss: 0.017977
Test set: Average loss: 0.0257, Accuracy: 9926/10000 (99%)
Train Epoch: 9 [0/60000 (0%)] Loss: 0.011956
Train Epoch: 9 [640/60000 (1%)] Loss: 0.044432
Train Epoch: 9 [1280/60000 (2%)] Loss: 0.014100
Train Epoch: 9 [1920/60000 (3%)] Loss: 0.007170
Train Epoch: 9 [2560/60000 (4%)] Loss: 0.002061
Train Epoch: 9 [3200/60000 (5%)] Loss: 0.000618
Train Epoch: 9 [3840/60000 (6%)] Loss: 0.064108
Train Epoch: 9 [4480/60000 (7%)] Loss: 0.014028
Train Epoch: 9 [5120/60000 (9%)] Loss: 0.000927
Train Epoch: 9 [5760/60000 (10%)] Loss: 0.087000
Train Epoch: 9 [6400/60000 (11%)] Loss: 0.044170
Train Epoch: 9 [7040/60000 (12%)] Loss: 0.003181
Train Epoch: 9 [7680/60000 (13%)] Loss: 0.020482
Train Epoch: 9 [8320/60000 (14%)] Loss: 0.000891
Train Epoch: 9 [8960/60000 (15%)] Loss: 0.017776
Train Epoch: 9 [9600/60000 (16%)] Loss: 0.040190
Train Epoch: 9 [10240/60000 (17%)] Loss: 0.032901
Train Epoch: 9 [10880/60000 (18%)] Loss: 0.003509
Train Epoch: 9 [11520/60000 (19%)] Loss: 0.002295
Train Epoch: 9 [12160/60000 (20%)] Loss: 0.013485
Train Epoch: 9 [12800/60000 (21%)] Loss: 0.046666
Train Epoch: 9 [13440/60000 (22%)] Loss: 0.057709
Train Epoch: 9 [14080/60000 (23%)] Loss: 0.047336
Train Epoch: 9 [14720/60000 (25%)] Loss: 0.003375
Train Epoch: 9 [15360/60000 (26%)] Loss: 0.039654
Train Epoch: 9 [16000/60000 (27%)] Loss: 0.006855
Train Epoch: 9 [16640/60000 (28%)] Loss: 0.023772
Train Epoch: 9 [17280/60000 (29%)] Loss: 0.011993
Train Epoch: 9 [17920/60000 (30%)] Loss: 0.024049
Train Epoch: 9 [18560/60000 (31%)] Loss: 0.047730
Train Epoch: 9 [19200/60000 (32%)] Loss: 0.008789
Train Epoch: 9 [19840/60000 (33%)] Loss: 0.216059
Train Epoch: 9 [20480/60000 (34%)] Loss: 0.004184
Train Epoch: 9 [21120/60000 (35%)] Loss: 0.072528
Train Epoch: 9 [21760/60000 (36%)] Loss: 0.009422
Train Epoch: 9 [22400/60000 (37%)] Loss: 0.013074
Train Epoch: 9 [23040/60000 (38%)] Loss: 0.017050
Train Epoch: 9 [23680/60000 (39%)] Loss: 0.000827
Train Epoch: 9 [24320/60000 (41%)] Loss: 0.071603
Train Epoch: 9 [24960/60000 (42%)] Loss: 0.108789
Train Epoch: 9 [25600/60000 (43%)] Loss: 0.005479
Train Epoch: 9 [26240/60000 (44%)] Loss: 0.013350
Train Epoch: 9 [26880/60000 (45%)] Loss: 0.006925
Train Epoch: 9 [27520/60000 (46%)] Loss: 0.172379
Train Epoch: 9 [28160/60000 (47%)] Loss: 0.007001
Train Epoch: 9 [28800/60000 (48%)] Loss: 0.013274
Train Epoch: 9 [29440/60000 (49%)] Loss: 0.003844
Train Epoch: 9 [30080/60000 (50%)] Loss: 0.032776
Train Epoch: 9 [30720/60000 (51%)] Loss: 0.002211
Train Epoch: 9 [31360/60000 (52%)] Loss: 0.004652
Train Epoch: 9 [32000/60000 (53%)] Loss: 0.011554
Train Epoch: 9 [32640/60000 (54%)] Loss: 0.010572
Train Epoch: 9 [33280/60000 (55%)] Loss: 0.007962
Train Epoch: 9 [33920/60000 (57%)] Loss: 0.006437
Train Epoch: 9 [34560/60000 (58%)] Loss: 0.003528
Train Epoch: 9 [35200/60000 (59%)] Loss: 0.011768
Train Epoch: 9 [35840/60000 (60%)] Loss: 0.053583
Train Epoch: 9 [36480/60000 (61%)] Loss: 0.014126
Train Epoch: 9 [37120/60000 (62%)] Loss: 0.013082
Train Epoch: 9 [37760/60000 (63%)] Loss: 0.004973
Train Epoch: 9 [38400/60000 (64%)] Loss: 0.020327
Train Epoch: 9 [39040/60000 (65%)] Loss: 0.002645
Train Epoch: 9 [39680/60000 (66%)] Loss: 0.001748
Train Epoch: 9 [40320/60000 (67%)] Loss: 0.030702
Train Epoch: 9 [40960/60000 (68%)] Loss: 0.020649
Train Epoch: 9 [41600/60000 (69%)] Loss: 0.017096
Train Epoch: 9 [42240/60000 (70%)] Loss: 0.040790
Train Epoch: 9 [42880/60000 (71%)] Loss: 0.031363
Train Epoch: 9 [43520/60000 (72%)] Loss: 0.005746
Train Epoch: 9 [44160/60000 (74%)] Loss: 0.014666
Train Epoch: 9 [44800/60000 (75%)] Loss: 0.012592
Train Epoch: 9 [45440/60000 (76%)] Loss: 0.075322
Train Epoch: 9 [46080/60000 (77%)] Loss: 0.002169
Train Epoch: 9 [46720/60000 (78%)] Loss: 0.035429
Train Epoch: 9 [47360/60000 (79%)] Loss: 0.002059
Train Epoch: 9 [48000/60000 (80%)] Loss: 0.005575
Train Epoch: 9 [48640/60000 (81%)] Loss: 0.010623
Train Epoch: 9 [49280/60000 (82%)] Loss: 0.015657
Train Epoch: 9 [49920/60000 (83%)] Loss: 0.342543
Train Epoch: 9 [50560/60000 (84%)] Loss: 0.210901
Train Epoch: 9 [51200/60000 (85%)] Loss: 0.046940
Train Epoch: 9 [51840/60000 (86%)] Loss: 0.009209
Train Epoch: 9 [52480/60000 (87%)] Loss: 0.012542
Train Epoch: 9 [53120/60000 (88%)] Loss: 0.080083
Train Epoch: 9 [53760/60000 (90%)] Loss: 0.016926
Train Epoch: 9 [54400/60000 (91%)] Loss: 0.049153
Train Epoch: 9 [55040/60000 (92%)] Loss: 0.005334
Train Epoch: 9 [55680/60000 (93%)] Loss: 0.009768
Train Epoch: 9 [56320/60000 (94%)] Loss: 0.123350
Train Epoch: 9 [56960/60000 (95%)] Loss: 0.008641
Train Epoch: 9 [57600/60000 (96%)] Loss: 0.038723
Train Epoch: 9 [58240/60000 (97%)] Loss: 0.018208
Train Epoch: 9 [58880/60000 (98%)] Loss: 0.004663
Train Epoch: 9 [59520/60000 (99%)] Loss: 0.026513
Test set: Average loss: 0.0267, Accuracy: 9923/10000 (99%)
Train Epoch: 10 [0/60000 (0%)] Loss: 0.009204
Train Epoch: 10 [640/60000 (1%)] Loss: 0.000488
Train Epoch: 10 [1280/60000 (2%)] Loss: 0.003041
Train Epoch: 10 [1920/60000 (3%)] Loss: 0.044894
Train Epoch: 10 [2560/60000 (4%)] Loss: 0.010415
Train Epoch: 10 [3200/60000 (5%)] Loss: 0.009566
Train Epoch: 10 [3840/60000 (6%)] Loss: 0.051050
Train Epoch: 10 [4480/60000 (7%)] Loss: 0.002764
Train Epoch: 10 [5120/60000 (9%)] Loss: 0.010052
Train Epoch: 10 [5760/60000 (10%)] Loss: 0.015958
Train Epoch: 10 [6400/60000 (11%)] Loss: 0.017731
Train Epoch: 10 [7040/60000 (12%)] Loss: 0.005033
Train Epoch: 10 [7680/60000 (13%)] Loss: 0.034701
Train Epoch: 10 [8320/60000 (14%)] Loss: 0.013517
Train Epoch: 10 [8960/60000 (15%)] Loss: 0.044430
Train Epoch: 10 [9600/60000 (16%)] Loss: 0.031159
Train Epoch: 10 [10240/60000 (17%)] Loss: 0.037950
Train Epoch: 10 [10880/60000 (18%)] Loss: 0.006554
Train Epoch: 10 [11520/60000 (19%)] Loss: 0.020985
Train Epoch: 10 [12160/60000 (20%)] Loss: 0.010484
Train Epoch: 10 [12800/60000 (21%)] Loss: 0.039093
Train Epoch: 10 [13440/60000 (22%)] Loss: 0.039096
Train Epoch: 10 [14080/60000 (23%)] Loss: 0.001877
Train Epoch: 10 [14720/60000 (25%)] Loss: 0.000859
Train Epoch: 10 [15360/60000 (26%)] Loss: 0.011563
Train Epoch: 10 [16000/60000 (27%)] Loss: 0.013502
Train Epoch: 10 [16640/60000 (28%)] Loss: 0.051948
Train Epoch: 10 [17280/60000 (29%)] Loss: 0.022418
Train Epoch: 10 [17920/60000 (30%)] Loss: 0.085264
Train Epoch: 10 [18560/60000 (31%)] Loss: 0.020911
Train Epoch: 10 [19200/60000 (32%)] Loss: 0.005612
Train Epoch: 10 [19840/60000 (33%)] Loss: 0.046620
Train Epoch: 10 [20480/60000 (34%)] Loss: 0.046337
Train Epoch: 10 [21120/60000 (35%)] Loss: 0.000846
Train Epoch: 10 [21760/60000 (36%)] Loss: 0.025868
Train Epoch: 10 [22400/60000 (37%)] Loss: 0.085982
Train Epoch: 10 [23040/60000 (38%)] Loss: 0.022696
Train Epoch: 10 [23680/60000 (39%)] Loss: 0.046949
Train Epoch: 10 [24320/60000 (41%)] Loss: 0.019006
Train Epoch: 10 [24960/60000 (42%)] Loss: 0.101759
Train Epoch: 10 [25600/60000 (43%)] Loss: 0.004660
Train Epoch: 10 [26240/60000 (44%)] Loss: 0.031396
Train Epoch: 10 [26880/60000 (45%)] Loss: 0.038672
Train Epoch: 10 [27520/60000 (46%)] Loss: 0.009012
Train Epoch: 10 [28160/60000 (47%)] Loss: 0.008041
Train Epoch: 10 [28800/60000 (48%)] Loss: 0.016530
Train Epoch: 10 [29440/60000 (49%)] Loss: 0.106844
Train Epoch: 10 [30080/60000 (50%)] Loss: 0.097697
Train Epoch: 10 [30720/60000 (51%)] Loss: 0.122220
Train Epoch: 10 [31360/60000 (52%)] Loss: 0.003784
Train Epoch: 10 [32000/60000 (53%)] Loss: 0.010800
Train Epoch: 10 [32640/60000 (54%)] Loss: 0.028731
Train Epoch: 10 [33280/60000 (55%)] Loss: 0.002334
Train Epoch: 10 [33920/60000 (57%)] Loss: 0.001850
Train Epoch: 10 [34560/60000 (58%)] Loss: 0.011677
Train Epoch: 10 [35200/60000 (59%)] Loss: 0.008625
Train Epoch: 10 [35840/60000 (60%)] Loss: 0.151467
Train Epoch: 10 [36480/60000 (61%)] Loss: 0.157283
Train Epoch: 10 [37120/60000 (62%)] Loss: 0.004965
Train Epoch: 10 [37760/60000 (63%)] Loss: 0.014678
Train Epoch: 10 [38400/60000 (64%)] Loss: 0.003421
Train Epoch: 10 [39040/60000 (65%)] Loss: 0.023539
Train Epoch: 10 [39680/60000 (66%)] Loss: 0.031885
Train Epoch: 10 [40320/60000 (67%)] Loss: 0.010425
Train Epoch: 10 [40960/60000 (68%)] Loss: 0.088723
Train Epoch: 10 [41600/60000 (69%)] Loss: 0.000976
Train Epoch: 10 [42240/60000 (70%)] Loss: 0.001700
Train Epoch: 10 [42880/60000 (71%)] Loss: 0.003436
Train Epoch: 10 [43520/60000 (72%)] Loss: 0.122106
Train Epoch: 10 [44160/60000 (74%)] Loss: 0.005674
Train Epoch: 10 [44800/60000 (75%)] Loss: 0.065215
Train Epoch: 10 [45440/60000 (76%)] Loss: 0.010673
Train Epoch: 10 [46080/60000 (77%)] Loss: 0.002360
Train Epoch: 10 [46720/60000 (78%)] Loss: 0.069567
Train Epoch: 10 [47360/60000 (79%)] Loss: 0.012620
Train Epoch: 10 [48000/60000 (80%)] Loss: 0.002036
Train Epoch: 10 [48640/60000 (81%)] Loss: 0.009742
Train Epoch: 10 [49280/60000 (82%)] Loss: 0.001294
Train Epoch: 10 [49920/60000 (83%)] Loss: 0.008810
Train Epoch: 10 [50560/60000 (84%)] Loss: 0.004116
Train Epoch: 10 [51200/60000 (85%)] Loss: 0.000975
Train Epoch: 10 [51840/60000 (86%)] Loss: 0.009720
Train Epoch: 10 [52480/60000 (87%)] Loss: 0.002065
Train Epoch: 10 [53120/60000 (88%)] Loss: 0.048539
Train Epoch: 10 [53760/60000 (90%)] Loss: 0.004084
Train Epoch: 10 [54400/60000 (91%)] Loss: 0.001950
Train Epoch: 10 [55040/60000 (92%)] Loss: 0.020956
Train Epoch: 10 [55680/60000 (93%)] Loss: 0.031521
Train Epoch: 10 [56320/60000 (94%)] Loss: 0.012408
Train Epoch: 10 [56960/60000 (95%)] Loss: 0.009591
Train Epoch: 10 [57600/60000 (96%)] Loss: 0.060889
Train Epoch: 10 [58240/60000 (97%)] Loss: 0.023143
Train Epoch: 10 [58880/60000 (98%)] Loss: 0.026583
Train Epoch: 10 [59520/60000 (99%)] Loss: 0.001925
Test set: Average loss: 0.0252, Accuracy: 9922/10000 (99%)
Train Epoch: 11 [0/60000 (0%)] Loss: 0.000643
Train Epoch: 11 [640/60000 (1%)] Loss: 0.041149
Train Epoch: 11 [1280/60000 (2%)] Loss: 0.004773
Train Epoch: 11 [1920/60000 (3%)] Loss: 0.002375
Train Epoch: 11 [2560/60000 (4%)] Loss: 0.033779
Train Epoch: 11 [3200/60000 (5%)] Loss: 0.050348
Train Epoch: 11 [3840/60000 (6%)] Loss: 0.002844
Train Epoch: 11 [4480/60000 (7%)] Loss: 0.020644
Train Epoch: 11 [5120/60000 (9%)] Loss: 0.006397
Train Epoch: 11 [5760/60000 (10%)] Loss: 0.018484
Train Epoch: 11 [6400/60000 (11%)] Loss: 0.024009
Train Epoch: 11 [7040/60000 (12%)] Loss: 0.009233
Train Epoch: 11 [7680/60000 (13%)] Loss: 0.010275
Train Epoch: 11 [8320/60000 (14%)] Loss: 0.011256
Train Epoch: 11 [8960/60000 (15%)] Loss: 0.024783
Train Epoch: 11 [9600/60000 (16%)] Loss: 0.012596
Train Epoch: 11 [10240/60000 (17%)] Loss: 0.028823
Train Epoch: 11 [10880/60000 (18%)] Loss: 0.003300
Train Epoch: 11 [11520/60000 (19%)] Loss: 0.020667
Train Epoch: 11 [12160/60000 (20%)] Loss: 0.025594
Train Epoch: 11 [12800/60000 (21%)] Loss: 0.003216
Train Epoch: 11 [13440/60000 (22%)] Loss: 0.003150
Train Epoch: 11 [14080/60000 (23%)] Loss: 0.000566
Train Epoch: 11 [14720/60000 (25%)] Loss: 0.026344
Train Epoch: 11 [15360/60000 (26%)] Loss: 0.007334
Train Epoch: 11 [16000/60000 (27%)] Loss: 0.005995
Train Epoch: 11 [16640/60000 (28%)] Loss: 0.098630
Train Epoch: 11 [17280/60000 (29%)] Loss: 0.006056
Train Epoch: 11 [17920/60000 (30%)] Loss: 0.019787
Train Epoch: 11 [18560/60000 (31%)] Loss: 0.057256
Train Epoch: 11 [19200/60000 (32%)] Loss: 0.048368
Train Epoch: 11 [19840/60000 (33%)] Loss: 0.030600
Train Epoch: 11 [20480/60000 (34%)] Loss: 0.025714
Train Epoch: 11 [21120/60000 (35%)] Loss: 0.013856
Train Epoch: 11 [21760/60000 (36%)] Loss: 0.001353
Train Epoch: 11 [22400/60000 (37%)] Loss: 0.035525
Train Epoch: 11 [23040/60000 (38%)] Loss: 0.007346
Train Epoch: 11 [23680/60000 (39%)] Loss: 0.010212
Train Epoch: 11 [24320/60000 (41%)] Loss: 0.011391
Train Epoch: 11 [24960/60000 (42%)] Loss: 0.008403
Train Epoch: 11 [25600/60000 (43%)] Loss: 0.004002
Train Epoch: 11 [26240/60000 (44%)] Loss: 0.089580
Train Epoch: 11 [26880/60000 (45%)] Loss: 0.008463
Train Epoch: 11 [27520/60000 (46%)] Loss: 0.006835
Train Epoch: 11 [28160/60000 (47%)] Loss: 0.031269
Train Epoch: 11 [28800/60000 (48%)] Loss: 0.055097
Train Epoch: 11 [29440/60000 (49%)] Loss: 0.022374
Train Epoch: 11 [30080/60000 (50%)] Loss: 0.068506
Train Epoch: 11 [30720/60000 (51%)] Loss: 0.001938
Train Epoch: 11 [31360/60000 (52%)] Loss: 0.000694
Train Epoch: 11 [32000/60000 (53%)] Loss: 0.036674
Train Epoch: 11 [32640/60000 (54%)] Loss: 0.006566
Train Epoch: 11 [33280/60000 (55%)] Loss: 0.018003
Train Epoch: 11 [33920/60000 (57%)] Loss: 0.002541
Train Epoch: 11 [34560/60000 (58%)] Loss: 0.015087
Train Epoch: 11 [35200/60000 (59%)] Loss: 0.019500
Train Epoch: 11 [35840/60000 (60%)] Loss: 0.005819
Train Epoch: 11 [36480/60000 (61%)] Loss: 0.004776
Train Epoch: 11 [37120/60000 (62%)] Loss: 0.004458
Train Epoch: 11 [37760/60000 (63%)] Loss: 0.002905
Train Epoch: 11 [38400/60000 (64%)] Loss: 0.005131
Train Epoch: 11 [39040/60000 (65%)] Loss: 0.010119
Train Epoch: 11 [39680/60000 (66%)] Loss: 0.007685
Train Epoch: 11 [40320/60000 (67%)] Loss: 0.005725
Train Epoch: 11 [40960/60000 (68%)] Loss: 0.135369
Train Epoch: 11 [41600/60000 (69%)] Loss: 0.001773
Train Epoch: 11 [42240/60000 (70%)] Loss: 0.017532
Train Epoch: 11 [42880/60000 (71%)] Loss: 0.000830
Train Epoch: 11 [43520/60000 (72%)] Loss: 0.010465
Train Epoch: 11 [44160/60000 (74%)] Loss: 0.007864
Train Epoch: 11 [44800/60000 (75%)] Loss: 0.003969
Train Epoch: 11 [45440/60000 (76%)] Loss: 0.011202
Train Epoch: 11 [46080/60000 (77%)] Loss: 0.065552
Train Epoch: 11 [46720/60000 (78%)] Loss: 0.008145
Train Epoch: 11 [47360/60000 (79%)] Loss: 0.044302
Train Epoch: 11 [48000/60000 (80%)] Loss: 0.008320
Train Epoch: 11 [48640/60000 (81%)] Loss: 0.011349
Train Epoch: 11 [49280/60000 (82%)] Loss: 0.010497
Train Epoch: 11 [49920/60000 (83%)] Loss: 0.060804
Train Epoch: 11 [50560/60000 (84%)] Loss: 0.005377
Train Epoch: 11 [51200/60000 (85%)] Loss: 0.041450
Train Epoch: 11 [51840/60000 (86%)] Loss: 0.039288
Train Epoch: 11 [52480/60000 (87%)] Loss: 0.004661
Train Epoch: 11 [53120/60000 (88%)] Loss: 0.059664
Train Epoch: 11 [53760/60000 (90%)] Loss: 0.001824
Train Epoch: 11 [54400/60000 (91%)] Loss: 0.063768
Train Epoch: 11 [55040/60000 (92%)] Loss: 0.014165
Train Epoch: 11 [55680/60000 (93%)] Loss: 0.012471
Train Epoch: 11 [56320/60000 (94%)] Loss: 0.010155
Train Epoch: 11 [56960/60000 (95%)] Loss: 0.090924
Train Epoch: 11 [57600/60000 (96%)] Loss: 0.012109
Train Epoch: 11 [58240/60000 (97%)] Loss: 0.011624
Train Epoch: 11 [58880/60000 (98%)] Loss: 0.012024
Train Epoch: 11 [59520/60000 (99%)] Loss: 0.004135
Test set: Average loss: 0.0259, Accuracy: 9924/10000 (99%)
Train Epoch: 12 [0/60000 (0%)] Loss: 0.001675
Train Epoch: 12 [640/60000 (1%)] Loss: 0.004651
Train Epoch: 12 [1280/60000 (2%)] Loss: 0.011113
Train Epoch: 12 [1920/60000 (3%)] Loss: 0.060214
Train Epoch: 12 [2560/60000 (4%)] Loss: 0.001232
Train Epoch: 12 [3200/60000 (5%)] Loss: 0.009105
Train Epoch: 12 [3840/60000 (6%)] Loss: 0.007360
Train Epoch: 12 [4480/60000 (7%)] Loss: 0.066242
Train Epoch: 12 [5120/60000 (9%)] Loss: 0.006429
Train Epoch: 12 [5760/60000 (10%)] Loss: 0.062769
Train Epoch: 12 [6400/60000 (11%)] Loss: 0.020854
Train Epoch: 12 [7040/60000 (12%)] Loss: 0.031601
Train Epoch: 12 [7680/60000 (13%)] Loss: 0.012003
Train Epoch: 12 [8320/60000 (14%)] Loss: 0.040986
Train Epoch: 12 [8960/60000 (15%)] Loss: 0.007916
Train Epoch: 12 [9600/60000 (16%)] Loss: 0.018470
Train Epoch: 12 [10240/60000 (17%)] Loss: 0.008284
Train Epoch: 12 [10880/60000 (18%)] Loss: 0.015258
Train Epoch: 12 [11520/60000 (19%)] Loss: 0.061269
Train Epoch: 12 [12160/60000 (20%)] Loss: 0.022815
Train Epoch: 12 [12800/60000 (21%)] Loss: 0.015148
Train Epoch: 12 [13440/60000 (22%)] Loss: 0.004950
Train Epoch: 12 [14080/60000 (23%)] Loss: 0.011374
Train Epoch: 12 [14720/60000 (25%)] Loss: 0.037866
Train Epoch: 12 [15360/60000 (26%)] Loss: 0.013882
Train Epoch: 12 [16000/60000 (27%)] Loss: 0.004361
Train Epoch: 12 [16640/60000 (28%)] Loss: 0.011929
Train Epoch: 12 [17280/60000 (29%)] Loss: 0.061392
Train Epoch: 12 [17920/60000 (30%)] Loss: 0.000897
Train Epoch: 12 [18560/60000 (31%)] Loss: 0.042927
Train Epoch: 12 [19200/60000 (32%)] Loss: 0.005629
Train Epoch: 12 [19840/60000 (33%)] Loss: 0.011283
Train Epoch: 12 [20480/60000 (34%)] Loss: 0.005701
Train Epoch: 12 [21120/60000 (35%)] Loss: 0.013442
Train Epoch: 12 [21760/60000 (36%)] Loss: 0.011944
Train Epoch: 12 [22400/60000 (37%)] Loss: 0.005858
Train Epoch: 12 [23040/60000 (38%)] Loss: 0.026175
Train Epoch: 12 [23680/60000 (39%)] Loss: 0.018708
Train Epoch: 12 [24320/60000 (41%)] Loss: 0.039083
Train Epoch: 12 [24960/60000 (42%)] Loss: 0.003156
Train Epoch: 12 [25600/60000 (43%)] Loss: 0.047728
Train Epoch: 12 [26240/60000 (44%)] Loss: 0.017620
Train Epoch: 12 [26880/60000 (45%)] Loss: 0.045216
Train Epoch: 12 [27520/60000 (46%)] Loss: 0.012947
Train Epoch: 12 [28160/60000 (47%)] Loss: 0.015450
Train Epoch: 12 [28800/60000 (48%)] Loss: 0.000721
Train Epoch: 12 [29440/60000 (49%)] Loss: 0.011115
Train Epoch: 12 [30080/60000 (50%)] Loss: 0.015272
Train Epoch: 12 [30720/60000 (51%)] Loss: 0.015517
Train Epoch: 12 [31360/60000 (52%)] Loss: 0.016967
Train Epoch: 12 [32000/60000 (53%)] Loss: 0.001368
Train Epoch: 12 [32640/60000 (54%)] Loss: 0.004827
Train Epoch: 12 [33280/60000 (55%)] Loss: 0.077198
Train Epoch: 12 [33920/60000 (57%)] Loss: 0.001666
Train Epoch: 12 [34560/60000 (58%)] Loss: 0.008841
Train Epoch: 12 [35200/60000 (59%)] Loss: 0.107419
Train Epoch: 12 [35840/60000 (60%)] Loss: 0.028854
Train Epoch: 12 [36480/60000 (61%)] Loss: 0.007597
Train Epoch: 12 [37120/60000 (62%)] Loss: 0.002414
Train Epoch: 12 [37760/60000 (63%)] Loss: 0.020209
Train Epoch: 12 [38400/60000 (64%)] Loss: 0.014578
Train Epoch: 12 [39040/60000 (65%)] Loss: 0.004163
Train Epoch: 12 [39680/60000 (66%)] Loss: 0.008141
Train Epoch: 12 [40320/60000 (67%)] Loss: 0.023545
Train Epoch: 12 [40960/60000 (68%)] Loss: 0.046212
Train Epoch: 12 [41600/60000 (69%)] Loss: 0.024903
Train Epoch: 12 [42240/60000 (70%)] Loss: 0.009293
Train Epoch: 12 [42880/60000 (71%)] Loss: 0.002976
Train Epoch: 12 [43520/60000 (72%)] Loss: 0.004085
Train Epoch: 12 [44160/60000 (74%)] Loss: 0.042448
Train Epoch: 12 [44800/60000 (75%)] Loss: 0.010914
Train Epoch: 12 [45440/60000 (76%)] Loss: 0.011886
Train Epoch: 12 [46080/60000 (77%)] Loss: 0.005287
Train Epoch: 12 [46720/60000 (78%)] Loss: 0.005735
Train Epoch: 12 [47360/60000 (79%)] Loss: 0.046429
Train Epoch: 12 [48000/60000 (80%)] Loss: 0.013356
Train Epoch: 12 [48640/60000 (81%)] Loss: 0.084353
Train Epoch: 12 [49280/60000 (82%)] Loss: 0.006819
Train Epoch: 12 [49920/60000 (83%)] Loss: 0.001213
Train Epoch: 12 [50560/60000 (84%)] Loss: 0.003828
Train Epoch: 12 [51200/60000 (85%)] Loss: 0.034852
Train Epoch: 12 [51840/60000 (86%)] Loss: 0.003583
Train Epoch: 12 [52480/60000 (87%)] Loss: 0.031002
Train Epoch: 12 [53120/60000 (88%)] Loss: 0.016651
Train Epoch: 12 [53760/60000 (90%)] Loss: 0.004567
Train Epoch: 12 [54400/60000 (91%)] Loss: 0.032221
Train Epoch: 12 [55040/60000 (92%)] Loss: 0.018444
Train Epoch: 12 [55680/60000 (93%)] Loss: 0.002694
Train Epoch: 12 [56320/60000 (94%)] Loss: 0.080163
Train Epoch: 12 [56960/60000 (95%)] Loss: 0.003757
Train Epoch: 12 [57600/60000 (96%)] Loss: 0.078489
Train Epoch: 12 [58240/60000 (97%)] Loss: 0.003875
Train Epoch: 12 [58880/60000 (98%)] Loss: 0.005629
Train Epoch: 12 [59520/60000 (99%)] Loss: 0.140195
Test set: Average loss: 0.0253, Accuracy: 9919/10000 (99%)
Train Epoch: 13 [0/60000 (0%)] Loss: 0.006849
Train Epoch: 13 [640/60000 (1%)] Loss: 0.014937
Train Epoch: 13 [1280/60000 (2%)] Loss: 0.045428
Train Epoch: 13 [1920/60000 (3%)] Loss: 0.039137
Train Epoch: 13 [2560/60000 (4%)] Loss: 0.007463
Train Epoch: 13 [3200/60000 (5%)] Loss: 0.002201
Train Epoch: 13 [3840/60000 (6%)] Loss: 0.000995
Train Epoch: 13 [4480/60000 (7%)] Loss: 0.008288
Train Epoch: 13 [5120/60000 (9%)] Loss: 0.046883
Train Epoch: 13 [5760/60000 (10%)] Loss: 0.007128
Train Epoch: 13 [6400/60000 (11%)] Loss: 0.000719
Train Epoch: 13 [7040/60000 (12%)] Loss: 0.182368
Train Epoch: 13 [7680/60000 (13%)] Loss: 0.101809
Train Epoch: 13 [8320/60000 (14%)] Loss: 0.002415
Train Epoch: 13 [8960/60000 (15%)] Loss: 0.065472
Train Epoch: 13 [9600/60000 (16%)] Loss: 0.034620
Train Epoch: 13 [10240/60000 (17%)] Loss: 0.030651
Train Epoch: 13 [10880/60000 (18%)] Loss: 0.101772
Train Epoch: 13 [11520/60000 (19%)] Loss: 0.001626
Train Epoch: 13 [12160/60000 (20%)] Loss: 0.004139
Train Epoch: 13 [12800/60000 (21%)] Loss: 0.009143
Train Epoch: 13 [13440/60000 (22%)] Loss: 0.016151
Train Epoch: 13 [14080/60000 (23%)] Loss: 0.016641
Train Epoch: 13 [14720/60000 (25%)] Loss: 0.002441
Train Epoch: 13 [15360/60000 (26%)] Loss: 0.074819
Train Epoch: 13 [16000/60000 (27%)] Loss: 0.045300
Train Epoch: 13 [16640/60000 (28%)] Loss: 0.002215
Train Epoch: 13 [17280/60000 (29%)] Loss: 0.007908
Train Epoch: 13 [17920/60000 (30%)] Loss: 0.110830
Train Epoch: 13 [18560/60000 (31%)] Loss: 0.015812
Train Epoch: 13 [19200/60000 (32%)] Loss: 0.005905
Train Epoch: 13 [19840/60000 (33%)] Loss: 0.004767
Train Epoch: 13 [20480/60000 (34%)] Loss: 0.042236
Train Epoch: 13 [21120/60000 (35%)] Loss: 0.000603
Train Epoch: 13 [21760/60000 (36%)] Loss: 0.070208
Train Epoch: 13 [22400/60000 (37%)] Loss: 0.003454
Train Epoch: 13 [23040/60000 (38%)] Loss: 0.141158
Train Epoch: 13 [23680/60000 (39%)] Loss: 0.003854
Train Epoch: 13 [24320/60000 (41%)] Loss: 0.007790
Train Epoch: 13 [24960/60000 (42%)] Loss: 0.003856
Train Epoch: 13 [25600/60000 (43%)] Loss: 0.016601
Train Epoch: 13 [26240/60000 (44%)] Loss: 0.000837
Train Epoch: 13 [26880/60000 (45%)] Loss: 0.003017
Train Epoch: 13 [27520/60000 (46%)] Loss: 0.250847
Train Epoch: 13 [28160/60000 (47%)] Loss: 0.011099
Train Epoch: 13 [28800/60000 (48%)] Loss: 0.079327
Train Epoch: 13 [29440/60000 (49%)] Loss: 0.031726
Train Epoch: 13 [30080/60000 (50%)] Loss: 0.048035
Train Epoch: 13 [30720/60000 (51%)] Loss: 0.024651
Train Epoch: 13 [31360/60000 (52%)] Loss: 0.001597
Train Epoch: 13 [32000/60000 (53%)] Loss: 0.029304
Train Epoch: 13 [32640/60000 (54%)] Loss: 0.012545
Train Epoch: 13 [33280/60000 (55%)] Loss: 0.069283
Train Epoch: 13 [33920/60000 (57%)] Loss: 0.042197
Train Epoch: 13 [34560/60000 (58%)] Loss: 0.001504
Train Epoch: 13 [35200/60000 (59%)] Loss: 0.017152
Train Epoch: 13 [35840/60000 (60%)] Loss: 0.092977
Train Epoch: 13 [36480/60000 (61%)] Loss: 0.013816
Train Epoch: 13 [37120/60000 (62%)] Loss: 0.004034
Train Epoch: 13 [37760/60000 (63%)] Loss: 0.011934
Train Epoch: 13 [38400/60000 (64%)] Loss: 0.003430
Train Epoch: 13 [39040/60000 (65%)] Loss: 0.033119
Train Epoch: 13 [39680/60000 (66%)] Loss: 0.062010
Train Epoch: 13 [40320/60000 (67%)] Loss: 0.155564
Train Epoch: 13 [40960/60000 (68%)] Loss: 0.004567
Train Epoch: 13 [41600/60000 (69%)] Loss: 0.019368
Train Epoch: 13 [42240/60000 (70%)] Loss: 0.057248
Train Epoch: 13 [42880/60000 (71%)] Loss: 0.064919
Train Epoch: 13 [43520/60000 (72%)] Loss: 0.027566
Train Epoch: 13 [44160/60000 (74%)] Loss: 0.062751
Train Epoch: 13 [44800/60000 (75%)] Loss: 0.004989
Train Epoch: 13 [45440/60000 (76%)] Loss: 0.027566
Train Epoch: 13 [46080/60000 (77%)] Loss: 0.010376
Train Epoch: 13 [46720/60000 (78%)] Loss: 0.000903
Train Epoch: 13 [47360/60000 (79%)] Loss: 0.145241
Train Epoch: 13 [48000/60000 (80%)] Loss: 0.004968
Train Epoch: 13 [48640/60000 (81%)] Loss: 0.009010
Train Epoch: 13 [49280/60000 (82%)] Loss: 0.002557
Train Epoch: 13 [49920/60000 (83%)] Loss: 0.015616
Train Epoch: 13 [50560/60000 (84%)] Loss: 0.002572
Train Epoch: 13 [51200/60000 (85%)] Loss: 0.028267
Train Epoch: 13 [51840/60000 (86%)] Loss: 0.001808
Train Epoch: 13 [52480/60000 (87%)] Loss: 0.000703
Train Epoch: 13 [53120/60000 (88%)] Loss: 0.109102
Train Epoch: 13 [53760/60000 (90%)] Loss: 0.018993
Train Epoch: 13 [54400/60000 (91%)] Loss: 0.005438
Train Epoch: 13 [55040/60000 (92%)] Loss: 0.015585
Train Epoch: 13 [55680/60000 (93%)] Loss: 0.001080
Train Epoch: 13 [56320/60000 (94%)] Loss: 0.000946
Train Epoch: 13 [56960/60000 (95%)] Loss: 0.002476
Train Epoch: 13 [57600/60000 (96%)] Loss: 0.007942
Train Epoch: 13 [58240/60000 (97%)] Loss: 0.008576
Train Epoch: 13 [58880/60000 (98%)] Loss: 0.020211
Train Epoch: 13 [59520/60000 (99%)] Loss: 0.025141
Test set: Average loss: 0.0259, Accuracy: 9924/10000 (99%)
Train Epoch: 14 [0/60000 (0%)] Loss: 0.001255
Train Epoch: 14 [640/60000 (1%)] Loss: 0.074888
Train Epoch: 14 [1280/60000 (2%)] Loss: 0.003073
Train Epoch: 14 [1920/60000 (3%)] Loss: 0.014478
Train Epoch: 14 [2560/60000 (4%)] Loss: 0.027479
Train Epoch: 14 [3200/60000 (5%)] Loss: 0.043085
Train Epoch: 14 [3840/60000 (6%)] Loss: 0.061583
Train Epoch: 14 [4480/60000 (7%)] Loss: 0.005525
Train Epoch: 14 [5120/60000 (9%)] Loss: 0.038227
Train Epoch: 14 [5760/60000 (10%)] Loss: 0.160576
Train Epoch: 14 [6400/60000 (11%)] Loss: 0.006663
Train Epoch: 14 [7040/60000 (12%)] Loss: 0.033710
Train Epoch: 14 [7680/60000 (13%)] Loss: 0.004818
Train Epoch: 14 [8320/60000 (14%)] Loss: 0.028156
Train Epoch: 14 [8960/60000 (15%)] Loss: 0.003839
Train Epoch: 14 [9600/60000 (16%)] Loss: 0.003640
Train Epoch: 14 [10240/60000 (17%)] Loss: 0.002657
Train Epoch: 14 [10880/60000 (18%)] Loss: 0.001865
Train Epoch: 14 [11520/60000 (19%)] Loss: 0.003178
Train Epoch: 14 [12160/60000 (20%)] Loss: 0.001151
Train Epoch: 14 [12800/60000 (21%)] Loss: 0.002490
Train Epoch: 14 [13440/60000 (22%)] Loss: 0.100219
Train Epoch: 14 [14080/60000 (23%)] Loss: 0.123331
Train Epoch: 14 [14720/60000 (25%)] Loss: 0.027646
Train Epoch: 14 [15360/60000 (26%)] Loss: 0.002390
Train Epoch: 14 [16000/60000 (27%)] Loss: 0.019805
Train Epoch: 14 [16640/60000 (28%)] Loss: 0.034430
Train Epoch: 14 [17280/60000 (29%)] Loss: 0.000317
Train Epoch: 14 [17920/60000 (30%)] Loss: 0.002433
Train Epoch: 14 [18560/60000 (31%)] Loss: 0.002735
Train Epoch: 14 [19200/60000 (32%)] Loss: 0.002088
Train Epoch: 14 [19840/60000 (33%)] Loss: 0.095927
Train Epoch: 14 [20480/60000 (34%)] Loss: 0.006197
Train Epoch: 14 [21120/60000 (35%)] Loss: 0.012814
Train Epoch: 14 [21760/60000 (36%)] Loss: 0.007367
Train Epoch: 14 [22400/60000 (37%)] Loss: 0.019995
Train Epoch: 14 [23040/60000 (38%)] Loss: 0.190696
Train Epoch: 14 [23680/60000 (39%)] Loss: 0.013575
Train Epoch: 14 [24320/60000 (41%)] Loss: 0.044315
Train Epoch: 14 [24960/60000 (42%)] Loss: 0.026246
Train Epoch: 14 [25600/60000 (43%)] Loss: 0.020449
Train Epoch: 14 [26240/60000 (44%)] Loss: 0.044301
Train Epoch: 14 [26880/60000 (45%)] Loss: 0.002407
Train Epoch: 14 [27520/60000 (46%)] Loss: 0.003667
Train Epoch: 14 [28160/60000 (47%)] Loss: 0.002547
Train Epoch: 14 [28800/60000 (48%)] Loss: 0.010491
Train Epoch: 14 [29440/60000 (49%)] Loss: 0.001074
Train Epoch: 14 [30080/60000 (50%)] Loss: 0.015989
Train Epoch: 14 [30720/60000 (51%)] Loss: 0.064361
Train Epoch: 14 [31360/60000 (52%)] Loss: 0.000942
Train Epoch: 14 [32000/60000 (53%)] Loss: 0.082821
Train Epoch: 14 [32640/60000 (54%)] Loss: 0.012558
Train Epoch: 14 [33280/60000 (55%)] Loss: 0.062665
Train Epoch: 14 [33920/60000 (57%)] Loss: 0.027698
Train Epoch: 14 [34560/60000 (58%)] Loss: 0.033962
Train Epoch: 14 [35200/60000 (59%)] Loss: 0.007041
Train Epoch: 14 [35840/60000 (60%)] Loss: 0.023287
Train Epoch: 14 [36480/60000 (61%)] Loss: 0.018226
Train Epoch: 14 [37120/60000 (62%)] Loss: 0.007956
Train Epoch: 14 [37760/60000 (63%)] Loss: 0.001503
Train Epoch: 14 [38400/60000 (64%)] Loss: 0.064215
Train Epoch: 14 [39040/60000 (65%)] Loss: 0.016337
Train Epoch: 14 [39680/60000 (66%)] Loss: 0.019652
Train Epoch: 14 [40320/60000 (67%)] Loss: 0.007127
Train Epoch: 14 [40960/60000 (68%)] Loss: 0.015364
Train Epoch: 14 [41600/60000 (69%)] Loss: 0.002786
Train Epoch: 14 [42240/60000 (70%)] Loss: 0.012069
Train Epoch: 14 [42880/60000 (71%)] Loss: 0.012717
Train Epoch: 14 [43520/60000 (72%)] Loss: 0.002783
Train Epoch: 14 [44160/60000 (74%)] Loss: 0.016118
Train Epoch: 14 [44800/60000 (75%)] Loss: 0.002617
Train Epoch: 14 [45440/60000 (76%)] Loss: 0.001912
Train Epoch: 14 [46080/60000 (77%)] Loss: 0.044160
Train Epoch: 14 [46720/60000 (78%)] Loss: 0.017263
Train Epoch: 14 [47360/60000 (79%)] Loss: 0.039953
Train Epoch: 14 [48000/60000 (80%)] Loss: 0.043392
Train Epoch: 14 [48640/60000 (81%)] Loss: 0.003833
Train Epoch: 14 [49280/60000 (82%)] Loss: 0.014092
Train Epoch: 14 [49920/60000 (83%)] Loss: 0.014904
Train Epoch: 14 [50560/60000 (84%)] Loss: 0.003285
Train Epoch: 14 [51200/60000 (85%)] Loss: 0.077817
Train Epoch: 14 [51840/60000 (86%)] Loss: 0.004313
Train Epoch: 14 [52480/60000 (87%)] Loss: 0.003841
Train Epoch: 14 [53120/60000 (88%)] Loss: 0.023167
Train Epoch: 14 [53760/60000 (90%)] Loss: 0.009064
Train Epoch: 14 [54400/60000 (91%)] Loss: 0.007248
Train Epoch: 14 [55040/60000 (92%)] Loss: 0.023903
Train Epoch: 14 [55680/60000 (93%)] Loss: 0.005857
Train Epoch: 14 [56320/60000 (94%)] Loss: 0.023585
Train Epoch: 14 [56960/60000 (95%)] Loss: 0.016657
Train Epoch: 14 [57600/60000 (96%)] Loss: 0.001826
Train Epoch: 14 [58240/60000 (97%)] Loss: 0.005145
Train Epoch: 14 [58880/60000 (98%)] Loss: 0.001573
Train Epoch: 14 [59520/60000 (99%)] Loss: 0.014263
Test set: Average loss: 0.0254, Accuracy: 9922/10000 (99%)
Raw
time
real 1m44.002s
user 2m42.714s
sys 0m6.409s
@sametbirol
Copy link

sametbirol commented Dec 26, 2023 •
edited

7900 XTX , Ubuntu 22.04 , installed 23.20.00.48 for Ubuntu 22.04.3 HWE with ROCm 5.7 from
https://www.amd.com/en/support/linux-drivers
then tested with this test script
https://gist.github.com/damico/484f7b0a148a0c5f707054cf9c0a0533
everything looked fine but running

tensor = torch.rand(size=(1000,1000,1000), device="cuda:0")
tensor

would kernel crash
but after trying your 'workarounds'

os.environ["HSA_OVERRIDE_GFX_VERSION"] = "11.0.0"

it works , thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment