dcslin

input="""R 1
D 1
L 1
D 1
L 2
U 2
D 2
R 1
D 1
L 2

dcslin

pass

dcslin

'''pass'''

dcslin

'''
pass
'''

# nvprof python3 examples/cnn/train_cnn.py cnn mnist -m1 -pfloat16

dcslin

'''
Diff https://github.com/kuangliu/pytorch-cifar/blob/master/main.py
'''

from apex import amp

net, optimizer = amp.initialize(net, optimizer, opt_level=args.opt_level)

#if device == 'cuda':
#    net = torch.nn.DataParallel(net)

dcslin

"""
This script is modified from https://github.com/pytorch/examples.git
"""
from __future__ import print_function
import argparse
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchvision import datasets, transforms

dcslin

qabot:

1. fix cos sim bug, value same as pytorch, ok now
2. lstm+cos sim+margin loss, loss ok, top1 accuracy 4%~10%, http://ncrs/:8888/notebooks/singa-etc/notebook/singa-qabot-train.ipynb
3. lstm+cos sim+pooling+margin, loss not ok, top1 accuracy not ok
4. pytorch: lstm+cos sim+pooling+margin, loss ok

kint:

1. follow numpy convension, if int tensor + int tensor, return int tensor, else return float
2. check input types and cast to float in the GenTensorScalarFn when necessary
3. pr merged

dcslin

qabot:

1. fix cos sim bug, value ok
2. lstm+cos sim+margin loss, loss ok, top1 accuracy 4%~10%
3. lstm+cos sim+pooling+margin, loss not ok, top1 accuracy not ok
4. pytorch: lstm+cos sim+pooling+margin, loss ok

kint:

1. follow numpy convension, if int tensor + int tensor, return int tensor, else return float
2. check input types and cast to float in the GenTensorScalarFn when necessary
3. pr merged

dcslin

root@39516c62233d:~/singa-hf2# cd build/
root@39516c62233d:~/singa-hf2/build# ./bin/test_singa --gtest_filter=*RNN*
Running main() from gtest_main.cc
Note: Google Test filter = *RNN*
[==========] Running 5 tests from 3 test cases.
[----------] Global test environment set-up.
[----------] 3 tests from TestCudnnRNN
[ RUN      ] TestCudnnRNN.Setup
[       OK ] TestCudnnRNN.Setup (0 ms)
[ RUN      ] TestCudnnRNN.Forward

dcslin

kint:

1. removed static_assert(std::is_same<SType, DType>::value,"The Scalar type must match the Tensor data type");, compilation is ok
2. static_assert SType == DType fails at compile time, now after remove, fails at runtime (will see more XX op Not implemented)
3. to expect some hidden bug when SType != DType
4. ok:
   • int tensor and int tensor operation (test_onnx_backend.py)
   • int tensor and int scalar
   • nrm2() on int tensor (used tmp tensor with type cast)
   • float tensor and int scalar ops
   • cuda + int (after added cuda+int in TYPE_LANG_SWITCH