8jul.md

8jul.md

kint:

1. removed static_assert(std::is_same<SType, DType>::value,"The Scalar type must match the Tensor data type");, compilation is ok
2. static_assert SType == DType fails at compile time, now after remove, fails at runtime (will see more XX op Not implemented)
3. to expect some hidden bug when SType != DType
4. ok:
   • int tensor and int tensor operation (test_onnx_backend.py)
   • int tensor and int scalar
   • nrm2() on int tensor (used tmp tensor with type cast)
   • float tensor and int scalar ops
   • cuda + int (after added cuda+int in TYPE_LANG_SWITCH
5. not ok:
   • int tensor and float tensor ops (is it useful?)
   • int tensor and float scalar (is it useful?)
6. drawback 1. need to add tmp tensor to cast type, 2. need to implement int version of function again for current float

qabot:

1. loss could descend for a few iterations, but later loss ascend to 0.2 quickly (margin = 0.2, so loss 0.2 is not usable)
2. implemented get_Wx(weights)/get_Wh(weights)/get_Bx(weights)... by getting cudnn rnn weight gpu memory offset.
3. use get_Wx(weights) to set exactly same weights with pytorch for comparison
4. given same input, singa rnn and pytorch rnn give same forward and backward output
5. given same input, singa loss fn and pytorch loss fn give same forward and backward output
6. given same input, singa cos sim fn and pytorch cos sim fn give same forward, but different backward output

tensor[x,y] = val

6d tensor:

apache/singa#750