Comparison of Chainer's memnn example between in FP32 mode and in FP16 mode

memnn_fp16

$ CHAINER_DTYPE=float16 python train_memnn.py tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_train.txt tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_test.txt -d 0
Training data: tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_train.txt: 2000
Test data: tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_test.txt: 200
epoch       main/loss   validation/main/loss  main/accuracy  validation/main/accuracy
1           nan         nan                   0.0017004      0
2           nan         nan                   0              0
3           nan         nan                   0              0
4           nan         nan                   0              0
5           nan         nan                   0              0
6           nan         nan                   0              0
7           nan         nan                   0              0
8           nan         nan                   0              0
9           nan         nan                   0              0
10          nan         nan                   0              0
11          nan         nan                   0              0
12          nan         nan                   0              0
13          nan         nan                   0              0
14          nan         nan                   0              0
15          nan         nan                   0              0
16          nan         nan                   0              0
17          nan         nan                   0              0
18          nan         nan                   0              0
19          nan         nan                   0              0
20          nan         nan                   0              0
21          nan         nan                   0              0
22          nan         nan                   0              0
23          nan         nan                   0              0
24          nan         nan                   0              0
25          nan         nan                   0              0
26          nan         nan                   0              0
27          nan         nan                   0              0
28          nan         nan                   0              0
29          nan         nan                   0              0
30          nan         nan                   0              0
...

memnn_fp32

$ python train_memnn.py tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_train.txt tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_test.txt -d 0
Training data: tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_train.txt: 2000
Test data: tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_test.txt: 200
epoch       main/loss   validation/main/loss  main/accuracy  validation/main/accuracy
1           2.41833     1.80678               0.2083         0.225
2           1.76084     1.71291               0.2972         0.381
3           1.62661     1.52482               0.4552         0.521
4           1.40628     1.29272               0.5494         0.573
5           0.937755    0.539432              0.728          0.863
6           0.230902    0.09883               0.9655         0.994
7           0.0569344   0.0417305             0.9993         0.997
8           0.0273535   0.0242202             0.9999         0.999
9           0.0167176   0.0162068             1              1
10          0.011597    0.011871              1              1
11          0.00858905  0.00896583            1              1
12          0.00666347  0.00704912            1              1
13          0.00533772  0.00574273            1              1
14          0.00437632  0.00473799            1              1
15          0.00365677  0.00395577            1              1
16          0.00309958  0.00336608            1              1
17          0.00266046  0.00289834            1              1
18          0.00230534  0.00251321            1              1
19          0.00201576  0.00220578            1              1
20          0.00177504  0.00193605            1              1
21          0.00157303  0.00172307            1              1
22          0.00140205  0.00153442            1              1
23          0.00125536  0.00137551            1              1
24          0.00112903  0.00123596            1              1
25          0.00101942  0.00111762            1              1
26          0.00092357  0.00101332            1              1
27          0.000839527  0.000921253           1              1
28          0.000765088  0.000840034           1              1
29          0.000699283  0.000767484           1              1
30          0.000640476  0.000703148           1              1
...

runtime_info

$ python
Python 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import chainer
>>> chainer.print_runtime_info()
Platform: Linux-4.15.0-50-generic-x86_64-with-debian-buster-sid
Chainer: 6.0.0
NumPy: 1.16.4
CuPy:
  CuPy Version          : 6.0.0
  CUDA Root             : /usr/local/cuda
  CUDA Build Version    : 10000
  CUDA Driver Version   : 10000
  CUDA Runtime Version  : 10000
  cuDNN Build Version   : 7500
  cuDNN Version         : 7500
  NCCL Build Version    : 2402
  NCCL Runtime Version  : 2402
iDeep: 2.0.0.post3
>>>