Manish manish-kumar-garg
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Local devices available to TensorFlow: | |
| 1/4: name: "/device:CPU:0" | |
| device_type: "CPU" | |
| memory_limit: 268435456 | |
| locality { | |
| } | |
| incarnation: 12898452358282468420 | |
| 2/4: name: "/device:XLA_GPU:0" | |
| device_type: "XLA_GPU" | |
| memory_limit: 17179869184 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. | |
| _np_qint8 = np.dtype([("qint8", np.int8, 1)]) | |
| /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. | |
| _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) | |
| /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. | |
| _np_qint16 = np.dtype([("qint16", np.int16, 1)]) | |
| /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing ( |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. | |
| _np_qint8 = np.dtype([("qint8", np.int8, 1)]) | |
| /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. | |
| _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) | |
| /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. | |
| _np_qint16 = np.dtype([("qint16", np.int16, 1)]) | |
| /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing ( |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| pretrain epoch 13, step 838, cost:ctc 4.202715049206063, cost:output/output_prob 6.60445280135491, error:ctc 0.9343339595943689, error:decision 0.0, error:output/output_prob 0.9061913704499602, loss 5760.2207, max_size:classes 44, max_size:data 1519, mem_usage:GPU:0 3.8GB, num_seqs 13, 2.951 sec/step, elapsed 0:42:39, exp. remaining 0:11:41, complete 78.49% | |
| pretrain epoch 13, step 839, cost:ctc 3.2496038947583656, cost:output/output_prob 6.663787345441051, error:ctc 0.9126015920192003, error:decision 0.0, error:output/output_prob 0.9085365515202284, loss 4877.3887, max_size:classes 48, max_size:data 1554, mem_usage:GPU:0 3.8GB, num_seqs 12, 3.061 sec/step, elapsed 0:42:42, exp. remaining 0:11:38, complete 78.57% | |
| pretrain epoch 13, step 840, cost:ctc 5.131707663990653, cost:output/output_prob 6.481258076130643, error:ctc 0.9176244990667327, error:decision 0.0, error:output/output_prob 0.9042145377025007, loss 6061.9683, max_size:classes 46, max_size:data 1498, mem_usage:GPU:0 3.8GB, num_seqs 13, 2.925 sec/step |
manish-kumar-garg
/ local_incorrect_shape.txt
Last active
December 30, 2019 07:46
asr local attention error
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| warning: unable to access '/home/ubuntu/.gitconfig': Is a directory | |
| warning: unable to access '/home/ubuntu/.gitconfig': Is a directory | |
| fatal: unknown error occurred while reading the configuration files | |
| RETURNN starting up, version unknown(git exception: CalledProcessError(128, ('git', 'show', '-s', '--format=%ci', 'HEAD'))), date/time 2019-12-30-07-45-03 (UTC+0000), pid 61826, cwd /home/ubuntu/rwth-i6/returnn-experiments/2018-asr-attention/librispeech/full-setup-attention, Python /home/ubuntu/tf1.13/bin/python3 | |
| RETURNN command line options: ['local_win05.config'] | |
| Hostname: ip-10-1-21-241 | |
| /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. | |
| _np_qint8 = np.dtype([("qint8", np.int8, 1)]) | |
| /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or |
manish-kumar-garg
/ asr_local_attention_error.txt
Created
December 26, 2019 06:47
Returnn - asr_local_attention
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| RETURNN starting up, version 20191217.234858--git-09b41c6f-dirty, date/time 2019-12-26-06-45-58 (UTC+0000), pid 20215, cwd /home/ubuntu/rwth-i6/returnn-experiments/2018-asr-attention/librispeech/full-setup-attention, Python /home/ubuntu/tf1.13/bin/python3 | |
| RETURNN command line options: ['local-heuristic.argmax.win05.exp3.ctc.config'] | |
| Hostname: ip-10-1-21-241 | |
| /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. | |
| _np_qint8 = np.dtype([("qint8", np.int8, 1)]) | |
| /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. | |
| _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) | |
| /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| rnn/rnn.py returnn-distributed.config | |
| [ip-10-1-21-241:21504] Warning: could not find environment variable "HOROVOD_TIMELINE" | |
| [ip-10-1-21-241:21504] Warning: could not find environment variable "DEBUG" | |
| -------------------------------------------------------------------------- | |
| WARNING: Linux kernel CMA support was requested via the | |
| btl_vader_single_copy_mechanism MCA variable, but CMA support is | |
| not available due to restrictive ptrace settings. | |
| The vader shared memory BTL will fall back on another single-copy | |
| mechanism if one is available. This may result in lower performance. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [ip-10-1-21-241:76137] Warning: could not find environment variable "HOROVOD_TIMELINE" | |
| [ip-10-1-21-241:76137] Warning: could not find environment variable "DEBUG" | |
| Horovod initialized. Hostname ip-10-1-21-241, pid 76142, rank 0 / size 2, local rank 0 / local size 2. | |
| Horovod initialized. Hostname ip-10-1-21-241, pid 76143, rank 1 / size 2, local rank 1 / local size 2. | |
| RETURNN starting up, version 20191217.234858--git-09b41c6f-dirty, date/time 2019-12-21-20-15-27 (UTC+0000), pid 76142, cwd /home/ubuntu/rwth-i6/returnn-experiments/2018-asr-attention/librispeech/full-setup-attention, Python /home/ubuntu/anaconda3/envs/tensorflow_p36/bin/python3 | |
| RETURNN starting up, version 20191217.234858--git-09b41c6f-dirty, date/time 2019-12-21-20-15-27 (UTC+0000), pid 76143, cwd /home/ubuntu/rwth-i6/returnn-experiments/2018-asr-attention/librispeech/full-setup-attention, Python /home/ubuntu/anaconda3/envs/tensorflow_p36/bin/python3 | |
| RETURNN command line options: ['returnn-distributed.config'] | |
| Hostname: ip-10-1-21-241 | |
| RETURNN com |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| RETURNN starting up, version 20191217.234858--git-09b41c6f-dirty, date/time 2019-12-20-15-38-17 (UTC+0000), pid 50071, cwd /home/ubuntu/rwth-i6/returnn-experiments/2018-asr-attention/librispeech/full-setup-attention, Python /home/ubuntu/tf1.13/bin/python3 | |
| RETURNN command line options: ['returnn.config'] | |
| Hostname: ip-10-1-21-241 | |
| TensorFlow: 1.15.0 (v1.15.0-rc3-22-g590d6ee) (<site-package> in /home/ubuntu/tf1.13/lib/python3.6/site-packages/tensorflow) | |
| Setup TF inter and intra global thread pools, num_threads None, session opts {'log_device_placement': False, 'device_count': {'GPU': 0}}. | |
| CUDA_VISIBLE_DEVICES is set to '1'. | |
| Local devices available to TensorFlow: | |
| 1/4: name: "/device:CPU:0" | |
| device_type: "CPU" | |
| memory_limit: 268435456 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (returnn) ubuntu@ip-10-1-21-241:~/rwth-i6/returnn-experiments/2018-asr-attention/librispeech/full-setup-attention$ python3 returnn/tests/test_TFNativeOp.py | |
| __file__: returnn/tests/test_TFNativeOp.py | |
| base path: /home/ubuntu/rwth-i6/returnn-experiments/2018-asr-attention/librispeech/full-setup-attention/returnn | |
| Env OPENBLAS_NUM_THREADS = None | |
| Env GOTO_NUM_THREADS = None | |
| Env OMP_NUM_THREADS = None | |
| /home/ubuntu/returnn/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. | |
| _np_qint8 = np.dtype([("qint8", np.int8, 1)]) | |
| /home/ubuntu/returnn/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. | |
| _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) |