Created
June 29, 2018 04:40
-
-
Save Giribushan/4db26aa0cbf3092a1c44ef26e9c56a28 to your computer and use it in GitHub Desktop.
CTC label index out of bound error
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sathish@sathish-linux-deep:~/gitRepos/OCR/CRNN_Tensorflow$ python3 tools/train_shadownet.py --dataset_dir /home/sathish/gitRepos/OCR/datasets/SynthText/SynthText/tfrecords_dir/ | |
/usr/local/lib/python3.5/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`. | |
from ._conv import register_converters as _register_converters | |
The dict_path - /home/sathish/gitRepos/OCR/CRNN_Tensorflow/data/char_dict/char_dict.json | |
The dict_path - /home/sathish/gitRepos/OCR/CRNN_Tensorflow/data/char_dict/char_dict.json | |
The labels read - SparseTensor(indices=Tensor("ParseSingleExample/ParseSingleExample:0", shape=(?, 1), dtype=int64), values=Tensor("Cast:0", shape=(?,), dtype=int32), dense_shape=Tensor("ParseSingleExample/ParseSingleExample:2", shape=(1,), dtype=int64)) | |
The image names - Tensor("ParseSingleExample/ParseSingleExample:3", shape=(1,), dtype=string) | |
2018-06-29 10:09:08.875204: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA | |
2018-06-29 10:09:09.445772: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero | |
2018-06-29 10:09:09.446455: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties: | |
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.607 | |
pciBusID: 0000:01:00.0 | |
totalMemory: 10.91GiB freeMemory: 10.75GiB | |
2018-06-29 10:09:09.446481: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0 | |
2018-06-29 10:09:09.619575: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix: | |
2018-06-29 10:09:09.619621: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929] 0 | |
2018-06-29 10:09:09.619627: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0: N | |
2018-06-29 10:09:09.619784: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9496 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1) | |
INFO: 06-29 10:09:09: train_shadownet.py:121 * 140532568536832 Training from scratch | |
The input labels - SparseTensorValue(indices=array([[ 0, 0], | |
[ 0, 1], | |
[ 0, 2], | |
[ 1, 0], | |
[ 1, 1], | |
[ 2, 0], | |
[ 2, 1], | |
[ 2, 2], | |
[ 3, 0], | |
[ 3, 1], | |
[ 3, 2], | |
[ 3, 3], | |
[ 3, 4], | |
[ 4, 0], | |
[ 4, 1], | |
[ 4, 2], | |
[ 4, 3], | |
[ 5, 0], | |
[ 5, 1], | |
[ 6, 0], | |
[ 6, 1], | |
[ 6, 2], | |
[ 7, 0], | |
[ 7, 1], | |
[ 7, 2], | |
[ 8, 0], | |
[ 9, 0], | |
[ 9, 1], | |
[ 9, 2], | |
[10, 0], | |
[10, 1], | |
[10, 2], | |
[10, 3], | |
[10, 4], | |
[10, 5], | |
[10, 6], | |
[10, 7], | |
[11, 0], | |
[11, 1], | |
[11, 2], | |
[12, 0], | |
[12, 1], | |
[12, 2], | |
[12, 3], | |
[12, 4], | |
[12, 5], | |
[13, 0], | |
[13, 1], | |
[13, 2], | |
[14, 0], | |
[14, 1], | |
[14, 2], | |
[15, 0], | |
[15, 1], | |
[15, 2], | |
[16, 0], | |
[16, 1], | |
[16, 2], | |
[17, 0], | |
[17, 1], | |
[17, 2], | |
[17, 3], | |
[18, 0], | |
[18, 1], | |
[18, 2], | |
[18, 3], | |
[18, 4], | |
[18, 5], | |
[19, 0], | |
[19, 1], | |
[20, 0], | |
[20, 1], | |
[20, 2], | |
[21, 0], | |
[21, 1], | |
[21, 2], | |
[22, 0], | |
[22, 1], | |
[22, 2], | |
[23, 0], | |
[23, 1], | |
[23, 2], | |
[24, 0], | |
[24, 1], | |
[24, 2], | |
[25, 0], | |
[25, 1], | |
[25, 2], | |
[26, 0], | |
[26, 1], | |
[26, 2], | |
[26, 3], | |
[27, 0], | |
[27, 1], | |
[27, 2], | |
[28, 0], | |
[28, 1], | |
[28, 2], | |
[28, 3], | |
[29, 0], | |
[29, 1], | |
[29, 2], | |
[30, 0], | |
[30, 1], | |
[30, 2], | |
[30, 3], | |
[30, 4], | |
[31, 0], | |
[31, 1], | |
[31, 2], | |
[31, 3]]), values=array([22, 8, 13, 27, 34, 13, 14, 19, 3, 0, 19, 4, 45, 12, 0, 13, 24, | |
0, 19, 19, 7, 4, 5, 14, 17, 8, 24, 14, 20, 0, 12, 4, 17, 8, | |
2, 0, 13, 24, 14, 20, 11, 8, 13, 4, 18, 45, 12, 0, 6, 7, 8, | |
18, 24, 14, 20, 6, 14, 19, 5, 4, 17, 13, 2, 8, 21, 8, 2, 18, | |
1, 24, 5, 14, 17, 19, 14, 14, 0, 13, 3, 19, 7, 4, 0, 13, 3, | |
10, 20, 46, 22, 8, 11, 11, 12, 0, 6, 0, 4, 17, 14, 19, 7, 4, | |
11, 14, 14, 13, 24, 22, 8, 11, 11], dtype=int32), dense_shape=array([32, 8])) | |
The output labels - (25, 32, 42) | |
2018-06-29 10:09:11.027875: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at ctc_loss_op.cc:166 : Invalid argument: Saw a non-null label (index >= num_classes - 1) following a null label, batch: 10 num_classes: 42 labels: | |
Traceback (most recent call last): | |
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1322, in _do_call | |
return fn(*args) | |
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1307, in _run_fn | |
options, feed_dict, fetch_list, target_list, run_metadata) | |
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1409, in _call_tf_sessionrun | |
run_metadata) | |
tensorflow.python.framework.errors_impl.InvalidArgumentError: Saw a non-null label (index >= num_classes - 1) following a null label, batch: 10 num_classes: 42 labels: | |
[[Node: CTCLoss = CTCLoss[ctc_merge_repeated=true, ignore_longer_outputs_than_inputs=false, preprocess_collapse_repeated=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shadow/LSTMLayers/transpose_time_major/_91, shuffle_batch/TakeManySparseFromTensorsMap, shuffle_batch/TakeManySparseFromTensorsMap:1, CTCLoss/sequence_length)]] | |
During handling of the above exception, another exception occurred: | |
Traceback (most recent call last): | |
File "tools/train_shadownet.py", line 192, in <module> | |
train_shadownet(args.dataset_dir, args.weights_path) | |
File "tools/train_shadownet.py", line 136, in train_shadownet | |
sess.run(optimizer) | |
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 900, in run | |
run_metadata_ptr) | |
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1135, in _run | |
feed_dict_tensor, options, run_metadata) | |
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1316, in _do_run | |
run_metadata) | |
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1335, in _do_call | |
raise type(e)(node_def, op, message) | |
tensorflow.python.framework.errors_impl.InvalidArgumentError: Saw a non-null label (index >= num_classes - 1) following a null label, batch: 10 num_classes: 42 labels: | |
[[Node: CTCLoss = CTCLoss[ctc_merge_repeated=true, ignore_longer_outputs_than_inputs=false, preprocess_collapse_repeated=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shadow/LSTMLayers/transpose_time_major/_91, shuffle_batch/TakeManySparseFromTensorsMap, shuffle_batch/TakeManySparseFromTensorsMap:1, CTCLoss/sequence_length)]] | |
Caused by op 'CTCLoss', defined at: | |
File "tools/train_shadownet.py", line 192, in <module> | |
train_shadownet(args.dataset_dir, args.weights_path) | |
File "tools/train_shadownet.py", line 71, in train_shadownet | |
cost = tf.reduce_mean(tf.nn.ctc_loss(labels=input_labels, inputs=net_out, sequence_length=25*np.ones(32))) | |
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/ctc_ops.py", line 158, in ctc_loss | |
ignore_longer_outputs_than_inputs=ignore_longer_outputs_than_inputs) | |
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_ctc_ops.py", line 285, in ctc_loss | |
name=name) | |
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper | |
op_def=op_def) | |
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3392, in create_op | |
op_def=op_def) | |
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1718, in __init__ | |
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access | |
InvalidArgumentError (see above for traceback): Saw a non-null label (index >= num_classes - 1) following a null label, batch: 10 num_classes: 42 labels: | |
[[Node: CTCLoss = CTCLoss[ctc_merge_repeated=true, ignore_longer_outputs_than_inputs=false, preprocess_collapse_repeated=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shadow/LSTMLayers/transpose_time_major/_91, shuffle_batch/TakeManySparseFromTensorsMap, shuffle_batch/TakeManySparseFromTensorsMap:1, CTCLoss/sequence_length)]] | |
sathish@sathish-linux-deep:~/gitRepos/OCR/CRNN_Tensorflow$ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment