Skip to content

Instantly share code, notes, and snippets.

@robin-p-schmitt

robin-p-schmitt/error_log

Created February 16, 2022 13:57
Show Gist options
  • Save robin-p-schmitt/c3da466ae7377344944f5f5430103146 to your computer and use it in GitHub Desktop.
Save robin-p-schmitt/c3da466ae7377344944f5f5430103146 to your computer and use it in GitHub Desktop.
Download ZIP
Compile Graph Error
Raw
error_log
Create graph...
Loading network, train flag False, eval flag False, search flag False
DEPRECATION WARNING: Missing "from" in layer definition: root/source
This will be disallowed with behavior_version 1.
layer root/'data' output: Data{'data', [B,T|'time'[B],F|F'feature:data'(40)]}
layer root/'source' output: Data{'data', [B,T|'time'[B],F|F'feature:data'(40)]}
layer root/'source0' output: Data{'source0_output', [B,T|'time'[B],F'feature:data'(40),F|F'source0_split_dims1'(1)]}
DEPRECATION WARNING: Explicitly specify in_spatial_dims when there is more than one spatial dim in the input.
This will be disallowed with behavior_version 8.
layer root/'conv0' output: Data{'conv0_output', [B,T|'time'[B],F'feature:data'(40),F|F'conv0:channel'(32)]}
layer root/'conv0p' output: Data{'conv0p_output', [B,T|'time'[B],'conv0p:conv:s1'(20),F|F'conv0:channel'(32)]}
layer root/'conv1' output: Data{'conv1_output', [B,T|'time'[B],'conv0p:conv:s1'(20),F|F'conv1:channel'(32)]}
layer root/'conv1p' output: Data{'conv1p_output', [B,T|'time'[B],'conv1p:conv:s1'(10),F|F'conv1:channel'(32)]}
layer root/'conv_merged' output: Data{'conv_merged_output', [B,T|'time'[B],F|F'conv1p:conv:s1*conv1:channel'(320)]}
DEPRECATION WARNING: MergeDimsLayer, only keep_order=True is allowed
This will be disallowed with behavior_version 6.
layer root/'lstm0_fw' output: Data{'lstm0_fw_output', [T|'time'[B],B,F|F'lstm0_fw:feature'(1024)]}
Device not set explicitly, and we found a GPU, which we will use.
layer root/'lstm0_bw' output: Data{'lstm0_bw_output', [T|'time'[B],B,F|F'lstm0_bw:feature'(1024)]}
layer root/'lstm0_pool' output: Data{'lstm0_pool_output', [B,T|'lstm0_pool:conv:s0'[?],F|F'lstm0_fw:feature+lstm0_bw:feature'(2048)]}
layer root/'lstm1_fw' output: Data{'lstm1_fw_output', [T|'lstm0_pool:conv:s0'[B],B,F|F'lstm1_fw:feature'(1024)]}
layer root/'lstm1_bw' output: Data{'lstm1_bw_output', [T|'lstm0_pool:conv:s0'[B],B,F|F'lstm1_bw:feature'(1024)]}
layer root/'lstm1_pool' output: Data{'lstm1_pool_output', [B,T|'lstm1_pool:conv:s0'[?],F|F'lstm1_fw:feature+lstm1_bw:feature'(2048)]}
layer root/'lstm2_fw' output: Data{'lstm2_fw_output', [T|'lstm1_pool:conv:s0'[B],B,F|F'lstm2_fw:feature'(1024)]}
layer root/'lstm2_bw' output: Data{'lstm2_bw_output', [T|'lstm1_pool:conv:s0'[B],B,F|F'lstm2_bw:feature'(1024)]}
layer root/'lstm2_pool' output: Data{'lstm2_pool_output', [B,T|'lstm1_pool:conv:s0'[B],F|F'lstm2_fw:feature+lstm2_bw:feature'(2048)]}
layer root/'lstm3_fw' output: Data{'lstm3_fw_output', [T|'lstm1_pool:conv:s0'[B],B,F|F'lstm3_fw:feature'(1024)]}
layer root/'lstm3_bw' output: Data{'lstm3_bw_output', [T|'lstm1_pool:conv:s0'[B],B,F|F'lstm3_bw:feature'(1024)]}
layer root/'lstm3_pool' output: Data{'lstm3_pool_output', [B,T|'lstm1_pool:conv:s0'[B],F|F'lstm3_fw:feature+lstm3_bw:feature'(2048)]}
layer root/'lstm4_fw' output: Data{'lstm4_fw_output', [T|'lstm1_pool:conv:s0'[B],B,F|F'lstm4_fw:feature'(1024)]}
layer root/'lstm4_bw' output: Data{'lstm4_bw_output', [T|'lstm1_pool:conv:s0'[B],B,F|F'lstm4_bw:feature'(1024)]}
layer root/'lstm4_pool' output: Data{'lstm4_pool_output', [B,T|'lstm1_pool:conv:s0'[B],F|F'lstm4_fw:feature+lstm4_bw:feature'(2048)]}
layer root/'lstm5_fw' output: Data{'lstm5_fw_output', [T|'lstm1_pool:conv:s0'[B],B,F|F'lstm5_fw:feature'(1024)]}
layer root/'lstm5_bw' output: Data{'lstm5_bw_output', [T|'lstm1_pool:conv:s0'[B],B,F|F'lstm5_bw:feature'(1024)]}
layer root/'encoder0' output: Data{'encoder0_output', [T|'lstm1_pool:conv:s0'[B],B,F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)]}
layer root/'encoder' output: Data{'encoder_output', [T|'lstm1_pool:conv:s0'[B],B,F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)]}
layer root/'data:targetb' output: Data{'targetb', [B,T|'time:var:extern_data:targetb'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}
layer root/'output' output: Data{'output_output', [T|'lstm1_pool:conv:s0'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}
New state var 'base_size_lstm1_pool__conv__s0': <tf.Variable 'IO/base_size_lstm1_pool__conv__s0:0' shape=(?,) dtype=int32>, shape Data{'base_size_lstm1_pool__conv__s0', [B?], dtype='int32'}
New state var 'base_value_encoder': <tf.Variable 'IO/base_value_encoder:0' shape=(?, ?, 2048) dtype=float32>, shape Data{'encoder_output', [B,T|'lstm1_pool:conv:s0_rec_step_by_step'[B],F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)]}
New state var 'i': <tf.Variable 'IO/i:0' shape=() dtype=int32>, shape Data{'i', [], dtype='int32'}
Delayed: Layer 'output_' depends on choices {'output'}, deps on prev frame set()
Delayed: Layer 'output_is_not_blank' depends on choices {'output'}, deps on prev frame set()
Delayed: Layer 'output_emit' depends on choices {'output'}, deps on prev frame set()
New state var 'stochastic_var_scores_output': <tf.Variable 'IO/stochastic_var_scores_output:0' shape=(?, 1031) dtype=float32>, shape Data{'output_log_prob_output', [B,F|F'(label_log_prob:feature-dense)+(emit_prob0:feature-dense)'(1031)], ctx=loop('lstm1_pool:conv:s0'[B])}
New state var 'stochastic_var_choice_output': <tf.Variable 'IO/stochastic_var_choice_output:0' shape=(?,) dtype=int32>, shape Data{'output_output', [B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])}
New state var 'state_delayed/output_emit/output': <tf.Variable 'IO/state_delayed/output_emit/output:0' shape=(?,) dtype=bool>, shape Data{'output_emit_output', [B], dtype='bool', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])}
New state var 'state_delayed/output_/output': <tf.Variable 'IO/state_delayed/output_/output:0' shape=(?,) dtype=int32>, shape Data{'output__output', [B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])}
New state var 'state_delayed/output_is_not_blank/output': <tf.Variable 'IO/state_delayed/output_is_not_blank/output:0' shape=(?,) dtype=bool>, shape Data{'output__output', [B], dtype='bool', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])}
New state var 'state/prev_non_blank_embed/extra/t': <tf.Variable 'IO/state/prev_non_blank_embed/extra/t:0' shape=(?,) dtype=int32>, shape Data{'state/prev_non_blank_embed/extra/t', [B?], dtype='int32'}
New state var 'state/lm_masked/extra/_output': <tf.Variable 'IO/state/lm_masked/extra/_output:0' shape=(?, 1024) dtype=float32>, shape Data{'state/lm_masked/extra/_output', [B?,F|F'feature:state/lm_masked/extra/_output'(1024)]}
New state var 'state/lm_masked/extra/input_embed0/state': <tf.Variable 'IO/state/lm_masked/extra/input_embed0/state:0' shape=(?, 1, 621) dtype=float32>, shape Data{'state/lm_masked/extra/input_embed0/state', [B?,'spatial0:static1:state/lm_masked/extra/input_embed0/state'(1),F|F'feature:state/lm_masked/extra/input_embed0/state'(621)]}
New state var 'state/lm_masked/extra/lm/state/c': <tf.Variable 'IO/state/lm_masked/extra/lm/state/c:0' shape=(?, 1024) dtype=float32>, shape Data{'state/lm_masked/extra/lm/state/c', [B?,F|F'feature:state/lm_masked/extra/lm/state/c'(1024)]}
New state var 'state/lm_masked/extra/lm/state/h': <tf.Variable 'IO/state/lm_masked/extra/lm/state/h:0' shape=(?, 1024) dtype=float32>, shape Data{'state/lm_masked/extra/lm/state/h', [B?,F|F'feature:state/lm_masked/extra/lm/state/h'(1024)]}
New state var 'state/lm_unmask/extra/t': <tf.Variable 'IO/state/lm_unmask/extra/t:0' shape=(?,) dtype=int32>, shape Data{'state/lm_unmask/extra/t', [B?], dtype='int32'}
New state var 'state/readout_masked/extra/_output': <tf.Variable 'IO/state/readout_masked/extra/_output:0' shape=(?, 500) dtype=float32>, shape Data{'state/readout_masked/extra/_output', [B?,F|F'feature:state/readout_masked/extra/_output'(500)]}
New state var 'state/prev_non_blank_embed_masked/extra/_output': <tf.Variable 'IO/state/prev_non_blank_embed_masked/extra/_output:0' shape=(?, 621) dtype=float32>, shape Data{'state/prev_non_blank_embed_masked/extra/_output', [B?,F|F'feature:state/prev_non_blank_embed_masked/extra/_output'(621)]}
New state var 'state/readout_unmask/extra/t': <tf.Variable 'IO/state/readout_unmask/extra/t:0' shape=(?,) dtype=int32>, shape Data{'state/readout_unmask/extra/t', [B?], dtype='int32'}
New state var 'state/att_masked/extra/_output': <tf.Variable 'IO/state/att_masked/extra/_output:0' shape=(?, 2048) dtype=float32>, shape Data{'state/att_masked/extra/_output', [B?,F|F'feature:state/att_masked/extra/_output'(2048)]}
New state var 'state/segment_starts/output': <tf.Variable 'IO/state/segment_starts/output:0' shape=(?,) dtype=int32>, shape Data{'segment_starts_output', [B], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])}
New state var 'state/att_unmask/extra/t': <tf.Variable 'IO/state/att_unmask/extra/t:0' shape=(?,) dtype=int32>, shape Data{'state/att_unmask/extra/t', [B?], dtype='int32'}
layer root/output(rec-subnet)(delayed-update)/'data:targetb' output: Data{'targetb', [B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False, ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)(delayed-update)/'output' output: Data{'output_output', [B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)(delayed-update)/'output_' output: Data{'output__output', [B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)(delayed-update)/'output_is_not_blank' output: Data{'output__output', [B], dtype='bool', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)(delayed-update)/'output_emit' output: Data{'output_emit_output', [B], dtype='bool', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])}
New state var 'cond': <tf.Variable 'IO/cond:0' shape=() dtype=bool>, shape Data{'cond', [], dtype='bool'}
layer root/output(rec-subnet)/'prev_out_non_blank' output: Data{'prev_out_non_blank_output', [B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'prev_non_blank_embed0' output: Data{'prev_non_blank_embed0_output', [B,F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'prev_non_blank_embed_masked' output: Data{'prev_non_blank_embed_masked_output', [B,F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'prev_non_blank_embed' output: Data{'prev_non_blank_embed_output', [B,F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'const0.0_0' output: Data{'const0.0_0_const', [B], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'const0.0' output: Data{'const0.0_output', [B,F|F'const0.0_expand_dims'(1)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'const1.0_0' output: Data{'const1.0_0_const', [B], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'const1.0' output: Data{'const1.0_output', [B,F|F'const1.0_expand_dims'(1)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'2d_emb1' output: Data{'2d_emb1_output', [B,F|F'const0.0_expand_dims+const1.0_expand_dims'(2)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'2d_emb0' output: Data{'2d_emb0_output', [B,F|F'const1.0_expand_dims+const0.0_expand_dims'(2)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'prev_out_is_non_blank' output: Data{'prev_out_is_non_blank_output', [B,F|F'const0.0_expand_dims+const1.0_expand_dims'(2)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'data:source' output: Data{'encoder_output', [B,F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'am' output: Data{'am_output', [B,F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'s' output: Data{'s_output', [B,F|F's:feature-dense'(128)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'emit_prob0' output: Data{'emit_prob0_output', [B,F|F'emit_prob0:feature-dense'(1)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'lm_masked' output: Data{'output_output', [B,F|F'lm:feature'(1024)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)(extra._internal.masked(lm_masked))/lm_masked(subnet)/'input_embed0' output: Data{'input_embed0_output', [B,'input_embed0:window'(1),F|F'prev_non_blank_embed0:feature-dense'(621)]}
layer root/output(rec-subnet)(extra._internal.masked(lm_masked))/lm_masked(subnet)/'input_embed' output: Data{'input_embed_output', [B,F|F'input_embed0:window*(prev_non_blank_embed0:feature-dense)'(621)]}
layer root/output(rec-subnet)(extra._internal.masked(lm_masked))/lm_masked(subnet)/'lm' output: Data{'lm_output', [B,F|F'lm:feature'(1024)]}
layer root/output(rec-subnet)(extra._internal.masked(lm_masked))/lm_masked(subnet)/'output' output: Data{'output_output', [B,F|F'lm:feature'(1024)]}
layer root/output(rec-subnet)/'lm_unmask' output: Data{'lm_unmask_output', [B,F|F'lm:feature'(1024)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'lm' output: Data{'lm_output', [B,F|F'lm:feature'(1024)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'segment_starts' output: Data{'segment_starts_output', [B], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'segment_lens0' output: Data{'segment_lens0_output', [B], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'const1' output: Data{'const1_const', [], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'segment_lens' output: Data{'segment_lens_output', [B], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'const_true' output: Data{'const_true_const', [B], dtype='bool', ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)/'att_masked' output: Data{'output_output', [B,F|F'att_heads*lstm5_fw:feature+att_heads*lstm5_bw:feature'(2048)], ctx=loop('lstm1_pool:conv:s0'[B])}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'segments0' output: Data{'segments0_gather_output', [B,T|'sliced-time:segments0'[?],F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'segments' output: Data{'segments_output', [B,T|'att_t'[?],F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'att_val' output: Data{'att_val_output', [B,T|'att_t'[B],F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'att_val_split0' output: Data{'att_val_split0_output', [B,T|'att_t'[B],F'att_val_split0_split_dims0'(1),F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'att_val_split' output: Data{'att_val_split_output', [B,T|'att_t'[B],'att_heads'(1),F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'att_ctx' output: Data{'att_ctx_output', [B,T|'att_t'[B],F|F'att_ctx:feature-dense'(1024)]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'att_query' output: Data{'att_query_output', [B,F|F'lm:feature'(1024)]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'att_energy_in' output: Data{'att_energy_in_output', [B,T|'att_t'[B],F|F'att_ctx:feature-dense'(1024)]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'energy_tanh' output: Data{'energy_tanh_output', [B,T|'att_t'[B],F|F'att_ctx:feature-dense'(1024)]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'att_energy0' output: Data{'att_energy0_output', [B,T|'att_t'[B],F|F'att_energy0:feature-dense'(1)]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'att_energy' output: Data{'att_energy_output', [B,T|'att_t'[B],F|'att_heads'(1)]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'att_weights0' output: Data{'att_weights0_output', [B,F|'att_heads'(1),T|'att_t'[B]]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'att_weights' output: Data{'att_weights_output', [B,F|'att_heads'(1),T|'att_t'[B]]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'att0' output: Data{'att0_output', [B,'att_heads'(1),F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'att' output: Data{'att_output', [B,F|F'att_heads*lstm5_fw:feature+att_heads*lstm5_bw:feature'(2048)]}
layer root/output(rec-subnet)(extra._internal.masked(att_masked))/att_masked(subnet)/'output' output: Data{'output_output', [B,F|F'att_heads*lstm5_fw:feature+att_heads*lstm5_bw:feature'(2048)]}
New state var 'base_size_time__var__extern_data__targetb': <tf.Variable 'IO/base_size_time__var__extern_data__targetb:0' shape=(?,) dtype=int32>, shape Data{'base_size_time__var__extern_data__targetb', [B?], dtype='int32'}
ERROR: Got exception during in-loop construction of layer 'cur_label':
ValueError: 'IO/base_value_data:targetb' is not a valid scope name
Template network (check out types / shapes):
output: <_TemplateLayer(ChoiceStateVarLayer)(:template:choice_state_var) output/'output' out_type=Data{[B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack None)>
output_log_prob: <_TemplateLayer(CopyLayer)(:template:copy) output/'output_log_prob' out_type=Data{[B,F|F'(label_log_prob:feature-dense)+(emit_prob0:feature-dense)'(1031)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'output')>
label_emit_log_prob: <_TemplateLayer(CombineLayer)(:template:combine) output/'label_emit_log_prob' out_type=Data{[B,F|F'label_log_prob:feature-dense'(1030)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'output_log_prob')>
label_log_prob: <_TemplateLayer(LinearLayer)(:template:linear) output/'label_log_prob' out_type=Data{[B,F|F'label_log_prob:feature-dense'(1030)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'label_emit_log_prob')>
readout: <_TemplateLayer(CopyLayer)(:template:copy) output/'readout' out_type=Data{[B,F|F'(readout_in:feature-dense)//2'(500)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'label_log_prob')>
readout_unmask: <_TemplateLayer(UnmaskLayer)(:template:unmask) output/'readout_unmask' out_type=Data{[B,F|F'(readout_in:feature-dense)//2'(500)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'readout')>
readout_masked: <_TemplateLayer(MaskedComputationLayer)(:template:masked_computation) output/'readout_masked' out_type=Data{[B,F|F'(readout_in:feature-dense)//2'(500)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'readout_unmask')>
prev_non_blank_embed: <_TemplateLayer(UnmaskLayer)(:template:unmask) output/'prev_non_blank_embed' out_type=Data{[B,F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'readout_masked')>
prev_non_blank_embed_masked: <_TemplateLayer(MaskedComputationLayer)(:template:masked_computation) output/'prev_non_blank_embed_masked' out_type=Data{[B,F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_non_blank_embed')>
prev_non_blank_embed0: <_TemplateLayer(LinearLayer)(:template:linear) output/'prev_non_blank_embed0' out_type=Data{[B,F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_non_blank_embed_masked')>
prev_out_non_blank: <_TemplateLayer(ReinterpretDataLayer)(:template:reinterpret_data) output/'prev_out_non_blank' out_type=Data{[B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_non_blank_embed0')>
output_: <_TemplateLayer(CopyLayer)(:template:copy) output/'output_' out_type=Data{[B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_out_non_blank')>
output_emit: <_TemplateLayer(CopyLayer)(:template:copy) output/'output_emit' out_type=Data{[B], dtype='bool', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_non_blank_embed_masked')>
output_is_not_blank: <_TemplateLayer(CompareLayer)(:template:compare) output/'output_is_not_blank' out_type=Data{[B], dtype='bool', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'output_emit')>
lm: <_TemplateLayer(CopyLayer)(:template:copy) output/'lm' out_type=Data{[B,F|F'lm:feature'(1024)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'readout_masked')>
lm_unmask: <_TemplateLayer(UnmaskLayer)(:template:unmask) output/'lm_unmask' out_type=Data{[B,F|F'lm:feature'(1024)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'lm')>
lm_masked: <_TemplateLayer(MaskedComputationLayer)(:template:masked_computation) output/'lm_masked' out_type=Data{[B,F|F'lm:feature'(1024)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'lm_unmask')>
att: <_TemplateLayer(CopyLayer)(:template:copy) output/'att' out_type=Data{[B,F|F'att_heads*lstm5_fw:feature+att_heads*lstm5_bw:feature'(2048)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'readout_masked')>
att_unmask: <_TemplateLayer(UnmaskLayer)(:template:unmask) output/'att_unmask' out_type=Data{[B,F|F'att_heads*lstm5_fw:feature+att_heads*lstm5_bw:feature'(2048)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'att')>
att_masked: <_TemplateLayer(MaskedComputationLayer)(:template:masked_computation) output/'att_masked' out_type=Data{[B,F|F'att_heads*lstm5_fw:feature+att_heads*lstm5_bw:feature'(2048)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'att_unmask')>
segment_starts: <_TemplateLayer(SwitchLayer)(:template:switch) output/'segment_starts' out_type=Data{[B], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'att_masked')>
:i: <_TemplateLayer(RecStepInfoLayer)(:template::i) output/':i' out_type=Data{[], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'segment_starts')>
segment_lens: <_TemplateLayer(CombineLayer)(:template:combine) output/'segment_lens' out_type=Data{[B], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'att_masked')>
segment_lens0: <_TemplateLayer(CombineLayer)(:template:combine) output/'segment_lens0' out_type=Data{[B], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'segment_lens')>
const1: <_TemplateLayer(ConstantLayer)(:template:constant) output/'const1' out_type=Data{[], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'segment_lens')>
const_true: <_TemplateLayer(ConstantLayer)(:template:constant) output/'const_true' out_type=Data{[B], dtype='bool', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'att_masked')>
is_label: <_TemplateLayer(CompareLayer)(:template:compare) output/'is_label' out_type=Data{[B], dtype='bool', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'att_unmask')>
cur_label: <_TemplateLayer(GatherLayer)(:template:gather) output/'cur_label' out_type=Data{[B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'is_label')>
emit_log_prob: <_TemplateLayer(ActivationLayer)(:template:activation) output/'emit_log_prob' out_type=Data{[B,F|F'emit_prob0:feature-dense'(1)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'label_emit_log_prob')>
emit_prob0: <_TemplateLayer(LinearLayer)(:template:linear) output/'emit_prob0' out_type=Data{[B,F|F'emit_prob0:feature-dense'(1)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'emit_log_prob')>
s: <_TemplateLayer(LinearLayer)(:template:linear) output/'s' out_type=Data{[B,F|F's:feature-dense'(128)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'emit_prob0')>
prev_out_is_non_blank: <_TemplateLayer(SwitchLayer)(:template:switch) output/'prev_out_is_non_blank' out_type=Data{[B,F|F'const0.0_expand_dims+const1.0_expand_dims'(2)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 's')>
2d_emb1: <_TemplateLayer(CopyLayer)(:template:copy) output/'2d_emb1' out_type=Data{[B,F|F'const0.0_expand_dims+const1.0_expand_dims'(2)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_out_is_non_blank')>
const0.0: <_TemplateLayer(ExpandDimsLayer)(:template:expand_dims) output/'const0.0' out_type=Data{[B,F|F'const0.0_expand_dims'(1)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack '2d_emb1')>
const0.0_0: <_TemplateLayer(ConstantLayer)(:template:constant) output/'const0.0_0' out_type=Data{[B], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'const0.0')>
const1.0: <_TemplateLayer(ExpandDimsLayer)(:template:expand_dims) output/'const1.0' out_type=Data{[B,F|F'const1.0_expand_dims'(1)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack '2d_emb1')>
const1.0_0: <_TemplateLayer(ConstantLayer)(:template:constant) output/'const1.0_0' out_type=Data{[B], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'const1.0')>
2d_emb0: <_TemplateLayer(CopyLayer)(:template:copy) output/'2d_emb0' out_type=Data{[B,F|F'const1.0_expand_dims+const0.0_expand_dims'(2)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_out_is_non_blank')>
am: <_TemplateLayer(CopyLayer)(:template:copy) output/'am' out_type=Data{[B,F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 's')>
data:source: <_TemplateLayer(SourceLayer)(:template:source) output/'data:source' out_type=Data{[B,F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'am')>
blank_log_prob: <_TemplateLayer(EvalLayer)(:template:eval) output/'blank_log_prob' out_type=Data{[B,F|F'emit_prob0:feature-dense'(1)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'output_log_prob')>
data:targetb: <_TemplateLayer(SourceLayer)(:template:source) output/'data:targetb' out_type=Data{[B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'output')>
Collected (unique) exceptions during template construction:
(Note that many of these can be ignored, or are expected.)
EXCEPTION
Traceback (most recent call last):
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1488, in _SubnetworkRecCell._construct_template.<locals>.GetLayer.__call__
line: res_layer = self.net.construct_layer(
net_dict=self.net_dict, name=name,
get_layer=get_layer, add_layer=get_layer.add_templated_layer)
locals:
res_layer = <local> None
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_net=<TFNetwork 'root' train=False> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_net=<TFNetwork 'root' train=False> train=False>>
net_dict = <not found>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'prev_non_blank_embed_masked', len = 27
get_layer = <local> <RecLayer construct template GetLayer>(allow_uninitialized_template False, parents 'output <- output_log_prob <- label_emit_log_prob <- label_log_prob <- readout <- readout_unmask <- readout_masked <- prev_non_blank_embed <- prev_non_blank_embed_masked')
add_layer = <not found>
get_layer.add_templated_layer = <local> <bound method _SubnetworkRecCell._construct_template.<locals>.GetLayer.add_templated_layer of <RecLayer construct template GetLayer>(allow_uninitialized_template False, parents 'output <- output_log_prob <- label_emit_log_prob <- label_log_prob <- readout <- readout_unmask <- readout_masked <- pr...
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 956, in TFNetwork.construct_layer
line: return add_layer(name=name_with_prefix, layer_class=layer_class, **layer_desc)
locals:
add_layer = <local> <bound method _SubnetworkRecCell._construct_template.<locals>.GetLayer.add_templated_layer of <RecLayer construct template GetLayer>(allow_uninitialized_template False, parents 'output <- output_log_prob <- label_emit_log_prob <- label_log_prob <- readout <- readout_unmask <- readout_masked <- pr...
name = <local> 'prev_non_blank_embed_masked', len = 27
name_with_prefix = <local> 'prev_non_blank_embed_masked', len = 27
layer_class = <local> <class 'returnn.tf.layers.rec.MaskedComputationLayer'>
layer_desc = <local> {'mask': <_TemplateLayer(CopyLayer)(:prev:copy) output/'prev:output_emit' out_type=Data{[], dtype='bool', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack None)>, 'unit': {'class': 'copy', 'from': 'data'}, '_network': <TFNetwork 'root/output(rec-subnet)' parent_net=<TFNetwork 'root' train=F..., len = 9
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1379, in _SubnetworkRecCell._construct_template.<locals>.GetLayer.add_templated_layer
line: layer_.kwargs["output"] = layer_class.get_out_data_from_opts(**layer_desc)
locals:
layer_ = <local> <_TemplateLayer 'output/prev_non_blank_embed_masked' uninitialized, construction stack 'prev_non_blank_embed'>
layer_.kwargs = <local> {'mask': <_TemplateLayer(CopyLayer)(:prev:copy) output/'prev:output_emit' out_type=Data{[], dtype='bool', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack None)>, 'unit': {'class': 'copy', 'from': 'data'}, '_network': <TFNetwork 'root/output(rec-subnet)' parent_net=<TFNetwork 'root' train=F..., len = 11
layer_class = <local> <class 'returnn.tf.layers.rec.MaskedComputationLayer'>
layer_class.get_out_data_from_opts = <local> <bound method MaskedComputationLayer.get_out_data_from_opts of <class 'returnn.tf.layers.rec.MaskedComputationLayer'>>
layer_desc = <local> {'mask': <_TemplateLayer(CopyLayer)(:prev:copy) output/'prev:output_emit' out_type=Data{[], dtype='bool', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack None)>, 'unit': {'class': 'copy', 'from': 'data'}, '_network': <TFNetwork 'root/output(rec-subnet)' parent_net=<TFNetwork 'root' train=F..., len = 11
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 7862, in MaskedComputationLayer.get_out_data_from_opts
line: output = output.copy_as_batch_major()
locals:
output = <local> Data{'prev_non_blank_embed_masked_output', [F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])}
output.copy_as_batch_major = <local> <bound method Data.copy_as_batch_major of Data{'prev_non_blank_embed_masked_output', [F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])}>
File "/u/schmitt/src/returnn/returnn/tf/util/data.py", line 3024, in Data.copy_as_batch_major
line: return self.copy_with_batch_dim_axis(0)
locals:
self = <local> Data{'prev_non_blank_embed_masked_output', [F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])}
self.copy_with_batch_dim_axis = <local> <bound method Data.copy_with_batch_dim_axis of Data{'prev_non_blank_embed_masked_output', [F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])}>
File "/u/schmitt/src/returnn/returnn/tf/util/data.py", line 3040, in Data.copy_with_batch_dim_axis
line: assert self.batch_dim_axis is not None
locals:
self = <local> Data{'prev_non_blank_embed_masked_output', [F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])}
self.batch_dim_axis = <local> None
AssertionError
ERROR: Got exception during in-loop construction of layer 'readout_masked':
Exception: Subnetwork{root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)}: Exception constructing template network (for deps and data shapes): ValueError 'IO/base_value_data:targetb' is not a valid scope name
Template network so far:
{}
EXCEPTION
Traceback (most recent call last):
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2875, in Subnetwork._construct_template_subnet
line: get_templated_layer("output")
locals:
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'output', len = 6
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2865, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return subnet.construct_layer(
net_dict=net_dict, name=name, get_layer=get_templated_layer, add_layer=add_templated_layer)
locals:
subnet = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
subnet.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>>
net_dict = <local> {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}
name = <local> 'output', len = 6
get_layer = <not found>
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
add_layer = <not found>
add_templated_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.add_templated_layer at 0x7ff534239d30>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
layer_class.transform_config_dict = <local> <bound method CopyLayer.transform_config_dict of <class 'returnn.tf.layers.basic.CopyLayer'>>
layer_desc = <local> {'_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'output'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 363, in CopyLayer.transform_config_dict
line: super(CopyLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
CopyLayer = <global> <class 'returnn.tf.layers.basic.CopyLayer'>
cls = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
transform_config_dict = <not found>
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'output'}
network = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'output'}
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <not found>
src_names = <local> ['readout'], _[0]: {len = 7}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in LayerBase.transform_config_dict.<locals>.<listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <local> 'readout', len = 7
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'readout', len = 7
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2865, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return subnet.construct_layer(
net_dict=net_dict, name=name, get_layer=get_templated_layer, add_layer=add_templated_layer)
locals:
subnet = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
subnet.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>>
net_dict = <local> {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}
name = <local> 'readout', len = 7
get_layer = <not found>
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
add_layer = <not found>
add_templated_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.add_templated_layer at 0x7ff534239d30>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.ReduceOutLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.ReduceOutLayer'>>
layer_desc = <local> {'mode': 'max', 'num_pieces': 2, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'mode': 'max', 'num_pieces': 2, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout'}
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <not found>
src_names = <local> ['readout_in'], _[0]: {len = 10}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in LayerBase.transform_config_dict.<locals>.<listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <local> 'readout_in', len = 10
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'readout_in', len = 10
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2865, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return subnet.construct_layer(
net_dict=net_dict, name=name, get_layer=get_templated_layer, add_layer=add_templated_layer)
locals:
subnet = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
subnet.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>>
net_dict = <local> {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}
name = <local> 'readout_in', len = 10
get_layer = <not found>
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
add_layer = <not found>
add_templated_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.add_templated_layer at 0x7ff534239d30>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.LinearLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.LinearLayer'>>
layer_desc = <local> {'activation': None, 'n_out': 1000, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout_in'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'activation': None, 'n_out': 1000, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout_in'}
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <not found>
src_names = <local> ['base:lm', 'base:att'], _[0]: {len = 7}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in LayerBase.transform_config_dict.<locals>.<listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <local> 'base:att', len = 8
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'base:att', len = 8
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2863, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return get_parent_layer(name[len("base:"):])
locals:
get_parent_layer = <local> <function MaskedComputationLayer._create_template.<locals>.sub_get_layer at 0x7ff534239b80>
name = <local> 'base:att', len = 8
len = <builtin> <built-in function len>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 7818, in MaskedComputationLayer._create_template.<locals>.sub_get_layer
line: layer = get_layer(sub_layer_name)
locals:
layer = <not found>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
sub_layer_name = <local> 'att'
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'att'
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
layer_class.transform_config_dict = <local> <bound method CopyLayer.transform_config_dict of <class 'returnn.tf.layers.basic.CopyLayer'>>
layer_desc = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 363, in CopyLayer.transform_config_dict
line: super(CopyLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
CopyLayer = <global> <class 'returnn.tf.layers.basic.CopyLayer'>
cls = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
transform_config_dict = <not found>
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att'}
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['att_unmask'], _[0]: {len = 10}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in LayerBase.transform_config_dict.<locals>.<listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'att_unmask', len = 10
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'att_unmask', len = 10
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.rec.UnmaskLayer'>
layer_class.transform_config_dict = <local> <bound method UnmaskLayer.transform_config_dict of <class 'returnn.tf.layers.rec.UnmaskLayer'>>
layer_desc = <local> {'mask': 'is_label', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att_unmask', 'sources': [<MaskedComput...
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 8039, in UnmaskLayer.transform_config_dict
line: d["mask"] = get_layer(d["mask"])
locals:
d = <local> {'mask': 'is_label', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att_unmask', 'sources': [<MaskedComput...
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'is_label', len = 8
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CompareLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.CompareLayer'>>
layer_desc = <local> {'kind': 'not_equal', 'value': 1030, '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'is_label'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'kind': 'not_equal', 'value': 1030, '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'is_label'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['cur_label'], _[0]: {len = 9}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in LayerBase.transform_config_dict.<locals>.<listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'cur_label', len = 9
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'cur_label', len = 9
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.GatherLayer'>
layer_class.transform_config_dict = <local> <bound method GatherLayer.transform_config_dict of <class 'returnn.tf.layers.basic.GatherLayer'>>
layer_desc = <local> {'axis': 't', 'position': ':i', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'cur_label'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 1486, in GatherLayer.transform_config_dict
line: super(GatherLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
GatherLayer = <global> <class 'returnn.tf.layers.basic.GatherLayer'>
cls = <local> <class 'returnn.tf.layers.basic.GatherLayer'>
transform_config_dict = <not found>
d = <local> {'axis': 't', 'position': ':i', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'cur_label'}
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'axis': 't', 'position': ':i', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'cur_label'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['base:data:targetb'], _[0]: {len = 17}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in LayerBase.transform_config_dict.<locals>.<listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'base:data:targetb', len = 17
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1797, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self._get_parent_layer(name[len("base:"):])
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self._get_parent_layer = <local> <bound method SubnetworkRecCellSingleStep._get_parent_layer of <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>>
name = <local> 'base:data:targetb', len = 17
len = <builtin> <built-in function len>
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 215, in SubnetworkRecCellSingleStep._get_parent_layer
line: state_var = rec_layer.create_state_var(
name="base_value_%s" % layer_name, initial_value=output.placeholder, data_shape=output)
locals:
state_var = <not found>
rec_layer = <local> <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>
rec_layer.create_state_var = <local> <bound method RecStepByStepLayer.create_state_var of <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>>
name = <not found>
layer_name = <local> 'data:targetb', len = 12
initial_value = <not found>
output = <local> Data{'targetb', [B,T|'time:var:extern_data:targetb_rec_step_by_step'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}
output.placeholder = <local> <tf.Tensor 'extern_data/placeholders/targetb/targetb:0' shape=(?, ?) dtype=int32>
data_shape = <not found>
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 1192, in RecStepByStepLayer.create_state_var
line: var = self.StateVar(parent=self, name=name, initial_value=initial_value, data_shape=data_shape)
locals:
var = <not found>
self = <local> <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>
self.StateVar = <local> <class '__main__.RecStepByStepLayer.StateVar'>
parent = <not found>
name = <local> 'base_value_data:targetb', len = 23
initial_value = <local> <tf.Tensor 'extern_data/placeholders/targetb/targetb:0' shape=(?, ?) dtype=int32>
data_shape = <local> Data{'targetb', [B,T|'time:var:extern_data:targetb_rec_step_by_step'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 994, in RecStepByStepLayer.StateVar.__init__
line: self.var = tf_compat.v1.get_variable(
name=name, initializer=zero_initializer, validate_shape=False) # type: tf.Variable
locals:
self = <local> <StateVar 'base_value_data:targetb', shape Data{'targetb', [B,T|'time:var:extern_data:targetb_rec_step_by_step'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}, initial <tf.Tensor 'extern_data/placeholders/targetb/targetb:0' shape=(?, ?) dtype=int32>>
self.var = <local> !AttributeError: 'StateVar' object has no attribute 'var'
tf_compat = <global> <module 'returnn.tf.compat' from '/u/schmitt/src/returnn/returnn/tf/compat.py'>
tf_compat.v1 = <global> <module 'tensorflow._api.v2.compat.v1' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/_api/v2/compat/v1/__init__.py'>
tf_compat.v1.get_variable = <global> <function get_variable at 0x7ff5b37baaf0>
name = <local> 'base_value_data:targetb', len = 23
initializer = <not found>
zero_initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
validate_shape = <not found>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 1556, in get_variable
line: return get_variable_scope().get_variable(
_get_default_variable_store(),
name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
trainable=trainable,
collections=collections,
caching_device=caching_device,
partitioner=partitioner,
validate_shape=validate_shape,
use_resource=use_resource,
custom_getter=custom_getter,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
get_variable_scope = <global> <function get_variable_scope at 0x7ff5b37ba8b0>
get_variable = <global> <function get_variable at 0x7ff5b37baaf0>
_get_default_variable_store = <global> <function _get_default_variable_store at 0x7ff5b37ba940>
name = <local> 'base_value_data:targetb', len = 23
shape = <local> None
dtype = <local> None
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
trainable = <local> None
collections = <local> None
caching_device = <local> None
partitioner = <local> None
validate_shape = <local> False
use_resource = <local> None
custom_getter = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 1299, in VariableScope.get_variable
line: return var_store.get_variable(
full_name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
reuse=reuse,
trainable=trainable,
collections=collections,
caching_device=caching_device,
partitioner=partitioner,
validate_shape=validate_shape,
use_resource=use_resource,
custom_getter=custom_getter,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
var_store = <local> <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>
var_store.get_variable = <local> <bound method _VariableStore.get_variable of <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>>
full_name = <local> 'IO/base_value_data:targetb', len = 26
shape = <local> None
dtype = <local> tf.float32
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
reuse = <local> None
trainable = <local> None
collections = <local> None
caching_device = <local> None
partitioner = <local> None
validate_shape = <local> False
use_resource = <local> None
custom_getter = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 554, in _VariableStore.get_variable
line: return _true_getter(
name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
reuse=reuse,
trainable=trainable,
collections=collections,
caching_device=caching_device,
partitioner=partitioner,
validate_shape=validate_shape,
use_resource=use_resource,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
_true_getter = <local> <function _VariableStore.get_variable.<locals>._true_getter at 0x7ff534109c10>
name = <local> 'IO/base_value_data:targetb', len = 26
shape = <local> None
dtype = <local> tf.float32
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
reuse = <local> None
trainable = <local> True
collections = <local> None
caching_device = <local> None
partitioner = <local> None
validate_shape = <local> False
use_resource = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 507, in _VariableStore.get_variable.<locals>._true_getter
line: return self._get_single_variable(
name=name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
reuse=reuse,
trainable=trainable,
collections=collections,
caching_device=caching_device,
validate_shape=validate_shape,
use_resource=use_resource,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
self = <local> <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>
self._get_single_variable = <local> <bound method _VariableStore._get_single_variable of <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>>
name = <local> 'IO/base_value_data:targetb', len = 26
shape = <local> None
dtype = <local> tf.float32
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
reuse = <local> None
trainable = <local> True
collections = <local> None
caching_device = <local> None
validate_shape = <local> False
use_resource = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 929, in _VariableStore._get_single_variable
line: v = variables.VariableV1(
initial_value=init_val,
name=name,
trainable=trainable,
collections=collections,
caching_device=caching_device,
dtype=variable_dtype,
validate_shape=validate_shape,
constraint=constraint,
use_resource=use_resource,
synchronization=synchronization,
aggregation=aggregation)
locals:
v = <not found>
variables = <global> <module 'tensorflow.python.ops.variables' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py'>
variables.VariableV1 = <global> <class 'tensorflow.python.ops.variables.VariableV1'>
initial_value = <not found>
init_val = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
name = <local> 'IO/base_value_data:targetb', len = 26
trainable = <local> True
collections = <local> None
caching_device = <local> None
dtype = <local> tf.float32
variable_dtype = <local> None
validate_shape = <local> False
constraint = <local> None
use_resource = <local> True
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 260, in VariableMetaclass.__call__
line: return cls._variable_v1_call(*args, **kwargs)
locals:
cls = <local> <class 'tensorflow.python.ops.variables.VariableV1'>
cls._variable_v1_call = <local> <bound method VariableMetaclass._variable_v1_call of <class 'tensorflow.python.ops.variables.VariableV1'>>
args = <local> ()
kwargs = <local> {'initial_value': <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>, 'name': 'IO/base_value_data:targetb', 'trainable': True, 'collections': None, 'caching_device': None, 'dtype': None, 'validate_shape': False, 'constraint': None, 'use_resource': True, 'synchronization': <VariableSynchr..., len = 11
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 206, in VariableMetaclass._variable_v1_call
line: return previous_getter(
initial_value=initial_value,
trainable=trainable,
collections=collections,
validate_shape=validate_shape,
caching_device=caching_device,
name=name,
variable_def=variable_def,
dtype=dtype,
expected_shape=expected_shape,
import_scope=import_scope,
constraint=constraint,
use_resource=use_resource,
synchronization=synchronization,
aggregation=aggregation,
shape=shape)
locals:
previous_getter = <local> <function VariableMetaclass._variable_v1_call.<locals>.<lambda> at 0x7ff534109d30>
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
trainable = <local> True
collections = <local> None
validate_shape = <local> False
caching_device = <local> None
name = <local> 'IO/base_value_data:targetb', len = 26
variable_def = <local> None
dtype = <local> None
expected_shape = <local> None
import_scope = <local> None
constraint = <local> None
use_resource = <local> True
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
shape = <local> None
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 199, in VariableMetaclass._variable_v1_call.<locals>.<lambda>
line: previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
locals:
previous_getter = <not found>
kwargs = <local> {'initial_value': <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>, 'trainable': True, 'collections': None, 'validate_shape': False, 'caching_device': None, 'name': 'IO/base_value_data:targetb', 'variable_def': None, 'dtype': None, 'expected_shape': None, 'import_scope': None, 'constra..., len = 15
default_variable_creator = <global> <function default_variable_creator at 0x7ff5b37bb8b0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 2583, in default_variable_creator
line: return resource_variable_ops.ResourceVariable(
initial_value=initial_value,
trainable=trainable,
collections=collections,
validate_shape=validate_shape,
caching_device=caching_device,
name=name,
dtype=dtype,
constraint=constraint,
variable_def=variable_def,
import_scope=import_scope,
distribute_strategy=distribute_strategy,
synchronization=synchronization,
aggregation=aggregation,
shape=shape)
locals:
resource_variable_ops = <global> <module 'tensorflow.python.ops.resource_variable_ops' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py'>
resource_variable_ops.ResourceVariable = <global> <class 'tensorflow.python.ops.resource_variable_ops.ResourceVariable'>
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
trainable = <local> True
collections = <local> None
validate_shape = <local> False
caching_device = <local> None
name = <local> 'IO/base_value_data:targetb', len = 26
dtype = <local> None
constraint = <local> None
variable_def = <local> None
import_scope = <local> None
distribute_strategy = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
shape = <local> None
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 264, in VariableMetaclass.__call__
line: return super(VariableMetaclass, cls).__call__(*args, **kwargs)
locals:
super = <builtin> <class 'super'>
VariableMetaclass = <global> <class 'tensorflow.python.ops.variables.VariableMetaclass'>
cls = <local> <class 'tensorflow.python.ops.resource_variable_ops.ResourceVariable'>
__call__ = <not found>
args = <local> ()
kwargs = <local> {'initial_value': <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>, 'trainable': True, 'collections': None, 'validate_shape': False, 'caching_device': None, 'name': 'IO/base_value_data:targetb', 'dtype': None, 'constraint': None, 'variable_def': None, 'import_scope': None, 'distribute_..., len = 14
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1507, in ResourceVariable.__init__
line: self._init_from_args(
initial_value=initial_value,
trainable=trainable,
collections=collections,
caching_device=caching_device,
name=name,
dtype=dtype,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation,
shape=shape,
distribute_strategy=distribute_strategy)
locals:
self = <local> !AttributeError: 'ResourceVariable' object has no attribute '_handle_name'
self._init_from_args = <local> !AttributeError: 'ResourceVariable' object has no attribute '_handle_name'
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
trainable = <local> True
collections = <local> None
caching_device = <local> None
name = <local> 'IO/base_value_data:targetb', len = 26
dtype = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
shape = <local> None
distribute_strategy = <local> None
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1626, in ResourceVariable._init_from_args
line: with ops.name_scope(
name,
"Variable", [] if init_from_fn else [initial_value],
skip_on_eager=False) as name:
locals:
ops = <global> <module 'tensorflow.python.framework.ops' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/framework/ops.py'>
ops.name_scope = <global> <function name_scope at 0x7ff5b594f940>
name = <local> 'IO/base_value_data:targetb', len = 26
init_from_fn = <local> False
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
skip_on_eager = <not found>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 6492, in internal_name_scope_v1.__enter__
line: return self._name_scope.__enter__()
locals:
self = <local> <tensorflow.python.framework.ops.internal_name_scope_v1 object at 0x7ff5340bc730>
self._name_scope = <local> <contextlib._GeneratorContextManager object at 0x7ff5340bc790>
self._name_scope.__enter__ = <local> <bound method _GeneratorContextManager.__enter__ of <contextlib._GeneratorContextManager object at 0x7ff5340bc790>>
File "/work/tools/asr/python/3.8.0/lib/python3.8/contextlib.py", line 113, in _GeneratorContextManager.__enter__
line: return next(self.gen)
locals:
next = <builtin> <built-in function next>
self = <local> <contextlib._GeneratorContextManager object at 0x7ff5340bc790>
self.gen = <local> <generator object Graph.name_scope at 0x7ff5340aa3c0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 4190, in Graph.name_scope
line: raise ValueError("'%s' is not a valid scope name" % name)
locals:
ValueError = <builtin> <class 'ValueError'>
name = <local> 'IO/base_value_data:targetb', len = 26
ValueError: 'IO/base_value_data:targetb' is not a valid scope name
Template network (check out types / shapes):
output: <_TemplateLayer(ChoiceStateVarLayer)(:template:choice_state_var) output/'output' out_type=Data{[B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack None)>
output_log_prob: <_TemplateLayer(CopyLayer)(:template:copy) output/'output_log_prob' out_type=Data{[B,F|F'(label_log_prob:feature-dense)+(emit_prob0:feature-dense)'(1031)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'output')>
label_emit_log_prob: <_TemplateLayer(CombineLayer)(:template:combine) output/'label_emit_log_prob' out_type=Data{[B,F|F'label_log_prob:feature-dense'(1030)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'output_log_prob')>
label_log_prob: <_TemplateLayer(LinearLayer)(:template:linear) output/'label_log_prob' out_type=Data{[B,F|F'label_log_prob:feature-dense'(1030)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'label_emit_log_prob')>
readout: <_TemplateLayer(CopyLayer)(:template:copy) output/'readout' out_type=Data{[B,F|F'(readout_in:feature-dense)//2'(500)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'label_log_prob')>
readout_unmask: <_TemplateLayer(UnmaskLayer)(:template:unmask) output/'readout_unmask' out_type=Data{[B,F|F'(readout_in:feature-dense)//2'(500)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'readout')>
readout_masked: <_TemplateLayer(MaskedComputationLayer)(:template:masked_computation) output/'readout_masked' out_type=Data{[B,F|F'(readout_in:feature-dense)//2'(500)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'readout_unmask')>
prev_non_blank_embed: <_TemplateLayer(UnmaskLayer)(:template:unmask) output/'prev_non_blank_embed' out_type=Data{[B,F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'readout_masked')>
prev_non_blank_embed_masked: <_TemplateLayer(MaskedComputationLayer)(:template:masked_computation) output/'prev_non_blank_embed_masked' out_type=Data{[B,F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_non_blank_embed')>
prev_non_blank_embed0: <_TemplateLayer(LinearLayer)(:template:linear) output/'prev_non_blank_embed0' out_type=Data{[B,F|F'prev_non_blank_embed0:feature-dense'(621)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_non_blank_embed_masked')>
prev_out_non_blank: <_TemplateLayer(ReinterpretDataLayer)(:template:reinterpret_data) output/'prev_out_non_blank' out_type=Data{[B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_non_blank_embed0')>
output_: <_TemplateLayer(CopyLayer)(:template:copy) output/'output_' out_type=Data{[B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_out_non_blank')>
output_emit: <_TemplateLayer(CopyLayer)(:template:copy) output/'output_emit' out_type=Data{[B], dtype='bool', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_non_blank_embed_masked')>
output_is_not_blank: <_TemplateLayer(CompareLayer)(:template:compare) output/'output_is_not_blank' out_type=Data{[B], dtype='bool', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'output_emit')>
lm: <_TemplateLayer(CopyLayer)(:template:copy) output/'lm' out_type=Data{[B,F|F'lm:feature'(1024)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'readout_masked')>
lm_unmask: <_TemplateLayer(UnmaskLayer)(:template:unmask) output/'lm_unmask' out_type=Data{[B,F|F'lm:feature'(1024)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'lm')>
lm_masked: <_TemplateLayer(MaskedComputationLayer)(:template:masked_computation) output/'lm_masked' out_type=Data{[B,F|F'lm:feature'(1024)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'lm_unmask')>
att: <_TemplateLayer(CopyLayer)(:template:copy) output/'att' out_type=Data{[B,F|F'att_heads*lstm5_fw:feature+att_heads*lstm5_bw:feature'(2048)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'readout_masked')>
att_unmask: <_TemplateLayer(UnmaskLayer)(:template:unmask) output/'att_unmask' out_type=Data{[B,F|F'att_heads*lstm5_fw:feature+att_heads*lstm5_bw:feature'(2048)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'att')>
att_masked: <_TemplateLayer(MaskedComputationLayer)(:template:masked_computation) output/'att_masked' out_type=Data{[B,F|F'att_heads*lstm5_fw:feature+att_heads*lstm5_bw:feature'(2048)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'att_unmask')>
segment_starts: <_TemplateLayer(SwitchLayer)(:template:switch) output/'segment_starts' out_type=Data{[B], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'att_masked')>
:i: <_TemplateLayer(RecStepInfoLayer)(:template::i) output/':i' out_type=Data{[], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'segment_starts')>
segment_lens: <_TemplateLayer(CombineLayer)(:template:combine) output/'segment_lens' out_type=Data{[B], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'att_masked')>
segment_lens0: <_TemplateLayer(CombineLayer)(:template:combine) output/'segment_lens0' out_type=Data{[B], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'segment_lens')>
const1: <_TemplateLayer(ConstantLayer)(:template:constant) output/'const1' out_type=Data{[], dtype='int32', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'segment_lens')>
const_true: <_TemplateLayer(ConstantLayer)(:template:constant) output/'const_true' out_type=Data{[B], dtype='bool', ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'att_masked')>
is_label: <_TemplateLayer(CompareLayer)(:template:compare) output/'is_label' out_type=Data{[B], dtype='bool', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'att_unmask')>
cur_label: <_TemplateLayer(GatherLayer)(:template:gather) output/'cur_label' out_type=Data{[B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'is_label')>
emit_log_prob: <_TemplateLayer(ActivationLayer)(:template:activation) output/'emit_log_prob' out_type=Data{[B,F|F'emit_prob0:feature-dense'(1)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'label_emit_log_prob')>
emit_prob0: <_TemplateLayer(LinearLayer)(:template:linear) output/'emit_prob0' out_type=Data{[B,F|F'emit_prob0:feature-dense'(1)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'emit_log_prob')>
s: <_TemplateLayer(LinearLayer)(:template:linear) output/'s' out_type=Data{[B,F|F's:feature-dense'(128)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'emit_prob0')>
prev_out_is_non_blank: <_TemplateLayer(SwitchLayer)(:template:switch) output/'prev_out_is_non_blank' out_type=Data{[B,F|F'const0.0_expand_dims+const1.0_expand_dims'(2)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 's')>
2d_emb1: <_TemplateLayer(CopyLayer)(:template:copy) output/'2d_emb1' out_type=Data{[B,F|F'const0.0_expand_dims+const1.0_expand_dims'(2)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_out_is_non_blank')>
const0.0: <_TemplateLayer(ExpandDimsLayer)(:template:expand_dims) output/'const0.0' out_type=Data{[B,F|F'const0.0_expand_dims'(1)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack '2d_emb1')>
const0.0_0: <_TemplateLayer(ConstantLayer)(:template:constant) output/'const0.0_0' out_type=Data{[B], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'const0.0')>
const1.0: <_TemplateLayer(ExpandDimsLayer)(:template:expand_dims) output/'const1.0' out_type=Data{[B,F|F'const1.0_expand_dims'(1)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack '2d_emb1')>
const1.0_0: <_TemplateLayer(ConstantLayer)(:template:constant) output/'const1.0_0' out_type=Data{[B], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'const1.0')>
2d_emb0: <_TemplateLayer(CopyLayer)(:template:copy) output/'2d_emb0' out_type=Data{[B,F|F'const1.0_expand_dims+const0.0_expand_dims'(2)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'prev_out_is_non_blank')>
am: <_TemplateLayer(CopyLayer)(:template:copy) output/'am' out_type=Data{[B,F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 's')>
data:source: <_TemplateLayer(SourceLayer)(:template:source) output/'data:source' out_type=Data{[B,F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'am')>
blank_log_prob: <_TemplateLayer(EvalLayer)(:template:eval) output/'blank_log_prob' out_type=Data{[B,F|F'emit_prob0:feature-dense'(1)], ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'output_log_prob')>
data:targetb: <_TemplateLayer(SourceLayer)(:template:source) output/'data:targetb' out_type=Data{[B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False, ctx=loop('lstm1_pool:conv:s0'[B])} (construction stack 'output')>
Exception creating layer root/'output' of class RecStepByStepLayer with opts:
{'_name': 'output',
'_network': <TFNetwork 'root' train=False>,
'_orig_sources': 'encoder',
'_target_layers': {'targetb': <DataNotAvailableLayer 'data:targetb' out_type=Data{[B,T|'time:var:extern_data:targetb'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}>},
'axis': Dim{'lstm1_pool:conv:s0'[B]},
'back_prop': False,
'include_eos': True,
'max_seq_len': <tf.Tensor 'mul:0' shape=() dtype=int32>,
'n_out': <class 'returnn.util.basic.NotSpecified'>,
'name': 'output',
'network': <TFNetwork 'root' train=False>,
'output': Data{'output_output', [T|'lstm1_pool:conv:s0'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}},
'size_target': None,
'sources': [<CopyLayer 'encoder' out_type=Data{[T|'lstm1_pool:conv:s0'[B],B,F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)]}>],
'target': 'targetb',
'unit': <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>}
Unhandled exception <class 'Exception'> in thread <_MainThread(MainThread, started 140694124906240)>, proc 19769.
Thread current, main, <_MainThread(MainThread, started 140694124906240)>:
(Excluded thread.)
That were all threads.
EXCEPTION
Traceback (most recent call last):
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2875, in Subnetwork._construct_template_subnet
line: get_templated_layer("output")
locals:
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'output', len = 6
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2865, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return subnet.construct_layer(
net_dict=net_dict, name=name, get_layer=get_templated_layer, add_layer=add_templated_layer)
locals:
subnet = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
subnet.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>>
net_dict = <local> {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}
name = <local> 'output', len = 6
get_layer = <not found>
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
add_layer = <not found>
add_templated_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.add_templated_layer at 0x7ff534239d30>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
layer_class.transform_config_dict = <local> <bound method CopyLayer.transform_config_dict of <class 'returnn.tf.layers.basic.CopyLayer'>>
layer_desc = <local> {'_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'output'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 363, in CopyLayer.transform_config_dict
line: super(CopyLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
CopyLayer = <global> <class 'returnn.tf.layers.basic.CopyLayer'>
cls = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
transform_config_dict = <not found>
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'output'}
network = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'output'}
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <not found>
src_names = <local> ['readout'], _[0]: {len = 7}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <local> 'readout', len = 7
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'readout', len = 7
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2865, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return subnet.construct_layer(
net_dict=net_dict, name=name, get_layer=get_templated_layer, add_layer=add_templated_layer)
locals:
subnet = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
subnet.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>>
net_dict = <local> {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}
name = <local> 'readout', len = 7
get_layer = <not found>
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
add_layer = <not found>
add_templated_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.add_templated_layer at 0x7ff534239d30>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.ReduceOutLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.ReduceOutLayer'>>
layer_desc = <local> {'mode': 'max', 'num_pieces': 2, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'mode': 'max', 'num_pieces': 2, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout'}
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <not found>
src_names = <local> ['readout_in'], _[0]: {len = 10}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <local> 'readout_in', len = 10
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'readout_in', len = 10
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2865, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return subnet.construct_layer(
net_dict=net_dict, name=name, get_layer=get_templated_layer, add_layer=add_templated_layer)
locals:
subnet = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
subnet.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>>
net_dict = <local> {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}
name = <local> 'readout_in', len = 10
get_layer = <not found>
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
add_layer = <not found>
add_templated_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.add_templated_layer at 0x7ff534239d30>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.LinearLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.LinearLayer'>>
layer_desc = <local> {'activation': None, 'n_out': 1000, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout_in'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'activation': None, 'n_out': 1000, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout_in'}
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <not found>
src_names = <local> ['base:lm', 'base:att'], _[0]: {len = 7}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <local> 'base:att', len = 8
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'base:att', len = 8
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2863, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return get_parent_layer(name[len("base:"):])
locals:
get_parent_layer = <local> <function MaskedComputationLayer._create_template.<locals>.sub_get_layer at 0x7ff534239b80>
name = <local> 'base:att', len = 8
len = <builtin> <built-in function len>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 7818, in MaskedComputationLayer._create_template.<locals>.sub_get_layer
line: layer = get_layer(sub_layer_name)
locals:
layer = <not found>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
sub_layer_name = <local> 'att'
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'att'
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
layer_class.transform_config_dict = <local> <bound method CopyLayer.transform_config_dict of <class 'returnn.tf.layers.basic.CopyLayer'>>
layer_desc = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 363, in CopyLayer.transform_config_dict
line: super(CopyLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
CopyLayer = <global> <class 'returnn.tf.layers.basic.CopyLayer'>
cls = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
transform_config_dict = <not found>
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att'}
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['att_unmask'], _[0]: {len = 10}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'att_unmask', len = 10
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'att_unmask', len = 10
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.rec.UnmaskLayer'>
layer_class.transform_config_dict = <local> <bound method UnmaskLayer.transform_config_dict of <class 'returnn.tf.layers.rec.UnmaskLayer'>>
layer_desc = <local> {'mask': 'is_label', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att_unmask', 'sources': [<MaskedComput...
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 8039, in UnmaskLayer.transform_config_dict
line: d["mask"] = get_layer(d["mask"])
locals:
d = <local> {'mask': 'is_label', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att_unmask', 'sources': [<MaskedComput...
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'is_label', len = 8
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CompareLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.CompareLayer'>>
layer_desc = <local> {'kind': 'not_equal', 'value': 1030, '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'is_label'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'kind': 'not_equal', 'value': 1030, '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'is_label'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['cur_label'], _[0]: {len = 9}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'cur_label', len = 9
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'cur_label', len = 9
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.GatherLayer'>
layer_class.transform_config_dict = <local> <bound method GatherLayer.transform_config_dict of <class 'returnn.tf.layers.basic.GatherLayer'>>
layer_desc = <local> {'axis': 't', 'position': ':i', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'cur_label'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 1486, in GatherLayer.transform_config_dict
line: super(GatherLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
GatherLayer = <global> <class 'returnn.tf.layers.basic.GatherLayer'>
cls = <local> <class 'returnn.tf.layers.basic.GatherLayer'>
transform_config_dict = <not found>
d = <local> {'axis': 't', 'position': ':i', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'cur_label'}
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'axis': 't', 'position': ':i', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'cur_label'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['base:data:targetb'], _[0]: {len = 17}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'base:data:targetb', len = 17
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1797, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self._get_parent_layer(name[len("base:"):])
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self._get_parent_layer = <local> <bound method SubnetworkRecCellSingleStep._get_parent_layer of <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>>
name = <local> 'base:data:targetb', len = 17
len = <builtin> <built-in function len>
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 215, in SubnetworkRecCellSingleStep._get_parent_layer
line: state_var = rec_layer.create_state_var(
name="base_value_%s" % layer_name, initial_value=output.placeholder, data_shape=output)
locals:
state_var = <not found>
rec_layer = <local> <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>
rec_layer.create_state_var = <local> <bound method RecStepByStepLayer.create_state_var of <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>>
name = <not found>
layer_name = <local> 'data:targetb', len = 12
initial_value = <not found>
output = <local> Data{'targetb', [B,T|'time:var:extern_data:targetb_rec_step_by_step'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}
output.placeholder = <local> <tf.Tensor 'extern_data/placeholders/targetb/targetb:0' shape=(?, ?) dtype=int32>
data_shape = <not found>
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 1192, in RecStepByStepLayer.create_state_var
line: var = self.StateVar(parent=self, name=name, initial_value=initial_value, data_shape=data_shape)
locals:
var = <not found>
self = <local> <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>
self.StateVar = <local> <class '__main__.RecStepByStepLayer.StateVar'>
parent = <not found>
name = <local> 'base_value_data:targetb', len = 23
initial_value = <local> <tf.Tensor 'extern_data/placeholders/targetb/targetb:0' shape=(?, ?) dtype=int32>
data_shape = <local> Data{'targetb', [B,T|'time:var:extern_data:targetb_rec_step_by_step'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 994, in RecStepByStepLayer.StateVar.__init__
line: self.var = tf_compat.v1.get_variable(
name=name, initializer=zero_initializer, validate_shape=False) # type: tf.Variable
locals:
self = <local> <StateVar 'base_value_data:targetb', shape Data{'targetb', [B,T|'time:var:extern_data:targetb_rec_step_by_step'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}, initial <tf.Tensor 'extern_data/placeholders/targetb/targetb:0' shape=(?, ?) dtype=int32>>
self.var = <local> !AttributeError: 'StateVar' object has no attribute 'var'
tf_compat = <global> <module 'returnn.tf.compat' from '/u/schmitt/src/returnn/returnn/tf/compat.py'>
tf_compat.v1 = <global> <module 'tensorflow._api.v2.compat.v1' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/_api/v2/compat/v1/__init__.py'>
tf_compat.v1.get_variable = <global> <function get_variable at 0x7ff5b37baaf0>
name = <local> 'base_value_data:targetb', len = 23
initializer = <not found>
zero_initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
validate_shape = <not found>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 1556, in get_variable
line: return get_variable_scope().get_variable(
_get_default_variable_store(),
name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
trainable=trainable,
collections=collections,
caching_device=caching_device,
partitioner=partitioner,
validate_shape=validate_shape,
use_resource=use_resource,
custom_getter=custom_getter,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
get_variable_scope = <global> <function get_variable_scope at 0x7ff5b37ba8b0>
get_variable = <global> <function get_variable at 0x7ff5b37baaf0>
_get_default_variable_store = <global> <function _get_default_variable_store at 0x7ff5b37ba940>
name = <local> 'base_value_data:targetb', len = 23
shape = <local> None
dtype = <local> None
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
trainable = <local> None
collections = <local> None
caching_device = <local> None
partitioner = <local> None
validate_shape = <local> False
use_resource = <local> None
custom_getter = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 1299, in VariableScope.get_variable
line: return var_store.get_variable(
full_name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
reuse=reuse,
trainable=trainable,
collections=collections,
caching_device=caching_device,
partitioner=partitioner,
validate_shape=validate_shape,
use_resource=use_resource,
custom_getter=custom_getter,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
var_store = <local> <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>
var_store.get_variable = <local> <bound method _VariableStore.get_variable of <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>>
full_name = <local> 'IO/base_value_data:targetb', len = 26
shape = <local> None
dtype = <local> tf.float32
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
reuse = <local> None
trainable = <local> None
collections = <local> None
caching_device = <local> None
partitioner = <local> None
validate_shape = <local> False
use_resource = <local> None
custom_getter = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 554, in _VariableStore.get_variable
line: return _true_getter(
name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
reuse=reuse,
trainable=trainable,
collections=collections,
caching_device=caching_device,
partitioner=partitioner,
validate_shape=validate_shape,
use_resource=use_resource,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
_true_getter = <local> <function _VariableStore.get_variable.<locals>._true_getter at 0x7ff534109c10>
name = <local> 'IO/base_value_data:targetb', len = 26
shape = <local> None
dtype = <local> tf.float32
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
reuse = <local> None
trainable = <local> True
collections = <local> None
caching_device = <local> None
partitioner = <local> None
validate_shape = <local> False
use_resource = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 507, in _VariableStore.get_variable.<locals>._true_getter
line: return self._get_single_variable(
name=name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
reuse=reuse,
trainable=trainable,
collections=collections,
caching_device=caching_device,
validate_shape=validate_shape,
use_resource=use_resource,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
self = <local> <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>
self._get_single_variable = <local> <bound method _VariableStore._get_single_variable of <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>>
name = <local> 'IO/base_value_data:targetb', len = 26
shape = <local> None
dtype = <local> tf.float32
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
reuse = <local> None
trainable = <local> True
collections = <local> None
caching_device = <local> None
validate_shape = <local> False
use_resource = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 929, in _VariableStore._get_single_variable
line: v = variables.VariableV1(
initial_value=init_val,
name=name,
trainable=trainable,
collections=collections,
caching_device=caching_device,
dtype=variable_dtype,
validate_shape=validate_shape,
constraint=constraint,
use_resource=use_resource,
synchronization=synchronization,
aggregation=aggregation)
locals:
v = <not found>
variables = <global> <module 'tensorflow.python.ops.variables' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py'>
variables.VariableV1 = <global> <class 'tensorflow.python.ops.variables.VariableV1'>
initial_value = <not found>
init_val = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
name = <local> 'IO/base_value_data:targetb', len = 26
trainable = <local> True
collections = <local> None
caching_device = <local> None
dtype = <local> tf.float32
variable_dtype = <local> None
validate_shape = <local> False
constraint = <local> None
use_resource = <local> True
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 260, in VariableMetaclass.__call__
line: return cls._variable_v1_call(*args, **kwargs)
locals:
cls = <local> <class 'tensorflow.python.ops.variables.VariableV1'>
cls._variable_v1_call = <local> <bound method VariableMetaclass._variable_v1_call of <class 'tensorflow.python.ops.variables.VariableV1'>>
args = <local> ()
kwargs = <local> {'initial_value': <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>, 'name': 'IO/base_value_data:targetb', 'trainable': True, 'collections': None, 'caching_device': None, 'dtype': None, 'validate_shape': False, 'constraint': None, 'use_resource': True, 'synchronization': <VariableSynchr..., len = 11
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 206, in VariableMetaclass._variable_v1_call
line: return previous_getter(
initial_value=initial_value,
trainable=trainable,
collections=collections,
validate_shape=validate_shape,
caching_device=caching_device,
name=name,
variable_def=variable_def,
dtype=dtype,
expected_shape=expected_shape,
import_scope=import_scope,
constraint=constraint,
use_resource=use_resource,
synchronization=synchronization,
aggregation=aggregation,
shape=shape)
locals:
previous_getter = <local> <function VariableMetaclass._variable_v1_call.<locals>.<lambda> at 0x7ff534109d30>
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
trainable = <local> True
collections = <local> None
validate_shape = <local> False
caching_device = <local> None
name = <local> 'IO/base_value_data:targetb', len = 26
variable_def = <local> None
dtype = <local> None
expected_shape = <local> None
import_scope = <local> None
constraint = <local> None
use_resource = <local> True
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
shape = <local> None
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 199, in VariableMetaclass._variable_v1_call.<locals>.<lambda>
line: previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
locals:
previous_getter = <not found>
kwargs = <local> {'initial_value': <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>, 'trainable': True, 'collections': None, 'validate_shape': False, 'caching_device': None, 'name': 'IO/base_value_data:targetb', 'variable_def': None, 'dtype': None, 'expected_shape': None, 'import_scope': None, 'constra..., len = 15
default_variable_creator = <global> <function default_variable_creator at 0x7ff5b37bb8b0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 2583, in default_variable_creator
line: return resource_variable_ops.ResourceVariable(
initial_value=initial_value,
trainable=trainable,
collections=collections,
validate_shape=validate_shape,
caching_device=caching_device,
name=name,
dtype=dtype,
constraint=constraint,
variable_def=variable_def,
import_scope=import_scope,
distribute_strategy=distribute_strategy,
synchronization=synchronization,
aggregation=aggregation,
shape=shape)
locals:
resource_variable_ops = <global> <module 'tensorflow.python.ops.resource_variable_ops' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py'>
resource_variable_ops.ResourceVariable = <global> <class 'tensorflow.python.ops.resource_variable_ops.ResourceVariable'>
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
trainable = <local> True
collections = <local> None
validate_shape = <local> False
caching_device = <local> None
name = <local> 'IO/base_value_data:targetb', len = 26
dtype = <local> None
constraint = <local> None
variable_def = <local> None
import_scope = <local> None
distribute_strategy = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
shape = <local> None
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 264, in VariableMetaclass.__call__
line: return super(VariableMetaclass, cls).__call__(*args, **kwargs)
locals:
super = <builtin> <class 'super'>
VariableMetaclass = <global> <class 'tensorflow.python.ops.variables.VariableMetaclass'>
cls = <local> <class 'tensorflow.python.ops.resource_variable_ops.ResourceVariable'>
__call__ = <not found>
args = <local> ()
kwargs = <local> {'initial_value': <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>, 'trainable': True, 'collections': None, 'validate_shape': False, 'caching_device': None, 'name': 'IO/base_value_data:targetb', 'dtype': None, 'constraint': None, 'variable_def': None, 'import_scope': None, 'distribute_..., len = 14
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1507, in ResourceVariable.__init__
line: self._init_from_args(
initial_value=initial_value,
trainable=trainable,
collections=collections,
caching_device=caching_device,
name=name,
dtype=dtype,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation,
shape=shape,
distribute_strategy=distribute_strategy)
locals:
self = <local> !AttributeError: 'ResourceVariable' object has no attribute '_handle_name'
self._init_from_args = <local> !AttributeError: 'ResourceVariable' object has no attribute '_handle_name'
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
trainable = <local> True
collections = <local> None
caching_device = <local> None
name = <local> 'IO/base_value_data:targetb', len = 26
dtype = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
shape = <local> None
distribute_strategy = <local> None
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1626, in ResourceVariable._init_from_args
line: with ops.name_scope(
name,
"Variable", [] if init_from_fn else [initial_value],
skip_on_eager=False) as name:
locals:
ops = <global> <module 'tensorflow.python.framework.ops' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/framework/ops.py'>
ops.name_scope = <global> <function name_scope at 0x7ff5b594f940>
name = <local> 'IO/base_value_data:targetb', len = 26
init_from_fn = <local> False
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
skip_on_eager = <not found>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 6492, in internal_name_scope_v1.__enter__
line: return self._name_scope.__enter__()
locals:
self = <local> <tensorflow.python.framework.ops.internal_name_scope_v1 object at 0x7ff5340bc730>
self._name_scope = <local> <contextlib._GeneratorContextManager object at 0x7ff5340bc790>
self._name_scope.__enter__ = <local> <bound method _GeneratorContextManager.__enter__ of <contextlib._GeneratorContextManager object at 0x7ff5340bc790>>
File "/work/tools/asr/python/3.8.0/lib/python3.8/contextlib.py", line 113, in _GeneratorContextManager.__enter__
line: return next(self.gen)
locals:
next = <builtin> <built-in function next>
self = <local> <contextlib._GeneratorContextManager object at 0x7ff5340bc790>
self.gen = <local> <generator object Graph.name_scope at 0x7ff5340aa3c0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 4190, in Graph.name_scope
line: raise ValueError("'%s' is not a valid scope name" % name)
locals:
ValueError = <builtin> <class 'ValueError'>
name = <local> 'IO/base_value_data:targetb', len = 26
ValueError: 'IO/base_value_data:targetb' is not a valid scope name
During handling of the above exception, another exception occurred:
EXCEPTION
Traceback (most recent call last):
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 1565, in <module>
line: main(sys.argv)
locals:
main = <local> <function main at 0x7ff59d441550>
sys = <local> <module 'sys' (built-in)>
sys.argv = <local> ['/u/schmitt/src/returnn/tools/compile_tf_graph.py', 'returnn.config', '--rec_step_by_step', 'output', '--rec_step_by_step_output_file', '/u/schmitt/experiments/transducer/work/i6_private/users/schmitt/returnn/tools/CompileTFGraphJob.q50Q3r76km5b/output/out-rec.info', '--output_file', '/u/schmitt..., len = 8, _[0]: {len = 48}
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 1491, in main
line: network = create_graph(train_flag=train_flag, eval_flag=eval_flag, search_flag=search_flag, net_dict=net_dict)
locals:
network = <not found>
create_graph = <global> <function create_graph at 0x7ff59d487c10>
train_flag = <local> False
eval_flag = <local> False
search_flag = <local> False
net_dict = <local> {'#info': {'l2': 0.0001, 'learning_rate': 0.001, 'lstm_dim': 1024, 'time_red': [3, 2]}, '1_targetb_base': {'class': 'copy', 'from': 'existing_alignment', 'register_as_extern_data': None}, '2_targetb_target': {'class': 'eval', 'eval': 'source(0)', 'from': 'data:targetb_base', 'register_as_extern_d..., len = 40
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 77, in create_graph
line: network, updater = Engine.create_network(
config=config, rnd_seed=1,
train_flag=train_flag, eval_flag=eval_flag, search_flag=search_flag,
net_dict=net_dict)
locals:
network = <not found>
updater = <not found>
Engine = <local> <class 'returnn.tf.engine.Engine'>
Engine.create_network = <local> <bound method Engine.create_network of <class 'returnn.tf.engine.Engine'>>
config = <global> <returnn.config.Config object at 0x7ff59d405c40>
rnd_seed = <not found>
train_flag = <local> False
eval_flag = <local> False
search_flag = <local> False
net_dict = <local> {'#info': {'l2': 0.0001, 'learning_rate': 0.001, 'lstm_dim': 1024, 'time_red': [3, 2]}, '1_targetb_base': {'class': 'copy', 'from': 'existing_alignment', 'register_as_extern_data': None}, '2_targetb_target': {'class': 'eval', 'eval': 'source(0)', 'from': 'data:targetb_base', 'register_as_extern_d..., len = 40
File "/u/schmitt/src/returnn/returnn/tf/engine.py", line 1342, in Engine.create_network
line: network.construct_from_dict(net_dict)
locals:
network = <local> <TFNetwork 'root' train=False>
network.construct_from_dict = <local> <bound method TFNetwork.construct_from_dict of <TFNetwork 'root' train=False>>
net_dict = <local> {'#info': {'l2': 0.0001, 'learning_rate': 0.001, 'lstm_dim': 1024, 'time_red': [3, 2]}, '1_targetb_base': {'class': 'copy', 'from': 'existing_alignment', 'register_as_extern_data': None}, '2_targetb_target': {'class': 'eval', 'eval': 'source(0)', 'from': 'data:targetb_base', 'register_as_extern_d..., len = 40
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 629, in TFNetwork.construct_from_dict
line: self.construct_layer(net_dict, name, get_layer=get_layer)
locals:
self = <local> <TFNetwork 'root' train=False>
self.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root' train=False>>
net_dict = <local> {'#info': {'l2': 0.0001, 'learning_rate': 0.001, 'lstm_dim': 1024, 'time_red': [3, 2]}, '1_targetb_base': {'class': 'copy', 'from': 'existing_alignment', 'register_as_extern_data': None}, '2_targetb_target': {'class': 'eval', 'eval': 'source(0)', 'from': 'data:targetb_base', 'register_as_extern_d..., len = 40
name = <local> 'output', len = 6
get_layer = <local> None
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 956, in TFNetwork.construct_layer
line: return add_layer(name=name_with_prefix, layer_class=layer_class, **layer_desc)
locals:
add_layer = <local> <bound method TFNetwork.add_layer of <TFNetwork 'root' train=False>>
name = <local> 'output', len = 6
name_with_prefix = <local> 'output', len = 6
layer_class = <local> <class '__main__.RecStepByStepLayer'>
layer_desc = <local> {'back_prop': False, 'include_eos': True, 'max_seq_len': <tf.Tensor 'mul:0' shape=() dtype=int32>, 'size_target': None, 'target': 'targetb', '_network': <TFNetwork 'root' train=False>, '_name': 'output', '_orig_sources': 'encoder', 'n_out': <class 'returnn.util.basic.NotSpecified'>, 'sources': [<..., len = 13
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 1103, in TFNetwork.add_layer
line: layer = self._create_layer(name=name, layer_class=layer_class, **layer_desc)
locals:
layer = <not found>
self = <local> <TFNetwork 'root' train=False>
self._create_layer = <local> <bound method TFNetwork._create_layer of <TFNetwork 'root' train=False>>
name = <local> 'output', len = 6
layer_class = <local> <class '__main__.RecStepByStepLayer'>
layer_desc = <local> {'back_prop': False, 'include_eos': True, 'max_seq_len': <tf.Tensor 'mul:0' shape=() dtype=int32>, 'size_target': None, 'target': 'targetb', '_network': <TFNetwork 'root' train=False>, '_name': 'output', '_orig_sources': 'encoder', 'n_out': <class 'returnn.util.basic.NotSpecified'>, 'sources': [<..., len = 13
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 1023, in TFNetwork._create_layer
line: layer = layer_class(**layer_desc)
locals:
layer = <not found>
layer_class = <local> <class '__main__.RecStepByStepLayer'>
layer_desc = <local> {'back_prop': False, 'include_eos': True, 'max_seq_len': <tf.Tensor 'mul:0' shape=() dtype=int32>, 'size_target': None, 'target': 'targetb', '_network': <TFNetwork 'root' train=False>, '_name': 'output', '_orig_sources': 'encoder', 'n_out': <class 'returnn.util.basic.NotSpecified'>, 'sources': [<..., len = 16
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 1129, in RecStepByStepLayer.__init__
line: super(RecStepByStepLayer, self).__init__(
sources=sources, network=network, name=name, output=output, unit=unit, axis=axis, **kwargs)
locals:
super = <builtin> <class 'super'>
RecStepByStepLayer = <global> <class '__main__.RecStepByStepLayer'>
self = <local> <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>
__init__ = <not found>
sources = <local> [<WrappedInternalLayer 'encoder' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B,F|F'lstm5_fw:feature+lstm5_bw:feature'(2048)]}>]
network = <local> <TFNetwork 'root' train=False>
name = <local> 'output', len = 6
output = <local> Data{'output_output', [T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}
unit = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
axis = <local> Dim{'lstm1_pool:conv:s0_rec_step_by_step'[B]}
kwargs = <local> {'back_prop': False, 'include_eos': True, 'max_seq_len': <tf.Tensor 'mul:0' shape=() dtype=int32>, 'size_target': None, 'target': 'targetb', '_network': <TFNetwork 'root' train=False>, '_name': 'output', 'n_out': <class 'returnn.util.basic.NotSpecified'>, '_target_layers': {'targetb': <DataNotAva..., len = 10
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 253, in RecLayer.__init__
line: y = self._get_output_subnet_unit(self.cell)
locals:
y = <not found>
self = <local> <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>
self._get_output_subnet_unit = <local> <bound method RecLayer._get_output_subnet_unit of <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>>
self.cell = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1055, in RecLayer._get_output_subnet_unit
line: output = cell.get_output()
locals:
output = <not found>
cell = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
cell.get_output = <local> <bound method _SubnetworkRecCell.get_output of <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 2891, in _SubnetworkRecCell.get_output
line: final_loop_vars = self._while_loop(
cond=cond,
body=body,
loop_vars=init_loop_vars,
shape_invariants=shape_invariants)
locals:
final_loop_vars = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self._while_loop = <local> <bound method SubnetworkRecCellSingleStep._while_loop of <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>>
cond = <local> <function _SubnetworkRecCell.get_output.<locals>.cond at 0x7ff544301af0>
body = <local> <function _SubnetworkRecCell.get_output.<locals>.body at 0x7ff5442b7f70>
loop_vars = <not found>
init_loop_vars = <local> (<tf.Tensor 'output/rec/initial_i:0' shape=() dtype=int32>, ([<tf.Tensor 'output/rec/output_/init_output__const/constant_with_shape/Mul:0' shape=(?,) dtype=int32>, <tf.Tensor 'output/rec/output_emit/init_output_emit_const/constant_with_shape/LogicalAnd:0' shape=(?,) dtype=bool>, <tf.Tensor 'outpu...
shape_invariants = <local> (TensorShape([]), ([TensorShape([Dimension(None)]), TensorShape([Dimension(None)]), TensorShape([Dimension(None)]), TensorShape([Dimension(None)])], [[TensorShape([Dimension(None), Dimension(2048)])], [TensorShape([Dimension(None)])], [TensorShape([Dimension(None), Dimension(1024)]), TensorShape(..., _[0]: {len = 0}
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 508, in SubnetworkRecCellSingleStep._while_loop
line: res = body(i, net_vars, acc_tas, seq_len_info)
locals:
res = <not found>
body = <local> <function _SubnetworkRecCell.get_output.<locals>.body at 0x7ff5442b7f70>
i = <local> <tf.Tensor 'output/rec/Identity:0' shape=() dtype=int32>
net_vars = <local> ([<tf.Tensor 'output/rec/delayed_state_update/Identity_1:0' shape=(?,) dtype=int32>, <tf.Tensor 'output/rec/delayed_state_update/Identity:0' shape=(?,) dtype=bool>, <tf.Tensor 'output/rec/delayed_state_update/Identity_2:0' shape=(?,) dtype=bool>, <tf.Tensor 'output/rec/Identity_13:0' shape=(?,) d...
acc_tas = <local> [<tf.TensorArray 'output/rec/subnet_base/acc_ta_output_output'>, <tf.TensorArray 'output/rec/subnet_base/acc_ta_output_emit_prob0'>, <tf.TensorArray 'output/rec/subnet_base/acc_ta_output_output_emit'>]
seq_len_info = <local> None
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 2727, in _SubnetworkRecCell.get_output.<locals>.body
line: self._construct(
prev_outputs=prev_outputs, prev_extra=prev_extra,
i=i,
data=data_,
inputs_moved_out_tas=input_layers_moved_out_tas,
needed_outputs=needed_outputs)
locals:
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self._construct = <local> <bound method SubnetworkRecCellSingleStep._construct of <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>>
prev_outputs = <local> {'output_': <tf.Tensor 'output/rec/while_loop_body/prev_outputs/identity_output_:0' shape=(?,) dtype=int32>, 'output_emit': <tf.Tensor 'output/rec/while_loop_body/prev_outputs/identity_output_emit:0' shape=(?,) dtype=bool>, 'output_is_not_blank': <tf.Tensor 'output/rec/while_loop_body/prev_output...
prev_extra = <local> {'att_masked': {'_output': <tf.Tensor 'output/rec/while_loop_body/prev_extra/identity_att_masked_.95.output:0' shape=(?, 2048) dtype=float32>}, 'att_unmask': {'t': <tf.Tensor 'output/rec/while_loop_body/prev_extra/identity_att_unmask_t:0' shape=(?,) dtype=int32>}, 'lm_masked': {'_output': <tf.Ten..., len = 8
i = <local> <tf.Tensor 'output/rec/Identity:0' shape=() dtype=int32>
data = <not found>
data_ = <local> {'source': <tf.Tensor 'output/rec/tile_transposed_2/Reshape:0' shape=(?, 2048) dtype=float32>}
inputs_moved_out_tas = <not found>
input_layers_moved_out_tas = <local> {}
needed_outputs = <local> {'output', 'emit_prob0', 'output_emit'}, len = 3
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 610, in SubnetworkRecCellSingleStep._construct
line: super(SubnetworkRecCellSingleStep, self)._construct(
prev_outputs=prev_outputs, prev_extra=prev_extra, i=i, data=data,
inputs_moved_out_tas=inputs_moved_out_tas, needed_outputs=needed_outputs)
locals:
super = <builtin> <class 'super'>
SubnetworkRecCellSingleStep = <global> <class '__main__.SubnetworkRecCellSingleStep'>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
_construct = <not found>
prev_outputs = <local> {'output_': <tf.Tensor 'output/rec/while_loop_body/prev_outputs/identity_output_:0' shape=(?,) dtype=int32>, 'output_emit': <tf.Tensor 'output/rec/while_loop_body/prev_outputs/identity_output_emit:0' shape=(?,) dtype=bool>, 'output_is_not_blank': <tf.Tensor 'output/rec/while_loop_body/prev_output...
prev_extra = <local> {'att_masked': {'_output': <tf.Tensor 'output/rec/while_loop_body/prev_extra/identity_att_masked_.95.output:0' shape=(?, 2048) dtype=float32>}, 'att_unmask': {'t': <tf.Tensor 'output/rec/while_loop_body/prev_extra/identity_att_unmask_t:0' shape=(?,) dtype=int32>}, 'lm_masked': {'_output': <tf.Ten..., len = 8
i = <local> <tf.Tensor 'output/rec/Identity:0' shape=() dtype=int32>
data = <local> {'source': <tf.Tensor 'output/rec/tile_transposed_2/Reshape:0' shape=(?, 2048) dtype=float32>}
inputs_moved_out_tas = <local> {}
needed_outputs = <local> {'output', 'emit_prob0', 'output_emit'}, len = 3
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1840, in _SubnetworkRecCell._construct
line: layer = get_layer(layer_name)
locals:
layer = <local> <LinearLayer output/'emit_prob0' out_type=Data{[B,F|F'emit_prob0:feature-dense'(1)], ctx=loop('lstm1_pool:conv:s0'[B])}>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
layer_name = <local> 'output', len = 6
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'output', len = 6
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class '__main__.ChoiceStateVarLayer'>
layer_class.transform_config_dict = <local> <bound method ChoiceStateVarLayer.transform_config_dict of <class '__main__.ChoiceStateVarLayer'>>
layer_desc = <local> {'beam_size': 12, 'cheating': None, 'custom_score_combine': None, 'explicit_search_sources': None, 'initial_output': 0, 'input_type': 'log_prob', 'target': ['targetb'], '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_..., len = 9
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 1428, in ChoiceStateVarLayer.transform_config_dict
line: super(ChoiceStateVarLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
ChoiceStateVarLayer = <global> <class '__main__.ChoiceStateVarLayer'>
cls = <local> <class '__main__.ChoiceStateVarLayer'>
transform_config_dict = <not found>
d = <local> {'beam_size': 12, 'cheating': None, 'custom_score_combine': None, 'explicit_search_sources': None, 'initial_output': 0, 'input_type': 'log_prob', 'target': ['targetb'], '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_..., len = 9
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'beam_size': 12, 'cheating': None, 'custom_score_combine': None, 'explicit_search_sources': None, 'initial_output': 0, 'input_type': 'log_prob', 'target': ['targetb'], '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_..., len = 9
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['output_log_prob'], _[0]: {len = 15}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'output_log_prob', len = 15
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'output_log_prob', len = 15
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
layer_class.transform_config_dict = <local> <bound method CopyLayer.transform_config_dict of <class 'returnn.tf.layers.basic.CopyLayer'>>
layer_desc = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'output_log_prob'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 363, in CopyLayer.transform_config_dict
line: super(CopyLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
CopyLayer = <global> <class 'returnn.tf.layers.basic.CopyLayer'>
cls = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
transform_config_dict = <not found>
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'output_log_prob'}
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'output_log_prob'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['label_emit_log_prob', 'blank_log_prob'], _[0]: {len = 19}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'label_emit_log_prob', len = 19
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'label_emit_log_prob', len = 19
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CombineLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.CombineLayer'>>
layer_desc = <local> {'kind': 'add', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'label_emit_log_prob'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'kind': 'add', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'label_emit_log_prob'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['label_log_prob', 'emit_log_prob'], _[0]: {len = 14}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'label_log_prob', len = 14
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'label_log_prob', len = 14
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.LinearLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.LinearLayer'>>
layer_desc = <local> {'activation': 'log_softmax', 'dropout': 0.3, 'n_out': 1030, '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name':...
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'activation': 'log_softmax', 'dropout': 0.3, 'n_out': 1030, '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name':...
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['readout'], _[0]: {len = 7}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'readout', len = 7
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'readout', len = 7
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
layer_class.transform_config_dict = <local> <bound method CopyLayer.transform_config_dict of <class 'returnn.tf.layers.basic.CopyLayer'>>
layer_desc = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'readout'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 363, in CopyLayer.transform_config_dict
line: super(CopyLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
CopyLayer = <global> <class 'returnn.tf.layers.basic.CopyLayer'>
cls = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
transform_config_dict = <not found>
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'readout'}
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'readout'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['readout_unmask'], _[0]: {len = 14}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'readout_unmask', len = 14
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'readout_unmask', len = 14
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.rec.UnmaskLayer'>
layer_class.transform_config_dict = <local> <bound method UnmaskLayer.transform_config_dict of <class 'returnn.tf.layers.rec.UnmaskLayer'>>
layer_desc = <local> {'mask': 'is_label', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'readout_unmask'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 8038, in UnmaskLayer.transform_config_dict
line: super(UnmaskLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
UnmaskLayer = <global> <class 'returnn.tf.layers.rec.UnmaskLayer'>
cls = <local> <class 'returnn.tf.layers.rec.UnmaskLayer'>
transform_config_dict = <not found>
d = <local> {'mask': 'is_label', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'readout_unmask'}
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'mask': 'is_label', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'readout_unmask'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['readout_masked'], _[0]: {len = 14}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'readout_masked', len = 14
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'readout_masked', len = 14
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.rec.MaskedComputationLayer'>
layer_class.transform_config_dict = <local> <bound method MaskedComputationLayer.transform_config_dict of <class 'returnn.tf.layers.rec.MaskedComputationLayer'>>
layer_desc = <local> {'mask': 'const_true', 'unit': {'class': 'subnetwork', 'from': 'data', 'subnetwork': {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm',..., len = 8
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 7741, in MaskedComputationLayer.transform_config_dict
line: d["_layer_class"], d["_layer_desc"] = cls._create_template(
name=d["_name"], network=network, sources=d["sources"],
masked_from=masked_from,
unit=d["unit"],
out_spatial_dim=d.get("out_spatial_dim", None),
get_layer=get_layer, _parent_layer_cache=parent_layer_cache)
locals:
d = <local> {'mask': 'const_true', 'unit': {'class': 'subnetwork', 'from': 'data', 'subnetwork': {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm',..., len = 8
cls = <local> <class 'returnn.tf.layers.rec.MaskedComputationLayer'>
cls._create_template = <local> <bound method MaskedComputationLayer._create_template of <class 'returnn.tf.layers.rec.MaskedComputationLayer'>>
name = <not found>
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
sources = <not found>
masked_from = <local> None
unit = <not found>
out_spatial_dim = <not found>
d.get = <local> <built-in method get of dict object at 0x7ff534229440>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
_parent_layer_cache = <not found>
parent_layer_cache = <local> {'lm': <CopyLayer output/'lm' out_type=Data{[B,F|F'lm:feature'(1024)], ctx=loop('lstm1_pool:conv:s0'[B])}>}
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 7844, in MaskedComputationLayer._create_template
line: layer_class.transform_config_dict(layer_desc, network=extra_net, get_layer=sub_get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.SubnetworkLayer'>
layer_class.transform_config_dict = <local> <bound method SubnetworkLayer.transform_config_dict of <class 'returnn.tf.layers.basic.SubnetworkLayer'>>
layer_desc = <local> {'from': 'data', 'subnetwork': {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}, '_network': <TFNetwork ...
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
extra_net = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
sub_get_layer = <local> <function MaskedComputationLayer._create_template.<locals>.sub_get_layer at 0x7ff534239b80>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 7911, in SubnetworkLayer.transform_config_dict
line: d["_output"] = subnet.construct_layer("output", parent_get_layer=get_layer)
locals:
d = <local> {'from': 'data', 'subnetwork': {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}, '_network': <TFNetwork ...
subnet = <local> Subnetwork{root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)}
subnet.construct_layer = <local> <bound method Subnetwork.construct_layer of Subnetwork{root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)}>
parent_get_layer = <not found>
get_layer = <local> <function MaskedComputationLayer._create_template.<locals>.sub_get_layer at 0x7ff534239b80>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2756, in Subnetwork.construct_layer
line: return self.get_sub_layer_func(parent_get_layer)(name)
locals:
self = <local> Subnetwork{root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)}
self.get_sub_layer_func = <local> <bound method Subnetwork.get_sub_layer_func of Subnetwork{root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)}>
parent_get_layer = <local> <function MaskedComputationLayer._create_template.<locals>.sub_get_layer at 0x7ff534239b80>
name = <local> 'output', len = 6
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork.get_sub_layer_func.<locals>.wrapped_get_layer at 0x7ff534239c10>
name = <local> 'output', len = 6
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2712, in Subnetwork.get_sub_layer_func.<locals>.wrapped_get_layer
line: self._construct_template_subnet(get_parent_layer=base_get_layer)
locals:
self = <local> Subnetwork{root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)}
self._construct_template_subnet = <local> <bound method Subnetwork._construct_template_subnet of Subnetwork{root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)}>
get_parent_layer = <not found>
base_get_layer = <local> <function MaskedComputationLayer._create_template.<locals>.sub_get_layer at 0x7ff534239b80>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2902, in Subnetwork._construct_template_subnet
line: raise new_exc
locals:
new_exc = <local> Exception('Subnetwork{root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)}: Exception constructing template network (for deps and data shapes): ValueError \'IO/base_value_data:targetb\' is not a valid scope name\nTemplate network so far:\n{}\n\x1b[31;1m...
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2875, in Subnetwork._construct_template_subnet
line: get_templated_layer("output")
locals:
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'output', len = 6
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2865, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return subnet.construct_layer(
net_dict=net_dict, name=name, get_layer=get_templated_layer, add_layer=add_templated_layer)
locals:
subnet = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
subnet.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>>
net_dict = <local> {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}
name = <local> 'output', len = 6
get_layer = <not found>
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
add_layer = <not found>
add_templated_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.add_templated_layer at 0x7ff534239d30>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
layer_class.transform_config_dict = <local> <bound method CopyLayer.transform_config_dict of <class 'returnn.tf.layers.basic.CopyLayer'>>
layer_desc = <local> {'_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'output'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 363, in CopyLayer.transform_config_dict
line: super(CopyLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
CopyLayer = <global> <class 'returnn.tf.layers.basic.CopyLayer'>
cls = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
transform_config_dict = <not found>
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'output'}
network = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'output'}
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <not found>
src_names = <local> ['readout'], _[0]: {len = 7}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <local> 'readout', len = 7
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'readout', len = 7
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2865, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return subnet.construct_layer(
net_dict=net_dict, name=name, get_layer=get_templated_layer, add_layer=add_templated_layer)
locals:
subnet = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
subnet.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>>
net_dict = <local> {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}
name = <local> 'readout', len = 7
get_layer = <not found>
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
add_layer = <not found>
add_templated_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.add_templated_layer at 0x7ff534239d30>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.ReduceOutLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.ReduceOutLayer'>>
layer_desc = <local> {'mode': 'max', 'num_pieces': 2, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'mode': 'max', 'num_pieces': 2, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout'}
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <not found>
src_names = <local> ['readout_in'], _[0]: {len = 10}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <local> 'readout_in', len = 10
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'readout_in', len = 10
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2865, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return subnet.construct_layer(
net_dict=net_dict, name=name, get_layer=get_templated_layer, add_layer=add_templated_layer)
locals:
subnet = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
subnet.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>>
net_dict = <local> {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}
name = <local> 'readout_in', len = 10
get_layer = <not found>
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
add_layer = <not found>
add_templated_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.add_templated_layer at 0x7ff534239d30>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.LinearLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.LinearLayer'>>
layer_desc = <local> {'activation': None, 'n_out': 1000, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout_in'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'activation': None, 'n_out': 1000, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout_in'}
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <not found>
src_names = <local> ['base:lm', 'base:att'], _[0]: {len = 7}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <local> 'base:att', len = 8
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'base:att', len = 8
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2863, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return get_parent_layer(name[len("base:"):])
locals:
get_parent_layer = <local> <function MaskedComputationLayer._create_template.<locals>.sub_get_layer at 0x7ff534239b80>
name = <local> 'base:att', len = 8
len = <builtin> <built-in function len>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 7818, in MaskedComputationLayer._create_template.<locals>.sub_get_layer
line: layer = get_layer(sub_layer_name)
locals:
layer = <not found>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
sub_layer_name = <local> 'att'
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'att'
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
layer_class.transform_config_dict = <local> <bound method CopyLayer.transform_config_dict of <class 'returnn.tf.layers.basic.CopyLayer'>>
layer_desc = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 363, in CopyLayer.transform_config_dict
line: super(CopyLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
CopyLayer = <global> <class 'returnn.tf.layers.basic.CopyLayer'>
cls = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
transform_config_dict = <not found>
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att'}
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['att_unmask'], _[0]: {len = 10}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'att_unmask', len = 10
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'att_unmask', len = 10
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.rec.UnmaskLayer'>
layer_class.transform_config_dict = <local> <bound method UnmaskLayer.transform_config_dict of <class 'returnn.tf.layers.rec.UnmaskLayer'>>
layer_desc = <local> {'mask': 'is_label', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att_unmask', 'sources': [<MaskedComput...
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 8039, in UnmaskLayer.transform_config_dict
line: d["mask"] = get_layer(d["mask"])
locals:
d = <local> {'mask': 'is_label', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att_unmask', 'sources': [<MaskedComput...
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'is_label', len = 8
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CompareLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.CompareLayer'>>
layer_desc = <local> {'kind': 'not_equal', 'value': 1030, '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'is_label'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'kind': 'not_equal', 'value': 1030, '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'is_label'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['cur_label'], _[0]: {len = 9}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'cur_label', len = 9
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'cur_label', len = 9
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.GatherLayer'>
layer_class.transform_config_dict = <local> <bound method GatherLayer.transform_config_dict of <class 'returnn.tf.layers.basic.GatherLayer'>>
layer_desc = <local> {'axis': 't', 'position': ':i', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'cur_label'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 1486, in GatherLayer.transform_config_dict
line: super(GatherLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
GatherLayer = <global> <class 'returnn.tf.layers.basic.GatherLayer'>
cls = <local> <class 'returnn.tf.layers.basic.GatherLayer'>
transform_config_dict = <not found>
d = <local> {'axis': 't', 'position': ':i', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'cur_label'}
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'axis': 't', 'position': ':i', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'cur_label'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['base:data:targetb'], _[0]: {len = 17}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in <listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'base:data:targetb', len = 17
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1797, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self._get_parent_layer(name[len("base:"):])
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self._get_parent_layer = <local> <bound method SubnetworkRecCellSingleStep._get_parent_layer of <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>>
name = <local> 'base:data:targetb', len = 17
len = <builtin> <built-in function len>
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 215, in SubnetworkRecCellSingleStep._get_parent_layer
line: state_var = rec_layer.create_state_var(
name="base_value_%s" % layer_name, initial_value=output.placeholder, data_shape=output)
locals:
state_var = <not found>
rec_layer = <local> <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>
rec_layer.create_state_var = <local> <bound method RecStepByStepLayer.create_state_var of <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>>
name = <not found>
layer_name = <local> 'data:targetb', len = 12
initial_value = <not found>
output = <local> Data{'targetb', [B,T|'time:var:extern_data:targetb_rec_step_by_step'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}
output.placeholder = <local> <tf.Tensor 'extern_data/placeholders/targetb/targetb:0' shape=(?, ?) dtype=int32>
data_shape = <not found>
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 1192, in RecStepByStepLayer.create_state_var
line: var = self.StateVar(parent=self, name=name, initial_value=initial_value, data_shape=data_shape)
locals:
var = <not found>
self = <local> <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>
self.StateVar = <local> <class '__main__.RecStepByStepLayer.StateVar'>
parent = <not found>
name = <local> 'base_value_data:targetb', len = 23
initial_value = <local> <tf.Tensor 'extern_data/placeholders/targetb/targetb:0' shape=(?, ?) dtype=int32>
data_shape = <local> Data{'targetb', [B,T|'time:var:extern_data:targetb_rec_step_by_step'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 994, in RecStepByStepLayer.StateVar.__init__
line: self.var = tf_compat.v1.get_variable(
name=name, initializer=zero_initializer, validate_shape=False) # type: tf.Variable
locals:
self = <local> <StateVar 'base_value_data:targetb', shape Data{'targetb', [B,T|'time:var:extern_data:targetb_rec_step_by_step'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}, initial <tf.Tensor 'extern_data/placeholders/targetb/targetb:0' shape=(?, ?) dtype=int32>>
self.var = <local> !AttributeError: 'StateVar' object has no attribute 'var'
tf_compat = <global> <module 'returnn.tf.compat' from '/u/schmitt/src/returnn/returnn/tf/compat.py'>
tf_compat.v1 = <global> <module 'tensorflow._api.v2.compat.v1' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/_api/v2/compat/v1/__init__.py'>
tf_compat.v1.get_variable = <global> <function get_variable at 0x7ff5b37baaf0>
name = <local> 'base_value_data:targetb', len = 23
initializer = <not found>
zero_initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
validate_shape = <not found>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 1556, in get_variable
line: return get_variable_scope().get_variable(
_get_default_variable_store(),
name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
trainable=trainable,
collections=collections,
caching_device=caching_device,
partitioner=partitioner,
validate_shape=validate_shape,
use_resource=use_resource,
custom_getter=custom_getter,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
get_variable_scope = <global> <function get_variable_scope at 0x7ff5b37ba8b0>
get_variable = <global> <function get_variable at 0x7ff5b37baaf0>
_get_default_variable_store = <global> <function _get_default_variable_store at 0x7ff5b37ba940>
name = <local> 'base_value_data:targetb', len = 23
shape = <local> None
dtype = <local> None
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
trainable = <local> None
collections = <local> None
caching_device = <local> None
partitioner = <local> None
validate_shape = <local> False
use_resource = <local> None
custom_getter = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 1299, in VariableScope.get_variable
line: return var_store.get_variable(
full_name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
reuse=reuse,
trainable=trainable,
collections=collections,
caching_device=caching_device,
partitioner=partitioner,
validate_shape=validate_shape,
use_resource=use_resource,
custom_getter=custom_getter,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
var_store = <local> <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>
var_store.get_variable = <local> <bound method _VariableStore.get_variable of <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>>
full_name = <local> 'IO/base_value_data:targetb', len = 26
shape = <local> None
dtype = <local> tf.float32
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
reuse = <local> None
trainable = <local> None
collections = <local> None
caching_device = <local> None
partitioner = <local> None
validate_shape = <local> False
use_resource = <local> None
custom_getter = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 554, in _VariableStore.get_variable
line: return _true_getter(
name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
reuse=reuse,
trainable=trainable,
collections=collections,
caching_device=caching_device,
partitioner=partitioner,
validate_shape=validate_shape,
use_resource=use_resource,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
_true_getter = <local> <function _VariableStore.get_variable.<locals>._true_getter at 0x7ff534109c10>
name = <local> 'IO/base_value_data:targetb', len = 26
shape = <local> None
dtype = <local> tf.float32
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
reuse = <local> None
trainable = <local> True
collections = <local> None
caching_device = <local> None
partitioner = <local> None
validate_shape = <local> False
use_resource = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 507, in _VariableStore.get_variable.<locals>._true_getter
line: return self._get_single_variable(
name=name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
reuse=reuse,
trainable=trainable,
collections=collections,
caching_device=caching_device,
validate_shape=validate_shape,
use_resource=use_resource,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
self = <local> <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>
self._get_single_variable = <local> <bound method _VariableStore._get_single_variable of <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>>
name = <local> 'IO/base_value_data:targetb', len = 26
shape = <local> None
dtype = <local> tf.float32
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
reuse = <local> None
trainable = <local> True
collections = <local> None
caching_device = <local> None
validate_shape = <local> False
use_resource = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 929, in _VariableStore._get_single_variable
line: v = variables.VariableV1(
initial_value=init_val,
name=name,
trainable=trainable,
collections=collections,
caching_device=caching_device,
dtype=variable_dtype,
validate_shape=validate_shape,
constraint=constraint,
use_resource=use_resource,
synchronization=synchronization,
aggregation=aggregation)
locals:
v = <not found>
variables = <global> <module 'tensorflow.python.ops.variables' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py'>
variables.VariableV1 = <global> <class 'tensorflow.python.ops.variables.VariableV1'>
initial_value = <not found>
init_val = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
name = <local> 'IO/base_value_data:targetb', len = 26
trainable = <local> True
collections = <local> None
caching_device = <local> None
dtype = <local> tf.float32
variable_dtype = <local> None
validate_shape = <local> False
constraint = <local> None
use_resource = <local> True
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 260, in VariableMetaclass.__call__
line: return cls._variable_v1_call(*args, **kwargs)
locals:
cls = <local> <class 'tensorflow.python.ops.variables.VariableV1'>
cls._variable_v1_call = <local> <bound method VariableMetaclass._variable_v1_call of <class 'tensorflow.python.ops.variables.VariableV1'>>
args = <local> ()
kwargs = <local> {'initial_value': <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>, 'name': 'IO/base_value_data:targetb', 'trainable': True, 'collections': None, 'caching_device': None, 'dtype': None, 'validate_shape': False, 'constraint': None, 'use_resource': True, 'synchronization': <VariableSynchr..., len = 11
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 206, in VariableMetaclass._variable_v1_call
line: return previous_getter(
initial_value=initial_value,
trainable=trainable,
collections=collections,
validate_shape=validate_shape,
caching_device=caching_device,
name=name,
variable_def=variable_def,
dtype=dtype,
expected_shape=expected_shape,
import_scope=import_scope,
constraint=constraint,
use_resource=use_resource,
synchronization=synchronization,
aggregation=aggregation,
shape=shape)
locals:
previous_getter = <local> <function VariableMetaclass._variable_v1_call.<locals>.<lambda> at 0x7ff534109d30>
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
trainable = <local> True
collections = <local> None
validate_shape = <local> False
caching_device = <local> None
name = <local> 'IO/base_value_data:targetb', len = 26
variable_def = <local> None
dtype = <local> None
expected_shape = <local> None
import_scope = <local> None
constraint = <local> None
use_resource = <local> True
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
shape = <local> None
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 199, in VariableMetaclass._variable_v1_call.<locals>.<lambda>
line: previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
locals:
previous_getter = <not found>
kwargs = <local> {'initial_value': <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>, 'trainable': True, 'collections': None, 'validate_shape': False, 'caching_device': None, 'name': 'IO/base_value_data:targetb', 'variable_def': None, 'dtype': None, 'expected_shape': None, 'import_scope': None, 'constra..., len = 15
default_variable_creator = <global> <function default_variable_creator at 0x7ff5b37bb8b0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 2583, in default_variable_creator
line: return resource_variable_ops.ResourceVariable(
initial_value=initial_value,
trainable=trainable,
collections=collections,
validate_shape=validate_shape,
caching_device=caching_device,
name=name,
dtype=dtype,
constraint=constraint,
variable_def=variable_def,
import_scope=import_scope,
distribute_strategy=distribute_strategy,
synchronization=synchronization,
aggregation=aggregation,
shape=shape)
locals:
resource_variable_ops = <global> <module 'tensorflow.python.ops.resource_variable_ops' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py'>
resource_variable_ops.ResourceVariable = <global> <class 'tensorflow.python.ops.resource_variable_ops.ResourceVariable'>
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
trainable = <local> True
collections = <local> None
validate_shape = <local> False
caching_device = <local> None
name = <local> 'IO/base_value_data:targetb', len = 26
dtype = <local> None
constraint = <local> None
variable_def = <local> None
import_scope = <local> None
distribute_strategy = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
shape = <local> None
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 264, in VariableMetaclass.__call__
line: return super(VariableMetaclass, cls).__call__(*args, **kwargs)
locals:
super = <builtin> <class 'super'>
VariableMetaclass = <global> <class 'tensorflow.python.ops.variables.VariableMetaclass'>
cls = <local> <class 'tensorflow.python.ops.resource_variable_ops.ResourceVariable'>
__call__ = <not found>
args = <local> ()
kwargs = <local> {'initial_value': <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>, 'trainable': True, 'collections': None, 'validate_shape': False, 'caching_device': None, 'name': 'IO/base_value_data:targetb', 'dtype': None, 'constraint': None, 'variable_def': None, 'import_scope': None, 'distribute_..., len = 14
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1507, in ResourceVariable.__init__
line: self._init_from_args(
initial_value=initial_value,
trainable=trainable,
collections=collections,
caching_device=caching_device,
name=name,
dtype=dtype,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation,
shape=shape,
distribute_strategy=distribute_strategy)
locals:
self = <local> !AttributeError: 'ResourceVariable' object has no attribute '_handle_name'
self._init_from_args = <local> !AttributeError: 'ResourceVariable' object has no attribute '_handle_name'
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
trainable = <local> True
collections = <local> None
caching_device = <local> None
name = <local> 'IO/base_value_data:targetb', len = 26
dtype = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
shape = <local> None
distribute_strategy = <local> None
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1626, in ResourceVariable._init_from_args
line: with ops.name_scope(
name,
"Variable", [] if init_from_fn else [initial_value],
skip_on_eager=False) as name:
locals:
ops = <global> <module 'tensorflow.python.framework.ops' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/framework/ops.py'>
ops.name_scope = <global> <function name_scope at 0x7ff5b594f940>
name = <local> 'IO/base_value_data:targetb', len = 26
init_from_fn = <local> False
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
skip_on_eager = <not found>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 6492, in internal_name_scope_v1.__enter__
line: return self._name_scope.__enter__()
locals:
self = <local> <tensorflow.python.framework.ops.internal_name_scope_v1 object at 0x7ff5340bc730>
self._name_scope = <local> <contextlib._GeneratorContextManager object at 0x7ff5340bc790>
self._name_scope.__enter__ = <local> <bound method _GeneratorContextManager.__enter__ of <contextlib._GeneratorContextManager object at 0x7ff5340bc790>>
File "/work/tools/asr/python/3.8.0/lib/python3.8/contextlib.py", line 113, in _GeneratorContextManager.__enter__
line: return next(self.gen)
locals:
next = <builtin> <built-in function next>
self = <local> <contextlib._GeneratorContextManager object at 0x7ff5340bc790>
self.gen = <local> <generator object Graph.name_scope at 0x7ff5340aa3c0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 4190, in Graph.name_scope
line: raise ValueError("'%s' is not a valid scope name" % name)
locals:
ValueError = <builtin> <class 'ValueError'>
name = <local> 'IO/base_value_data:targetb', len = 26
Exception: Subnetwork{root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)}: Exception constructing template network (for deps and data shapes): ValueError 'IO/base_value_data:targetb' is not a valid scope name
Template network so far:
{}
EXCEPTION
Traceback (most recent call last):
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2875, in Subnetwork._construct_template_subnet
line: get_templated_layer("output")
locals:
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'output', len = 6
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2865, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return subnet.construct_layer(
net_dict=net_dict, name=name, get_layer=get_templated_layer, add_layer=add_templated_layer)
locals:
subnet = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
subnet.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>>
net_dict = <local> {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}
name = <local> 'output', len = 6
get_layer = <not found>
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
add_layer = <not found>
add_templated_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.add_templated_layer at 0x7ff534239d30>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
layer_class.transform_config_dict = <local> <bound method CopyLayer.transform_config_dict of <class 'returnn.tf.layers.basic.CopyLayer'>>
layer_desc = <local> {'_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'output'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 363, in CopyLayer.transform_config_dict
line: super(CopyLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
CopyLayer = <global> <class 'returnn.tf.layers.basic.CopyLayer'>
cls = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
transform_config_dict = <not found>
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'output'}
network = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'output'}
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <not found>
src_names = <local> ['readout'], _[0]: {len = 7}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in LayerBase.transform_config_dict.<locals>.<listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <local> 'readout', len = 7
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'readout', len = 7
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2865, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return subnet.construct_layer(
net_dict=net_dict, name=name, get_layer=get_templated_layer, add_layer=add_templated_layer)
locals:
subnet = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
subnet.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>>
net_dict = <local> {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}
name = <local> 'readout', len = 7
get_layer = <not found>
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
add_layer = <not found>
add_templated_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.add_templated_layer at 0x7ff534239d30>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.ReduceOutLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.ReduceOutLayer'>>
layer_desc = <local> {'mode': 'max', 'num_pieces': 2, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'mode': 'max', 'num_pieces': 2, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout'}
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <not found>
src_names = <local> ['readout_in'], _[0]: {len = 10}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in LayerBase.transform_config_dict.<locals>.<listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <local> 'readout_in', len = 10
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'readout_in', len = 10
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2865, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return subnet.construct_layer(
net_dict=net_dict, name=name, get_layer=get_templated_layer, add_layer=add_templated_layer)
locals:
subnet = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
subnet.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>>
net_dict = <local> {'output': {'class': 'copy', 'from': 'readout'}, 'readout': {'class': 'reduce_out', 'from': 'readout_in', 'mode': 'max', 'num_pieces': 2}, 'readout_in': {'activation': None, 'class': 'linear', 'from': ['base:lm', 'base:att'], 'n_out': 1000}}
name = <local> 'readout_in', len = 10
get_layer = <not found>
get_templated_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
add_layer = <not found>
add_templated_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.add_templated_layer at 0x7ff534239d30>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.LinearLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.LinearLayer'>>
layer_desc = <local> {'activation': None, 'n_out': 1000, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout_in'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'activation': None, 'n_out': 1000, '_network': <TFNetwork 'root/output(rec-subnet)(extra._internal_template.masked(readout_masked))/readout_masked(subnet)' parent_layer=<InternalLayer output/'readout_masked' out_type=Data{[B?]}> train=False>, '_name': 'readout_in'}
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <not found>
src_names = <local> ['base:lm', 'base:att'], _[0]: {len = 7}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in LayerBase.transform_config_dict.<locals>.<listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function Subnetwork.get_layer_func.<locals>.wrapped_get_layer at 0x7ff534239e50>
src_name = <local> 'base:att', len = 8
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2742, in Subnetwork.get_layer_func.<locals>.wrapped_get_layer
line: return get_layer(name)
locals:
get_layer = <local> <function Subnetwork._construct_template_subnet.<locals>.get_templated_layer at 0x7ff534239dc0>
name = <local> 'base:att', len = 8
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 2863, in Subnetwork._construct_template_subnet.<locals>.get_templated_layer
line: return get_parent_layer(name[len("base:"):])
locals:
get_parent_layer = <local> <function MaskedComputationLayer._create_template.<locals>.sub_get_layer at 0x7ff534239b80>
name = <local> 'base:att', len = 8
len = <builtin> <built-in function len>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 7818, in MaskedComputationLayer._create_template.<locals>.sub_get_layer
line: layer = get_layer(sub_layer_name)
locals:
layer = <not found>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
sub_layer_name = <local> 'att'
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'att'
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
layer_class.transform_config_dict = <local> <bound method CopyLayer.transform_config_dict of <class 'returnn.tf.layers.basic.CopyLayer'>>
layer_desc = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 363, in CopyLayer.transform_config_dict
line: super(CopyLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
CopyLayer = <global> <class 'returnn.tf.layers.basic.CopyLayer'>
cls = <local> <class 'returnn.tf.layers.basic.CopyLayer'>
transform_config_dict = <not found>
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att'}
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['att_unmask'], _[0]: {len = 10}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in LayerBase.transform_config_dict.<locals>.<listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'att_unmask', len = 10
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'att_unmask', len = 10
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.rec.UnmaskLayer'>
layer_class.transform_config_dict = <local> <bound method UnmaskLayer.transform_config_dict of <class 'returnn.tf.layers.rec.UnmaskLayer'>>
layer_desc = <local> {'mask': 'is_label', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att_unmask', 'sources': [<MaskedComput...
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 8039, in UnmaskLayer.transform_config_dict
line: d["mask"] = get_layer(d["mask"])
locals:
d = <local> {'mask': 'is_label', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'att_unmask', 'sources': [<MaskedComput...
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'is_label', len = 8
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.CompareLayer'>
layer_class.transform_config_dict = <local> <bound method LayerBase.transform_config_dict of <class 'returnn.tf.layers.basic.CompareLayer'>>
layer_desc = <local> {'kind': 'not_equal', 'value': 1030, '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'is_label'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'kind': 'not_equal', 'value': 1030, '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'is_label'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['cur_label'], _[0]: {len = 9}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in LayerBase.transform_config_dict.<locals>.<listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'cur_label', len = 9
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1808, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self.net.construct_layer(self.net_dict, name=name, get_layer=get_layer)
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self.net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
self.net.construct_layer = <local> <bound method TFNetwork.construct_layer of <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>>
self.net_dict = <local> {'2d_emb0': {'class': 'copy', 'from': ['const1.0', 'const0.0']}, '2d_emb1': {'class': 'copy', 'from': ['const0.0', 'const1.0']}, 'am': {'class': 'copy', 'from': 'data:source'}, 'att': {'class': 'copy', 'from': 'att_unmask'}, 'att_masked': {'class': 'masked_computation', 'from': 'prev_non_blank_em..., len = 46
name = <local> 'cur_label', len = 9
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/network.py", line 949, in TFNetwork.construct_layer
line: layer_class.transform_config_dict(layer_desc, network=net, get_layer=get_layer)
locals:
layer_class = <local> <class 'returnn.tf.layers.basic.GatherLayer'>
layer_class.transform_config_dict = <local> <bound method GatherLayer.transform_config_dict of <class 'returnn.tf.layers.basic.GatherLayer'>>
layer_desc = <local> {'axis': 't', 'position': ':i', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'cur_label'}
network = <not found>
net = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/basic.py", line 1486, in GatherLayer.transform_config_dict
line: super(GatherLayer, cls).transform_config_dict(d, network=network, get_layer=get_layer)
locals:
super = <builtin> <class 'super'>
GatherLayer = <global> <class 'returnn.tf.layers.basic.GatherLayer'>
cls = <local> <class 'returnn.tf.layers.basic.GatherLayer'>
transform_config_dict = <not found>
d = <local> {'axis': 't', 'position': ':i', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'cur_label'}
network = <local> <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 575, in LayerBase.transform_config_dict
line: d["sources"] = [
get_layer(src_name)
for src_name in src_names
if not src_name == "none"]
locals:
d = <local> {'axis': 't', 'position': ':i', '_network': <TFNetwork 'root/output(rec-subnet)' parent_layer=<RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}> train=False>, '_name': 'cur_label'}
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <not found>
src_names = <local> ['base:data:targetb'], _[0]: {len = 17}
File "/u/schmitt/src/returnn/returnn/tf/layers/base.py", line 576, in LayerBase.transform_config_dict.<locals>.<listcomp>
line: get_layer(src_name)
locals:
get_layer = <local> <function _SubnetworkRecCell._construct.<locals>.get_layer at 0x7ff544136ee0>
src_name = <local> 'base:data:targetb', len = 17
File "/u/schmitt/src/returnn/returnn/tf/layers/rec.py", line 1797, in _SubnetworkRecCell._construct.<locals>.get_layer
line: layer = self._get_parent_layer(name[len("base:"):])
locals:
layer = <not found>
self = <local> <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>
self._get_parent_layer = <local> <bound method SubnetworkRecCellSingleStep._get_parent_layer of <SubnetworkRecCellSingleStep 'root/output(rec-subnet)'>>
name = <local> 'base:data:targetb', len = 17
len = <builtin> <built-in function len>
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 215, in SubnetworkRecCellSingleStep._get_parent_layer
line: state_var = rec_layer.create_state_var(
name="base_value_%s" % layer_name, initial_value=output.placeholder, data_shape=output)
locals:
state_var = <not found>
rec_layer = <local> <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>
rec_layer.create_state_var = <local> <bound method RecStepByStepLayer.create_state_var of <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>>
name = <not found>
layer_name = <local> 'data:targetb', len = 12
initial_value = <not found>
output = <local> Data{'targetb', [B,T|'time:var:extern_data:targetb_rec_step_by_step'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}
output.placeholder = <local> <tf.Tensor 'extern_data/placeholders/targetb/targetb:0' shape=(?, ?) dtype=int32>
data_shape = <not found>
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 1192, in RecStepByStepLayer.create_state_var
line: var = self.StateVar(parent=self, name=name, initial_value=initial_value, data_shape=data_shape)
locals:
var = <not found>
self = <local> <RecStepByStepLayer 'output' out_type=Data{[T|'lstm1_pool:conv:s0_rec_step_by_step'[B],B], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}}>
self.StateVar = <local> <class '__main__.RecStepByStepLayer.StateVar'>
parent = <not found>
name = <local> 'base_value_data:targetb', len = 23
initial_value = <local> <tf.Tensor 'extern_data/placeholders/targetb/targetb:0' shape=(?, ?) dtype=int32>
data_shape = <local> Data{'targetb', [B,T|'time:var:extern_data:targetb_rec_step_by_step'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}
File "/u/schmitt/src/returnn/tools/compile_tf_graph.py", line 994, in RecStepByStepLayer.StateVar.__init__
line: self.var = tf_compat.v1.get_variable(
name=name, initializer=zero_initializer, validate_shape=False) # type: tf.Variable
locals:
self = <local> <StateVar 'base_value_data:targetb', shape Data{'targetb', [B,T|'time:var:extern_data:targetb_rec_step_by_step'[B]], dtype='int32', sparse_dim=Dim{F'targetb:sparse-dim'(1031)}, available_for_inference=False}, initial <tf.Tensor 'extern_data/placeholders/targetb/targetb:0' shape=(?, ?) dtype=int32>>
self.var = <local> !AttributeError: 'StateVar' object has no attribute 'var'
tf_compat = <global> <module 'returnn.tf.compat' from '/u/schmitt/src/returnn/returnn/tf/compat.py'>
tf_compat.v1 = <global> <module 'tensorflow._api.v2.compat.v1' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/_api/v2/compat/v1/__init__.py'>
tf_compat.v1.get_variable = <global> <function get_variable at 0x7ff5b37baaf0>
name = <local> 'base_value_data:targetb', len = 23
initializer = <not found>
zero_initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
validate_shape = <not found>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 1556, in get_variable
line: return get_variable_scope().get_variable(
_get_default_variable_store(),
name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
trainable=trainable,
collections=collections,
caching_device=caching_device,
partitioner=partitioner,
validate_shape=validate_shape,
use_resource=use_resource,
custom_getter=custom_getter,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
get_variable_scope = <global> <function get_variable_scope at 0x7ff5b37ba8b0>
get_variable = <global> <function get_variable at 0x7ff5b37baaf0>
_get_default_variable_store = <global> <function _get_default_variable_store at 0x7ff5b37ba940>
name = <local> 'base_value_data:targetb', len = 23
shape = <local> None
dtype = <local> None
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
trainable = <local> None
collections = <local> None
caching_device = <local> None
partitioner = <local> None
validate_shape = <local> False
use_resource = <local> None
custom_getter = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 1299, in VariableScope.get_variable
line: return var_store.get_variable(
full_name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
reuse=reuse,
trainable=trainable,
collections=collections,
caching_device=caching_device,
partitioner=partitioner,
validate_shape=validate_shape,
use_resource=use_resource,
custom_getter=custom_getter,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
var_store = <local> <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>
var_store.get_variable = <local> <bound method _VariableStore.get_variable of <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>>
full_name = <local> 'IO/base_value_data:targetb', len = 26
shape = <local> None
dtype = <local> tf.float32
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
reuse = <local> None
trainable = <local> None
collections = <local> None
caching_device = <local> None
partitioner = <local> None
validate_shape = <local> False
use_resource = <local> None
custom_getter = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 554, in _VariableStore.get_variable
line: return _true_getter(
name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
reuse=reuse,
trainable=trainable,
collections=collections,
caching_device=caching_device,
partitioner=partitioner,
validate_shape=validate_shape,
use_resource=use_resource,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
_true_getter = <local> <function _VariableStore.get_variable.<locals>._true_getter at 0x7ff534109c10>
name = <local> 'IO/base_value_data:targetb', len = 26
shape = <local> None
dtype = <local> tf.float32
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
reuse = <local> None
trainable = <local> True
collections = <local> None
caching_device = <local> None
partitioner = <local> None
validate_shape = <local> False
use_resource = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 507, in _VariableStore.get_variable.<locals>._true_getter
line: return self._get_single_variable(
name=name,
shape=shape,
dtype=dtype,
initializer=initializer,
regularizer=regularizer,
reuse=reuse,
trainable=trainable,
collections=collections,
caching_device=caching_device,
validate_shape=validate_shape,
use_resource=use_resource,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation)
locals:
self = <local> <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>
self._get_single_variable = <local> <bound method _VariableStore._get_single_variable of <tensorflow.python.ops.variable_scope._VariableStore object at 0x7ff59d2af880>>
name = <local> 'IO/base_value_data:targetb', len = 26
shape = <local> None
dtype = <local> tf.float32
initializer = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
regularizer = <local> None
reuse = <local> None
trainable = <local> True
collections = <local> None
caching_device = <local> None
validate_shape = <local> False
use_resource = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 929, in _VariableStore._get_single_variable
line: v = variables.VariableV1(
initial_value=init_val,
name=name,
trainable=trainable,
collections=collections,
caching_device=caching_device,
dtype=variable_dtype,
validate_shape=validate_shape,
constraint=constraint,
use_resource=use_resource,
synchronization=synchronization,
aggregation=aggregation)
locals:
v = <not found>
variables = <global> <module 'tensorflow.python.ops.variables' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py'>
variables.VariableV1 = <global> <class 'tensorflow.python.ops.variables.VariableV1'>
initial_value = <not found>
init_val = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
name = <local> 'IO/base_value_data:targetb', len = 26
trainable = <local> True
collections = <local> None
caching_device = <local> None
dtype = <local> tf.float32
variable_dtype = <local> None
validate_shape = <local> False
constraint = <local> None
use_resource = <local> True
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 260, in VariableMetaclass.__call__
line: return cls._variable_v1_call(*args, **kwargs)
locals:
cls = <local> <class 'tensorflow.python.ops.variables.VariableV1'>
cls._variable_v1_call = <local> <bound method VariableMetaclass._variable_v1_call of <class 'tensorflow.python.ops.variables.VariableV1'>>
args = <local> ()
kwargs = <local> {'initial_value': <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>, 'name': 'IO/base_value_data:targetb', 'trainable': True, 'collections': None, 'caching_device': None, 'dtype': None, 'validate_shape': False, 'constraint': None, 'use_resource': True, 'synchronization': <VariableSynchr..., len = 11
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 206, in VariableMetaclass._variable_v1_call
line: return previous_getter(
initial_value=initial_value,
trainable=trainable,
collections=collections,
validate_shape=validate_shape,
caching_device=caching_device,
name=name,
variable_def=variable_def,
dtype=dtype,
expected_shape=expected_shape,
import_scope=import_scope,
constraint=constraint,
use_resource=use_resource,
synchronization=synchronization,
aggregation=aggregation,
shape=shape)
locals:
previous_getter = <local> <function VariableMetaclass._variable_v1_call.<locals>.<lambda> at 0x7ff534109d30>
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
trainable = <local> True
collections = <local> None
validate_shape = <local> False
caching_device = <local> None
name = <local> 'IO/base_value_data:targetb', len = 26
variable_def = <local> None
dtype = <local> None
expected_shape = <local> None
import_scope = <local> None
constraint = <local> None
use_resource = <local> True
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
shape = <local> None
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 199, in VariableMetaclass._variable_v1_call.<locals>.<lambda>
line: previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
locals:
previous_getter = <not found>
kwargs = <local> {'initial_value': <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>, 'trainable': True, 'collections': None, 'validate_shape': False, 'caching_device': None, 'name': 'IO/base_value_data:targetb', 'variable_def': None, 'dtype': None, 'expected_shape': None, 'import_scope': None, 'constra..., len = 15
default_variable_creator = <global> <function default_variable_creator at 0x7ff5b37bb8b0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 2583, in default_variable_creator
line: return resource_variable_ops.ResourceVariable(
initial_value=initial_value,
trainable=trainable,
collections=collections,
validate_shape=validate_shape,
caching_device=caching_device,
name=name,
dtype=dtype,
constraint=constraint,
variable_def=variable_def,
import_scope=import_scope,
distribute_strategy=distribute_strategy,
synchronization=synchronization,
aggregation=aggregation,
shape=shape)
locals:
resource_variable_ops = <global> <module 'tensorflow.python.ops.resource_variable_ops' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py'>
resource_variable_ops.ResourceVariable = <global> <class 'tensorflow.python.ops.resource_variable_ops.ResourceVariable'>
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
trainable = <local> True
collections = <local> None
validate_shape = <local> False
caching_device = <local> None
name = <local> 'IO/base_value_data:targetb', len = 26
dtype = <local> None
constraint = <local> None
variable_def = <local> None
import_scope = <local> None
distribute_strategy = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
shape = <local> None
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 264, in VariableMetaclass.__call__
line: return super(VariableMetaclass, cls).__call__(*args, **kwargs)
locals:
super = <builtin> <class 'super'>
VariableMetaclass = <global> <class 'tensorflow.python.ops.variables.VariableMetaclass'>
cls = <local> <class 'tensorflow.python.ops.resource_variable_ops.ResourceVariable'>
__call__ = <not found>
args = <local> ()
kwargs = <local> {'initial_value': <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>, 'trainable': True, 'collections': None, 'validate_shape': False, 'caching_device': None, 'name': 'IO/base_value_data:targetb', 'dtype': None, 'constraint': None, 'variable_def': None, 'import_scope': None, 'distribute_..., len = 14
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1507, in ResourceVariable.__init__
line: self._init_from_args(
initial_value=initial_value,
trainable=trainable,
collections=collections,
caching_device=caching_device,
name=name,
dtype=dtype,
constraint=constraint,
synchronization=synchronization,
aggregation=aggregation,
shape=shape,
distribute_strategy=distribute_strategy)
locals:
self = <local> !AttributeError: 'ResourceVariable' object has no attribute '_handle_name'
self._init_from_args = <local> !AttributeError: 'ResourceVariable' object has no attribute '_handle_name'
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
trainable = <local> True
collections = <local> None
caching_device = <local> None
name = <local> 'IO/base_value_data:targetb', len = 26
dtype = <local> None
constraint = <local> None
synchronization = <local> <VariableSynchronization.AUTO: 0>
aggregation = <local> <VariableAggregation.NONE: 0>
shape = <local> None
distribute_strategy = <local> None
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1626, in ResourceVariable._init_from_args
line: with ops.name_scope(
name,
"Variable", [] if init_from_fn else [initial_value],
skip_on_eager=False) as name:
locals:
ops = <global> <module 'tensorflow.python.framework.ops' from '/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/framework/ops.py'>
ops.name_scope = <global> <function name_scope at 0x7ff5b594f940>
name = <local> 'IO/base_value_data:targetb', len = 26
init_from_fn = <local> False
initial_value = <local> <tf.Tensor 'output/rec/zeros_4:0' shape=(?, ?) dtype=int32>
skip_on_eager = <not found>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 6492, in internal_name_scope_v1.__enter__
line: return self._name_scope.__enter__()
locals:
self = <local> <tensorflow.python.framework.ops.internal_name_scope_v1 object at 0x7ff5340bc730>
self._name_scope = <local> <contextlib._GeneratorContextManager object at 0x7ff5340bc790>
self._name_scope.__enter__ = <local> <bound method _GeneratorContextManager.__enter__ of <contextlib._GeneratorContextManager object at 0x7ff5340bc790>>
File "/work/tools/asr/python/3.8.0/lib/python3.8/contextlib.py", line 113, in _GeneratorContextManager.__enter__
line: return next(self.gen)
locals:
next = <builtin> <built-in function next>
self = <local> <contextlib._GeneratorContextManager object at 0x7ff5340bc790>
self.gen = <local> <generator object Graph.name_scope at 0x7ff5340aa3c0>
File "/work/tools/asr/python/3.8.0_tf_2.3-v1-generic+cuda10.1/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 4190, in Graph.name_scope
line: raise ValueError("'%s' is not a valid scope name" % name)
locals:
ValueError = <builtin> <class 'ValueError'>
name = <local> 'IO/base_value_data:targetb', len = 26
ValueError: 'IO/base_value_data:targetb' is not a valid scope name
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment