Skip to content

Instantly share code, notes, and snippets.

@anijain2305

anijain2305/gist:4e2639684bbc13d28ef6de1fb5efc2b4

Created June 17, 2024 23:35
Show Gist options
  • Save anijain2305/4e2639684bbc13d28ef6de1fb5efc2b4 to your computer and use it in GitHub Desktop.
Save anijain2305/4e2639684bbc13d28ef6de1fb5efc2b4 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
TORCHDYNAMO_INLINE_INBUILT_NN_MODULES=0 TORCH_LOGS=recompiles python benchmarks/dynamo/torchbench.py --accuracy --no-translation-validation --inference --bfloat16 --backend eager --disable-cudagraphs --device cuda --only=hf_T5_generate
loading model: 0it [00:29, ?it/s]
cuda eval hf_T5_generate
V0617 16:32:34.360000 140419876123776 torch/_dynamo/guards.py:2619] [8/1] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:1639
V0617 16:32:34.360000 140419876123776 torch/_dynamo/guards.py:2619] [8/1] [__recompiles] triggered by the following guard failure(s):
V0617 16:32:34.360000 140419876123776 torch/_dynamo/guards.py:2619] [8/1] [__recompiles] - ___check_obj_id(L['past_key_values'], 8825760)
V0617 16:32:38.606000 140419876123776 torch/_dynamo/guards.py:2619] [10/1] [__recompiles] Recompiling function _update_model_kwargs_for_generation in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/generation/utils.py:628
V0617 16:32:38.606000 140419876123776 torch/_dynamo/guards.py:2619] [10/1] [__recompiles] triggered by the following guard failure(s):
V0617 16:32:38.606000 140419876123776 torch/_dynamo/guards.py:2619] [10/1] [__recompiles] - len(L['model_kwargs']) == 4
V0617 16:32:38.693000 140419876123776 torch/_dynamo/guards.py:2619] [11/1] [__recompiles] Recompiling function __call__ in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/generation/stopping_criteria.py:503
V0617 16:32:38.693000 140419876123776 torch/_dynamo/guards.py:2619] [11/1] [__recompiles] triggered by the following guard failure(s):
V0617 16:32:38.693000 140419876123776 torch/_dynamo/guards.py:2619] [11/1] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 2, actual 3
V0617 16:32:38.761000 140419876123776 torch/_dynamo/guards.py:2619] [8/2] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:1639
V0617 16:32:38.761000 140419876123776 torch/_dynamo/guards.py:2619] [8/2] [__recompiles] triggered by the following guard failure(s):
V0617 16:32:38.761000 140419876123776 torch/_dynamo/guards.py:2619] [8/2] [__recompiles] - tensor 'L['past_key_values'][0][0]' stride mismatch at index 0. expected 512, actual 1024
V0617 16:32:38.761000 140419876123776 torch/_dynamo/guards.py:2619] [8/2] [__recompiles] - ___check_obj_id(L['past_key_values'], 8825760)
V0617 16:32:45.954000 140419876123776 torch/_dynamo/guards.py:2619] [8/3] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:1639
V0617 16:32:45.954000 140419876123776 torch/_dynamo/guards.py:2619] [8/3] [__recompiles] triggered by the following guard failure(s):
V0617 16:32:45.954000 140419876123776 torch/_dynamo/guards.py:2619] [8/3] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 3, actual 4
V0617 16:32:45.954000 140419876123776 torch/_dynamo/guards.py:2619] [8/3] [__recompiles] - tensor 'L['past_key_values'][0][0]' stride mismatch at index 0. expected 512, actual 1536
V0617 16:32:45.954000 140419876123776 torch/_dynamo/guards.py:2619] [8/3] [__recompiles] - ___check_obj_id(L['past_key_values'], 8825760)
V0617 16:32:50.597000 140419876123776 torch/_dynamo/guards.py:2619] [8/4] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:1639
V0617 16:32:50.597000 140419876123776 torch/_dynamo/guards.py:2619] [8/4] [__recompiles] triggered by the following guard failure(s):
V0617 16:32:50.597000 140419876123776 torch/_dynamo/guards.py:2619] [8/4] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 4, actual 5
V0617 16:32:50.597000 140419876123776 torch/_dynamo/guards.py:2619] [8/4] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 3, actual 5
V0617 16:32:50.597000 140419876123776 torch/_dynamo/guards.py:2619] [8/4] [__recompiles] - tensor 'L['past_key_values'][0][0]' stride mismatch at index 0. expected 512, actual 2048
V0617 16:32:50.597000 140419876123776 torch/_dynamo/guards.py:2619] [8/4] [__recompiles] - ___check_obj_id(L['past_key_values'], 8825760)
V0617 16:32:56.893000 140419876123776 torch/_dynamo/guards.py:2619] [8/5] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:1639
V0617 16:32:56.893000 140419876123776 torch/_dynamo/guards.py:2619] [8/5] [__recompiles] triggered by the following guard failure(s):
V0617 16:32:56.893000 140419876123776 torch/_dynamo/guards.py:2619] [8/5] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 5, actual 6
V0617 16:32:56.893000 140419876123776 torch/_dynamo/guards.py:2619] [8/5] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 4, actual 6
V0617 16:32:56.893000 140419876123776 torch/_dynamo/guards.py:2619] [8/5] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 3, actual 6
V0617 16:32:56.893000 140419876123776 torch/_dynamo/guards.py:2619] [8/5] [__recompiles] - tensor 'L['past_key_values'][0][0]' stride mismatch at index 0. expected 512, actual 2560
V0617 16:32:56.893000 140419876123776 torch/_dynamo/guards.py:2619] [8/5] [__recompiles] - ___check_obj_id(L['past_key_values'], 8825760)
V0617 16:33:02.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/6] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:1639
V0617 16:33:02.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/6] [__recompiles] triggered by the following guard failure(s):
V0617 16:33:02.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/6] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 6, actual 7
V0617 16:33:02.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/6] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 5, actual 7
V0617 16:33:02.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/6] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 4, actual 7
V0617 16:33:02.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/6] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 3, actual 7
V0617 16:33:02.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/6] [__recompiles] - tensor 'L['past_key_values'][0][0]' stride mismatch at index 0. expected 512, actual 3072
V0617 16:33:02.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/6] [__recompiles] - ___check_obj_id(L['past_key_values'], 8825760)
V0617 16:33:09.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/7] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:1639
V0617 16:33:09.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/7] [__recompiles] triggered by the following guard failure(s):
V0617 16:33:09.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/7] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 7, actual 8
V0617 16:33:09.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/7] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 6, actual 8
V0617 16:33:09.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/7] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 5, actual 8
V0617 16:33:09.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/7] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 4, actual 8
V0617 16:33:09.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/7] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 3, actual 8
V0617 16:33:09.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/7] [__recompiles] - tensor 'L['past_key_values'][0][0]' stride mismatch at index 0. expected 512, actual 3584
V0617 16:33:09.184000 140419876123776 torch/_dynamo/guards.py:2619] [8/7] [__recompiles] - ___check_obj_id(L['past_key_values'], 8825760)
V0617 16:33:14.488000 140419876123776 torch/_dynamo/guards.py:2619] [8/8] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:1639
V0617 16:33:14.488000 140419876123776 torch/_dynamo/guards.py:2619] [8/8] [__recompiles] triggered by the following guard failure(s):
V0617 16:33:14.488000 140419876123776 torch/_dynamo/guards.py:2619] [8/8] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 8, actual 9
V0617 16:33:14.488000 140419876123776 torch/_dynamo/guards.py:2619] [8/8] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 7, actual 9
V0617 16:33:14.488000 140419876123776 torch/_dynamo/guards.py:2619] [8/8] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 6, actual 9
V0617 16:33:14.488000 140419876123776 torch/_dynamo/guards.py:2619] [8/8] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 5, actual 9
V0617 16:33:14.488000 140419876123776 torch/_dynamo/guards.py:2619] [8/8] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 4, actual 9
V0617 16:33:14.488000 140419876123776 torch/_dynamo/guards.py:2619] [8/8] [__recompiles] - tensor 'L['decoder_input_ids']' stride mismatch at index 0. expected 3, actual 9
V0617 16:33:14.488000 140419876123776 torch/_dynamo/guards.py:2619] [8/8] [__recompiles] - tensor 'L['past_key_values'][0][0]' stride mismatch at index 0. expected 512, actual 4096
V0617 16:33:14.488000 140419876123776 torch/_dynamo/guards.py:2619] [8/8] [__recompiles] - ___check_obj_id(L['past_key_values'], 8825760)
W0617 16:33:14.489000 140419876123776 torch/_dynamo/convert_frame.py:762] [8/8] torch._dynamo hit config.cache_size_limit (8)
W0617 16:33:14.489000 140419876123776 torch/_dynamo/convert_frame.py:762] [8/8] function: 'forward' (/home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:1639)
W0617 16:33:14.489000 140419876123776 torch/_dynamo/convert_frame.py:762] [8/8] last reason: ___check_obj_id(L['past_key_values'], 8825760)
W0617 16:33:14.489000 140419876123776 torch/_dynamo/convert_frame.py:762] [8/8] To log all recompilation reasons, use TORCH_LOGS="recompiles".
W0617 16:33:14.489000 140419876123776 torch/_dynamo/convert_frame.py:762] [8/8] To diagnose recompilation issues, see https://pytorch.org/docs/main/torch.compiler_troubleshooting.html.
V0617 16:33:23.915000 140419876123776 torch/_dynamo/guards.py:2619] [17/1] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:970
V0617 16:33:23.915000 140419876123776 torch/_dynamo/guards.py:2619] [17/1] [__recompiles] triggered by the following guard failure(s):
V0617 16:33:23.915000 140419876123776 torch/_dynamo/guards.py:2619] [17/1] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 9, actual 10
V0617 16:33:30.434000 140419876123776 torch/_dynamo/guards.py:2619] [17/2] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:970
V0617 16:33:30.434000 140419876123776 torch/_dynamo/guards.py:2619] [17/2] [__recompiles] triggered by the following guard failure(s):
V0617 16:33:30.434000 140419876123776 torch/_dynamo/guards.py:2619] [17/2] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 10, actual 11
V0617 16:33:30.434000 140419876123776 torch/_dynamo/guards.py:2619] [17/2] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 9, actual 11
V0617 16:33:36.046000 140419876123776 torch/_dynamo/guards.py:2619] [17/3] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:970
V0617 16:33:36.046000 140419876123776 torch/_dynamo/guards.py:2619] [17/3] [__recompiles] triggered by the following guard failure(s):
V0617 16:33:36.046000 140419876123776 torch/_dynamo/guards.py:2619] [17/3] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 11, actual 12
V0617 16:33:36.046000 140419876123776 torch/_dynamo/guards.py:2619] [17/3] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 10, actual 12
V0617 16:33:36.046000 140419876123776 torch/_dynamo/guards.py:2619] [17/3] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 9, actual 12
V0617 16:33:40.306000 140419876123776 torch/_dynamo/guards.py:2619] [17/4] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:970
V0617 16:33:40.306000 140419876123776 torch/_dynamo/guards.py:2619] [17/4] [__recompiles] triggered by the following guard failure(s):
V0617 16:33:40.306000 140419876123776 torch/_dynamo/guards.py:2619] [17/4] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 12, actual 13
V0617 16:33:40.306000 140419876123776 torch/_dynamo/guards.py:2619] [17/4] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 11, actual 13
V0617 16:33:40.306000 140419876123776 torch/_dynamo/guards.py:2619] [17/4] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 10, actual 13
V0617 16:33:40.306000 140419876123776 torch/_dynamo/guards.py:2619] [17/4] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 9, actual 13
V0617 16:33:42.851000 140419876123776 torch/_dynamo/guards.py:2619] [17/5] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:970
V0617 16:33:42.851000 140419876123776 torch/_dynamo/guards.py:2619] [17/5] [__recompiles] triggered by the following guard failure(s):
V0617 16:33:42.851000 140419876123776 torch/_dynamo/guards.py:2619] [17/5] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 13, actual 14
V0617 16:33:42.851000 140419876123776 torch/_dynamo/guards.py:2619] [17/5] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 12, actual 14
V0617 16:33:42.851000 140419876123776 torch/_dynamo/guards.py:2619] [17/5] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 11, actual 14
V0617 16:33:42.851000 140419876123776 torch/_dynamo/guards.py:2619] [17/5] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 10, actual 14
V0617 16:33:42.851000 140419876123776 torch/_dynamo/guards.py:2619] [17/5] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 9, actual 14
V0617 16:33:48.765000 140419876123776 torch/_dynamo/guards.py:2619] [17/6] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:970
V0617 16:33:48.765000 140419876123776 torch/_dynamo/guards.py:2619] [17/6] [__recompiles] triggered by the following guard failure(s):
V0617 16:33:48.765000 140419876123776 torch/_dynamo/guards.py:2619] [17/6] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 14, actual 15
V0617 16:33:48.765000 140419876123776 torch/_dynamo/guards.py:2619] [17/6] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 13, actual 15
V0617 16:33:48.765000 140419876123776 torch/_dynamo/guards.py:2619] [17/6] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 12, actual 15
V0617 16:33:48.765000 140419876123776 torch/_dynamo/guards.py:2619] [17/6] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 11, actual 15
V0617 16:33:48.765000 140419876123776 torch/_dynamo/guards.py:2619] [17/6] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 10, actual 15
V0617 16:33:48.765000 140419876123776 torch/_dynamo/guards.py:2619] [17/6] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 9, actual 15
V0617 16:33:56.082000 140419876123776 torch/_dynamo/guards.py:2619] [17/7] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:970
V0617 16:33:56.082000 140419876123776 torch/_dynamo/guards.py:2619] [17/7] [__recompiles] triggered by the following guard failure(s):
V0617 16:33:56.082000 140419876123776 torch/_dynamo/guards.py:2619] [17/7] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 15, actual 16
V0617 16:33:56.082000 140419876123776 torch/_dynamo/guards.py:2619] [17/7] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 14, actual 16
V0617 16:33:56.082000 140419876123776 torch/_dynamo/guards.py:2619] [17/7] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 13, actual 16
V0617 16:33:56.082000 140419876123776 torch/_dynamo/guards.py:2619] [17/7] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 12, actual 16
V0617 16:33:56.082000 140419876123776 torch/_dynamo/guards.py:2619] [17/7] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 11, actual 16
V0617 16:33:56.082000 140419876123776 torch/_dynamo/guards.py:2619] [17/7] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 10, actual 16
V0617 16:33:56.082000 140419876123776 torch/_dynamo/guards.py:2619] [17/7] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 9, actual 16
V0617 16:34:01.274000 140419876123776 torch/_dynamo/guards.py:2619] [17/8] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:970
V0617 16:34:01.274000 140419876123776 torch/_dynamo/guards.py:2619] [17/8] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:01.274000 140419876123776 torch/_dynamo/guards.py:2619] [17/8] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 16, actual 17
V0617 16:34:01.274000 140419876123776 torch/_dynamo/guards.py:2619] [17/8] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 15, actual 17
V0617 16:34:01.274000 140419876123776 torch/_dynamo/guards.py:2619] [17/8] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 14, actual 17
V0617 16:34:01.274000 140419876123776 torch/_dynamo/guards.py:2619] [17/8] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 13, actual 17
V0617 16:34:01.274000 140419876123776 torch/_dynamo/guards.py:2619] [17/8] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 12, actual 17
V0617 16:34:01.274000 140419876123776 torch/_dynamo/guards.py:2619] [17/8] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 11, actual 17
V0617 16:34:01.274000 140419876123776 torch/_dynamo/guards.py:2619] [17/8] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 10, actual 17
V0617 16:34:01.274000 140419876123776 torch/_dynamo/guards.py:2619] [17/8] [__recompiles] - tensor 'L['input_ids']' stride mismatch at index 0. expected 9, actual 17
W0617 16:34:01.275000 140419876123776 torch/_dynamo/convert_frame.py:762] [17/8] torch._dynamo hit config.cache_size_limit (8)
W0617 16:34:01.275000 140419876123776 torch/_dynamo/convert_frame.py:762] [17/8] function: 'forward' (/home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:970)
W0617 16:34:01.275000 140419876123776 torch/_dynamo/convert_frame.py:762] [17/8] last reason: tensor 'L['input_ids']' stride mismatch at index 0. expected 9, actual 17
W0617 16:34:01.275000 140419876123776 torch/_dynamo/convert_frame.py:762] [17/8] To log all recompilation reasons, use TORCH_LOGS="recompiles".
W0617 16:34:01.275000 140419876123776 torch/_dynamo/convert_frame.py:762] [17/8] To diagnose recompilation issues, see https://pytorch.org/docs/main/torch.compiler_troubleshooting.html.
V0617 16:34:08.884000 140419876123776 torch/_dynamo/guards.py:2619] [18/1] [__recompiles] Recompiling function get_extended_attention_mask in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/modeling_utils.py:1031
V0617 16:34:08.884000 140419876123776 torch/_dynamo/guards.py:2619] [18/1] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:08.884000 140419876123776 torch/_dynamo/guards.py:2619] [18/1] [__recompiles] - tensor 'L['attention_mask']' stride mismatch at index 0. expected 17, actual 18
V0617 16:34:09.541000 140419876123776 torch/_dynamo/guards.py:2619] [21/6] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:09.541000 140419876123776 torch/_dynamo/guards.py:2619] [21/6] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:09.541000 140419876123776 torch/_dynamo/guards.py:2619] [21/6] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:09.541000 140419876123776 torch/_dynamo/guards.py:2619] [21/6] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:09.541000 140419876123776 torch/_dynamo/guards.py:2619] [21/6] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:09.541000 140419876123776 torch/_dynamo/guards.py:2619] [21/6] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:09.541000 140419876123776 torch/_dynamo/guards.py:2619] [21/6] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:09.541000 140419876123776 torch/_dynamo/guards.py:2619] [21/6] [__recompiles] - tensor 'L['attention_mask']' stride mismatch at index 0. expected 17, actual 18
V0617 16:34:10.736000 140419876123776 torch/_dynamo/guards.py:2619] [21/7] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:10.736000 140419876123776 torch/_dynamo/guards.py:2619] [21/7] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:10.736000 140419876123776 torch/_dynamo/guards.py:2619] [21/7] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:10.736000 140419876123776 torch/_dynamo/guards.py:2619] [21/7] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:10.736000 140419876123776 torch/_dynamo/guards.py:2619] [21/7] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:10.736000 140419876123776 torch/_dynamo/guards.py:2619] [21/7] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:10.736000 140419876123776 torch/_dynamo/guards.py:2619] [21/7] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:10.736000 140419876123776 torch/_dynamo/guards.py:2619] [21/7] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 144
V0617 16:34:10.736000 140419876123776 torch/_dynamo/guards.py:2619] [21/7] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:11.583000 140419876123776 torch/_dynamo/guards.py:2619] [21/8] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:11.583000 140419876123776 torch/_dynamo/guards.py:2619] [21/8] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:11.583000 140419876123776 torch/_dynamo/guards.py:2619] [21/8] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:11.583000 140419876123776 torch/_dynamo/guards.py:2619] [21/8] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:11.583000 140419876123776 torch/_dynamo/guards.py:2619] [21/8] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:11.583000 140419876123776 torch/_dynamo/guards.py:2619] [21/8] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:11.583000 140419876123776 torch/_dynamo/guards.py:2619] [21/8] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:11.583000 140419876123776 torch/_dynamo/guards.py:2619] [21/8] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 144
V0617 16:34:11.583000 140419876123776 torch/_dynamo/guards.py:2619] [21/8] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:11.583000 140419876123776 torch/_dynamo/guards.py:2619] [21/8] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:13.287000 140419876123776 torch/_dynamo/guards.py:2619] [21/9] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:13.287000 140419876123776 torch/_dynamo/guards.py:2619] [21/9] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:13.287000 140419876123776 torch/_dynamo/guards.py:2619] [21/9] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:13.287000 140419876123776 torch/_dynamo/guards.py:2619] [21/9] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:13.287000 140419876123776 torch/_dynamo/guards.py:2619] [21/9] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:13.287000 140419876123776 torch/_dynamo/guards.py:2619] [21/9] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:13.287000 140419876123776 torch/_dynamo/guards.py:2619] [21/9] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:13.287000 140419876123776 torch/_dynamo/guards.py:2619] [21/9] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 144
V0617 16:34:13.287000 140419876123776 torch/_dynamo/guards.py:2619] [21/9] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:13.287000 140419876123776 torch/_dynamo/guards.py:2619] [21/9] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:13.287000 140419876123776 torch/_dynamo/guards.py:2619] [21/9] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:14.169000 140419876123776 torch/_dynamo/guards.py:2619] [21/10] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:14.169000 140419876123776 torch/_dynamo/guards.py:2619] [21/10] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:14.169000 140419876123776 torch/_dynamo/guards.py:2619] [21/10] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:14.169000 140419876123776 torch/_dynamo/guards.py:2619] [21/10] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:14.169000 140419876123776 torch/_dynamo/guards.py:2619] [21/10] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:14.169000 140419876123776 torch/_dynamo/guards.py:2619] [21/10] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:14.169000 140419876123776 torch/_dynamo/guards.py:2619] [21/10] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:14.169000 140419876123776 torch/_dynamo/guards.py:2619] [21/10] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 144
V0617 16:34:14.169000 140419876123776 torch/_dynamo/guards.py:2619] [21/10] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:14.169000 140419876123776 torch/_dynamo/guards.py:2619] [21/10] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:14.169000 140419876123776 torch/_dynamo/guards.py:2619] [21/10] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:14.169000 140419876123776 torch/_dynamo/guards.py:2619] [21/10] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:14.665000 140419876123776 torch/_dynamo/guards.py:2619] [21/11] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:14.665000 140419876123776 torch/_dynamo/guards.py:2619] [21/11] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:14.665000 140419876123776 torch/_dynamo/guards.py:2619] [21/11] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:14.665000 140419876123776 torch/_dynamo/guards.py:2619] [21/11] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:14.665000 140419876123776 torch/_dynamo/guards.py:2619] [21/11] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:14.665000 140419876123776 torch/_dynamo/guards.py:2619] [21/11] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:14.665000 140419876123776 torch/_dynamo/guards.py:2619] [21/11] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:14.665000 140419876123776 torch/_dynamo/guards.py:2619] [21/11] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 144
V0617 16:34:14.665000 140419876123776 torch/_dynamo/guards.py:2619] [21/11] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:14.665000 140419876123776 torch/_dynamo/guards.py:2619] [21/11] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:14.665000 140419876123776 torch/_dynamo/guards.py:2619] [21/11] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:14.665000 140419876123776 torch/_dynamo/guards.py:2619] [21/11] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:14.665000 140419876123776 torch/_dynamo/guards.py:2619] [21/11] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:29.238000 140419876123776 torch/_dynamo/guards.py:2619] [3/1] [__recompiles] Recompiling function generate in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/generation/utils.py:1440
V0617 16:34:29.238000 140419876123776 torch/_dynamo/guards.py:2619] [3/1] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:29.238000 140419876123776 torch/_dynamo/guards.py:2619] [3/1] [__recompiles] - L['generation_config'].max_length == 20
V0617 16:34:47.547000 140419876123776 torch/_dynamo/guards.py:2619] [18/2] [__recompiles] Recompiling function get_extended_attention_mask in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/modeling_utils.py:1031
V0617 16:34:47.547000 140419876123776 torch/_dynamo/guards.py:2619] [18/2] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:47.547000 140419876123776 torch/_dynamo/guards.py:2619] [18/2] [__recompiles] - 2 <= L['attention_mask'].size()[1] # _dynamo/output_graph.py:452 in init_ambient_guards
V0617 16:34:47.547000 140419876123776 torch/_dynamo/guards.py:2619] [18/2] [__recompiles] - tensor 'L['attention_mask']' stride mismatch at index 0. expected 17, actual 1
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] - ___check_type_id(L['past_key_value'], 8815232)
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:48.243000 140419876123776 torch/_dynamo/guards.py:2619] [21/12] [__recompiles] - tensor 'L['attention_mask']' stride mismatch at index 0. expected 17, actual 1
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] - ___check_type_id(L['past_key_value'], 8815232)
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 8
V0617 16:34:49.167000 140419876123776 torch/_dynamo/guards.py:2619] [21/13] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - ___check_type_id(L['past_key_value'], 8815232)
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 8
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:50.019000 140419876123776 torch/_dynamo/guards.py:2619] [21/14] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_type_id(L['past_key_value'], 8815232)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 8
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:50.814000 140419876123776 torch/_dynamo/guards.py:2619] [21/15] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_type_id(L['past_key_value'], 8815232)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 8
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:51.735000 140419876123776 torch/_dynamo/guards.py:2619] [21/16] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_type_id(L['past_key_value'], 8815232)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 8
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:52.870000 140419876123776 torch/_dynamo/guards.py:2619] [21/17] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - tensor 'L['attention_mask']' stride mismatch at index 0. expected 1, actual 2
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - tensor 'L['past_key_value'][0]' stride mismatch at index 2. expected 64, actual 512
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:55.618000 140419876123776 torch/_dynamo/guards.py:2619] [21/18] [__recompiles] - tensor 'L['attention_mask']' stride mismatch at index 0. expected 17, actual 2
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 8, actual 16
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - tensor 'L['past_key_value'][0]' stride mismatch at index 2. expected 64, actual 512
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 16
V0617 16:34:56.731000 140419876123776 torch/_dynamo/guards.py:2619] [21/19] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 8, actual 16
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - tensor 'L['past_key_value'][0]' stride mismatch at index 2. expected 64, actual 512
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 16
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:57.762000 140419876123776 torch/_dynamo/guards.py:2619] [21/20] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] triggered by the following guard failure(s):
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 8, actual 16
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - tensor 'L['past_key_value'][0]' stride mismatch at index 2. expected 64, actual 512
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 16
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:34:58.796000 140419876123776 torch/_dynamo/guards.py:2619] [21/21] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] triggered by the following guard failure(s):
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 8, actual 16
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - tensor 'L['past_key_value'][0]' stride mismatch at index 2. expected 64, actual 512
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459539472)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 16
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:35:00.260000 140419876123776 torch/_dynamo/guards.py:2619] [21/22] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] Recompiling function forward in /home/anijain/local/miniconda3/envs/pytorch2/lib/python3.11/site-packages/transformers/models/t5/modeling_t5.py:655
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] triggered by the following guard failure(s):
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 8, actual 16
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - tensor 'L['past_key_value'][0]' stride mismatch at index 2. expected 64, actual 512
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - tensor 'L['position_bias']' stride mismatch at index 0. expected 136, actual 16
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459527888)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459336016)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415449084368)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459385360)
V0617 16:35:01.271000 140419876123776 torch/_dynamo/guards.py:2619] [21/23] [__recompiles] - ___check_obj_id(L['self'], 140415459508112)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment