Skip to content

Instantly share code, notes, and snippets.

@ezyang

ezyang/final.csv Secret

Created November 12, 2022 22:44
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ezyang/05e1e20dd0d4ff77b537efbb194289b6 to your computer and use it in GitHub Desktop.
Save ezyang/05e1e20dd0d4ff77b537efbb194289b6 to your computer and use it in GitHub Desktop.
9dbb7e488e0a8036e1b90f4711f6738d9593d94b
torchbench BERT_pytorch FAIL torch/_inductor/ir.py TypeError: sub expected 2 arguments, got 0
torchbench Background_Matting FAIL benchmarks/dynamo/common.py NotImplementedError: Eager model failed to run
torchbench LearningToPaint PASS
torchbench Super_SloMo FAIL torch/_inductor/ir.py TypeError: sym_float() missing 1 required positional argument: 'a'
torchbench alexnet FAIL ../../../../tmp/torchinductor_ezyang/ku/ckuz4jrmsdsdkq6aqwcv7l6h2sdtmkrq7fef66thx5m6d3pzhpa2.py NameError: name 's0' is not defined
torchbench attention_is_all_you_need_pytorch FAIL ../../../../tmp/torchinductor_ezyang/gf/cgfclnbsox4ewuabxrlllokirj4b3shtaxrvgfwmvfqkov34ov7w.py NameError: name 's0' is not defined
torchbench dcgan PASS
torchbench densenet121 FAIL benchmarks/dynamo/common.py NotImplementedError: Eager model failed to run
torchbench dlrm FAIL torch/autograd/__init__.py NotImplementedError: Cannot access storage of SparseTensorImpl
torchbench drq PASS
torchbench fastNLP_Bert FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
torchbench functorch_dp_cifar10 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
torchbench functorch_maml_omniglot PASS
torchbench hf_Albert FAIL torch/_inductor/triton_ops/autotune.py TypeError: do_bench() got an unexpected keyword argument 'fast_flush'
torchbench hf_Bart FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
torchbench hf_Bert FAIL torch/_inductor/triton_ops/autotune.py TypeError: do_bench() got an unexpected keyword argument 'fast_flush'
torchbench hf_BigBird FAIL torch/_inductor/triton_ops/autotune.py TypeError: do_bench() got an unexpected keyword argument 'fast_flush'
torchbench hf_DistilBert FAIL torch/_inductor/triton_ops/autotune.py TypeError: do_bench() got an unexpected keyword argument 'fast_flush'
torchbench hf_GPT2 FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
torchbench hf_GPT2_large PASS
torchbench hf_Longformer FAIL functorch/_src/aot_autograd.py AssertionError: aot_autograd expected to have an entirely functional graph, but found %index_put_ : [#users=0] = call_function[target=torch.ops.aten.index_put_.default](args = (%clone_1, [%_unsafe_view], %view_1, True), kwargs = {})
torchbench hf_Reformer FAIL torch/_inductor/overrides.py RuntimeError: Cannot call sizes() on tensor with symbolic sizes/strides
torchbench hf_T5 FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
torchbench hf_T5_base FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
torchbench hf_T5_large PASS
torchbench lennard_jones PASS
torchbench maml_omniglot PASS
torchbench mnasnet1_0 PASS
torchbench mobilenet_v2 PASS
torchbench mobilenet_v2_quantized_qat FAIL functorch/_src/aot_autograd.py AssertionError: aot_autograd expected to have an entirely functional graph, but found %copy_ : [#users=0] = call_function[target=torch.ops.aten.copy_.default](args = (%primals_161, %getitem_4), kwargs = {})
torchbench mobilenet_v3_large PASS
torchbench moco FAIL torch/_subclasses/fake_tensor.py RuntimeError: c10d.allgather_.default - couldn't find symbolic meta function/decomposition
torchbench nvidia_deeprecommender PASS
torchbench pytorch_CycleGAN_and_pix2pix FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
torchbench pytorch_stargan FAIL functorch/_src/aot_autograd.py AssertionError: aot_autograd expected to have an entirely functional graph, but found %copy_ : [#users=0] = call_function[target=torch.ops.aten.copy_.default](args = (%primals_53, %alias_1), kwargs = {})
torchbench pytorch_struct FAIL sympy/core/numbers.py TypeError: Argument of Integer should be of numeric type, got s0.
torchbench pytorch_unet PASS
torchbench resnet18 PASS
torchbench resnet50 PASS
torchbench resnet50_quantized_qat FAIL functorch/_src/aot_autograd.py AssertionError: aot_autograd expected to have an entirely functional graph, but found %copy_ : [#users=0] = call_function[target=torch.ops.aten.copy_.default](args = (%primals_164, %getitem_4), kwargs = {})
torchbench resnext50_32x4d PASS
torchbench shufflenet_v2_x1_0 PASS
torchbench soft_actor_critic FAIL ACCURACY
torchbench speech_transformer FAIL torch/_inductor/triton_ops/autotune.py TypeError: do_bench() got an unexpected keyword argument 'fast_flush'
torchbench squeezenet1_1 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(
torchbench tacotron2 FAIL torch/_inductor/overrides.py RuntimeError: Cannot call sizes() on tensor with symbolic sizes/strides
torchbench timm_efficientdet FAIL torch/nn/functional.py RuntimeError: Cannot call sizes() on tensor with symbolic sizes/strides
torchbench timm_efficientnet FAIL ../../../../tmp/torchinductor_ezyang/y7/cy744iypaq6pia4kjrruby3k4cb3ziexdrlrdviqcbvysrobxwvx.py NameError: name 's0' is not defined
torchbench timm_regnet FAIL ../../../../tmp/torchinductor_ezyang/bk/cbk5re3gdhcty7hgwbnbgsopsnqzecmyeo6fymnzlbwzcdsjc5af.py NameError: name 's0' is not defined
torchbench timm_resnest PASS
torchbench timm_vision_transformer FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
torchbench timm_vision_transformer_large PASS
torchbench timm_vovnet FAIL ../../../../tmp/torchinductor_ezyang/uv/cuvtybarkzcwdhsxc5whteiph4hj2qqfjkswssifcpjubzhdwxh6.py NameError: name 's0' is not defined
torchbench tts_angular FAIL torch/nn/modules/rnn.py RuntimeError: !has_symbolic_sizes_strides_ INTERNAL ASSERT FAILED at "/scratch/ezyang/work/pytorch/aten/src/ATen/TensorGeometry.h":63, please report a bug to PyTorch.
torchbench vgg16 FAIL ../../../../tmp/torchinductor_ezyang/76/c76wejbtwpulr4tdpbwgmnso6qz7akcxbjavgdqqndu4trq7vuws.py NameError: name 's0' is not defined
torchbench vision_maskrcnn FAIL torch/_inductor/ir.py TypeError: sym_float() missing 1 required positional argument: 'a'
torchbench yolov3 FAIL torch/nn/functional.py RuntimeError: Cannot call sizes() on tensor with symbolic sizes/strides
huggingface AlbertForMaskedLM FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
huggingface AlbertForQuestionAnswering PASS
huggingface AllenaiLongformerBase FAIL ../../../../tmp/torchinductor_ezyang/ku/ckukwz2mqyj7zmejyspjuxivzdxxkk63ssh6ltdknjvxlqg776jg.py NameError: name 's0' is not defined
huggingface BartForCausalLM FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface BartForConditionalGeneration FAIL sympy/core/numbers.py TypeError: Argument of Integer should be of numeric type, got s0.
huggingface BertForMaskedLM FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
huggingface BertForQuestionAnswering PASS
huggingface BigBird FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
huggingface BlenderbotSmallForCausalLM FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface BlenderbotSmallForConditionalGeneration FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface CamemBert FAIL ../../../../tmp/torchinductor_ezyang/2p/c2pey6u5sll3iv7yltwsy7vs4imig2ccoaixo77jgfstpnor67xi.py NameError: name 's0' is not defined
huggingface DebertaForMaskedLM FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
huggingface DebertaForQuestionAnswering FAIL torch/_inductor/lowering.py AssertionError: <class 'sympy.core.numbers.Integer'>
huggingface DistilBertForMaskedLM FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
huggingface DistilBertForQuestionAnswering PASS
huggingface DistillGPT2 FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface ElectraForCausalLM FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
huggingface ElectraForQuestionAnswering PASS
huggingface GPT2ForSequenceClassification FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface GoogleFnet FAIL torch/_subclasses/fake_tensor.py RuntimeError: aten.view_as_real.default - couldn't find symbolic meta function/decomposition
huggingface LayoutLMForMaskedLM FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
huggingface LayoutLMForSequenceClassification FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
huggingface M2M100ForConditionalGeneration FAIL torch/_inductor/lowering.py AssertionError: <class 'sympy.core.numbers.Integer'>
huggingface MBartForCausalLM FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface MBartForConditionalGeneration FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface MT5ForConditionalGeneration FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface MegatronBertForCausalLM FAIL torch/_inductor/lowering.py AssertionError: <class 'sympy.core.numbers.Integer'>
huggingface MegatronBertForQuestionAnswering FAIL torch/_inductor/lowering.py AssertionError: <class 'sympy.core.numbers.Integer'>
huggingface MobileBertForMaskedLM FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
huggingface MobileBertForQuestionAnswering PASS
huggingface OPTForCausalLM FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface PLBartForCausalLM FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface PLBartForConditionalGeneration FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface PegasusForCausalLM FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface PegasusForConditionalGeneration FAIL torch/_inductor/lowering.py AssertionError: <class 'sympy.core.numbers.Integer'>
huggingface RobertaForCausalLM FAIL ../../../../tmp/torchinductor_ezyang/x4/cx46zeqrb3k5e3gvg5dvqbbehylu2ij5jvcccg6snpbo4mmc26ja.py NameError: name 's0' is not defined
huggingface RobertaForQuestionAnswering FAIL ../../../../tmp/torchinductor_ezyang/e3/ce3tl4i2nggrjuchlgel6zeumzwuxk6s2aum2gxawmctgkuihnul.py NameError: name 's0' is not defined
huggingface Speech2Text2ForCausalLM FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface T5ForConditionalGeneration FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface T5Small FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface TrOCRForCausalLM FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface XGLMForCausalLM FAIL torch/_ops.py RuntimeError: Overloaded torch operator invoked from Python failed to many any schema:
huggingface XLNetLMHeadModel FAIL functorch/_src/aot_autograd.py AssertionError: aot_autograd expected to have an entirely functional graph, but found %index_put_ : [#users=0] = call_function[target=torch.ops.aten.index_put_.default](args = (%clone_48, [None, None, None, %arange_23], %mul_465, True), kwargs = {})
huggingface YituTechConvBert FAIL torch/_inductor/ir.py TypeError: sub expected 2 arguments, got 0
timm_models adv_inception_v3 FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models beit_base_patch16_224 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models botnet26t_256 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models cait_m36_384 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models coat_lite_mini FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models convit_base FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models convmixer_768_32 FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models convnext_base FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models crossvit_9_240 FAIL torch/nn/functional.py RuntimeError: Cannot call sizes() on tensor with symbolic sizes/strides
timm_models cspdarknet53 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models deit_base_distilled_patch16_224 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models dla102 FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models dm_nfnet_f0 FAIL torch/_inductor/ir.py TypeError: truediv expected 2 arguments, got 0
timm_models dpn107 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(
timm_models eca_botnext26ts_256 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models eca_halonext26ts FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models ese_vovnet19b_dw FAIL ../../../../tmp/torchinductor_ezyang/cl/ccl62ryau2zr2pz57teorbbu7veagdzxnguzh5kmb6awr5lhj2wy.py NameError: name 's0' is not defined
timm_models fbnetc_100 FAIL ../../../../tmp/torchinductor_ezyang/kh/ckhoymlporstyrbsfuflq66m3mojdpog6r33ufjmkhmdjgwphpqx.py NameError: name 's0' is not defined
timm_models fbnetv3_b FAIL ../../../../tmp/torchinductor_ezyang/vd/cvdnk5zlp7vgcocwryus3u2enx24fxx54cm2ax6szqfzh723h57x.py NameError: name 's0' is not defined
timm_models gernet_l FAIL ../../../../tmp/torchinductor_ezyang/wz/cwzpjwhuchb3fwpkhc7uycoe5ji4coku3v5pwbjhhncu7vm4x6xy.py NameError: name 's0' is not defined
timm_models ghostnet_100 FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models gluon_inception_v3 FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models gluon_xception65 FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models gmixer_24_224 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models gmlp_s16_224 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models hrnet_w18 FAIL torch/nn/functional.py RuntimeError: Cannot call sizes() on tensor with symbolic sizes/strides
timm_models inception_v3 FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models jx_nest_base FAIL torch/_inductor/ir.py TypeError: sym_sqrt() missing 1 required positional argument: 'a'
timm_models lcnet_050 FAIL ../../../../tmp/torchinductor_ezyang/wd/cwdp5i2ylagsa6eysvya367lxp3t3t6sdp3ighxuunutaa2m54dl.py NameError: name 's0' is not defined
timm_models levit_128 FAIL torch/autograd/__init__.py RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.
timm_models mixer_b16_224 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models mixnet_l FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models mnasnet_100 FAIL ../../../../tmp/torchinductor_ezyang/fy/cfyunmd2yikti7momtzx6taipkyn43anv6aqcuw33glheqsqxj5c.py NameError: name 's0' is not defined
timm_models mobilenetv2_100 FAIL ../../../../tmp/torchinductor_ezyang/u6/cu66p2ccxjgbd7hwb4e2anqtqpdpthntq3bkx57wktrwykhni6y7.py NameError: name 's0' is not defined
timm_models mobilenetv3_large_100 FAIL ../../../../tmp/torchinductor_ezyang/jg/cjgjinbjzzdwaxm5awcn5viuzlj7jwt4lcvyurus2w7yramr3te2.py NameError: name 's0' is not defined
timm_models mobilevit_s FAIL torch/_inductor/ir.py TypeError: truediv expected 2 arguments, got 0
timm_models nfnet_l0 FAIL ../../../../tmp/torchinductor_ezyang/tg/ctgrrctxtvnuyc352qnbihegso6bdqth4bhibpsyf5u2a3tm5ahs.py NameError: name 's0' is not defined
timm_models pit_b_224 FAIL torch/_inductor/lowering.py AssertionError: <class 'sympy.core.numbers.Integer'>
timm_models pnasnet5large FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models poolformer_m36 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models regnety_002 FAIL ../../../../tmp/torchinductor_ezyang/f2/cf26tkgyz3kvu4yfgi5x4hba3c3s3r6mo67toawfurryvwxnqssp.py NameError: name 's0' is not defined
timm_models repvgg_a2 FAIL ../../../../tmp/torchinductor_ezyang/tg/ctgfru36bxjr6suj7pkxn635iqrcxh4b7amlia7hnnxsyz2zgsqz.py NameError: name 's0' is not defined
timm_models res2net101_26w_4s FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models res2net50_14w_8s FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models res2next50 FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models resmlp_12_224 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models resnest101e FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models rexnet_100 FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models sebotnet33ts_256 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models selecsls42b FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models spnasnet_100 FAIL ../../../../tmp/torchinductor_ezyang/c2/cc2rj47sq3umyuqdteafe35zyunr7hovyxey3w3pn6th2ifcefpa.py NameError: name 's0' is not defined
timm_models swin_base_patch4_window7_224 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models swsl_resnext101_32x16d FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models tf_efficientnet_b0 FAIL torch/_inductor/ir.py TypeError: truediv expected 2 arguments, got 0
timm_models tf_mixnet_l FAIL torch/_inductor/ir.py TypeError: truediv expected 2 arguments, got 0
timm_models tinynet_a FAIL ../../../../tmp/torchinductor_ezyang/k2/ck2vlgdeeskh6ldjyryqdhi36zu464ux2jsnbu3q5db2fpvryxqh.py NameError: name 's0' is not defined
timm_models tnt_s_patch16_224 FAIL torch/_inductor/ir.py TypeError: sub expected 2 arguments, got 0
timm_models twins_pcpvt_base FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models visformer_small FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models vit_base_patch16_224 FAIL torch/_inductor/graph.py AssertionError: [TensorBox(StorageBox(
timm_models volo_d1_224 FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
timm_models xcit_large_24_p8_224 FAIL torch/_inductor/graph.py AttributeError: 'SymFloat' object has no attribute 'size'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment