Skip to content

Instantly share code, notes, and snippets.

@msaroufim

msaroufim/gist:2e764de83ab98f337e8e0d093c08f705

Created August 14, 2024 16:25
Show Gist options
  • Save msaroufim/2e764de83ab98f337e8e0d093c08f705 to your computer and use it in GitHub Desktop.
Save msaroufim/2e764de83ab98f337e8e0d093c08f705 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
run_vit_b.py run_vit_b_quant.py
(ao) [marksaroufim@devvm4567.ash0 ~/ao/tutorials/quantize_vit (main)]$ python run_vit_b_quant.py
Downloading: "https://download.pytorch.org/models/vit_b_16-c867db91.pth" to /home/marksaroufim/.cache/torch/hub/checkpoints/vit_b_16-c867db91.pth
100%|█████████████████████████████████████████████████████████████████████████████████| 330M/330M [00:01<00:00, 209MB/s]
AUTOTUNE convolution(1x3x224x224, 768x3x16x16)
triton_convolution_4 0.1184 ms 100.0%
convolution 0.1450 ms 81.7%
triton_convolution_3 0.2024 ms 58.5%
triton_convolution_5 0.2268 ms 52.2%
triton_convolution_6 0.2445 ms 48.4%
triton_convolution_2 0.2745 ms 43.1%
triton_convolution_0 0.4813 ms 24.6%
triton_convolution_1 0.6060 ms 19.5%
SingleProcess AUTOTUNE benchmarking takes 1.3321 seconds and 0.3501 seconds precompiling
/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py:663: UserWarning: Graph break due to unsupported builtin None.TensorBase._make_wrapper_subclass. This function is either a Python builtin (e.g. _warnings.warn) or a third-party C/C++ Python extension (perhaps created with pybind). If it is a Python builtin, please file an issue on GitHub so the PyTorch team can add support for it and see the next case for a workaround. If it is a third-party C/C++ Python extension, please either wrap it into a PyTorch-understood custom operator (see https://pytorch.org/tutorials/advanced/custom_ops_landing_page.html for more details) or, if it is traceable, use torch.compiler.allow_in_graph.
torch._dynamo.utils.warn_once(msg)
Traceback (most recent call last):
File "/home/marksaroufim/ao/tutorials/quantize_vit/run_vit_b_quant.py", line 44, in <module>
benchmark_model(model, 20, inputs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchao/utils.py", line 64, in benchmark_model
model(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 433, in _fn
return fn(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchvision/models/vision_transformer.py", line 298, in forward
x = self.encoder(x)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchvision/models/vision_transformer.py", line 157, in forward
return self.ln(self.layers(self.dropout(input)))
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/container.py", line 219, in forward
input = module(input)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchvision/models/vision_transformer.py", line 113, in forward
x, _ = self.self_attention(x, x, x, need_weights=False)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/activation.py", line 1211, in forward
self.out_proj.weight,
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/utils/parametrize.py", line 379, in get_parametrized
return parametrization()
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/utils/parametrize.py", line 279, in forward
x = self[0](*originals)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchao/utils.py", line 229, in forward
rebuilt = tp.__tensor_unflatten__(inner_tensors, meta, None, None)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchao/dtypes/affine_quantized_tensor.py", line 398, in __tensor_unflatten__
return cls(int_data, scale, zero_point, layout_type)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1116, in __call__
return self._torchdynamo_orig_callable(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 948, in __call__
result = self._inner_convert(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 472, in __call__
return _compile(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_utils_internal.py", line 84, in wrapper_function
return StrobelightCompileTimeProfiler.profile_compile_time(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_strobelight/compile_time_profiler.py", line 129, in profile_compile_time
return func(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 846, in _compile
raise InternalTorchDynamoError(str(e)).with_traceback(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 817, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
r = func(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 636, in compile_inner
out_code = transform_code_object(code, transform)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1185, in transform_code_object
transformations(instructions, code_options)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 178, in _fn
return fn(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 582, in transform
tracer.run()
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2451, in run
super().run()
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 893, in run
while self.step():
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 805, in step
self.dispatch_table[inst.opcode](self, inst)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1569, in STORE_ATTR
if isinstance(obj, NNModuleVariable) and not isinstance(val, ConstantVariable):
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/base.py", line 106, in __instancecheck__
instance = instance.realize()
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/lazy.py", line 58, in realize
self._cache.realize()
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/lazy.py", line 24, in realize
self.vt = VariableBuilder(tx, self.source)(self.value)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 314, in __call__
vt = self._wrap(value)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 471, in _wrap
return self.wrap_tensor(value)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1268, in wrap_tensor
self.assert_not_wrapped_by_this_graph(value)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1199, in assert_not_wrapped_by_this_graph
if is_fake(value) and maybe_get_fake_mode(value) is self.tx.fake_mode:
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 155, in is_fake
attrs, _ = type(x).__tensor_flatten__(x)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchao/dtypes/affine_quantized_tensor.py", line 390, in __tensor_flatten__
return ["int_data", "scale", "zero_point"], [self.layout_type]
torch._dynamo.exc.InternalTorchDynamoError: 'PlainAQTLayout' object has no attribute 'layout_type'
from user code:
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchao/dtypes/affine_quantized_tensor.py", line 384, in __init__
self.int_data = int_data
Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
You can suppress this exception and fall back to eager by setting:
import torch._dynamo
torch._dynamo.config.suppress_errors = True
(ao) [marksaroufim@devvm4567.ash0 ~/ao/tutorials/quantize_vit (main)]$ ls
bfloat16_code.py bfloat16.json.gz quant_code.py quant.json.gz run.sh run_vit_b.py run_vit_b_quant.py
(ao) [marksaroufim@devvm4567.ash0 ~/ao/tutorials/quantize_vit (main)]$ pip list
Package Version Editable project location
------------------------ ---------------- -------------------------------------
aiohappyeyeballs 2.3.5
aiohttp 3.10.2
aiosignal 1.3.1
antlr4-python3-runtime 4.9.3
async-timeout 4.0.3
attrs 24.2.0
bitsandbytes 0.43.3
blobfile 2.1.1
certifi 2024.7.4
charset-normalizer 3.3.2
contourpy 1.2.1
cycler 0.12.1
datasets 2.20.0
dill 0.3.8
exceptiongroup 1.2.2
expecttest 0.2.1
filelock 3.15.4
fire 0.6.0
fonttools 4.53.1
frozenlist 1.4.1
fsspec 2024.5.0
huggingface-hub 0.24.5
hypothesis 6.110.1
idna 3.7
iniconfig 2.0.0
Jinja2 3.1.4
kiwisolver 1.4.5
lxml 4.9.4
MarkupSafe 2.1.5
matplotlib 3.9.1.post1
mpmath 1.3.0
multidict 6.0.5
multiprocess 0.70.16
networkx 3.3
ninja 1.11.1.1
numpy 1.26.4
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 9.1.0.70
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.20.5
nvidia-nvjitlink-cu12 12.6.20
nvidia-nvtx-cu12 12.1.105
omegaconf 2.3.0
packaging 24.1
pandas 2.2.2
parameterized 0.9.0
pillow 10.4.0
pip 24.0
pluggy 1.5.0
pyarrow 17.0.0
pyarrow-hotfix 0.6
pycryptodomex 3.20.0
pyparsing 3.1.2
pytest 7.4.0
python-dateutil 2.9.0.post0
pytz 2024.1
PyYAML 6.0.2
regex 2024.7.24
requests 2.32.3
safetensors 0.4.4
sentencepiece 0.2.0
setuptools 72.1.0
six 1.16.0
sortedcontainers 2.4.0
sympy 1.13.1
tabulate 0.9.0
termcolor 2.4.0
tiktoken 0.7.0
tokenizers 0.19.1
tomli 2.0.1
torch 2.4.0
torchao 0.4.0+git174e630
torchtune 0.0.0 /home/marksaroufim/test/ebs-torchtune
torchvision 0.19.0
tqdm 4.66.5
transformers 4.44.0
triton 3.0.0
typing_extensions 4.12.2
tzdata 2024.1
unittest-xml-reporting 3.2.0
urllib3 2.2.2
wheel 0.43.0
xxhash 3.4.1
yarl 1.9.4
(ao) [marksaroufim@devvm4567.ash0 ~/ao/tutorials/quantize_vit (main)]$ pip3 install --pre torch --index-url https://down
load.pytorch.org/whl/nightly/cu121 --force-reinstall
Looking in indexes: https://download.pytorch.org/whl/nightly/cu121
Collecting torch
Downloading https://download.pytorch.org/whl/nightly/cu121/torch-2.5.0.dev20240814%2Bcu121-cp310-cp310-linux_x86_64.whl (778.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 778.6/778.6 MB 7.3 MB/s eta 0:00:00
Collecting filelock (from torch)
Downloading https://download.pytorch.org/whl/nightly/filelock-3.13.1-py3-none-any.whl (11 kB)
Collecting typing-extensions>=4.8.0 (from torch)
Downloading https://download.pytorch.org/whl/nightly/typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Collecting networkx (from torch)
Downloading https://download.pytorch.org/whl/nightly/networkx-3.3-py3-none-any.whl (1.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 104.7 MB/s eta 0:00:00
Collecting jinja2 (from torch)
Downloading https://download.pytorch.org/whl/nightly/Jinja2-3.1.4-py3-none-any.whl (133 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.3/133.3 kB 24.1 MB/s eta 0:00:00
Collecting fsspec (from torch)
Downloading https://download.pytorch.org/whl/nightly/fsspec-2024.6.1-py3-none-any.whl (177 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 177.6/177.6 kB 32.4 MB/s eta 0:00:00
Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch)
Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.7/23.7 MB 103.6 MB/s eta 0:00:00
Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch)
Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 823.6/823.6 kB 84.0 MB/s eta 0:00:00
Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch)
Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.1/14.1 MB 111.2 MB/s eta 0:00:00
Collecting nvidia-cudnn-cu12==9.1.0.70 (from torch)
Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl (664.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 664.8/664.8 MB 8.2 MB/s eta 0:00:00
Collecting nvidia-cublas-cu12==12.1.3.1 (from torch)
Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 410.6/410.6 MB 11.3 MB/s eta 0:00:00
Collecting nvidia-cufft-cu12==11.0.2.54 (from torch)
Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.6/121.6 MB 15.4 MB/s eta 0:00:00
Collecting nvidia-curand-cu12==10.3.2.106 (from torch)
Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.5/56.5 MB 62.5 MB/s eta 0:00:00
Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch)
Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 MB 9.5 MB/s eta 0:00:00
Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch)
Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.0/196.0 MB 11.4 MB/s eta 0:00:00
Collecting nvidia-nccl-cu12==2.21.5 (from torch)
Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_nccl_cu12-2.21.5-py3-none-manylinux2014_x86_64.whl (188.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 188.7/188.7 MB 26.6 MB/s eta 0:00:00
Collecting nvidia-nvtx-cu12==12.1.105 (from torch)
Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 99.1/99.1 kB 18.1 MB/s eta 0:00:00
Collecting pytorch-triton==3.0.0+dedb7bdf33 (from torch)
Downloading https://download.pytorch.org/whl/nightly/pytorch_triton-3.0.0%2Bdedb7bdf33-cp310-cp310-linux_x86_64.whl (239.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 239.4/239.4 MB 22.3 MB/s eta 0:00:00
Collecting sympy==1.13.1 (from torch)
Downloading https://download.pytorch.org/whl/nightly/sympy-1.13.1-py3-none-any.whl (6.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.2/6.2 MB 119.5 MB/s eta 0:00:00
Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch)
Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_nvjitlink_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (19.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.8/19.8 MB 92.7 MB/s eta 0:00:00
Collecting mpmath<1.4,>=1.1.0 (from sympy==1.13.1->torch)
Downloading https://download.pytorch.org/whl/nightly/mpmath-1.3.0-py3-none-any.whl (536 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 53.9 MB/s eta 0:00:00
Collecting MarkupSafe>=2.0 (from jinja2->torch)
Downloading https://download.pytorch.org/whl/nightly/MarkupSafe-2.1.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Installing collected packages: mpmath, typing-extensions, sympy, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, networkx, MarkupSafe, fsspec, filelock, pytorch-triton, nvidia-cusparse-cu12, nvidia-cudnn-cu12, jinja2, nvidia-cusolver-cu12, torch
Attempting uninstall: mpmath
Found existing installation: mpmath 1.3.0
Uninstalling mpmath-1.3.0:
Successfully uninstalled mpmath-1.3.0
Attempting uninstall: typing-extensions
Found existing installation: typing_extensions 4.12.2
Uninstalling typing_extensions-4.12.2:
Successfully uninstalled typing_extensions-4.12.2
Attempting uninstall: sympy
Found existing installation: sympy 1.13.1
Uninstalling sympy-1.13.1:
Successfully uninstalled sympy-1.13.1
Attempting uninstall: nvidia-nvtx-cu12
Found existing installation: nvidia-nvtx-cu12 12.1.105
Uninstalling nvidia-nvtx-cu12-12.1.105:
Successfully uninstalled nvidia-nvtx-cu12-12.1.105
Attempting uninstall: nvidia-nvjitlink-cu12
Found existing installation: nvidia-nvjitlink-cu12 12.6.20
Uninstalling nvidia-nvjitlink-cu12-12.6.20:
Successfully uninstalled nvidia-nvjitlink-cu12-12.6.20
Attempting uninstall: nvidia-nccl-cu12
Found existing installation: nvidia-nccl-cu12 2.20.5
Uninstalling nvidia-nccl-cu12-2.20.5:
Successfully uninstalled nvidia-nccl-cu12-2.20.5
Attempting uninstall: nvidia-curand-cu12
Found existing installation: nvidia-curand-cu12 10.3.2.106
Uninstalling nvidia-curand-cu12-10.3.2.106:
Successfully uninstalled nvidia-curand-cu12-10.3.2.106
Attempting uninstall: nvidia-cufft-cu12
Found existing installation: nvidia-cufft-cu12 11.0.2.54
Uninstalling nvidia-cufft-cu12-11.0.2.54:
Successfully uninstalled nvidia-cufft-cu12-11.0.2.54
Attempting uninstall: nvidia-cuda-runtime-cu12
Found existing installation: nvidia-cuda-runtime-cu12 12.1.105
Uninstalling nvidia-cuda-runtime-cu12-12.1.105:
Successfully uninstalled nvidia-cuda-runtime-cu12-12.1.105
Attempting uninstall: nvidia-cuda-nvrtc-cu12
Found existing installation: nvidia-cuda-nvrtc-cu12 12.1.105
Uninstalling nvidia-cuda-nvrtc-cu12-12.1.105:
Successfully uninstalled nvidia-cuda-nvrtc-cu12-12.1.105
Attempting uninstall: nvidia-cuda-cupti-cu12
Found existing installation: nvidia-cuda-cupti-cu12 12.1.105
Uninstalling nvidia-cuda-cupti-cu12-12.1.105:
Successfully uninstalled nvidia-cuda-cupti-cu12-12.1.105
Attempting uninstall: nvidia-cublas-cu12
Found existing installation: nvidia-cublas-cu12 12.1.3.1
Uninstalling nvidia-cublas-cu12-12.1.3.1:
Successfully uninstalled nvidia-cublas-cu12-12.1.3.1
Attempting uninstall: networkx
Found existing installation: networkx 3.3
Uninstalling networkx-3.3:
Successfully uninstalled networkx-3.3
Attempting uninstall: MarkupSafe
Found existing installation: MarkupSafe 2.1.5
Uninstalling MarkupSafe-2.1.5:
Successfully uninstalled MarkupSafe-2.1.5
Attempting uninstall: fsspec
Found existing installation: fsspec 2024.5.0
Uninstalling fsspec-2024.5.0:
Successfully uninstalled fsspec-2024.5.0
Attempting uninstall: filelock
Found existing installation: filelock 3.15.4
Uninstalling filelock-3.15.4:
Successfully uninstalled filelock-3.15.4
Attempting uninstall: nvidia-cusparse-cu12
Found existing installation: nvidia-cusparse-cu12 12.1.0.106
Uninstalling nvidia-cusparse-cu12-12.1.0.106:
Successfully uninstalled nvidia-cusparse-cu12-12.1.0.106
Attempting uninstall: nvidia-cudnn-cu12
Found existing installation: nvidia-cudnn-cu12 9.1.0.70
Uninstalling nvidia-cudnn-cu12-9.1.0.70:
Successfully uninstalled nvidia-cudnn-cu12-9.1.0.70
Attempting uninstall: jinja2
Found existing installation: Jinja2 3.1.4
Uninstalling Jinja2-3.1.4:
Successfully uninstalled Jinja2-3.1.4
Attempting uninstall: nvidia-cusolver-cu12
Found existing installation: nvidia-cusolver-cu12 11.4.5.107
Uninstalling nvidia-cusolver-cu12-11.4.5.107:
Successfully uninstalled nvidia-cusolver-cu12-11.4.5.107
Attempting uninstall: torch
Found existing installation: torch 2.4.0
Uninstalling torch-2.4.0:
Successfully uninstalled torch-2.4.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
datasets 2.20.0 requires fsspec[http]<=2024.5.0,>=2023.1.0, but you have fsspec 2024.6.1 which is incompatible.
torchtune 0.0.0 requires torchao==0.3.1, but you have torchao 0.4.0+git174e630 which is incompatible.
torchvision 0.19.0 requires torch==2.4.0, but you have torch 2.5.0.dev20240814+cu121 which is incompatible.
Successfully installed MarkupSafe-2.1.5 filelock-3.13.1 fsspec-2024.6.1 jinja2-3.1.4 mpmath-1.3.0 networkx-3.3 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-9.1.0.70 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.21.5 nvidia-nvjitlink-cu12-12.1.105 nvidia-nvtx-cu12-12.1.105 pytorch-triton-3.0.0+dedb7bdf33 sympy-1.13.1 torch-2.5.0.dev20240814+cu121 typing-extensions-4.12.2
(ao) [marksaroufim@devvm4567.ash0 ~/ao/tutorials/quantize_vit (main)]$ python run_vit_b_quant.py
AUTOTUNE convolution(1x3x224x224, 768x3x16x16)
triton_convolution2d_4 0.1188 ms 100.0% ALLOW_TF32=True, BLOCK_K=16, BLOCK_M=64, BLOCK_N=64, GROUPS=1, KERNEL_H=16, KERNEL_W=16, PADDING_H=0, PADDING_W=0, STRIDE_H=16, STRIDE_W=16, UNROLL=False, num_stages=2, num_warps=4
convolution 0.1450 ms 81.9%
triton_convolution2d_3 0.2023 ms 58.7% ALLOW_TF32=True, BLOCK_K=16, BLOCK_M=128, BLOCK_N=128, GROUPS=1, KERNEL_H=16, KERNEL_W=16, PADDING_H=0, PADDING_W=0, STRIDE_H=16, STRIDE_W=16, UNROLL=False, num_stages=2, num_warps=8
triton_convolution2d_5 0.2263 ms 52.5% ALLOW_TF32=True, BLOCK_K=16, BLOCK_M=64, BLOCK_N=256, GROUPS=1, KERNEL_H=16, KERNEL_W=16, PADDING_H=0, PADDING_W=0, STRIDE_H=16, STRIDE_W=16, UNROLL=False, num_stages=2, num_warps=8
triton_convolution2d_6 0.2441 ms 48.7% ALLOW_TF32=True, BLOCK_K=16, BLOCK_M=256, BLOCK_N=64, GROUPS=1, KERNEL_H=16, KERNEL_W=16, PADDING_H=0, PADDING_W=0, STRIDE_H=16, STRIDE_W=16, UNROLL=False, num_stages=2, num_warps=8
triton_convolution2d_2 0.2752 ms 43.2% ALLOW_TF32=True, BLOCK_K=16, BLOCK_M=1024, BLOCK_N=16, GROUPS=1, KERNEL_H=16, KERNEL_W=16, PADDING_H=0, PADDING_W=0, STRIDE_H=16, STRIDE_W=16, UNROLL=False, num_stages=1, num_warps=8
triton_convolution2d_0 0.4095 ms 29.0% ALLOW_TF32=True, BLOCK_K=16, BLOCK_M=64, BLOCK_N=256, GROUPS=1, KERNEL_H=16, KERNEL_W=16, PADDING_H=0, PADDING_W=0, STRIDE_H=16, STRIDE_W=16, UNROLL=False, num_stages=2, num_warps=4
triton_convolution2d_1 0.6156 ms 19.3% ALLOW_TF32=True, BLOCK_K=16, BLOCK_M=256, BLOCK_N=64, GROUPS=1, KERNEL_H=16, KERNEL_W=16, PADDING_H=0, PADDING_W=0, STRIDE_H=16, STRIDE_W=16, UNROLL=False, num_stages=2, num_warps=4
SingleProcess AUTOTUNE benchmarking takes 1.2322 seconds and 0.1219 seconds precompiling
Traceback (most recent call last):
File "/home/marksaroufim/ao/tutorials/quantize_vit/run_vit_b_quant.py", line 44, in <module>
benchmark_model(model, 20, inputs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchao/utils.py", line 64, in benchmark_model
model(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 465, in _fn
return fn(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchvision/models/vision_transformer.py", line 298, in forward
x = self.encoder(x)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchvision/models/vision_transformer.py", line 157, in forward
return self.ln(self.layers(self.dropout(input)))
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/container.py", line 250, in forward
input = module(input)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchvision/models/vision_transformer.py", line 113, in forward
x, _ = self.self_attention(x, x, x, need_weights=False)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/modules/activation.py", line 1368, in forward
attn_output, attn_output_weights = F.multi_head_attention_forward(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/functional.py", line 5984, in multi_head_attention_forward
return handle_torch_function(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/overrides.py", line 1736, in handle_torch_function
result = torch_func_method(public_api, types, args, kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchao/dtypes/utils.py", line 57, in _dispatch__torch_function__
return func(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/nn/functional.py", line 6285, in multi_head_attention_forward
attn_output = linear(attn_output, out_proj_weight, out_proj_bias)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__
return self._torchdynamo_orig_callable(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1039, in __call__
result = self._inner_convert(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__
return _compile(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 929, in _compile
raise InternalTorchDynamoError(str(e)).with_traceback(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner
return _compile_inner(code, one_graph, hooks, transform)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_utils_internal.py", line 85, in wrapper_function
return StrobelightCompileTimeProfiler.profile_compile_time(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_strobelight/compile_time_profiler.py", line 129, in profile_compile_time
return func(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner
out_code = transform_code_object(code, transform)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object
transformations(instructions, code_options)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn
return fn(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform
tracer.run()
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2731, in run
super().run()
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 958, in run
while self.step():
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 870, in step
self.dispatch_table[inst.opcode](self, inst)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 558, in wrapper
return inner_fn(self, inst)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1565, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 805, in call_function
self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/lazy.py", line 156, in realize_and_forward
return getattr(self.realize(), name)(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function
return super().call_function(tx, args, kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function
return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 811, in inline_user_function_return
return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2946, in inline_call
return cls.inline_call_(parent, func, args, kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3062, in inline_call_
tracer.run()
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 958, in run
while self.step():
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 870, in step
self.dispatch_table[inst.opcode](self, inst)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 558, in wrapper
return inner_fn(self, inst)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1565, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 805, in call_function
self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function
return super().call_function(tx, args, kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function
return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 811, in inline_user_function_return
return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2946, in inline_call
return cls.inline_call_(parent, func, args, kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3062, in inline_call_
tracer.run()
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 958, in run
while self.step():
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 870, in step
self.dispatch_table[inst.opcode](self, inst)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 558, in wrapper
return inner_fn(self, inst)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1565, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 805, in call_function
self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/misc.py", line 954, in call_function
return self.obj.call_method(tx, self.name, args, kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/variables/tensor.py", line 537, in call_method
tx.output.create_proxy(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 543, in create_proxy
return self.current_tracer.create_proxy(*args, **kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1916, in create_proxy
rv = super().create_proxy(
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/fx/proxy.py", line 205, in create_proxy
args_ = self.create_arg(args)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 425, in create_arg
return super().create_arg(a)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/fx/proxy.py", line 272, in create_arg
return type(a)(self.create_arg(elem) for elem in a)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/fx/proxy.py", line 272, in <genexpr>
return type(a)(self.create_arg(elem) for elem in a)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 425, in create_arg
return super().create_arg(a)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torch/fx/proxy.py", line 308, in create_arg
raise NotImplementedError(f"argument of type: {type(a)}")
torch._dynamo.exc.InternalTorchDynamoError: argument of type: <class 'builtin_function_or_method'>
from user code:
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchao/dtypes/utils.py", line 69, in _dispatch__torch_dispatch__
return cls._ATEN_OP_OR_TORCH_FN_TABLE[func](func, types, args, kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchao/dtypes/utils.py", line 37, in wrapper
return func(f, types, args, kwargs)
File "/home/marksaroufim/.conda/envs/ao/lib/python3.10/site-packages/torchao/quantization/linear_activation_quantized_tensor.py", line 175, in _
func, args, kwargs, args[0]._apply_fn_to_data(torch.t)
Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
You can suppress this exception and fall back to eager by setting:
import torch._dynamo
torch._dynamo.config.suppress_errors = True
(ao) [marksaroufim@devvm4567.ash0 ~/ao/tutorials/quantize_vit (main)]$
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment