Benchmarking done using ghcr.io/pytorch/tensorrt/torch_tensorrt:release_2.1
and the following dependencies:
- accelerate
- transformers==4.33.2
- diffusers==0.21.4
Timing:
With compilation: False, and TensorRT: False in 3.767 seconds
With compilation: True, and TensorRT: False in 3.045 seconds
With compilation: True, and TensorRT: True in 1.157 seconds
Timing for SDXL:
With compilation: False, and TensorRT: False in 6.713 seconds
With compilation: True, and TensorRT: False in 6.417 seconds
With compilation: True, and TensorRT: True in 5.537 seconds