Skip to content

Instantly share code, notes, and snippets.

@pzread
Created January 11, 2023 21:42
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save pzread/0340c388ac1d85e82ab2c569afab51e8 to your computer and use it in GitHub Desktop.
Save pzread/0340c388ac1d85e82ab2c569afab51e8 to your computer and use it in GitHub Desktop.

Full Benchmark Summary

Raw Latencies

Benchmark Name Average Latency (ms) Median Latency (ms) Latency Standard Deviation (ms)
BertForMaskedLMTF [fp32,seqlen512,tensorflow] (exported\_tf) [default-flags][full-inference,default-flags] with IREE-CUDA @ Unknown (GPU-NVIDIA-A100-SXM4-40GB) 9.613 9.612 0.036
EfficientNetV2STF [fp32,cnn,tensorflow] (exported\_tf) [default-flags][full-inference,default-flags] with IREE-CUDA @ Unknown (GPU-NVIDIA-A100-SXM4-40GB) 11.945 11.926 0.069
MiniLML12H384Uncased [int32,seqlen128] (exported\_tf) [default-flags][full-inference,default-flags] with IREE-CUDA @ Unknown (GPU-NVIDIA-A100-SXM4-40GB) 1.708 1.705 0.009
Resnet50TF [fp32] (exported\_tf) [default-flags][full-inference,default-flags] with IREE-CUDA @ Unknown (GPU-NVIDIA-A100-SXM4-40GB) 7.956 7.940 0.062

All Compilation Metrics

Benchmark Name Compilation Time (ms) Total Dispatch Size (bytes) Total Artifact Size (bytes)
DeepLabV3_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 29233 154264 2935411
MobileSSD_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 53716 267512 18207731
PoseNet_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 18853 87064 5140209
MobileBertSquad_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 92318 80264 98483129
MobileBertSquad_int8 [int8] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 218345 6026936 31229689
MobileBertSquad_fp16 [fp16] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 88958 73352 99928569
MobileNetV1_fp32 [fp32,imagenet] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 35192 89624 17006077
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 36813 153560 14137083
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 48005 228488 10449978
PersonDetect_int8 [int8] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 22844 90168 337526
EfficientNet_int8 [int8] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 42949 197624 5048054
MiniLML12H384Uncased [int32,seqlen128] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 15509 68248 133570373
BertForMaskedLMTF [fp32,seqlen512,tensorflow] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 16367 64728 438235269
EfficientNetV2STF [fp32,cnn,tensorflow] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 50517 238936 86841797
Resnet50TF [fp32] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 26951 145352 102629125
DeepLabV3_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 27561 163208 2939507
MobileSSD_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 54140 291208 18226035
PoseNet_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 18982 92216 5141041
MobileBertSquad_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 89147 80264 98483129
MobileBertSquad_int8 [int8] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 212856 6026936 31229689
MobileBertSquad_fp16 [fp16] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 90469 73352 99928569
MobileNetV1_fp32 [fp32,imagenet] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 33951 109672 17023357
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 33610 176568 14155451
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 51796 259304 10479290
PersonDetect_int8 [int8] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 21634 90168 337526
EfficientNet_int8 [int8] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 44532 204200 5054518
MiniLML12H384Uncased [int32,seqlen128] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 18263 68248 133570373
BertForMaskedLMTF [fp32,seqlen512,tensorflow] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 18742 64728 438235269
EfficientNetV2STF [fp32,cnn,tensorflow] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 55594 546488 87141189
MiniLML12H384Uncased [int32,seqlen128] (exported_tf) [gpu-cuda-sm_80-linux-gnu] default-flags,compile-stats 13317 127844 133638697
BertForMaskedLMTF [fp32,seqlen512,tensorflow] (exported_tf) [gpu-cuda-sm_80-linux-gnu] default-flags,compile-stats 14530 173292 438350719
EfficientNetV2STF [fp32,cnn,tensorflow] (exported_tf) [gpu-cuda-sm_80-linux-gnu] default-flags,compile-stats 40872 788828 87431870
Resnet50TF [fp32] (exported_tf) [gpu-cuda-sm_80-linux-gnu] default-flags,compile-stats 20473 356252 102857621
DeepLabV3_fp32 [fp32] (exported_tflite) [cpu-riscv_64-generic-linux-gnu] default-flags,compile-stats 20462 47176 2828405
MobileBertSquad_fp32 [fp32] (exported_tflite) [cpu-riscv_64-generic-linux-gnu] default-flags,compile-stats 88422 22336 98425211
MobileNetV1_fp32 [fp32,imagenet] (exported_tflite) [cpu-riscv_64-generic-linux-gnu] default-flags,compile-stats 31613 36944 16953343
MobileBertSquad_int8 [int8] (exported_tflite) [cpu-riscv_64-generic-linux-gnu] default-flags,compile-stats 144965 1980856 27183611
PersonDetect_int8 [int8] (exported_tflite) [cpu-riscv_64-generic-linux-gnu] default-flags,compile-stats 22146 64632 311992
EfficientNet_int8 [int8] (exported_tflite) [cpu-riscv_64-generic-linux-gnu] default-flags,compile-stats 40049 143648 4994104
PersonDetect_int8 [int8] (exported_tflite) [cpu-riscv_32-generic-linux-gnu] default-flags,compile-stats 49621 913356 1160696
EfficientNet_int8 [int8] (exported_tflite) [cpu-riscv_32-generic-linux-gnu] default-flags,compile-stats 110795 2555024 7405496
DeepLabV3_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 33025 96136 2877299
MobileSSD_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 74889 287352 18227571
PoseNet_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 20645 33880 5087089
MobileBertSquad_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 96009 109096 98511993
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 41640 98952 14082555
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 49901 88488 10310010
MobileBertSquad_int8 [int8] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 161095 1887912 27090681
DeepLabV3_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,compile-stats 40473 87792 2886515
MobileSSD_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,compile-stats 67236 178112 18138739
PoseNet_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,compile-stats 25475 43680 5102769
MobileBertSquad_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,compile-stats 86481 38544 98623353
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,compile-stats 44584 111424 14109947
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,compile-stats 63462 138560 10380090
MobileBertSquad_int8 [int8] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,dotprod,compile-stats 244267 4233952 29639737
DeepLabV3_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] default-flags,compile-stats 17971 389644 3189359
MobileSSD_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] default-flags,compile-stats 32193 542338 18516251
PoseNet_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] default-flags,compile-stats 13552 137108 5201067
MobileBertSquad_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] default-flags,compile-stats 81410 226556 98640839
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [gpu-adreno-generic-linux-android31] default-flags,compile-stats 27708 329596 14331695
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [gpu-adreno-generic-linux-android31] default-flags,compile-stats 32724 394688 10647977
DeepLabV3_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,compile-stats 15201 412560 3204232
MobileSSD_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,compile-stats 27662 604820 18568322
PoseNet_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,compile-stats 11799 159596 5216797
MobileBertSquad_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,compile-stats 97206 226556 98640839
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,compile-stats 24204 374542 14367681
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,compile-stats 31642 411722 10660782
MobileSSD_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 34815 604820 18651522
PoseNet_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 12076 159596 5245405
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 24858 374542 14414721
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 32620 411722 10739758
DeepLabV3_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,compile-stats 16614 178304 2978159
MobileSSD_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,compile-stats 40221 353770 18327707
PoseNet_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,compile-stats 14292 95576 5159595
MobileBertSquad_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,compile-stats 76995 129860 98544135
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,compile-stats 27807 194320 14196527
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,compile-stats 32574 263908 10517289
MobileBertSquad_fp16 [fp16] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,demote-f32-to-f16,compile-stats 94123 2972578 52951438
MobileBertSquad_int8 [int8] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,demote-f32-to-f16,compile-stats 114553 2541160 27983220
DeepLabV3_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,compile-stats 13531 202756 2994568
MobileSSD_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,compile-stats 35770 378628 18342146
PoseNet_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,compile-stats 11942 118720 5176029
MobileBertSquad_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,compile-stats 87227 129860 98544135
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,compile-stats 20218 213966 14207169
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,compile-stats 31647 288866 10538094
MobileBertSquad_fp16 [fp16] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,demote-f32-to-f16,compile-stats 114822 2972578 52951438
MobileBertSquad_int8 [int8] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,demote-f32-to-f16,compile-stats 112275 2541160 27983220
DeepLabV3_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 17125 202756 3098760
MobileSSD_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 31575 378628 18513986
PoseNet_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 11001 118720 5235101
MobileBertSquad_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 102328 129860 99849671
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 28262 213966 14304385
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 33691 288866 10701294
MobileBertSquad_fp16 [fp16] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,demote-f32-to-f16,compile-stats 119625 2972578 54256910
MobileBertSquad_int8 [int8] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,demote-f32-to-f16,compile-stats 166517 2541160 30538612
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [cpu-vmvx-generic-vmvx] default-flags,compile-stats 28986 96949 14080564
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [cpu-vmvx-generic-vmvx] default-flags,compile-stats 33311 165749 10387251
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment