Skip to content

Instantly share code, notes, and snippets.

@pzread
Created January 10, 2023 23:28
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save pzread/4534cdcfcd4e23c9fd90dcc8191383aa to your computer and use it in GitHub Desktop.
Save pzread/4534cdcfcd4e23c9fd90dcc8191383aa to your computer and use it in GitHub Desktop.

Full Benchmark Summary

Raw Latencies

Benchmark Name Average Latency (ms) Median Latency (ms) Latency Standard Deviation (ms)
BertForMaskedLMTF [fp32,seqlen512,tensorflow] (exported\_tf) [default-flags][full-inference,default-flags] with IREE-CUDA @ Unknown (GPU-NVIDIA-A100-SXM4-40GB) 9.481 9.477 0.018
EfficientNetV2STF [fp32,cnn,tensorflow] (exported\_tf) [default-flags][full-inference,default-flags] with IREE-CUDA @ Unknown (GPU-NVIDIA-A100-SXM4-40GB) 11.862 11.850 0.037
MiniLML12H384Uncased [int32,seqlen128] (exported\_tf) [default-flags][full-inference,default-flags] with IREE-CUDA @ Unknown (GPU-NVIDIA-A100-SXM4-40GB) 1.707 1.708 0.007
Resnet50TF [fp32] (exported\_tf) [default-flags][full-inference,default-flags] with IREE-CUDA @ Unknown (GPU-NVIDIA-A100-SXM4-40GB) 7.906 7.892 0.045

All Compilation Metrics

Benchmark Name Compilation Time (ms) Total Dispatch Size (bytes) Total Artifact Size (bytes)
DeepLabV3_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 35148 143560 2924723
MobileSSD_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 62343 259128 18199603
PoseNet_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 26477 82088 5135281
MobileBertSquad_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 105630 80264 98483577
MobileBertSquad_int8 [int8] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 234997 6026680 31231353
MobileBertSquad_fp16 [fp16] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 106222 73352 99928633
MobileNetV1_fp32 [fp32,imagenet] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 39125 85064 17001533
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 40236 146472 14130043
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 52822 205016 10426426
PersonDetect_int8 [int8] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 28058 90024 337718
EfficientNet_int8 [int8] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 49494 197624 5048054
MiniLML12H384Uncased [int32,seqlen128] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 17199 67864 133572805
BertForMaskedLMTF [fp32,seqlen512,tensorflow] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 18745 63320 438233861
EfficientNetV2STF [fp32,cnn,tensorflow] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 59674 234616 86837125
Resnet50TF [fp32] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] default-flags,compile-stats 28748 141160 102624965
DeepLabV3_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 36099 152552 2928883
MobileSSD_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 66297 282856 18218035
PoseNet_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 22239 87208 5136049
MobileBertSquad_fp32 [fp32] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 109307 80264 98483577
MobileBertSquad_int8 [int8] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 233240 6026680 31231353
MobileBertSquad_fp16 [fp16] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 112560 73352 99928633
MobileNetV1_fp32 [fp32,imagenet] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 39401 105112 17018749
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 40335 169496 14148347
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 57130 236536 10456442
PersonDetect_int8 [int8] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 27408 90024 337718
EfficientNet_int8 [int8] (exported_tflite) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 49549 204200 5054518
MiniLML12H384Uncased [int32,seqlen128] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 21066 67864 133572805
BertForMaskedLMTF [fp32,seqlen512,tensorflow] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 19926 63320 438233861
EfficientNetV2STF [fp32,cnn,tensorflow] (exported_tf) [cpu-x86_64-cascadelake-linux-gnu] experimental-flags,fuse-padding,compile-stats 63920 542120 87136453
MiniLML12H384Uncased [int32,seqlen128] (exported_tf) [gpu-cuda-sm_80-linux-gnu] default-flags,compile-stats 15282 129656 133643369
BertForMaskedLMTF [fp32,seqlen512,tensorflow] (exported_tf) [gpu-cuda-sm_80-linux-gnu] default-flags,compile-stats 15913 173292 438350719
EfficientNetV2STF [fp32,cnn,tensorflow] (exported_tf) [gpu-cuda-sm_80-linux-gnu] default-flags,compile-stats 49618 788828 87431550
Resnet50TF [fp32] (exported_tf) [gpu-cuda-sm_80-linux-gnu] default-flags,compile-stats 21435 356252 102857621
DeepLabV3_fp32 [fp32] (exported_tflite) [cpu-riscv_64-generic-linux-gnu] default-flags,compile-stats 25340 47176 2828405
MobileBertSquad_fp32 [fp32] (exported_tflite) [cpu-riscv_64-generic-linux-gnu] default-flags,compile-stats 99281 22336 98425659
MobileNetV1_fp32 [fp32,imagenet] (exported_tflite) [cpu-riscv_64-generic-linux-gnu] default-flags,compile-stats 32337 36992 16953407
MobileBertSquad_int8 [int8] (exported_tflite) [cpu-riscv_64-generic-linux-gnu] default-flags,compile-stats 166097 1980576 27185211
PersonDetect_int8 [int8] (exported_tflite) [cpu-riscv_64-generic-linux-gnu] default-flags,compile-stats 26937 64504 312184
EfficientNet_int8 [int8] (exported_tflite) [cpu-riscv_64-generic-linux-gnu] default-flags,compile-stats 48775 143648 4994104
PersonDetect_int8 [int8] (exported_tflite) [cpu-riscv_32-generic-linux-gnu] default-flags,compile-stats 64010 913916 1161592
EfficientNet_int8 [int8] (exported_tflite) [cpu-riscv_32-generic-linux-gnu] default-flags,compile-stats 130979 2567136 7417592
DeepLabV3_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 39578 96136 2877299
MobileSSD_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 83365 286856 18227315
PoseNet_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 24421 33880 5087089
MobileBertSquad_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 112979 109096 98512441
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 58189 98952 14082555
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 56854 88392 10309818
MobileBertSquad_int8 [int8] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] default-flags,compile-stats 189132 1887768 27092409
DeepLabV3_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,compile-stats 51044 87808 2886515
MobileSSD_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,compile-stats 79418 177984 18138995
PoseNet_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,compile-stats 29702 43680 5102769
MobileBertSquad_fp32 [fp32] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,compile-stats 104803 38544 98623865
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,compile-stats 54870 111424 14109947
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,compile-stats 73934 137808 10381114
MobileBertSquad_int8 [int8] (exported_tflite) [cpu-armv8.2-a-generic-linux-android29] experimental-flags,mmt4d,dotprod,compile-stats 270558 4240304 29647929
DeepLabV3_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] default-flags,compile-stats 22314 389644 3189359
MobileSSD_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] default-flags,compile-stats 40842 541738 18515931
PoseNet_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] default-flags,compile-stats 17847 137108 5201067
MobileBertSquad_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] default-flags,compile-stats 98551 226556 98641287
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [gpu-adreno-generic-linux-android31] default-flags,compile-stats 36202 329596 14331695
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [gpu-adreno-generic-linux-android31] default-flags,compile-stats 40582 393992 10647209
DeepLabV3_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,compile-stats 19592 412560 3204232
MobileSSD_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,compile-stats 38784 604220 18568130
PoseNet_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,compile-stats 14362 159596 5216797
MobileBertSquad_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,compile-stats 111208 226556 98641287
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,compile-stats 33321 374542 14367681
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,compile-stats 37966 411026 10660014
MobileSSD_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 43624 604220 18651266
PoseNet_fp32 [fp32] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 16444 159596 5245405
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 28616 374542 14414721
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [gpu-adreno-generic-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 41453 411026 10738990
DeepLabV3_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,compile-stats 22020 178304 2978159
MobileSSD_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,compile-stats 49302 353302 18327515
PoseNet_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,compile-stats 17419 95576 5159595
MobileBertSquad_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,compile-stats 103882 129860 98544647
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,compile-stats 36143 194320 14196527
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,compile-stats 41022 263212 10516521
MobileBertSquad_fp16 [fp16] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,demote-f32-to-f16,compile-stats 116421 2972578 52951502
MobileBertSquad_int8 [int8] (exported_tflite) [gpu-mali-valhall-linux-android31] default-flags,demote-f32-to-f16,compile-stats 140589 2540222 27983813
DeepLabV3_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,compile-stats 19218 202756 2994568
MobileSSD_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,compile-stats 45589 378160 18342018
PoseNet_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,compile-stats 14939 118720 5176029
MobileBertSquad_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,compile-stats 112367 129860 98544647
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,compile-stats 28789 213966 14207169
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,compile-stats 38280 288170 10537326
MobileBertSquad_fp16 [fp16] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,demote-f32-to-f16,compile-stats 136577 2972578 52951502
MobileBertSquad_int8 [int8] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,demote-f32-to-f16,compile-stats 140696 2540222 27983813
DeepLabV3_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 20695 202756 3098760
MobileSSD_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 40465 378160 18513922
PoseNet_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 14250 118720 5235101
MobileBertSquad_fp32 [fp32] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 124354 129860 99850119
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 31789 213966 14304385
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,compile-stats 41692 288170 10700526
MobileBertSquad_fp16 [fp16] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,demote-f32-to-f16,compile-stats 137195 2972578 54256974
MobileBertSquad_int8 [int8] (exported_tflite) [gpu-mali-valhall-linux-android31] experimental-flags,fuse-padding,repeated-kernel,demote-f32-to-f16,compile-stats 194707 2540222 30539205
MobileNetV2_fp32 [fp32,imagenet] (exported_tflite) [cpu-vmvx-generic-vmvx] default-flags,compile-stats 36860 96949 14080564
MobileNetV3Small_fp32 [fp32,imagenet] (exported_tflite) [cpu-vmvx-generic-vmvx] default-flags,compile-stats 42201 165493 10386931
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment