Skip to content

Instantly share code, notes, and snippets.

@bjacob
Last active January 17, 2021 17:44
Show Gist options
  • Save bjacob/ca8e8e3129b496c00891f0558bbde965 to your computer and use it in GitHub Desktop.
Save bjacob/ca8e8e3129b496c00891f0558bbde965 to your computer and use it in GitHub Desktop.
Matmul shapes in MobileNet-v3-large, EfficientNet-Lite2 and EfficientNet-B4, 8bit quantized, by decreasing CPU time % on Pixel4
Note: EfficientNet-B4 is a huge model, unlikely to be deployed in any application except ones that would require the highest possible accuracy.
Still, it is currently an optimal trade-off point on the model-accuracy-vs-cost curves.
* 4.95% matmul shape: 1632x272x49
* 4.63% matmul shape: 960x160x196
* 4.10% matmul shape: 192x32x3136
* 3.81% matmul shape: 272x1632x49
* 3.19% matmul shape: 672x112x196
* 2.95% matmul shape: 144x24x12544
* 2.79% matmul shape: 160x960x196
* 2.42% matmul shape: 336x56x784
* 1.73% matmul shape: 32x192x3136
* 1.57% matmul shape: 2688x448x49
* 1.44% matmul shape: 48x27x12544
* 1.41% matmul shape: 112x672x196
* 1.22% matmul shape: 448x2688x49
* 1.06% matmul shape: 24x48x12544
* 1.06% matmul shape: 1792x448x49
* 1.04% matmul shape: 24x24x12544
* 0.96% matmul shape: 448x1632x49
* 0.67% matmul shape: 56x336x784
* 0.67% matmul shape: 32x144x3136
* 0.29% matmul shape: 160x672x196
* 0.16% matmul shape: 112x336x196
* 0.13% matmul shape: 272x960x49
* 0.08% matmul shape: 56x192x784
EfficientNet-Lite2 is a more reasonably sized model compared to EfficientNet-B4, while having higher accuracy than MobileNet-v3-large. It could be used in applications demanding high accuracy.
* 6.10% matmul shape: 144x24x4225
* 5.49% matmul shape: 96x16x16900
* 5.40% matmul shape: 720x120x289
* 5.31% matmul shape: 1248x208x81
* 3.33% matmul shape: 208x1248x81
* 3.33% matmul shape: 288x48x1089
* 3.10% matmul shape: 32x27x16900
* 3.05% matmul shape: 528x88x289
* 2.82% matmul shape: 120x720x289
* 2.39% matmul shape: 24x144x4225
* 2.07% matmul shape: 16x32x16900
* 1.74% matmul shape: 1280x352x81
* 1.55% matmul shape: 88x528x289
* 1.55% matmul shape: 48x288x1089
* 1.27% matmul shape: 352x1248x81
* 0.80% matmul shape: 208x720x81
* 0.70% matmul shape: 24x96x4225
* 0.66% matmul shape: 120x528x289
* 0.66% matmul shape: 48x144x1089
* 0.38% matmul shape: 88x288x289
Note: MobiletNet-v3-large is a typical model to deploy in current applications thanks to its moderate size.
* 7.63% matmul shape: 64x16x12544
* 4.35% matmul shape: 672x112x196
* 4.18% matmul shape: 16x27x12544
* 4.18% matmul shape: 960x160x49
* 4.15% matmul shape: 72x24x3136
* 3.76% matmul shape: 16x16x12544
* 2.08% matmul shape: 120x40x784
* 1.91% matmul shape: 24x72x3136
* 1.71% matmul shape: 160x960x49
* 1.68% matmul shape: 240x40x784
* 1.57% matmul shape: 480x80x196
* 1.46% matmul shape: 40x120x784
* 1.35% matmul shape: 24x64x3136
* 1.12% matmul shape: 112x480x196
* 1.12% matmul shape: 112x672x196
* 0.81% matmul shape: 184x80x196
* 0.73% matmul shape: 200x80x196
* 0.70% matmul shape: 160x672x49
* 0.56% matmul shape: 80x200x196
* 0.56% matmul shape: 80x184x196
* 0.53% matmul shape: 40x72x784
* 0.53% matmul shape: 80x240x196

Visualization of the MobileNet-v3-large shapes (ordered similarly by decreasing time percentage, so the most important shapes come first).

mobilenet-v3-large-matmuls-ordered

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment