Created
November 7, 2017 19:59
-
-
Save brian-armstrong-discord/1eb10046edb91167b7187513cc306d65 to your computer and use it in GitHub Desktop.
Lilliput vs. Pillow-simd
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Setup: Intel Haswell, Debian. Both tests ran against the same image libraries (libjpeg-turbo etc). | |
Both ran on a single thread only. Save qualities were 85 for JPEG and WEBP, compress level 7 for PNG. | |
Both tests ran many iterations and then averaged results. | |
Benchmarking code can be found at https://github.com/discordapp/lilliput-bench | |
Test types: | |
- Header reading: We don't actually know what's in a blob of image bytes when we get it. Reading the header allows us | |
to decide if we want to resize the image. | |
- Resize, 256x256 => 32x32: We have lots of icon-sized assets that we need resized into various smaller formats. | |
These make up a pretty sizable percentage of our resizes. | |
- Resize, 1920x1080 => 800x600: This test is a somewhat typical case for some arbitrary remote image that we've | |
fetched and have to crop and resize down to a size the client can use. This gives us a test case for large images. | |
- Transcode: A few various test cases that are useful for us. Just image decode/encode only, no resizing. | |
Pillow-simd | |
====================== | |
JPEG 1920x1080 header read: 1920x1080, avg: 0.037861 ms min: 0.033855 ms max: 13.420820 ms | |
PNG 1920x1080 header read: 1920x1080, avg: 0.041355 ms min: 0.038862 ms max: 0.269175 ms | |
WEBP 1920x1080 header read: 1920x1080, avg: 37.851110 ms min: 35.255909 ms max: 65.973997 ms | |
GIF 1920x1080 header read: 1920x1080, avg: 0.034195 ms min: 0.031948 ms max: 0.703096 ms | |
JPEG 256x256 => 32x32: 1083 Bytes, avg: 0.97 ms min: 0.89 ms max: 1.75 ms | |
PNG 256x256 => 32x32: 2090 Bytes, avg: 1.39 ms min: 1.31 ms max: 2.42 ms | |
WEBP 256x256 => 32x32: 1366 Bytes, avg: 2.88 ms min: 2.65 ms max: 4.70 ms | |
GIF 256x256 => 32x32: 61644 Bytes, avg: 71.76 ms min: 68.54 ms max: 96.00 ms | |
JPEG 1920x1080 => 800x600: 123522 Bytes, avg: 39.03 ms min: 34.09 ms max: 42.21 ms | |
PNG 1920x1080 => 800x600: 856122 Bytes, avg: 278.56 ms min: 272.84 ms max: 298.88 ms | |
WEBP 1920x1080 => 800x600: 93564 Bytes, avg: 147.06 ms min: 142.81 ms max: 171.15 ms | |
GIF 1920x1080 => 800x600: 4017933 Bytes, avg: 1819.24 ms min: 1734.70 ms max: 1915.06 ms | |
PNG 256x256 => WEBP 256x256: 9790 Bytes, avg: 20.67 ms min: 20.12 ms max: 31.40 ms | |
JPEG 256x256 => PNG 256x256: 38724 Bytes, avg: 7.13 ms min: 6.76 ms max: 9.91 ms | |
GIF 256x256 => PNG 256x256: 8511 Bytes, avg: 1.50 ms min: 1.44 ms max: 1.92 ms | |
Lilliput | |
====================== | |
JPEG 1920x1080 header read: 1920x1080, avg: 0.005083 ms, min: 0.004398 ms, max: 0.663941 ms | |
PNG 1920x1080 header read: 1920x1080, avg: 0.003636 ms, min: 0.003362 ms, max: 0.134671 ms | |
WEBP 1920x1080 header read: 1920x1080, avg: 0.002145 ms, min: 0.001795 ms, max: 1.949846 ms | |
GIF 1920x1080 header read: 1920x1080, avg: 0.003795 ms, min: 0.003521 ms, max: 0.140039 ms | |
JPEG 256x256 => 32x32: 1078 Bytes, avg: 0.62 ms, min: 0.58 ms, max: 1.07 ms | |
PNG 256x256 => 32x32: 1411 Bytes, avg: 0.92 ms, min: 0.88 ms, max: 1.43 ms | |
WEBP 256x256 => 32x32: 946 Bytes, avg: 3.19 ms, min: 2.87 ms, max: 4.57 ms | |
GIF 256x256 => 32x32: 18310 Bytes, avg: 27.96 ms, min: 27.31 ms, max: 39.77 ms | |
JPEG 1920x1080 => 800x600: 117991 Bytes, avg: 37.30 ms, min: 36.06 ms, max: 50.85 ms | |
PNG 1920x1080 => 800x600: 722169 Bytes, avg: 178.61 ms, min: 176.41 ms, max: 203.03 ms | |
WEBP 1920x1080 => 800x600: 88172 Bytes, avg: 129.14 ms, min: 124.67 ms, max: 171.35 ms | |
GIF 1920x1080 => 800x600: 2372725 Bytes, avg: 3255.42 ms, min: 3220.95 ms, max: 3301.01 ms | |
PNG 256x256 => WEBP 256x256: 9790 Bytes, avg: 22.52 ms, min: 21.53 ms, max: 31.95 ms | |
JPEG 256x256 => PNG 256x256: 40134 Bytes, avg: 6.38 ms, min: 6.20 ms, max: 9.17 ms | |
GIF 256x256 => PNG 256x256: 18053 Bytes, avg: 5.57 ms, min: 5.44 ms, max: 7.88 ms | |
Conclusion: | |
Lilliput seems to be a suitable replacement for pillow simd for our use cases. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@homm For what it's worth, I'm not seeing the same results you are, using:
On a virtualized Ubuntu 20.04 system with 6 cores (host CPU: Ryzen 5 2600X):
lscpu output
lsb-release output
uname output
Here are my results with Lilliput:
and Pillow-SIMD:
A few of my observations:
JPEG 1920x1080 => 800x600
Lilliput: 20.43ms, Pillow-SIMD: 14.77msGIF 1920x1080 => 800x600
Lilliput: 72.85ms, Pillow-SIMD: 69.07msGIF 256x256 => PNG 256x256
Lilliput: 11.13ms, Pillow-SIMD: 3.06ms (this is significant, but not a common real world use case)JPEG 256x256 => PNG 256x256
, Lilliput is about 2x faster. InPNG 1920x1080 => 800x600
, Lilliput is over 70% faster. InPNG 256x256 => 32x32
, Lilliput is over 35% faster. InGIF 256x256 => 32x32
, Lilliput is almost 60% faster.Overall this shows a very compelling advantage to Lilliput, even in a fair competition.