Skip to content

Instantly share code, notes, and snippets.

Created September 10, 2022 09:44
What would you like to do?
Google Summer of Code 2022 | OpenCV - Improved Imagecodecs

Google Summer of Code 2022 | OpenCV - Improved Imagecodecs

This gist contains the work that i have done in Google Summer of Code 2022 with the OpenCV organization.


Additional information about the work done is given at the below.

  1. Added optional png encoder using libspng library which is much more compact and performs better in image decoding.
  2. Added performance tests for png and jpg codecs to measure changes and added pngsuite to make sure decoding images are implemented correctly on png codecs
  3. Upgraded libjpeg-turbo version to 2.1.13 and enabled inline assembly/SIMD extension which resulted in almost 2x performance improvement on supported platforms.
  4. Proposed new api for image reading that is based on Vadim Pisarevsky -OE 34- named parameters to enable modern C++ features and to keep source-level compability when new parameters are added.
  5. Implemented a new container-like api to decode multipage image pages on demand. (Multi-page Image Decoder API)

Could Not Deliver

  1. Provide optional bit-exactness tests for OpenCV jpeg decoder to make sure that we can read jpeg images and get the same results on each platform.
  2. Insert some bits of pngcrush into png encoder to achieve better compression
  3. Multi-page image decoder Python API

Performance Comparision

PNG vs libspng comparision

With the 3024x4032 size and 11mb image, the perf results are:

Geometric mean (ms)

Name of Test libpng spng spng vs previous (x-factor) spng vs previous (score)
decode::PNG 27.878 ms 20.700 ms 1.35 FASTER
encode::PNG 54.362 ms 72.509 ms 0.75 SLOWER

libjpeg vs libjpeg-turbo SIMD enabled comparision

With the 3888x2592 size and 718kb image, the perf results are:

Geometric mean (ms)

Name of Test libjpeg libjpeg-turbo libjpeg-turbo vs previous (x-factor)
Decode::JPEG 62.426 ms 38.651 ms 1.62
Encode::JPEG 58.559 ms 33.405 ms 1.75

libjpeg-turbo vs libjpeg-turbo SIMD enabled comparision

With the 3888x2592 size and 718kb image, the perf results are:

Geometric mean (ms)

Name of Test libjpeg-turbo libjpeg-turbo with simd simd vs no simd (x-factor)
Decode::JPEG 67.759 ms 39.150 ms 1.73
Encode::JPEG 119.672 ms 33.812 ms 3.54

Additional Information

  1. libspng is an alternative to libpng. I have used this library to add an optional png codec to OpenCV. The users can enable libspng codec with -DWITH_SPNG flag. It will be built from the source.
  2. To verify simd extension works properly, I have added a new performance test for jpeg under imgcodecs module. Also, libspng does not support many features that libpng does. Such as conversion from RGB to Grayscale images. I had to implement that kind of stuff and to verify that it works, i have added pngsuite.
  3. SIMD extension is not available on every platform but it increases performance significantly when it is supported. I have copied some parts of CMake file in the libjpeg-turbo repository and created a new flag(-DWITH_LIBJPEG_TURBO_SIMD) to enable/disable SIMD extension. It is enabled by default.
  4. With the new proposed API, C++20 and above users can use designated initializers to write much more compact and readable code. It also helps developers add more parameters while keeping source level compatibility. For more information, please refer to this article that is written by Vadim Pisarevsky.
  5. Dealing with the multipage images are complicated. Existing API is a little bit complicated to do on-demand decoding on multipage images. With this API, the users can decode their multipage images much more easily. This API is inspired by standart container APIs. The user can decode any page, release that page to release memory, query image collection size and iterate with range-based for loops.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment