Google Summer of Code 2022 | OpenCV - Improved Imagecodecs

GSoC22_OpenCV.md

Google Summer of Code 2022 | OpenCV - Improved Imagecodecs

This gist contains the work that i have done in Google Summer of Code 2022 with the OpenCV organization.

Delivered

Additional information about the work done is given at the below.

1. Added optional png encoder using libspng library which is much more compact and performs better in image decoding.
2. Added performance tests for png and jpg codecs to measure changes and added pngsuite to make sure decoding images are implemented correctly on png codecs
3. Upgraded libjpeg-turbo version to 2.1.13 and enabled inline assembly/SIMD extension which resulted in almost 2x performance improvement on supported platforms.
4. Proposed new api for image reading that is based on Vadim Pisarevsky -OE 34- named parameters to enable modern C++ features and to keep source-level compability when new parameters are added.
5. Implemented a new container-like api to decode multipage image pages on demand. (Multi-page Image Decoder API)

Could Not Deliver

1. Provide optional bit-exactness tests for OpenCV jpeg decoder to make sure that we can read jpeg images and get the same results on each platform.
2. Insert some bits of pngcrush into png encoder to achieve better compression
3. Multi-page image decoder Python API

Performance Comparision

PNG vs libspng comparision

With the 3024x4032 size and 11mb image, the perf results are:

Geometric mean (ms)

Name of Test	libpng	spng	spng vs previous (x-factor)	spng vs previous (score)
decode::PNG	27.878 ms	20.700 ms	1.35	FASTER
encode::PNG	54.362 ms	72.509 ms	0.75	SLOWER

libjpeg vs libjpeg-turbo SIMD enabled comparision

With the 3888x2592 size and 718kb image, the perf results are:

Geometric mean (ms)

Name of Test	libjpeg	libjpeg-turbo	libjpeg-turbo vs previous (x-factor)
Decode::JPEG	62.426 ms	38.651 ms	1.62
Encode::JPEG	58.559 ms	33.405 ms	1.75

libjpeg-turbo vs libjpeg-turbo SIMD enabled comparision

With the 3888x2592 size and 718kb image, the perf results are:

Geometric mean (ms)

Name of Test	libjpeg-turbo	libjpeg-turbo with simd	simd vs no simd (x-factor)
Decode::JPEG	67.759 ms	39.150 ms	1.73
Encode::JPEG	119.672 ms	33.812 ms	3.54

Additional Information

1. libspng is an alternative to libpng. I have used this library to add an optional png codec to OpenCV. The users can enable libspng codec with -DWITH_SPNG flag. It will be built from the source.
2. To verify simd extension works properly, I have added a new performance test for jpeg under imgcodecs module. Also, libspng does not support many features that libpng does. Such as conversion from RGB to Grayscale images. I had to implement that kind of stuff and to verify that it works, i have added pngsuite.
3. SIMD extension is not available on every platform but it increases performance significantly when it is supported. I have copied some parts of CMake file in the libjpeg-turbo repository and created a new flag(-DWITH_LIBJPEG_TURBO_SIMD) to enable/disable SIMD extension. It is enabled by default.
4. With the new proposed API, C++20 and above users can use designated initializers to write much more compact and readable code. It also helps developers add more parameters while keeping source level compatibility. For more information, please refer to this article that is written by Vadim Pisarevsky.
5. Dealing with the multipage images are complicated. Existing API is a little bit complicated to do on-demand decoding on multipage images. With this API, the users can decode their multipage images much more easily. This API is inspired by standart container APIs. The user can decode any page, release that page to release memory, query image collection size and iterate with range-based for loops.