This gist contains the work that i have done in Google Summer of Code 2022 with the OpenCV organization.
Additional information about the work done is given at the below.
- Added optional png encoder using libspng library which is much more compact and performs better in image decoding.
- Added performance tests for png and jpg codecs to measure changes and added pngsuite to make sure decoding images are implemented correctly on png codecs
- Upgraded libjpeg-turbo version to 2.1.13 and enabled inline assembly/SIMD extension which resulted in almost 2x performance improvement on supported platforms.
- Proposed new api for image reading that is based on Vadim Pisarevsky -OE 34- named parameters to enable modern C++ features and to keep source-level compability when new parameters are added.
- Implemented a new container-like api to decode multipage image pages on demand. (Multi-page Image Decoder API)
- Provide optional bit-exactness tests for OpenCV jpeg decoder to make sure that we can read jpeg images and get the same results on each platform.
- Insert some bits of pngcrush into png encoder to achieve better compression
- Multi-page image decoder Python API
With the 3024x4032 size and 11mb image, the perf results are:
Geometric mean (ms)
| Name of Test | libpng | spng | spng vs previous (x-factor) | spng vs previous (score) |
|---|---|---|---|---|
| decode::PNG | 27.878 ms | 20.700 ms | 1.35 | FASTER |
| encode::PNG | 54.362 ms | 72.509 ms | 0.75 | SLOWER |
With the 3888x2592 size and 718kb image, the perf results are:
Geometric mean (ms)
| Name of Test | libjpeg | libjpeg-turbo | libjpeg-turbo vs previous (x-factor) |
|---|---|---|---|
| Decode::JPEG | 62.426 ms | 38.651 ms | 1.62 |
| Encode::JPEG | 58.559 ms | 33.405 ms | 1.75 |
With the 3888x2592 size and 718kb image, the perf results are:
Geometric mean (ms)
| Name of Test | libjpeg-turbo | libjpeg-turbo with simd | simd vs no simd (x-factor) |
|---|---|---|---|
| Decode::JPEG | 67.759 ms | 39.150 ms | 1.73 |
| Encode::JPEG | 119.672 ms | 33.812 ms | 3.54 |
- libspng is an alternative to libpng. I have used this library to add an optional png codec to OpenCV. The users can enable libspng codec with -DWITH_SPNG flag. It will be built from the source.
- To verify simd extension works properly, I have added a new performance test for jpeg under imgcodecs module. Also, libspng does not support many features that libpng does. Such as conversion from RGB to Grayscale images. I had to implement that kind of stuff and to verify that it works, i have added pngsuite.
- SIMD extension is not available on every platform but it increases performance significantly when it is supported. I have copied some parts of CMake file in the libjpeg-turbo repository and created a new flag(-DWITH_LIBJPEG_TURBO_SIMD) to enable/disable SIMD extension. It is enabled by default.
- With the new proposed API, C++20 and above users can use
designated initializersto write much more compact and readable code. It also helps developers add more parameters while keeping source level compatibility. For more information, please refer to this article that is written by Vadim Pisarevsky. - Dealing with the multipage images are complicated. Existing API is a little bit complicated to do on-demand decoding on multipage images. With this API, the users can decode their multipage images much more easily. This API is inspired by standart container APIs. The user can decode any page, release that page to release memory, query image collection size and iterate with range-based for loops.