FFmpeg DNN module supports the dnn-based filters, it has three backends, namely TensorFlow, native and OpenVINO. Currently, OpenVINO inference on GPU has been enabled. This project is aim to improve the performance by making inference, decode and encode only on GPU, without memory copy between CPU and GPU.
Move to this repo to see all my work.
- Add support for VA RemoteBlob in OpenVINO C API .
- GPU decode & inference in dnn_detect, dnn_classify and dnn_processing filter.
- Transfer ffmpeg DNN openvino backend to ov2.0 C API for better performance.
- Add support for directly encode frame which is from openvino output on GPU to finish the full gpu pipeline.
As OpenVINO C API doesn't support RemoteBlob( it's named RemoteTensor in ov2.0 ), we need to write some extra code to wrap openvino C++ API and expose it in C API. I finish it in my repo forked from OpenVINO but not merged. OpenVINO does not support output result in VA memory and color format converts in postprocess, so only GPU decode & inference in FFmpeg DNN module is enabled now. In ov2.0, preprocessing operations are added to inference models and fully occupies the accelerator, freeing CPU resources. I also made some try to compare preprocess performance between ov1.0 and ov2.0, for those input image which need resize, ov2.0 will work more efficiently. Additionally, ov2.0 has more features and keeps the ov1.0 API in namespace IE which means that we can keep the original functionality when upgrading the OpenVINO version. Hope to achieve such goals together with the help of the community