Spawning multiple ffmpeg processes with xargs:
On standard NVIDIA GPUs (Not the Quadros and Tesla lines), NVENC encodes are limited to two simultaneous sessions. The sample below illustrates how to pass a list of AVI files to ffmpeg and encode them to HEVC on two encode sessions:
$ find Videos/ -type f -name \*.avi -print | sed 's/.avi$//' |\ xargs -n 1 -I@ -P 2 ffmpeg -i "@.avi" -c:a aac -c:v hevc_nvenc "@.mp4"
This will find all files with the ending
.avi in the directory
Videos/ and transcode them into HEVC/H265+AAC files with the ending .mp4. The noteworthy part here is the
-P 2 to xargs, which starts up to two processes in parallel.
You may modify the command above to specify any other file wildcards to suit your case.
Note that you can also combine the encode process with CUDA's accelerated NPP for faster image scaling on top of it. So instead of doing this:
ffmpeg -i input.avi \ -c:a aac -b:a 128k \ -s hd480 \ -c:v hevc_nvenc -b:v 1024k \ -y output.mp4
You can now replace the
"-s hd480" with a (somewhat complicated) filter and do:
ffmpeg -i input.avi \ -c:a aac -b:a 128k \ -filter:v hwupload_cuda,scale_npp=w=852:h=480:format=nv12:interp_algo=lanczos,hwdownload,format=nv12 \ -c:v hevc_nvenc -b:v 1024k \ -y output.mp4
This uploads each frame to the CUDA engine where it then scales these with the lanczos algorithm to a size of 852x480 pixels (same as
hd480). The format "nv12" as well as "
yupv420p" can be used here and needs to be specified in order for it to work. "nv12" seems to run a bit faster than "yuv420p" but both formats are recognized by the NVENC encoder.
Now, running two ffmpeg processes in parallel and using CUDA based NPP (Nvidia Performance Primitives extensions, see this for the build guide) for scaling, you can rapidly encode video(s) using an NVENC capable GPU.
find Videos/ -type f -name \*.avi -print0 | sed 's/.avi$//' | xargs -0 -n 1 -I@ -P 2 \ ffmpeg -i "@.avi" \ -c:a aac -b:a 128k \ -filter:v hwupload_cuda,scale_npp=w=852:h=480:format=nv12:interp_algo=lanczos,hwdownload,format=nv12 \ -c:v hevc_nvenc -b:v 1024k \ -y "@.mp4"
Of course, if you have a Tesla or Quadro GPU, it's possible to launch more than two simultaneous encode sessions, as your need may arise. Merely modify the
-P $(n) argument passed to
$(n) is the number of simultaneous ffmpeg processes.
Good use of
👍 . I've used it before to parallelize rsync.